Carbon Emissions Metrics

The module tracks carbon emissions to provide transparency into the environmental impact of each LLM call. Carbon emissions are being estimated and tracked in three ways: Carbon Emissions of the Processors, SCI per API call, and SCI per 10K API calls. These metrics help users understand and optimize their environmental footprint alongside the cost of LLM operations.

• Carbon Emissions of the Processor: This metric shows the carbon emissions associated with the specific processor selected when running the scans. It helps users assess the environmental impact of their infrastructure choices.

• SCI per API Call: The SCI (Software Carbon Intensity) metric is a metric proposed by the Green Software Foundation and tracks carbon emissions per API call. It measures the amount of carbon dioxide (CO2 in KG equivalent) emissions generated per individual LLM API call. This metric provides insight into the carbon impact of each query processed.

• SCI per 10K API Calls: This metric gives a broader view, showing the cumulative carbon emissions associated with 10,000 API calls. It helps estimate the environmental impact over high-volume usage patterns.

Processor-Specific Carbon Emissions

The carbon emissions for the processor selected are calculated based on the type of hardware being used. This calculation factors in the processor's Thermal Design Power (TDP) and the carbon intensity of the cloud provider and region hosting the processor. The steps for calculating the carbon emissions are as follows:

1. TDP: The processor's Thermal Design Power (TDP) is taken from the processor details. TDP represents the amount of heat the processor generates under maximum load, which is used to estimate the energy consumption.

2. Carbon Intensity of the Cloud Provider and Region: The carbon intensity factor is retrieved by a lookup of the carbon impact of the selected cloud provider and region from the carbon impact dataset. The data provides the average carbon intensity of electricity generation in the selected region and cloud provider, measured in kilograms of CO2 emitted (kgCO2 eq.).

3. Carbon Emissions Calculation: The carbon emissions are then calculated by multiplying the TDP of the processor by the carbon intensity factor of the cloud provider and region, and the time the processor is active. This gives the total carbon emissions in kilograms, which is then converted from grams to kilograms:

   • TDP: Thermal Design Power (in watts)
   • Impact: Carbon intensity factor (in gCO2/kWh)
   • Time: Time the processor is active to run all the LLM calls

For more details on the carbon intensity dataset, you can refer to the carbon impact dataset for a comprehensive lookup of the carbon emissions by cloud provider and region.

SCI per API Call

The SCI per API call is calculated by considering both the energy consumed during the execution of a single API call and the carbon intensity factor of the electricity used to power the processor, as well as the embodied emissions of the hardware. The formula is as follows:

SCI = (E * I) + M per R

This gives the carbon emissions in kilograms of CO2 (kgCO2) for each individual API call.

SCI per 10K API Calls

The SCI per 10K API calls is a broader measure, calculated by multiplying the SCI per API call by 10,000. This helps estimate the environmental impact over high-volume usage scenarios and provides a clearer picture of the overall carbon emissions for large-scale operations.

SCI Calculation

The SCI (Software Carbon Intensity) is calculated using the following formula:

SCI = (E * I) + M per R

Where:

• (E) - Energy consumption (in kilowatt hours) for different components of the software boundary over a given time period.
  Examples include:

  • CPU/GPUs at different percentages of utilization
  • Data storage
  • Memory allocation
  • Data transferred over a network

• (I) - Energy Carbon Intensity: The carbon intensity of electricity, which measures how much carbon dioxide (CO2) emissions are produced per kilowatt-hour (kWh) of electricity consumed. This can be obtained from regional or marginal emissions data, often provided as gCO2/kWh.

• (M) - Embodied Carbon (M): Embodied carbon, also known as embedded carbon, refers to the amount of carbon emitted during the creation and disposal of a hardware device. When software runs on a device, a fraction of the total embodied emissions of the device is allocated to the software. This fraction is the value of M used in the SCI equation, representing the embodied emissions associated with running the software on a particular hardware component.

• (R) - The Functional Unit: This defines how your application scales. For example, if your application scales by APIs, you would choose API as your functional unit.

SCI Guidance and Reference Datasets

Internally, the SCI guidance project provides pointers to various reference datasets available in public forums or from Original Equipment Manufacturers (OEMs). These GSF recommended datasets provide reference values for the components E, I, M, and R, which help in calculating the SCI score.

Note

Here's a list of assumptions that we make today while making the Carbon Calculations (SCI values):

• 100% Machine (CPU and GPU) usage when API call is running
• Assuming every model is deployed on the same selected infrastructure in Optimize:ai
• No carbon offsetting taken into account from the cloud providers.
• SCI Guidance estimates hardware in use for 4 years, or 35040 hours. Our time unit is 1 hour; hence time share is 1/35040.
• Selected Infrastructure is only used for this project throughout the infrastructure lifespan (No resource share taken into account). -API call has a latency of x seconds (hence 3600/x API calls per hour).

For more information on the SCI specification and the datasets used on the website, visit the Green Software Foundation SCI Guide.

For more information and to see how Trustwise leverages the SCI metric to optimize the system for better carbon efficiency, visit Trustwise GSF Case Study.