Ecological impact

If you want to track the ecological impact of your LLM Usage, you can enable it in the Otoroshi LLM Extension (it should be enabled by default).

Configuration

impacts {
  enabled = true
  enabled = ${?CLOUD_APIM_EXTENSIONS_LLM_EXTENSION_IMPACTS_ENABLED}
  embed-description-in-json = true
  embed-description-in-json = ${?CLOUD_APIM_EXTENSIONS_LLM_EXTENSION_IMPACTS_EMBED_DESCRIPTION_IN_JSON}
  electricity-mix = "WOR"
  electricity-mix = ${?CLOUD_APIM_EXTENSIONS_LLM_EXTENSION_IMPACTS_ELECTRICITY_MIX}
  embed-impacts-in-responses = false
  embed-impacts-in-responses = ${?CLOUD_APIM_EXTENSIONS_LLM_EXTENSION_IMPACTS_EMBED_IMPACTS_IN_RESPONSES}
  custom-models = "{\"aliases\": [],\"models\": []}"
  custom-models = ${?CLOUD_APIM_EXTENSIONS_LLM_EXTENSION_IMPACTS_CUSTOM_MODELS}
  custom-electricity-mix = "{\"electricity_mixes\": []}"
  custom-electricity-mix = ${?CLOUD_APIM_EXTENSIONS_LLM_EXTENSION_IMPACTS_CUSTOM_ELECTRICITY_MIX}
}

Parameter	Type	Default	Description
`enabled`	boolean	`true`	Enable or disable ecological impact tracking
`embed-description-in-json`	boolean	`true`	Include human-readable descriptions in the impact JSON output
`electricity-mix`	string	`"WOR"`	Default electricity mix zone code (see available zones below)
`embed-impacts-in-responses`	boolean	`false`	Always embed impacts in LLM responses
`custom-models`	string (JSON)	`"{}"`	Custom model architectures for impact computation (see below)
`custom-electricity-mix`	string (JSON)	`"{}"`	Custom electricity mix definitions (see below)

Once it's enabled, audit events of kind LLMUsageAudit will have an impacts field. You can also embed the impacts value in your LLM responses using the impacts.embed-impacts-in-responses config or by adding ?embed_impacts=true to your query. Please remember that the impacts value is just an estimate of the real ecological impact of your LLM usage.

Impact metrics

The ecological impact output contains the following metrics:

Metric	Unit	Description
`energy`	kWh	Energy: related to the final electricity consumption
`gwp`	kgCO2eq	Global Warming Potential (GWP): related to climate change, commonly known as GHG emissions
`adpe`	kgSbeq	Abiotic Depletion Potential for Elements (ADPe): related to the depletion of minerals and metals
`wcf`	L	Water Consumption Footprint (WCF): water used to cool the hardware and to produce the electricity it drew. Reported under `usage` and at the top level — unlike the others it has no embodied counterpart
`pe`	MJ	Primary Energy (PE): related to the energy consumed from primary sources like oil, gas or coal

Each metric contains a value with min, max, and avg estimates. The output is split into:

Top-level metrics: combined usage + embodied impacts
usage: impacts of the energy consumption during model execution
embodied: impacts related to resource extraction, manufacturing, and transportation of the hardware

Supported providers

The ecological impact computation is based on project EcoLogits and only supports right now the following providers:

OpenAI
Anthropic
Mistral
Gemini
Cohere
HuggingFace

For unsupported providers, you can set metadata on your provider entities to map them to a supported provider/model:

eco-impacts-provider: the provider used for eco impacts computation
eco-impacts-model: the model used for eco impacts computation
eco-impacts-electricity-mix-zone: the electricity mix zone to use for this provider

Available electricity mix zones

Zones are ISO 3166-1 alpha-3 country codes, plus WOR for the world average — which is the default and the right choice when you do not know where the provider actually runs.

The full list is the one shipped by ecologits and lives in data/eg-elec.json, currently 215 zones. Listing them here would only go stale at the next refresh; a few common ones:

Code	Zone	Code	Zone
`WOR`	World average (default)	`USA`	United States
`FRA`	France	`DEU`	Germany
`GBR`	United Kingdom	`SWE`	Sweden
`IRL`	Ireland	`NLD`	Netherlands
`CAN`	Canada	`IND`	India

Each zone carries four factors: adpe, pe, gwp and wue. When one of them could not be measured for a zone, upstream falls back to the world value and says so — 165 of the 215 zones have an estimated water factor this way. Those warnings are attached to the impacts of every request using the zone, so an estimate is never published as a measurement.

Custom electricity mix

You can define custom electricity mix zones using the custom-electricity-mix config. The format is the one ecologits ships, so a zone can be copied straight from upstream:

{
  "electricity_mixes": [
    {
      "name": "MYZONE",
      "adpe": 4.858e-08,
      "pe": 9.3135,
      "gwp": 0.04144,
      "wue": 3.6737,
      "warnings": []
    }
  ]
}

adpe: Abiotic Depletion Potential for Elements (kgSbeq/kWh)
pe: Primary Energy (MJ/kWh)
gwp: Global Warming Potential (kgCO2eq/kWh)
wue: Water Usage Effectiveness of the grid (L/kWh)
warnings: which factors are estimated rather than measured, surfaced in the impacts of every request using the zone

The previous CSV shape (name,adpe,pe,gwp[,wue]) is still accepted, so an existing configuration keeps working.

Example of ecological impact embed in responses

NOTE: you can embed ecological impact informations by using embed_impacts=true query param

$ curl --request POST \
  --url 'http://test.oto.tools:8080/v1/chat/completions?embed_impacts=true' \
  --header 'content-type: application/json' \
  --data '{
  "messages": [
    {
      "role": "user",
      "content": "tell me a joke"
    }
  ]
}'

{
  "id": "chatcmpl-POPo2920QC8UdFT6MEF8MnnMBOGXDNdg",
  "object": "chat.completion",
  "created": 1743161221,
  "model": "gpt-4o-mini",
  "system_fingerprint": "fp-8hSBjQkQnl3InFEJ88ckzfm3JUtNC2bH",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Why did the scarecrow win an award?\n\nBecause he was outstanding in his field!"
      },
      "logprobs": null,
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 11,
    "completion_tokens": 18,
    "total_tokens": 29,
    "completion_tokens_details": {
      "reasoning_tokens": 0
    }
  },
  "impacts": {
    "energy": {
      "value": {
        "min": 0.000045859386,
        "max": 0.000140325354,
        "avg": 0.00009309237000000001
      },
      "unit": "kWh"
    },
    "gwp": {
      "value": {
        "min": 0.00002877619380504681,
        "max": 0.00008550637626962295,
        "avg": 0.00005714128503733488
      },
      "unit": "kgCO2eq"
    },
    "adpe": {
      "value": {
        "min": 1.183887945516329e-10,
        "max": 1.8973743554267144e-10,
        "avg": 1.5406311504715217e-10
      },
      "unit": "kgSbeq"
    },
    "pe": {
      "value": {
        "min": 0.00047959646949585386,
        "max": 0.001435187811341041,
        "avg": 0.0009573921404184475
      },
      "unit": "MJ"
    },
    "usage": {
      "energy": {
        "value": {
          "min": 0.000045859386,
          "max": 0.000140325354,
          "avg": 0.00009309237000000001
        },
        "unit": "kWh"
      },
      "gwp": {
        "value": {
          "min": 0.000027078958526507997,
          "max": 0.00008285903437921199,
          "avg": 0.000054968996452859996
        },
        "unit": "kgCO2eq"
      },
      "adpe": {
        "value": {
          "min": 3.3830835927287995e-12,
          "max": 1.0351913624863199e-11,
          "avg": 6.8674986087959995e-12
        },
        "unit": "kgSbeq"
      },
      "pe": {
        "value": {
          "min": 0.000458043547368,
          "max": 0.001401569635752,
          "avg": 0.00092980659156
        },
        "unit": "MJ"
      }
    },
    "embodied": {
      "gwp": {
        "value": {
          "min": 0.0000016972352785388126,
          "max": 0.000002647341890410959,
          "avg": 0.0000021722885844748857
        },
        "unit": "kgCO2eq"
      },
      "adpe": {
        "value": {
          "min": 1.150057109589041e-10,
          "max": 1.7938552191780823e-10,
          "avg": 1.4719561643835617e-10
        },
        "unit": "kgSbeq"
      },
      "pe": {
        "value": {
          "min": 0.00002155292212785388,
          "max": 0.0000336181755890411,
          "avg": 0.00002758554885844749
        },
        "unit": "MJ"
      }
    },
    "warnings": null
  }
}

Example of LLMUsageAudit event with ecological impact

{
  "@id" : "1905575568334651655",
  "@timestamp" : 1743159594030,
  "@type" : "AuditEvent",
  "@product" : "otoroshi",
  "@serviceId" : "",
  "@service" : "Otoroshi",
  "@env" : "dev",
  "audit" : "LLMUsageAudit",
  "provider_kind" : "openai",
  "provider" : "provider_10bbc76d-7cd8-4cb7-b760-61e749a1b691",
  "duration" : 497,
  "model" : "gpt-4o-mini",
  "rate_limit" : {
    "requests_limit" : 10000,
    "requests_remaining" : 9998,
    "tokens_limit" : 200000,
    "tokens_remaining" : 199993
  },
  "usage" : {
    "prompt_tokens" : 11,
    "generation_tokens" : 18,
    "reasoning_tokens" : 0
  },
  "error" : null,
  "consumed_using" : "chat/completion/blocking",
  "user" : null,
  "apikey" : null,
  "route" : {
    "id" : "route_e4a9d6cb3-d859-4203-a860-8d1dd6d09557",
    "name" : "test",
    "..."
  },
  "input_prompt" : [ {
    "role" : "user",
    "content" : "tell me a joke"
  } ],
  "output" : {
    "generations" : [ {
      "message" : {
        "role" : "assistant",
        "content" : "Why did the scarecrow win an award?\n\nBecause he was outstanding in his field!"
      }
    } ],
    "metadata" : {
      "rate_limit" : { "..." },
      "usage" : { "..." }
    }
  },
  "provider_details" : {
    "id" : "provider_10bbc76d-7cd8-4cb7-b760-61e749a1b691",
    "name" : "OpenAI",
    "provider" : "openai",
    "..."
  },
  "impacts" : {
    "energy" : {
      "value" : {
        "min" : 0.000045859386,
        "max" : 0.000140325354,
        "avg" : 0.00009309237000000001
      },
      "unit" : "kWh",
      "description" : "Energy: related to the final electricity consumption in kWh"
    },
    "gwp" : {
      "value" : {
        "min" : 0.00002877619380504681,
        "max" : 0.00008550637626962295,
        "avg" : 0.00005714128503733488
      },
      "unit" : "kgCO2eq",
      "description" : "Global Warming Potential (GWP): related to climate change, commonly known as GHG emissions in kgCO2eq"
    },
    "adpe" : {
      "value" : {
        "min" : 1.183887945516329E-10,
        "max" : 1.8973743554267144E-10,
        "avg" : 1.5406311504715217E-10
      },
      "unit" : "kgSbeq",
      "description" : "Abiotic Depletion Potential for Elements (ADPe): related to the depletion of minerals and metals in kgSbeq"
    },
    "pe" : {
      "value" : {
        "min" : 0.00047959646949585386,
        "max" : 0.001435187811341041,
        "avg" : 0.0009573921404184475
      },
      "unit" : "MJ",
      "description" : "Primary Energy (PE): related to the energy consumed from primary sources like oil, gas or coal in MJ"
    },
    "usage" : {
      "energy" : { "..." },
      "gwp" : { "..." },
      "adpe" : { "..." },
      "pe" : { "..." },
      "description" : "related to the impacts of the energy consumption during model execution"
    },
    "embodied" : {
      "gwp" : { "..." },
      "adpe" : { "..." },
      "pe" : { "..." },
      "description" : "related to resource extraction, manufacturing and transportation of the hardware"
    },
    "warnings" : null
  },
  "costs" : null,
  "budgets" : null,
  "consumer_rate_limit" : null
}

Dashboard example

The extension ships a ready-made Ecological impact dashboard in Otoroshi's user dashboards — see Built-in dashboards.

The AI Gateway ecological impact dashboard in Otoroshi

Kept in sync with ecologits

The impact model and its data are a port of ecologits: the formulas, the constants, the model characteristics and the electricity mixes all come from upstream and are meant to move with it, not to be tuned locally.

Two things worth knowing about the latest refresh:

The numbers changed, substantially. Upstream moved the model quantization from 4 to 16 bits — which multiplies the memory a model needs, hence the number of GPUs — and shortened the hardware lifespan from 5 to 3 years, which raises the embodied share. Every hardware constant moved too. Impacts reported before the refresh were not comparable to the ones reported now.
Models now carry measured deployment characteristics. When upstream ships a tps (tokens per second) and ttft (time to first token) for a model, they are used directly instead of the fitted latency formula, which only ever stood in for a measurement. Models without them still use the formula.

Configuration​

Impact metrics​

Supported providers​

Available electricity mix zones​

Custom electricity mix​

Example of ecological impact embed in responses​

Example of LLMUsageAudit event with ecological impact​

Dashboard example​

Kept in sync with ecologits​