Ecological impact

If you want to track the ecological impact of your LLM Usage, you can enable it in the Otoroshi LLM Extension (it should be enabled by default).

Configuration

impacts {
  enabled = true
  enabled = ${?CLOUD_APIM_EXTENSIONS_LLM_EXTENSION_IMPACTS_ENABLED}
  embed-description-in-json = true
  embed-description-in-json = ${?CLOUD_APIM_EXTENSIONS_LLM_EXTENSION_IMPACTS_EMBED_DESCRIPTION_IN_JSON}
  electricity-mix = "WOR"
  electricity-mix = ${?CLOUD_APIM_EXTENSIONS_LLM_EXTENSION_IMPACTS_ELECTRICITY_MIX}
  embed-impacts-in-responses = false
  embed-impacts-in-responses = ${?CLOUD_APIM_EXTENSIONS_LLM_EXTENSION_IMPACTS_EMBED_IMPACTS_IN_RESPONSES}
  custom-models = "{\"aliases\": [],\"models\": []}"
  custom-models = ${?CLOUD_APIM_EXTENSIONS_LLM_EXTENSION_IMPACTS_CUSTOM_MODELS}
  custom-electricity-mix = "name,adpe,pe,gwp\n"
  custom-electricity-mix = ${?CLOUD_APIM_EXTENSIONS_LLM_EXTENSION_IMPACTS_CUSTOM_ELECTRICITY_MIX}
}

Parameter	Type	Default	Description
`enabled`	boolean	`true`	Enable or disable ecological impact tracking
`embed-description-in-json`	boolean	`true`	Include human-readable descriptions in the impact JSON output
`electricity-mix`	string	`"WOR"`	Default electricity mix zone code (see available zones below)
`embed-impacts-in-responses`	boolean	`false`	Always embed impacts in LLM responses
`custom-models`	string (JSON)	`"{}"`	Custom model architectures for impact computation (see below)
`custom-electricity-mix`	string (CSV)	`""`	Custom electricity mix definitions (see below)

Once it's enabled, audit events of kind LLMUsageAudit will have an impacts field. You can also embed the impacts value in your LLM responses using the impacts.embed-impacts-in-responses config or by adding ?embed_impacts=true to your query. Please remember that the impacts value is just an estimate of the real ecological impact of your LLM usage.

Impact metrics

The ecological impact output contains the following metrics:

Metric	Unit	Description
`energy`	kWh	Energy: related to the final electricity consumption
`gwp`	kgCO2eq	Global Warming Potential (GWP): related to climate change, commonly known as GHG emissions
`adpe`	kgSbeq	Abiotic Depletion Potential for Elements (ADPe): related to the depletion of minerals and metals
`pe`	MJ	Primary Energy (PE): related to the energy consumed from primary sources like oil, gas or coal

Each metric contains a value with min, max, and avg estimates. The output is split into:

Top-level metrics: combined usage + embodied impacts
usage: impacts of the energy consumption during model execution
embodied: impacts related to resource extraction, manufacturing, and transportation of the hardware

Supported providers

The ecological impact computation is based on project EcoLogits and only supports right now the following providers:

OpenAI
Anthropic
Mistral
Gemini
Cohere
HuggingFace

For unsupported providers, you can set metadata on your provider entities to map them to a supported provider/model:

eco-impacts-provider: the provider used for eco impacts computation
eco-impacts-model: the model used for eco impacts computation
eco-impacts-electricity-mix-zone: the electricity mix zone to use for this provider

Available electricity mix zones

The following ISO country codes are available as electricity mix zones:

Code	Country	Code	Country	Code	Country
`ARG`	Argentina	`GBR`	United Kingdom	`NLD`	Netherlands
`AUS`	Australia	`GRC`	Greece	`NOR`	Norway
`AUT`	Austria	`HUN`	Hungary	`NZL`	New Zealand
`BEL`	Belgium	`IDN`	Indonesia	`POL`	Poland
`BGR`	Bulgaria	`IND`	India	`PRT`	Portugal
`BRA`	Brazil	`IRL`	Ireland	`ROU`	Romania
`CAN`	Canada	`ISL`	Iceland	`RUS`	Russia
`CHE`	Switzerland	`ITA`	Italy	`SVK`	Slovakia
`CHL`	Chile	`JPN`	Japan	`SVN`	Slovenia
`CHN`	China	`KOR`	South Korea	`SWE`	Sweden
`CYP`	Cyprus	`LTU`	Lithuania	`THA`	Thailand
`CZE`	Czech Republic	`LUX`	Luxembourg	`TUR`	Turkey
`DEU`	Germany	`LVA`	Latvia	`TWN`	Taiwan
`DNK`	Denmark	`MEX`	Mexico	`UKR`	Ukraine
`ESP`	Spain	`MLT`	Malta	`USA`	United States
`EST`	Estonia	`MYS`	Malaysia	`WOR`	World (default)
`FIN`	Finland
`FRA`	France

Custom models

You can define custom model architectures for impact computation using the custom-models config. The format is:

{
  "aliases": [],
  "models": [
    {
      "type": "model",
      "provider": "openai",
      "name": "my-custom-model",
      "architecture": {
        "type": "dense",
        "parameters": 1760000000000
      },
      "warnings": [],
      "sources": []
    }
  ]
}

The architecture.type can be "dense" or "moe" (Mixture of Experts). For MoE models, use:

{
  "architecture": {
    "type": "moe",
    "total": 1760000000000,
    "active": 440000000000
  }
}

Custom electricity mix

You can define custom electricity mix zones using the custom-electricity-mix config. The format is CSV with columns: name,adpe,pe,gwp:

name,adpe,pe,gwp
MYZONE,0.00000004858,9.3135,0.04418

adpe: Abiotic Depletion Potential for Elements (kgSbeq/kWh)
pe: Primary Energy (MJ/kWh)
gwp: Global Warming Potential (kgCO2eq/kWh)

Example of ecological impact embed in responses

NOTE: you can embed ecological impact informations by using embed_impacts=true query param

$ curl --request POST \
  --url 'http://test.oto.tools:8080/v1/chat/completions?embed_impacts=true' \
  --header 'content-type: application/json' \
  --data '{
  "messages": [
    {
      "role": "user",
      "content": "tell me a joke"
    }
  ]
}'

{
  "id": "chatcmpl-POPo2920QC8UdFT6MEF8MnnMBOGXDNdg",
  "object": "chat.completion",
  "created": 1743161221,
  "model": "gpt-4o-mini",
  "system_fingerprint": "fp-8hSBjQkQnl3InFEJ88ckzfm3JUtNC2bH",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Why did the scarecrow win an award?\n\nBecause he was outstanding in his field!"
      },
      "logprobs": null,
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 11,
    "completion_tokens": 18,
    "total_tokens": 29,
    "completion_tokens_details": {
      "reasoning_tokens": 0
    }
  },
  "impacts": {
    "energy": {
      "value": {
        "min": 0.000045859386,
        "max": 0.000140325354,
        "avg": 0.00009309237000000001
      },
      "unit": "kWh"
    },
    "gwp": {
      "value": {
        "min": 0.00002877619380504681,
        "max": 0.00008550637626962295,
        "avg": 0.00005714128503733488
      },
      "unit": "kgCO2eq"
    },
    "adpe": {
      "value": {
        "min": 1.183887945516329e-10,
        "max": 1.8973743554267144e-10,
        "avg": 1.5406311504715217e-10
      },
      "unit": "kgSbeq"
    },
    "pe": {
      "value": {
        "min": 0.00047959646949585386,
        "max": 0.001435187811341041,
        "avg": 0.0009573921404184475
      },
      "unit": "MJ"
    },
    "usage": {
      "energy": {
        "value": {
          "min": 0.000045859386,
          "max": 0.000140325354,
          "avg": 0.00009309237000000001
        },
        "unit": "kWh"
      },
      "gwp": {
        "value": {
          "min": 0.000027078958526507997,
          "max": 0.00008285903437921199,
          "avg": 0.000054968996452859996
        },
        "unit": "kgCO2eq"
      },
      "adpe": {
        "value": {
          "min": 3.3830835927287995e-12,
          "max": 1.0351913624863199e-11,
          "avg": 6.8674986087959995e-12
        },
        "unit": "kgSbeq"
      },
      "pe": {
        "value": {
          "min": 0.000458043547368,
          "max": 0.001401569635752,
          "avg": 0.00092980659156
        },
        "unit": "MJ"
      }
    },
    "embodied": {
      "gwp": {
        "value": {
          "min": 0.0000016972352785388126,
          "max": 0.000002647341890410959,
          "avg": 0.0000021722885844748857
        },
        "unit": "kgCO2eq"
      },
      "adpe": {
        "value": {
          "min": 1.150057109589041e-10,
          "max": 1.7938552191780823e-10,
          "avg": 1.4719561643835617e-10
        },
        "unit": "kgSbeq"
      },
      "pe": {
        "value": {
          "min": 0.00002155292212785388,
          "max": 0.0000336181755890411,
          "avg": 0.00002758554885844749
        },
        "unit": "MJ"
      }
    },
    "warnings": null
  }
}

Example of LLMUsageAudit event with ecological impact

{
  "@id" : "1905575568334651655",
  "@timestamp" : 1743159594030,
  "@type" : "AuditEvent",
  "@product" : "otoroshi",
  "@serviceId" : "",
  "@service" : "Otoroshi",
  "@env" : "dev",
  "audit" : "LLMUsageAudit",
  "provider_kind" : "openai",
  "provider" : "provider_10bbc76d-7cd8-4cb7-b760-61e749a1b691",
  "duration" : 497,
  "model" : "gpt-4o-mini",
  "rate_limit" : {
    "requests_limit" : 10000,
    "requests_remaining" : 9998,
    "tokens_limit" : 200000,
    "tokens_remaining" : 199993
  },
  "usage" : {
    "prompt_tokens" : 11,
    "generation_tokens" : 18,
    "reasoning_tokens" : 0
  },
  "error" : null,
  "consumed_using" : "chat/completion/blocking",
  "user" : null,
  "apikey" : null,
  "route" : {
    "id" : "route_e4a9d6cb3-d859-4203-a860-8d1dd6d09557",
    "name" : "test",
    "..."
  },
  "input_prompt" : [ {
    "role" : "user",
    "content" : "tell me a joke"
  } ],
  "output" : {
    "generations" : [ {
      "message" : {
        "role" : "assistant",
        "content" : "Why did the scarecrow win an award?\n\nBecause he was outstanding in his field!"
      }
    } ],
    "metadata" : {
      "rate_limit" : { "..." },
      "usage" : { "..." }
    }
  },
  "provider_details" : {
    "id" : "provider_10bbc76d-7cd8-4cb7-b760-61e749a1b691",
    "name" : "OpenAI",
    "provider" : "openai",
    "..."
  },
  "impacts" : {
    "energy" : {
      "value" : {
        "min" : 0.000045859386,
        "max" : 0.000140325354,
        "avg" : 0.00009309237000000001
      },
      "unit" : "kWh",
      "description" : "Energy: related to the final electricity consumption in kWh"
    },
    "gwp" : {
      "value" : {
        "min" : 0.00002877619380504681,
        "max" : 0.00008550637626962295,
        "avg" : 0.00005714128503733488
      },
      "unit" : "kgCO2eq",
      "description" : "Global Warming Potential (GWP): related to climate change, commonly known as GHG emissions in kgCO2eq"
    },
    "adpe" : {
      "value" : {
        "min" : 1.183887945516329E-10,
        "max" : 1.8973743554267144E-10,
        "avg" : 1.5406311504715217E-10
      },
      "unit" : "kgSbeq",
      "description" : "Abiotic Depletion Potential for Elements (ADPe): related to the depletion of minerals and metals in kgSbeq"
    },
    "pe" : {
      "value" : {
        "min" : 0.00047959646949585386,
        "max" : 0.001435187811341041,
        "avg" : 0.0009573921404184475
      },
      "unit" : "MJ",
      "description" : "Primary Energy (PE): related to the energy consumed from primary sources like oil, gas or coal in MJ"
    },
    "usage" : {
      "energy" : { "..." },
      "gwp" : { "..." },
      "adpe" : { "..." },
      "pe" : { "..." },
      "description" : "related to the impacts of the energy consumption during model execution"
    },
    "embodied" : {
      "gwp" : { "..." },
      "adpe" : { "..." },
      "pe" : { "..." },
      "description" : "related to resource extraction, manufacturing and transportation of the hardware"
    },
    "warnings" : null
  },
  "costs" : null,
  "budgets" : null,
  "consumer_rate_limit" : null
}

Ecological impact

Configuration​

Impact metrics​

Supported providers​

Available electricity mix zones​

Custom models​

Custom electricity mix​

Example of ecological impact embed in responses​

Example of LLMUsageAudit event with ecological impact​

Dashboard example​