Model Usage & Cost Tracking

Langfuse tracks the usage and cost of every LLM call in your application, so you can monitor spend across models, use cases, and over time.

For most users, cost tracking works out of the box. Langfuse ships with prices for popular models from OpenAI, Anthropic, and Google, and most integrations capture usage and cost automatically, so in most cases you don't need to configure anything.

What you can do with cost data

Once usage and cost are tracked, you can put the data to work:

• Create dashboards: monitor cost across models, tags, or users.
• Set up alerts: get notified automatically when spend crosses a threshold.
• Query with the Metrics API: retrieve aggregated usage and cost, filtered by application type, user, or tags, for analytics, billing, and rate-limiting.

How cost tracking works

For every LLM generation, Langfuse records two things, each broken down by usage type (for example input and output, or more specific types like cached_tokens and audio_tokens that vary by provider):

• Usage details: number of units consumed per usage type
• Cost details: USD cost per usage type

Both are captured on observations of type generation and embedding, and each can be either:

• Ingested: you send the usage and cost from the LLM response, via the API, SDKs, or an integration.
• Inferred: Langfuse works them out from the generation's model parameter, using a model definition that stores prices per usage type. Langfuse ships with definitions for popular OpenAI, Anthropic, and Google models, and you can add your own.

When both are available, ingested values take priority over inferred ones:

Model definitions and prices

When cost is not ingested directly, Langfuse infers it. The model parameter of the generation is matched to a model definition, which stores a price per usage type. Langfuse then multiplies those prices by the observation's usage to calculate cost. This is especially useful for model providers or self-hosted models that do not include cost in the response.

Langfuse comes with a list of predefined popular models and their tokenizers including OpenAI, Anthropic, Google. Check out the full list (you need to sign in).

You can manage prices in Project Settings > Models, add your own custom model definitions, or request official support for new models via GitHub.

Usage

If a tokenizer is specified for the model, Langfuse automatically calculates token amounts for ingested generations.

The following tokenizers are currently supported:

Cost

Model definitions include prices per usage type. Usage types must match exactly with the keys in the usage_details object of the generation.

Langfuse automatically calculates cost for ingested generations at the time of ingestion if (1) usage is ingested or inferred, (2) and a matching model definition includes prices.

Adding custom model definitions

You can flexibly add your own model definitions (incl. pricing tiers) to Langfuse. This is especially useful for self-hosted or fine-tuned models which are not included in the list of Langfuse maintained models.

GET    /api/public/models
POST   /api/public/models
GET    /api/public/models/{id}
DELETE /api/public/models/{id}

Models are matched to generations based on:

User-defined models take priority over models maintained by Langfuse.

{
  "tokenizerModel": "gpt-3.5-turbo", // tiktoken model name
  "tokensPerName": -1, // OpenAI Chatmessage tokenization config
  "tokensPerMessage": 4 // OpenAI Chatmessage tokenization config
}

Pricing Tiers

Some model providers charge different rates depending on context length or request attributes. For example, Anthropic's Claude Sonnet 4.5 and Google's Gemini 2.5 Pro apply higher pricing when more than 200K input tokens are used, while OpenAI Fast mode charges a premium based on the request's service_tier.

Langfuse supports pricing tiers for models, enabling accurate cost calculation across usage thresholds, model parameters, and metadata.

How tier matching works

Each model can have multiple pricing tiers, each with:

When calculating cost, Langfuse evaluates tiers in priority order (excluding the default tier). The first tier whose conditions are satisfied is used. If no conditional tier matches, the default tier is applied.

Each condition selects one of these sources:

For model parameters and metadata, select the source in the pricing-tier editor, enter the Top-level key, and add comma-separated exact values. String, number, and boolean values are compared as strings. Nested paths and non-primitive values are not supported. When a tier has multiple conditions, all of them must match.

Examples:

Frequently asked questions

Ingest usage and cost manually

While Langfuse infers this data automatically (if it unexpectedly doesn't, open an issue on GitHub), you might want to ingest it yourself in some cases, for example custom or self-hosted models Langfuse cannot price, private pricing, or exact provider counts you want recorded verbatim.

from langfuse import get_client
import anthropic

langfuse = get_client()
anthropic_client = anthropic.Anthropic()

with langfuse.start_as_current_observation(
    as_type="generation",
    name="anthropic-completion",
    model="claude-3-opus-20240229",
    input=[{"role": "user", "content": "Hello, Claude"}]
) as generation:
    response = anthropic_client.messages.create(
        model="claude-3-opus-20240229",
        max_tokens=1024,
        messages=[{"role": "user", "content": "Hello, Claude"}]
    )

    generation.update(
        output=response.content[0].text,
        usage_details={
            "input": response.usage.input_tokens,
            "output": response.usage.output_tokens,
            "cache_read_input_tokens": response.usage.cache_read_input_tokens
            # "total": int,  # if not set, it is derived as the sum of all usage types
        },
        # Optionally, also ingest USD cost. Alternatively, infer it via a model definition in Langfuse.
        cost_details={
            # Here we assume the input and output cost are 1 USD each and half the price for cached tokens.
            "input": 1,
            "cache_read_input_tokens": 0.5,
            "output": 1,
            # "total": float, # if not set, it is derived as the sum of all usage types
        }
    )

from langfuse import observe, get_client
import anthropic

langfuse = get_client()
anthropic_client = anthropic.Anthropic()

@observe(as_type="generation")
def anthropic_completion(**kwargs):
  # optional, extract some fields from kwargs
  kwargs_clone = kwargs.copy()
  input = kwargs_clone.pop('messages', None)
  model = kwargs_clone.pop('model', None)
  langfuse.update_current_generation(
      input=input,
      model=model,
      metadata=kwargs_clone
  )

  response = anthropic_client.messages.create(**kwargs)

  langfuse.update_current_generation(
      usage_details={
          "input": response.usage.input_tokens,
          "output": response.usage.output_tokens,
          "cache_read_input_tokens": response.usage.cache_read_input_tokens
        },
      cost_details={
          "input": 1,
          "cache_read_input_tokens": 0.5,
          "output": 1,
      }
  )

  # return result
  return response.content[0].text

@observe()
def main():
  return anthropic_completion(
      model="claude-3-opus-20240229",
      max_tokens=1024,
      messages=[
          {"role": "user", "content": "Hello, Claude"}
      ]
  )

main()

import { startObservation } from "@langfuse/tracing";

const generation = startObservation(
  "llm-call",
  {
    model: "gpt-4",
    input: [{ role: "user", content: "What is the capital of France?" }],
  },
  { asType: "generation" }
);

// ... LLM call logic ...

generation.update({
  usageDetails: {
    input: 10,
    output: 5,
    cache_read_input_tokens: 2,
    some_other_token_count: 10,
    total: 27, // optional, it is derived as the sum of all usage types
  },
  costDetails: {
    // Optional. If omitted, cost is inferred from a model definition.
    input: 1,
    output: 1,
    cache_read_input_tokens: 0.5,
    some_other_token_count: 1,
    total: 3.5,
  },
  output: { content: "The capital of France is Paris." },
});

generation.end();

import { startActiveObservation, startObservation } from "@langfuse/tracing";

await startActiveObservation("context-manager", async (span) => {
  span.update({
    input: { query: "What is the capital of France?" },
  });

  const generation = startObservation(
    "llm-call",
    {
      model: "gpt-4",
      input: [{ role: "user", content: "What is the capital of France?" }],
    },
    { asType: "generation" }
  );

  // ... LLM call logic ...

  generation.update({
    usageDetails: { input: 10, output: 5, total: 15 },
    costDetails: { input: 1, output: 1, total: 2 },
    output: { content: "The capital of France is Paris." },
  });

  generation.end();
});

import { observe, updateActiveObservation } from "@langfuse/tracing";

async function fetchData(source: string) {
  updateActiveObservation(
    {
      usageDetails: { input: 10, output: 5, total: 15 },
      costDetails: { input: 1, output: 1, total: 2 },
    },
    { asType: "generation" }
  );

  // ... logic to fetch data
  return { data: `some data from ${source}` };
}

const tracedFetchData = observe(fetchData, {
  name: "observe-wrapper",
  asType: "generation",
});

const result = await tracedFetchData("API");

You can also update usage and cost later via generation.update().

Usage types are mutually exclusive buckets

Langfuse treats every key in usage_details as a separate, non-overlapping bucket: each token must be counted in exactly one key. This means that

• input excludes any input_* values (such as input_cached_tokens)
• output excludes any output_* values (such as output_reasoning_tokens)

total is the sum of the buckets, not a bucket itself.

If buckets overlap, usage and inferred cost will be counted double, and cost shown in Langfuse overstates what your provider actually charged. Directly ingested cost_details are unchanged.

Compatibility with OpenAI

For increased compatibility with OpenAI, you can also use the OpenAI Usage schema. Langfuse maps its fields to its own usage types:

Since OpenAI reports the detail counts inclusively, Langfuse subtracts them from input and output respectively, so that the stored buckets are mutually exclusive.

from langfuse import get_client

langfuse = get_client()

with langfuse.start_as_current_observation(
    as_type="generation",
    name="openai-style-generation",
    model="gpt-4o"
) as generation:
    # Simulate LLM call
    # response = openai_client.chat.completions.create(...)

    generation.update(
        usage_details={
            # usage (OpenAI-style schema)
            "prompt_tokens": 10,
            "completion_tokens": 25,
            "total_tokens": 35,
            "prompt_tokens_details": {
                "cached_tokens": 5,
                "audio_tokens": 2,
            },
            "completion_tokens_details": {
                "reasoning_tokens": 15,
            },
        }
    )

import { startObservation } from "@langfuse/tracing";

const generation = startObservation(
  "openai-style-generation",
  {
    model: "gpt-4o",
    usageDetails: {
      // usage (OpenAI-style schema)
      prompt_tokens: 10,
      completion_tokens: 25,
      total_tokens: 35,
      prompt_tokens_details: {
        cached_tokens: 5,
        audio_tokens: 2,
      },
      completion_tokens_details: {
        reasoning_tokens: 15,
      },
    },
  },
  { asType: "generation" },
);
generation.end();

You can also ingest OpenAI-style usage via generation.update() and generation.end().

Troubleshooting

If you don't see cost on your generations, check the following:

• Usage or cost must be present. Langfuse can only infer cost if usage is ingested or a tokenizer is available for the model. If neither cost nor usage is ingested and the model has no tokenizer, no cost is calculated.
• The model must match a definition. Inferred cost requires a model definition whose match_pattern matches the generation's model value. If no model matches, add a custom model definition.
• Model definition changes only apply to new generations. If you change or add a model definition, the updated costs are applied only to generations logged afterward.
• Only generation and embedding observations track cost. Other observation types do not carry usage or cost.
• Reasoning models require ingested usage. Langfuse cannot infer cost for reasoning models such as the OpenAI o1 family without token counts. See cost inference for reasoning models.
• Gateways: If you use OpenRouter, Langfuse can directly capture the OpenRouter cost information. If you use LiteLLM, Langfuse directly captures the cost information returned in each LiteLLM response.

GitHub Discussions