ModulesAI ModuleAPIsExtracting Metrics

Extracting Provider Metrics

To send accurate telemetry to the OTM API, you need to grab real-time metrics (like prompt tokens, completion tokens, and timing) directly from your LLM provider’s response.

Every SDK hides these metrics in slightly different places. This guide shows you exactly where to find them so you don’t have to spend hours digging through documentation or console logging response objects.

Pro Tip: Always capture the startTime right before you make the API call and firstTokenTime when you receive the first chunk if you’re streaming. This ensures your TTFT (Time To First Token) metrics are spot on.

OpenAI

OpenAI makes it relatively easy to get token usage, provided you aren’t streaming. For streaming, you’ll need to look at the final chunk.

const response = await openai.chat.completions.create({
  model: "gpt-4o",
  messages: [{ role: "user", content: "Hello!" }],
});
 
// Here is the gold mine:
const { prompt_tokens, completion_tokens, total_tokens } = response.usage;
 
console.log(`Used ${prompt_tokens} input and ${completion_tokens} output tokens.`);

Anthropic

Anthropic calls their usage fields input_tokens and output_tokens. If you use prompt caching, they also provide cache_creation_input_tokens and cache_read_input_tokens.

const message = await anthropic.messages.create({
  model: "claude-3-5-sonnet-20240620",
  max_tokens: 1024,
  messages: [{ role: "user", content: "Hi Claude!" }],
});
 
// Standard usage extraction
const { input_tokens, output_tokens } = message.usage;
 
// If you use caching, keep an eye on these too:
const cacheRead = message.usage.cache_read_input_tokens || 0;

Google Gemini

Google wraps their usage in a usageMetadata object (CamelCase in JS/Go, snake_case in Python).

const result = await model.generateContent("Explain telemetry.");
const response = await result.response;
 
// Gemini calls them promptTokenCount and candidatesTokenCount
const { promptTokenCount, candidatesTokenCount } = response.usageMetadata;
 
console.log(`Prompt: ${promptTokenCount}, Output: ${candidatesTokenCount}`);

Alibaba Cloud (DashScope)

Alibaba Cloud’s DashScope API returns token usage inside a usage object. You’ll typically find the number of input and output tokens in input_tokens and output_tokens.

import OpenAI from "openai";
 
const client = new OpenAI({
  apiKey: process.env.DASHSCOPE_API_KEY,
  baseURL: "https://dashscope-intl.aliyuncs.com/compatible-mode/v1",
});
 
const response = await client.chat.completions.create({
  model: "qwen-plus",
  messages: [
    { role: "user", content: "Explain telemetry." },
  ],
});
 
// Usage information
const { prompt_tokens, completion_tokens, total_tokens } = response.usage;
 
console.log(
  `Prompt: ${prompt_tokens}, Output: ${completion_tokens}, Total: ${total_tokens}`
);

Cohere

Cohere exposes token usage through the meta.billed_units object, which contains separate counts for input and output tokens.

import { CohereClient } from "cohere-ai";
 
const cohere = new CohereClient({
  token: process.env.COHERE_API_KEY,
});
 
const response = await cohere.chat({
  model: "command-r-plus",
  message: "Explain telemetry.",
});
 
// Token usage
const inputTokens = response.meta.billedUnits.inputTokens;
const outputTokens = response.meta.billedUnits.outputTokens;
 
console.log(
  `Prompt: ${inputTokens}, Output: ${outputTokens}`
);

Meta (Llama)

Meta’s Llama models are commonly accessed through OpenAI-compatible APIs (such as Together AI, Fireworks AI, Groq, or other compatible providers). These APIs expose token usage through the standard usage object.

import OpenAI from "openai";
 
const client = new OpenAI({
  apiKey: process.env.META_API_KEY,
  baseURL: process.env.META_BASE_URL,
});
 
const response = await client.chat.completions.create({
  model: "meta-llama/llama-4-maverick",
  messages: [
    {
      role: "user",
      content: "Explain telemetry.",
    },
  ],
});
 
// Usage information
const { prompt_tokens, completion_tokens, total_tokens } = response.usage;
 
console.log(
  `Prompt: ${prompt_tokens}, Output: ${completion_tokens}, Total: ${total_tokens}`,
);

Mistral

Mistral’s official SDK returns token usage through the usage object containing prompt, completion, and total token counts.

import { Mistral } from "@mistralai/mistralai";
 
const client = new Mistral({
  apiKey: process.env.MISTRAL_API_KEY,
});
 
const response = await client.chat.complete({
  model: "mistral-large-latest",
  messages: [
    {
      role: "user",
      content: "Explain telemetry.",
    },
  ],
});
 
// Usage information
const { promptTokens, completionTokens, totalTokens } = response.usage;
 
console.log(
  `Prompt: ${promptTokens}, Output: ${completionTokens}, Total: ${totalTokens}`,
);

xAI (Grok)

xAI provides an OpenAI-compatible API. Token usage is available through the standard usage object.

import OpenAI from "openai";
 
const client = new OpenAI({
  apiKey: process.env.XAI_API_KEY,
  baseURL: "https://api.x.ai/v1",
});
 
const response = await client.chat.completions.create({
  model: "grok-4",
  messages: [
    {
      role: "user",
      content: "Explain telemetry.",
    },
  ],
});
 
// Usage information
const { prompt_tokens, completion_tokens, total_tokens } = response.usage;
 
console.log(
  `Prompt: ${prompt_tokens}, Output: ${completion_tokens}, Total: ${total_tokens}`,
);

Sarvam AI

Sarvam AI exposes an OpenAI-compatible Chat Completions API. Token usage is returned in the standard usage object.

import OpenAI from "openai";
 
const client = new OpenAI({
  apiKey: process.env.SARVAM_API_KEY,
  baseURL: "https://api.sarvam.ai/v1",
});
 
const response = await client.chat.completions.create({
  model: "sarvam-105b",
  messages: [
    {
      role: "user",
      content: "Explain telemetry.",
    },
  ],
});
 
// Usage information
const { prompt_tokens, completion_tokens, total_tokens } = response.usage;
 
console.log(
  `Prompt: ${prompt_tokens}, Output: ${completion_tokens}, Total: ${total_tokens}`,
);

Next Steps

Now that you’ve got the raw numbers, you’re ready to format them for the Telemetry Usage API. Just map these values to the tokens.input and tokens.output fields in your payload, and you’re good to go!

Telemetry UsageSecurity