OpenRoute is recruiting agents — share the AI gateway market →Bare-metal GPU rentals from 1 hour — idle-compute scheduling platform coming soon →
OROpenRoute
  • Models
  • Bare Metal
  • Observability
  • Docs
  • Blog
Observability & Evaluation

Every call, clearly visible

Ontology Route for AI Agent has observability and evaluation natively built into the gateway layer. Without touching your business code, capture the full call chain of every run, score output quality continuously, and turn latency, tokens and cost into analyzable metrics.

Connect modelsExplore guardrails

Core capabilities

From tracing to evaluation — the complete observability loop after launch.

Distributed tracing

Rebuild the full call tree of a run with Trace / Span, seamlessly spanning models, tools and sub-tasks.

Automated evaluation

Built-in faithfulness, relevancy, toxicity, conciseness and more, plus LLM-as-a-Judge and custom scoring functions.

Real-time metrics

Latency, tokens, cost and success rate aggregated by model, task and tenant, refreshed in seconds.

Anomaly alerts

Set thresholds for latency, error rate and cost; get notified via webhook, email or Slack.

Replay & annotation

Replay any call end to end; human annotations flow back into continuous evaluation.

Cost attribution

Attribute every yuan of cost precisely to tasks, features and end users.

Full-chain trace waterfall

Every run produces a call tree that clearly shows where time goes — planning, retrieval, model inference and tool execution.

Trace#8f2a…c1
2.84s · 6 spans
agent.run
2.84s
—
planner · gpt-4o
640ms
1.2k tok
tool: search_docs
510ms
—
retriever.embed
280ms
512 tok
executor · claude-3.5
980ms
3.4k tok
tool: run_code
390ms
—
RootLLMToolVector

Automated evaluation

Run evaluators on sampled or full traffic and track quantitative quality scores across versions.

Faithfulness0.94

Whether the answer stays true to the retrieved context

Relevancy0.88

How well the answer matches the user's intent

Toxicity0.99

Safety and compliance score of the output

Conciseness0.82

Whether the answer is concise without redundancy

Metrics dashboard

Key production metrics aggregated in real time, with automatic alerts on anomalies.

P50 latency

1.24s

P95 latency

3.10s

Success rate

99.2%

Cost (24h)

¥350

When P95 latency exceeds 3s or the error rate exceeds 1%, alerts are pushed automatically via Slack and webhook, with the corresponding trace link attached.

Observability and guardrails, a natural pair

Every guardrail block is recorded in the call chain — risks traceable to the exact request.

Explore guardrails
OROpenRoute· Ontology Route for AI Agent
ModelsProvidersObservabilityGuardrailsPricingBare MetalTerms of ServicePrivacy Policy

Prices are shown in CNY (¥) per million tokens and are for reference only. Live pricing, quotas and billing are subject to the console.

Copyright ©2025-2027 天津聚链科技有限公司版权所有

津ICP备2023007973号-5|beian津公网安备12011402001495号