Overview
500 traces with at least one agent run, tool call or model call, the latest first.
Per release
| Release | Variant | Traces | Cost (USD) | Input tokens | Output tokens | p95 (ms) | Error rate |
|---|
| no release |
|
82 |
0.0137 |
57252 |
15994 |
6764 |
26% |
| no release |
e2e-orchestrator-prompt-label=control |
20 |
0.0035 |
8556 |
5197 |
4803 |
50% |
| no release |
e2e-orchestrator-prompt-label=latest |
18 |
0.0049 |
11854 |
7375 |
4814 |
0% |
| no release |
e2e-orchestrator-shadow=cooler |
2 |
0.0011 |
2636 |
1639 |
3905 |
0% |
| ac81f39c7a72 |
|
33 |
0.0000 |
25603 |
5139 |
2802 |
0% |
| d6eaba2a6071 |
|
343 |
0.0000 |
255351 |
38087 |
2941 |
0% |
| dev |
|
2 |
0.0000 |
2014 |
486 |
5620 |
0% |
Agents
| Name | Release | Variant | Calls | Cost (USD) | Input tokens | Output tokens | p95 (ms) | Error rate |
|---|
| explainer_agent |
no release |
|
12 |
0.0069 |
23552 |
9168 |
14327 |
8% |
| math_agent |
no release |
|
11 |
0.0024 |
16381 |
1464 |
2927 |
9% |
| orchestrator |
no release |
e2e-orchestrator-prompt-label=control |
20 |
0.0035 |
8556 |
5197 |
3956 |
50% |
| orchestrator |
no release |
e2e-orchestrator-prompt-label=latest |
18 |
0.0049 |
11854 |
7375 |
4293 |
0% |
| orchestrator |
no release |
e2e-orchestrator-shadow=cooler |
4 |
0.0011 |
2636 |
1639 |
3158 |
0% |
| orchestrator |
no release |
|
29 |
0.0044 |
17319 |
5362 |
8987 |
14% |
Models
| Name | Release | Variant | Calls | Cost (USD) | Input tokens | Output tokens | p95 (ms) | Error rate |
|---|
| chat-default |
no release |
e2e-orchestrator-prompt-label=control |
20 |
0.0035 |
8556 |
5197 |
3490 |
0% |
| chat-default |
no release |
e2e-orchestrator-prompt-label=latest |
18 |
0.0049 |
11854 |
7375 |
3743 |
0% |
| chat-default |
no release |
e2e-orchestrator-shadow=cooler |
4 |
0.0011 |
2636 |
1639 |
2989 |
0% |
| chat-default |
no release |
|
73 |
0.0137 |
57252 |
15994 |
3734 |
0% |
| judge-default |
ac81f39c7a72 |
|
33 |
0.0000 |
25603 |
5139 |
2802 |
0% |
| judge-default |
d6eaba2a6071 |
|
343 |
0.0000 |
255351 |
38087 |
2941 |
0% |
| judge-default |
dev |
|
5 |
0.0000 |
2014 |
486 |
1862 |
0% |
Tools
| Name | Release | Variant | Calls | Cost (USD) | Input tokens | Output tokens | p95 (ms) | Error rate |
|---|
| gcd |
no release |
|
18 |
0.0000 |
0 |
0 |
345 |
0% |
| get_claim |
no release |
|
16 |
0.0000 |
0 |
0 |
288 |
44% |
| get_customer_360 |
no release |
|
5 |
0.0000 |
0 |
0 |
28 |
0% |
| get_open_escalations |
no release |
|
6 |
0.0000 |
0 |
0 |
10 |
0% |
| lcm |
no release |
|
30 |
0.0000 |
0 |
0 |
528 |
33% |
| resume_claim |
no release |
|
3 |
0.0000 |
0 |
0 |
0 |
100% |
Agents in the manifest
| Agent | Description | Model | Exposure | Tools | Agents it forwards to |
|---|
| orchestrator |
Answers a customer of the insurer, asks a clarifying question, or forwards to a specialist agent. |
chat-default |
internal, external |
|
|
Experiments in the manifest
No experiments in platform.yaml.