Welcome to the Inference Cost Desk
Score routing, cache, GPU waste, and token unit economics. You leave with a cut plan, not a slide.
Company Profile
Company Name
Industry
Company Size
Monthly inference spend
What you leave with
Cost score (1–5)
Routing, cache/batch, GPU pool, unit economics.
Peer unit cost
$ / 1M tokens vs same-industry desks.
Waste map
Where tokens, GPUs, and retries burn cash.
90-day cut plan
Ranked moves with estimated monthly save.
Routing — model choice per request
2 questionsRoutingStep 1 of 4
1. How do you choose a model for each request?
2. What happens on retries and timeouts?
Cache & batch — token waste
2 questionsCache & batchStep 2 of 4
6. Prompt / KV cache hit rate on production traffic?
7. Do you batch or coalesce non-interactive jobs?
GPU pool — idle and oversize
2 questionsGPU poolStep 3 of 4
11. How is GPU / reserved capacity utilized?
12. Do you track $ per 1M tokens by product?
Unit economics — who owns the bill
1 questionUnit economicsStep 4 of 4
16. Who can stop a spend spike?
Cost score — Acme Technologies
Routing
3.2
Cache & batch
3.8
GPU pool
3.5
Unit economics
3.0
5-level cost desk
3.4
1
BlindInvoice shock
2
WatchingSpreadsheet
3
RoutingSome waste cut
4
ChargebackOwners feel $
5
Hard capUnit cost SLO
Industry Peer Benchmarks — Financial Services
Your Score
3.4
Peer Median
3.1
Top Quartile
4.2
Your Percentile
62nd
| Dimension | Your Score | Peer Median | Top Quartile | Gap to Top |
|---|---|---|---|---|
| Routing | 3.2 | 3.0 | 4.1 | -0.9 |
| Cache & batch | 3.8 | 3.2 | 4.4 | -0.6 |
| GPU pool | 3.5 | 3.0 | 4.2 | -0.7 |
| Unit economics | 3.0 | 2.8 | 4.0 | -1.0 (largest gap) |
Gap Analysis — Top Priority Improvements
| Gap | Dimension | Impact | Effort | ROI Score | Priority |
|---|---|---|---|---|---|
| Hard spend cap + route freeze | Unit economics | High | Low | 9.2 | #1 |
| Learned router off frontier for 40% of calls | Routing | High | Medium | 8.4 | #2 |
| Kill idle reserved GPUs nights/weekends | GPU pool | Medium | Medium | 7.1 | #3 |
| Semantic cache SLO to 55% hit | Cache & batch | Medium | Low | 7.8 | #4 |
| Move reports to batch API | Cache & batch | Medium | High | 5.4 | #5 |
Personalized 90-Day Roadmap
Days 1-14 — Cut waste
1. Freeze the top spend route. 2. Move night jobs off online. 3. Turn on prefix + semantic cache measurement.
Days 15-45 — Route cheaper
4. Task-class router (cheap / mid / frontier). 5. Chargeback $ / 1M tokens to product owners.
Days 46-75 — Right-size GPU
6. Kill idle reserved capacity. 7. Prefill/decode split on the two heaviest endpoints.
Days 76-90 — Hard cap
8. Board pack: unit cost vs peers. 9. Target Level 4 chargeback. 10. Re-score in 90 days.
Expected Score Gain
+0.6
3.4 → 4.0 in 90d
Estimated ROI
$480K
Based on peer outcomes
Target Percentile
Top 25%
Export & Share Your Report
Report Preview
Company
Acme Technologies
Overall Score
3.4 / 5.0
Percentile
62nd (Industry: Financial Services)
Top Priority
Hard spend cap + route freeze
90d ROI Potential
$480K
Export Options