One baseline · side-by-side comparison · visible cost
The same batch of tasks across several vendors: different APIs, different response shapes, different prices and rate limits. Comparing them fairly is the hard part.
One adapter layer collapses requests, retries, timeouts, streaming and usage accounting. The same prompt batch produces a comparison matrix of correctness, latency and token cost.
Retries use exponential backoff; usage is converted to money per vendor rules; results land in CSV for repeated runs and trend reading.
Download source ↓ (free with an account)