InferenceIndex

Early access · Private preview

Reference

FAQ

Short answers for teams evaluating Compiled Agent Models.

The questions teams ask most before they try a Compiled Agent Model.

Four question cards on a dark plate

What is a Compiled Agent Model?

A Compiled Agent Model is a model built for one workflow your agent already runs. It does that job again on new inputs, much cheaper and faster than a frontier model, and anything it does not fit goes to your current model. On our tests, repeat runs are often 10-100x cheaper.

Is this caching?

No. A cache returns a stored answer, so it misses or returns the old answer when the values change. A Compiled Agent Model does the job again on the new values. A new invoice with a new vendor and total gets the new total, not yesterday's.

Is this a cheaper open-source model?

No. A cheaper general model lowers the price per token but has never seen your workflow, so quality on your job is whatever it turns out to be. A Compiled Agent Model is built for your workflow, and anything outside it goes to the frontier model you use today, so your quality bar stays where it is.

How do I get an API key?

Open the dashboard at https://inferenceindex-app.web.app. Sign in with the same Google account as the InferenceIndex site, enter your invite code, and get an API key there. Each invite code works once.

What base URL and model do I send?

The base URL is https://api.inferenceindex.com/v1. The model is workflow/auto. The call is OpenAI-compatible chat completions. Python uses base_url. TypeScript uses baseURL.

What happens when a request is not a fit?

It goes to your fallback model, which you choose in the dashboard. You get the result you would get today, at about today's cost, with no second code path.

Can I use it from Cursor, Claude Code, or Codex?

Cursor takes the OpenAI-compatible base URL in Settings, then Models. Codex takes it in ~/.codex/config.toml. Claude Code's own base URL speaks the Anthropic Messages API, so use the quickstart call from the code you are writing. The coding-agents page is https://inferenceindex.com/docs/coding-agents.

What do calls cost?

Calls spend credits, and the dashboard shows the balance. Published tests are the figures to quote: repeat runs are often 10-100x cheaper on our tests, and one measured invoice reconciliation run was about 450x cheaper than Opus 5.5.

Where are logs?

Logs are in the dashboard. They list your API calls: when the call ran, the model name, and whether it completed.

Is the API generally available?

The API is in early access. It is a private preview. Get a key at https://inferenceindex-app.web.app. The home page waitlist is https://inferenceindex.com/#waitlist.

What about security reviews?

SOC 2 is in progress.