Skip to main content

OpenAI-Compatible Inference API

Overview​

Foundry v2 Inference provides an OpenAI-compatible API, allowing applications to use the standard OpenAI SDK and familiar OpenAI request formats to interact with hosted inference models.

The base inference endpoint is:

https://sdk.foundry-sandbox.compactif.ai/api/v2/inference

The API supports standard OpenAI-compatible operations, including:

  • Chat completions
  • Model listing
  • Standard OpenAI message formats
  • OpenAI-compatible request and response structures

There are two Foundry-specific requirements to be aware of:

  1. Authentication uses your Multiverse IAM access token, passed in the x-api-key header. This is the same token you create under Profile → Access Tokens — see Create and manage access tokens. You may also see it called a personal access token (PAT) or an API key.
  2. The x-organization-id and x-project-id headers are required on every request. Copy both IDs from the Multiverse IAM dashboard — see Find your organization and project IDs.

1. Configure Environment Variables​

Before using the API, configure the required environment variables:

export FOUNDRY_BASE_URL=https://sdk.foundry-sandbox.compactif.ai
export FOUNDRY_API_KEY=xxxxxxxx
export FOUNDRY_ORGANIZATION_ID=xxxxxxxxxxxx
export FOUNDRY_PROJECT_ID=xxxxxxxxxxxx

FOUNDRY_API_KEY holds your access token — the value shown once when you create it under Profile → Access Tokens.

FOUNDRY_ORGANIZATION_ID and FOUNDRY_PROJECT_ID identify the organization and project you're calling on behalf of — see Find your organization and project IDs.

For convenience, these values can also be stored in a .env file:

FOUNDRY_BASE_URL=https://sdk.foundry-sandbox.compactif.ai
FOUNDRY_API_KEY=xxxxxxxx
FOUNDRY_ORGANIZATION_ID=xxxxxxxxxxxx
FOUNDRY_PROJECT_ID=xxxxxxxxxxxx

Load the variables into your shell with:

source .env

Security: Never commit .env files or access tokens to source control. Add .env to .gitignore.

2. Install the OpenAI SDK​

Install the standard OpenAI Python SDK and httpx:

pip install openai httpx

No Foundry-specific SDK is required to access the inference API.

3. Authentication​

The standard OpenAI Python client normally sends its API key using:

Authorization: Bearer <api_key>

Foundry v2 Inference uses a different authentication flow. Your access token must be provided using:

x-api-key: <your-access-token>

The gateway exchanges the access token for a JWT.

Because the OpenAI SDK automatically generates an Authorization: Bearer ... header, the example below explicitly removes that header before the request is sent. The Foundry-specific authentication headers are then added to the request.

The following headers are required for every inference request:

x-api-key: <FOUNDRY_API_KEY>
x-organization-id: <FOUNDRY_ORGANIZATION_ID>
x-project-id: <FOUNDRY_PROJECT_ID>

4. Inference via the OpenAI Python SDK​

"""Call the Foundry v2 Inference API with the standard OpenAI client."""

import os
import sys

import httpx
from openai import OpenAI


BASE_URL = os.environ.get("FOUNDRY_BASE_URL", "http://localhost:9000")
MODEL = os.environ.get("FOUNDRY_MODEL", "Qwen/Qwen3-0.6B")

SYSTEM_PROMPT = "You are a helpful assistant."
USER_MESSAGE = "What is the capital of Colombia?"


def _require(name: str) -> str:
    value = os.environ.get(name, "").strip()
    if not value:
        raise SystemExit(f"{name} is not set — add it to .env and run: source .env")
    return value


def _drop_bearer(request: httpx.Request) -> None:
    """Remove the SDK's Authorization header so Foundry uses x-api-key."""
    request.headers.pop("authorization", None)


def build_client() -> OpenAI:
    return OpenAI(
        base_url=f"{BASE_URL.rstrip('/')}/api/v2/inference",
        # The SDK requires this argument, but Foundry ignores it — your access
        # token goes in the x-api-key header below.
        api_key="unused",
        default_headers={
            "x-api-key": _require("FOUNDRY_API_KEY"),
            "x-organization-id": _require("FOUNDRY_ORGANIZATION_ID"),
            "x-project-id": _require("FOUNDRY_PROJECT_ID"),
        },
        http_client=httpx.Client(event_hooks={"request": [_drop_bearer]}),
    )


def main() -> None:
    client = build_client()

    if "--models" in sys.argv:
        print(client.models.list().to_json())
        return

    response = client.chat.completions.create(
        model=MODEL,
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": USER_MESSAGE},
        ],
    )
    print(response.to_json())


if __name__ == "__main__":
    main()

Save as inference_v2_openai_example.py and run:

python inference_v2_openai_example.py

5. Listing Available Models​

python inference_v2_openai_example.py --models

Or from Python directly:

client = build_client()
models = client.models.list()
for model in models.data:
    print(model.id)

You can also configure the model through an environment variable:

export FOUNDRY_MODEL=Qwen/Qwen3-0.6B

6. Inference via cURL​

curl -X POST \
  "${FOUNDRY_BASE_URL}/api/v2/inference/chat/completions" \
  -H "Content-Type: application/json" \
  -H "x-api-key: ${FOUNDRY_API_KEY}" \
  -H "x-organization-id: ${FOUNDRY_ORGANIZATION_ID}" \
  -H "x-project-id: ${FOUNDRY_PROJECT_ID}" \
  -d '{
    "model": "Qwen/Qwen3-0.6B",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "What is the capital of Colombia?"}
    ]
  }'

7. Common Issues​

401 Unauthorized​

  • Verify FOUNDRY_API_KEY is set correctly.
  • Confirm your access token is being sent through the x-api-key header.
  • When using the OpenAI SDK, ensure the Authorization: Bearer ... header is removed.
  • Check the access token has access to the specified organization and project.

Missing organization or project​

Both headers are required on every inference request:

x-organization-id: <organization-id>
x-project-id: <project-id>

Copy both values from the Multiverse IAM dashboard: the ID is shown when you create an organization or project, and for existing ones you can select the organization or project and copy it. See Find your organization and project IDs.

Model not found​

First retrieve the available model IDs:

python inference_v2_openai_example.py --models

Then use one of the returned IDs in your request.

Incorrect endpoint​

When using the OpenAI Python client, the base_url must include the Foundry inference path:

base_url=f"{BASE_URL.rstrip('/')}/api/v2/inference"

The complete inference base URL is:

https://sdk.foundry-sandbox.compactif.ai/api/v2/inference

8. Minimal Application Example​

Once the client is configured, application code remains very close to standard OpenAI usage:

response = client.chat.completions.create(
    model="Qwen/Qwen3-0.6B",
    messages=[
        {"role": "user", "content": "Explain quantum computing in one paragraph."},
    ],
)
print(response.choices[0].message.content)

This compatibility allows existing applications built around the OpenAI SDK to connect to Foundry inference with minimal changes.