Skip to main content

LiteLLM

Connect LiteLLM to AI Studio using its SDK or proxy.

This quickstart guide walks you through connecting Hyperstack AI Studio with LiteLLM. You’ll learn how to authenticate, configure environments, and make chat/completion requests to AI Studio models using either the LiteLLM SDK or Proxy.

Full Guide

For a full walkthrough, including production-ready proxy workflows and advanced configurations, see How to Integrate Hyperstack AI Studio as a Provider in LiteLLM.

Why Use LiteLLM with Hyperstack​

LiteLLM simplifies the developer experience when working with OpenAI-compatible models by abstracting away provider-specific logic. Hyperstack AI Studio complements this by offering a catalog of text and image models behind an OpenAI-compatible API.

When used together, LiteLLM provides a streamlined interface to Hyperstack’s powerful backend infrastructure. Hyperstack AI Studio serves the models, while LiteLLM handles routing, configuration, and API abstraction - ideal for everything from experimentation to production deployment.

Hyperstack AI StudioLiteLLM
AI backend runs inferenceInterface layer routes requests and provides SDK/proxy access
OpenAI-compatible APISupports OpenAI schema out of the box
Scalable, secure model infrastructureUnified function calls and proxy endpoints
Catalog of text and image modelsEasy local or cloud integration

To integrate LiteLLM with Hyperstack, you can choose between two supported workflows depending on your environment:

Each method is compatible with any AI Studio model via OpenAI-compatible endpoints.

Option 1: Python SDK Integration​

Use this approach for local dev, testing, or scripts.

  1. Install LiteLLM​

    Install the core LiteLLM SDK via pip:

    terminal
    pip install litellm

  2. Get Your API Key and Model Details​

    Generate the necessary credentials to authenticate with Hyperstack AI Studio.

    1. Log in to the Hyperstack Console
    2. Navigate to the AI Studio Playground and select your desired model (e.g., meta-llama/Llama-4-Scout-17B-16E-Instruct)
    3. Click on the API tab to retrieve your Base URL and Model ID
    4. Visit the API Keys page and generate a new key
    5. Copy and securely store the generated API key

  3. Set Up Environment Variables​

    Configure your authentication and base endpoint as environment variables:

    python
    import os
    os.environ["OPENAI_API_KEY"] = "<your-hyperstack-api-key>"
    os.environ["OPENAI_API_BASE"] = "https://console.hyperstack.cloud/ai/api/v1"

  4. Make a Completion Call​

    Run a basic test query to verify the setup:

    python
    from litellm import completion

    response = completion(
      model="openai/meta-llama/Llama-4-Scout-17B-16E-Instruct",
      messages=[{"role": "user", "content": "Who won the World Cup in 2022?"}]
    )

    print(response["choices"][0]["message"]["content"])

Option 2: Proxy Server Integration​

For team-based or production workloads, use the LiteLLM proxy server. It allows centralized management of models, API keys, usage tracking, and routing.

  1. Download Config Files​

    You will need two files: one for Docker Compose and another for Prometheus monitoring.

    terminal
    curl -O https://raw.githubusercontent.com/BerriAI/litellm/main/docker-compose.yml
    curl -O https://raw.githubusercontent.com/BerriAI/litellm/main/prometheus.yml

  2. Create a .env File​

    Define your admin credentials and encryption salt in a .env file:

    terminal
    echo 'LITELLM_MASTER_KEY="sk-admin"' > .env
    echo 'LITELLM_SALT_KEY="sk-salt"' >> .env
    source .env

  3. Start the Proxy Server​

    Run the following to launch the proxy:

    terminal
    docker compose up

    Once started, access the admin dashboard at:

    http://0.0.0.0:4000

  4. Add Your Hyperstack Model​

    After logging into the admin dashboard with your master key:

    1. Go to the Models section and click Add Model

    2. Select the OpenAI-Compatible Provider option

    3. Enter the following details:

      • Model Name: meta-llama/Llama-4-Scout-17B-16E-Instruct
      • Base URL: https://console.hyperstack.cloud/ai/api/v1
      • API Key: The key you generated in the Hyperstack Console

    4. Click Add Model to register it

  5. Test Your Model in the Proxy Playground​

    Navigate to the Playground tab in the LiteLLM dashboard:

    1. Select the model you just added

    2. Enter a prompt like:

      What's the capital of Nepal?

    3. Confirm the model response:

    The capital of Nepal is Kathmandu.

  6. Make cURL or SDK Requests​

    You can now access the Hyperstack model via REST API requests to the proxy endpoint:

    terminal
    curl --location 'http://0.0.0.0:4000/chat/completions' \
    --header 'Content-Type: application/json' \
    --data '{
      "model": "meta-llama/Llama-4-Scout-17B-16E-Instruct",
      "messages": [
        {"role": "user", "content": "What LLM are you?"}
      ]
    }'

    This will return a valid OpenAI-compatible response routed to your AI Studio model.

Next Steps

For details on how to move from prototype to production - including caching, load balancing, and managing multiple models - see How to Integrate Hyperstack AI Studio as a Provider in LiteLLM.

On this page