Prototyping AI Agents with GitHub Models (for Free!) 🤖💸

I love when companies roll out generous free tiers. It feels like they’re saying, “Hey, go build your thing — we’ve got your back.” And if you’re a student, between jobs, or just tired of racking up charges for every API call (yep, been there too), free tiers can be a total game-changer.

That’s exactly why GitHub Models stood out to me — it’s like an AI candy shop, completely free to explore, as long as you have a GitHub account.

Here’s what’s on the shelf:

🔮 OpenAI models like gpt-4o, o3-mini

🧠 Research favorites like Phi and LLaMA

🌍 Multimodal models like llama-vision-instruct

📚 Embeddings from Cohere and OpenAI

⚡ Plus providers like Mistral, Jamba, Codestral and more

Oh, and the best part? Many of these models support function calling, making them perfect for agent-style apps.

Now here’s the real kicker:

GitHub Models speak OpenAI-compatible API — which means any Python framework that already works with OpenAI's ChatCompletion API... just works out of the box.

Example 1: Connecting openai SDK to GitHub Models

import openai
import os

client = openai.OpenAI(
  api_key=os.environ["GITHUB_TOKEN"],
  base_url="https://models.inference.ai.azure.com"
)

Now go ahead and use it like you would with OpenAI:

client.chat.completions.create(...)

Simple, clean, no surprises.

Example 2: Running AutoGen with GitHub Models

import autogen_ext.models.openai
import autogen_agentchat.agents
import os

client = autogen_ext.models.openai.OpenAIChatCompletionClient(
  model="gpt-4o",
  api_key=os.environ["GITHUB_TOKEN"],
  base_url="https://models.inference.ai.azure.com"
)

math_teacher = autogen_agentchat.agents.AssistantAgent(
    name="Math_teacher",
    model_client=client,
    system_message="You only teach maths."
)

Just like that, your agent is ready to go.

You can plug GitHub Models into tons of other Python libraries too — LangGraph, PydanticAI, LlamaIndex, you name it.

Go build something fun. Happy tinkering!