Microsoft has introduced Azure AI Foundry, rebranding the existing Azure AI Studio. This platform serves as a new central hub for designing, customizing, and managing AI applications and agents effectively at scale.

What is Azure AI Foundry?

With Azure AI Foundry, you can:

  • Explore a wide variety of models, services, and capabilities.
  • Build AI applications tailored to your specific goals.
  • Facilitate scalability, transforming proofs of concept into full-fledged production applications with ease.
  • Leverage continuous monitoring and refinement features to support long-term success.

Core Components

Azure AI Foundry primarily consists of two components:

  1. Azure AI Foundry Hub: This is the core, region-specific infrastructure used to interface with AI models. You need to create a Hub first.
  2. Azure AI Foundry Project: Built upon a Hub, Projects are where you deploy specific AI models (like Phi-3, DeepSeek, Mistral). These are also region-specific.

The native Azure AI Foundry SDK aims to provide a simpler and more unified experience for developers building Generative AI applications.

The Challenge: Private Connections

While the documentation for this service is comprehensive, configuring private network connections for model inference seemed a bit unclear based on my experience. This post aims to provide guidance on achieving this.

Step-by-Step: Setting Up Private Endpoints

To establish a private connection for inference to models deployed in an Azure AI Foundry Project, follow these steps:

  1. Navigate to your Azure AI Foundry Hub resource within the Azure portal.
  2. From the left-side menu, select Settings, then Networking. Click on the Private endpoint connections tab and select + Private endpoint.
  3. When filling out the forms to create the private endpoint:
    • On the Basics tab, ensure the selected Region matches the region of your virtual network.
    • On the Resource tab, select amlworkspace as the Target sub-resource. (Internally, it leverages Azure Machine Learning Workspace infrastructure).
    • On the Virtual Network tab, select the target Virtual network and Subnet you wish to connect from.
  4. Configure any additional network settings as required, review your settings on the Review + create tab, and then click Create.

This process should automatically configure the necessary DNS records within your private DNS zones associated with the virtual network.

And that's it! No further configuration should be necessary. Attempts to reach your serverless model endpoint (e.g., xxx.region.models.ai.azure.com) should now resolve privately.

How it Works

Image description

The reason this works is that Azure automatically creates a CNAME DNS record for your endpoint, similar to:

..models..privatelink.api.azureml.ms

The Private Endpoint you created specifically handles DNS resolution for this .privatelink address, ensuring traffic stays within your private network.

Hopefully, this guide clarifies the process for setting up private connections in Azure AI Foundry.

Documentation for reference : Azure AI Foundry Private Endpoints