Build, Scale, and Govern AI Applications in the Enterprise
Quick Start • The Journey • Architecture • Prerequisites • Resources
Meet Promptling — your friendly guide through the world of Microsoft Foundry! This repository is your progressive learning path from simple inference to building a full fleet of intelligent, governed AI agents.
|
By the end of this journey, you'll have created agents that are:
|
The Architect — Ready to build! |
|
This repository implements an enterprise-first hub-and-spoke architecture:
Why this pattern?
|
┌─────────────────────────────────────┐
│ LANDING ZONE (HUB) │
│ (lab1a-foundry-lz-hub) │
├─────────────────────────────────────┤
│ │
│ ┌─────────────────────────────┐ │
│ │ API Management (APIM) │ │
│ │ • StandardV2 tier │ │
│ │ • Rate limiting policies │ │
│ │ • Managed Identity auth │ │
│ │ • Multi-backend routing │ │
│ └─────────────┬─────────────────┘ │
│ │ │
┌───────────────────┼────────────────┼────────────────────┐│
│ │ │ ││
▼ ▼ ▼ ▼│
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ AI Hub eastus2 │ │ AI Hub norwayeast│ │ AI Hub westus3 │ │ Shared Services │
├─────────────────┤ ├─────────────────┤ ├─────────────────┤ ├─────────────────┤
│ • gpt-4.1-mini │ │ • o3-deep- │ │ • DeepSeek-V3.2 │ │ • Storage │
│ • gpt-4.1 │ │ research │ │ │ │ • Key Vault │
│ • gpt-4.1-nano │ │ │ │ │ │ • App Insights │
│ • text-embed-3 │ │ │ │ │ │ │
│ • model-router │ │ │ │ │ │ │
└─────────────────┘ └─────────────────┘ └─────────────────┘ └─────────────────┘
│
┌───────────────────┼───────────────────┐
│ │ │
▼ ▼ ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ SPOKE PROJECTS (TEAMS) │
├─────────────────────┬─────────────────────┬─────────────────────────────────┤
│ Contoso Team │ Fabrikam Team │ Woodgrove Team │
│ ┌───────────────┐ │ ┌───────────────┐ │ ┌───────────────┐ │
│ │ Inventory AI │ │ │ Doc Studio │ │ │ Risk Analytics│ │
│ └───────────────┘ │ └───────────────┘ │ └───────────────┘ │
│ │ │ │
│ APIM Connection │ APIM Connection │ APIM Connection │
│ ┌───────────────┐ │ ┌───────────────┐ │ ┌───────────────┐ │
│ │landing-zone- │ │ │landing-zone- │ │ │landing-zone- │ │
│ │apim │ │ │apim │ │ │apim │ │
│ └───────────────┘ │ └───────────────┘ │ └───────────────┘ │
│ │ │ │
│ Allowed Models: │ Allowed Models: │ Allowed Models: │
│ • gpt-4.1-mini │ • gpt-4o │ • o1 │
│ • model-router │ • gpt-4o-mini │ • o3-deep-research │
└─────────────────────┴─────────────────────┴─────────────────────────────────┘
│
┌───────────────────┼───────────────────┐
│ │ │
▼ ▼ ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ FEATURE SPOKES (CAPABILITIES) │
├─────────────────────┬─────────────────────┬─────────────────────────────────┤
│ Foundry IQ │ Content Under- │ Built-in Tools │
│ Spoke (Lab 6) │ standing (Lab 9) │ Spoke (Lab 7A) │
│ ┌───────────────┐ │ ┌───────────────┐ │ ┌───────────────┐ │
│ │ AI Search │ │ │ Local Models │ │ │ Local gpt-4.1 │ │
│ │ Knowledge │ │ │ • gpt-4.1 │ │ │ File Search │ │
│ │ Bases │ │ │ • embeddings │ │ │ Code Interp. │ │
│ └───────────────┘ │ └───────────────┘ │ └───────────────┘ │
│ │ │ │
│ Uses APIM for │ Requires local │ Requires local │
│ chat + embeddings │ deployments │ deployments │
└─────────────────────┴─────────────────────┴─────────────────────────────────┘
Each phase builds upon the last. Follow Promptling through each adventure!
|
Plant your flag and establish your foundation
|
||||||||||||||||||||||||
|
Deploy models and scale your team
|
||||||||||||||||||||||||
|
Protect your infrastructure with policies
Deploy Azure Policy to prevent unauthorized model deployments in spokes. Keep your architecture secure and compliant! |
||||||||||||||||||||||||
|
Deploy containerized agents with Azure Developer CLI
Use the official |
||||||||||||||||||||||||
|
Give your agents the gift of memory
|
||||||||||||||||||||||||
|
Knowledge retrieval and RAG pipelines
Connect your agent to a Foundry IQ knowledge base for intelligent Retrieval Augmented Generation (RAG). |
||||||||||||||||||||||||
|
Equip your agents with powerful tools
|
||||||||||||||||||||||||
|
Multi-step agentic research with citations
Build deep research pipelines with |
||||||||||||||||||||||||
|
Extract insights from any content type
|
||||||||||||||||||||||||
|
Connect to Microsoft 365 ecosystem
Extend your agents to work with Microsoft 365 services and Copilot.
|
||||||||||||||||||||||||
|
Centralized agent discovery and management
Build a private tool catalog to manage agent discovery and organization-wide tools. |
||||||||||||||||||||||||
|
Multi-agent collaboration with Microsoft Agent Framework
Build a Planet Slideshow Builder with orchestrated agents: 🎯 Planner → 🔍 Researcher → 📝 Reviewer → ⚖️ Judge |
||||||||||||||||||||||||
|
See everything with OpenTelemetry tracing
Trace multi-agent hops, tool latencies, and system performance in the Foundry portal. |
||||||||||||||||||||||||
|
|
Step 14: Evaluation |
||||||||||||||||||||||||
|
Model Router automatically selects the best LLM for each request, optimizing for your priorities. See Lab 1C to configure it.
|
||||||||||||||||||||||||
|
|
Red Teaming Human-in-Loop |
||||||||||||||||||||||||
Before you begin, ensure you have:
| Requirement | Description |
|---|---|
| ☁️ Azure Subscription | Owner or User Access Administrator permissions |
| 🐍 Python 3.10+ | For running notebooks and scripts |
| 🔧 Azure CLI | Download here |
| 💻 VS Code | With Microsoft Foundry Extension |
# 1. Clone the repository
git clone https://github.com/Azure-Samples/AI-Engineer-Zero-to-Hero.git
cd AI-Engineer-Zero-to-Hero
# 2. Authenticate with Azure
az login
# 3. Create your .env file
cp .env.sample .env
# Edit .env with your Foundry connection string
# 4. Start with Step 01!
cd 01-project-setupThe following table shows how each lab's features integrate with the centralized Landing Zone architecture:
| Lab | Feature | Support Level | Notes |
|---|---|---|---|
| 00 - Image Gen | gpt-image-1.5 | APIM | Supports APIM (notebook not yet updated) |
| 01a - Landing Zone | Hub deployment | APIM | Deploys central APIM gateway + model deployments |
| 01b - Project Spoke | Spoke + APIM connection | Connection | Spoke consumes models via APIM connection |
| 01c - Model Router | model-router deployment | APIM | Intelligent routing across models |
| 02a - Team Spokes | Multi-team model access | Connection | Teams access models via <connection>/<model> |
| 02b - Direct APIM | REST API access | APIM | Direct APIM calls without Foundry SDK |
| 05 - Agent Memory | Memory API | Hybrid | Chat via APIM, but Memory API needs local embedding model |
| 06 - Foundry IQ | Knowledge bases | Connection | RAG via APIM for both chat and embeddings |
| 07a - Built-in Tools | File Search, Code Interpreter | Hybrid | File Search requires local model, Code Interpreter works via APIM |
| 07b - AI Gateway MCP | MCP tool governance | APIM | Extends APIM to govern MCP tool calls |
| 07c - Web Search | Bing grounding | Local | Web search tool requires local model |
| 08 - Deep Research | o3-deep-research | APIM | Full APIM support via Norway East hub |
| 09 - Content Understanding | Document/video analysis | Local | CU requires GPT-4.1 + embeddings in same resource |
| 12 - Agent Workflow | Multi-agent orchestration | APIM | Microsoft Agent Framework via APIM gateway |
| 13 - Human-in-Loop | Function approval | APIM | Works with APIM-based agents |
| 16 - Local Evaluation | Azure AI Evaluation SDK | APIM | Evaluators use APIM for judge model |
AI-Engineer-Zero-to-Hero/
├── 📖 README.md # You are here!
├── 🎨 00-image-generation/ # Promptling mascot images
├── 🏗️ 01-project-setup/ # Phase 1: Foundations
│ ├── lab-1a-landing-zone/
│ ├── lab-1b-project-spoke/
│ └── lab-1c-model-router/
├── 🔄 02-inference/ # Unified inference
├── 🛡️ 03-governance-policy/ # Azure Policy
├── 📦 04-hosted-agents/ # Hosted agents
├── 🧠 05-agent-memory/ # Memory service
├── 🔍 06-foundry-iq/ # Knowledge & RAG
├── 🔧 07-tool-catalog/ # Tools & MCP
├── 📚 08-deep-research/ # Research pipelines
├── 📄 09-content-understanding/ # Document/video analysis
├── 📋 10-agent-registry/ # Agent discovery
├── 🔗 11-agent-365/ # M365 integration (Step 10)
├── 🔀 12-agent-workflow/ # Multi-agent orchestration
├── 👤 13-human-in-loop/ # Safety: Human approval
├── 14-m365-integration/ # Additional M365
├── 14b-m365-foundry-publish/ # Publish Agents via Activity Protocol
├── 15-observability/ # Tracing & monitoring
├── 📊 16-evaluation/ # AI quality evaluation
└── 🔴 16-red-teaming/ # Safety: Red teaming
- Advanced Observability Lab — Deep dive into tracing with Foundry Agents and Application Insights integration
- Cloud Evaluations Lab — Run evaluations at scale using Foundry's cloud-based evaluation infrastructure
- Speech Capabilities Lab — Explore Foundry's voice features including the Voice Live API
- Basic vs Standard Agent Deployment — Configure secure access to resources used within Agents (VNet integration, private endpoints)
Have ideas for new labs? Open an issue or submit a PR!
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.
When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.
Microsoft takes the security of our software products and services seriously, which includes all source code repositories managed through our GitHub organizations, which include Microsoft, Azure, DotNet, AspNet and Xamarin.
If you believe you have found a security vulnerability in any Microsoft-owned repository that meets Microsoft's definition of a security vulnerability, please report it to us as described in SECURITY.md.
This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies. Any use of third-party trademarks or logos are subject to those third-party's policies.
Happy Building! 🚀
— Promptling & Team
Built with 💙 for AI