Duration: 75 minutes
Want to build trustworthy AI agents? The hard part lies NOT in making them work the first time, but in keeping them working continuously over the lifetime of the application - even as models get updated, prompts get refined, retrieval pipelines drift, and real-world use uncovers edge cases. To do this, we need a unified platform that provides end-to-end observability - with rich developer tools that take us seamlessly from detecting AI quality issues to diagnosing them, and optimizing solutions for performance in an iterative fashion.
In this workshop, we'll get hands-on experience with the Microsoft Foundry observabilty platform by exploring one of two available paths:
- Using Foundry SDK - use a code-first approach to learn how you can build, evaluate, trace, and red-team your AI agent solution.
- Using Foundry Skills - use a coding-agent approach to orchestrate a full eval-driven optimization cycle from a base AI agent.
The first option is more traditional, giving you a sense for the concepts, tools and workflows involved. The second option offers an early preview into an enhanced developer experience where the coding agent accelerates the "evaluate-optimize" loop to take actions (like creating datasets, running batch evaluations, comparing versions, optimizing prompts or instructions etc.) based on observed results rather than guesswork. Along the way, you'll build your intuition for these tools by looking at the agent reasoning process and guiding its decision-making when prompted.
We'll use a consistent scenario across all labs, allowing us to think about features and outcomes in the context of a real-world use case.
Contoso Travel is a fictitious mid-size travel agency whose team of human advisors can no longer keep up with the volume of customer inquiries for booking travel. They need an AI-powered travel assistant — a system of intelligent agents that can search relevant inventory (e.g., hotels, flights, car rentals) to make personalized recommendations, and deliver customized itimeraries across multi-turn conversations.
In this workshop, we'll trace the AI developer journey from planning to prototyping to production. By the end of the workshop, you should be able to:
- Observe agentic execution with OpenTelemetry traces
- Optimize agentic performance assisted by Foundry skills
- Protect agents from attacks using Red Teaming scans
- Deploy agents, then monitor & analyze insights with Ask AI
We'll achieve this using the Microsoft Foundry platform tools and workflows. By the end of this workshop you should be able to:
- Setup a single prompt agent with no code - using the Foundry portal.
- Build a multi-agent solution code-first - using the Foundry SDK
- Observe & Optimize an AI agent with coding agents - using Foundry skills
This workshop has two paths to choose from:
- Base Path: Complete Steps 1 & 2 to setup your infra & dev environment.
- Path A: Do Step 3. Uses Foundry Skills to automate the eval-optimize loop.
- Path B: Do Step 4. Uses Foundry SDK for traditional coding (manual).
Completing the end-to-end journey with either path can take 60 minutes or more. Pick one to complete - then try the second if time permits.
| Step | Instructions | Tool · Outcome |
|---|---|---|
| 1. | Infrastructure Setup | Foundry Portal · Setup Foundry project |
| 2. | Dev Environment Setup | GitHub Codespaces · Setup local .env |
| 3. | Activate Observe Skill | Foundry Skills · Run eval-optimize loop |
| 4. | Build It Step-By-Step | Foundry SDK · Go plan-production manually |
The current workshop (v1) is setup for use with prompt agents. The next version (v2) will expand this to showcase hosted agents that use custom code and runtimes with a containerized environment for maximum developer control. Fork & watch the repo for updates in May/June 2026.
The key takeaway is that the Microsoft Foundry Observability platform will work effectively with any agent (build using any programming language or framework) provided it supports OpenTelemetry-compliant traces and Responses API compliant endpoints.
The Microsoft Foundry Control Plane provides tools and features to support security, compliance, fleet management and observability for your agentic AI solutions - along with a unified role-aware management interface accessed through the "Operate" tab of the Microsoft Foundry portal.
In this workshop, we put the spotlight on Observability - but we encourage you to explore the resources below to dive deeper into the various components involved.
| Resource | Description |
|---|---|
| Foundry Control Plane | Enterprise-wide visibility, governance, and control of AI agents, models & tools |
| Observability | Monitor, understand, and troubleshoot your AI agents |
| Agent Tracing | OpenTelemetry (OTel) protocols & semantic conventions support in Foundry |
| Evaluations | Built-in and custom evaluators for quality, safety & agentic performance |
| Red Teaming | Adversarial testing for targeted risk categories & attack strategies |
This workshop is intended to be an evergreen resource that will evolve to reflect the latest Microsoft Foundry platform updates. For convenience, we will maintain branches for prior workshop versions that were delivered at specific events.
| Date | Branch | Description |
|---|---|---|
| Mar 27 2026 | 2026-03-mvp-summit | Prompt agent with observe skill |
| Apr 04 2026 | 2026-04-aie-europe | Prompt agent with observe skill |
| Apr 18 2026 | 2026-04-msft-tw | Prompt agent with observe skill |
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit Contributor License Agreements.
When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.
This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.