🎯 TL;DR: Four Agent Types, Four Different Workflows

GitHub Copilot in VS Code now supports four distinct agent types, each designed for different workflows and levels of autonomy. Local Agent is your interactive coding partner, running in VS Code with full access to all your tools, MCP servers, and three personas (Agent, Plan, Ask). Coding Agent (Cloud) runs on GitHub’s cloud infrastructure via Actions runners, works fully autonomously on issues, and creates PRs while you’re away. Background Agent (Copilot CLI) runs locally but outside the VS Code process; it survives restarts, supports parallel sessions, and can hand off work to cloud agents with /delegate. Sub-Agents are the secret weapon for context management, running as isolated subtasks within a parent agent session, keeping the main agent’s context window clean while handling research, analysis, or parallel tasks.

Key insight: If you’re using a 1x premium model like Claude Sonnet 4, sub-agent calls are effectively free, making them the most cost-efficient way to scale complex multi-step workflows without burning through your premium request budget.

GitHub Copilot has evolved far beyond simple code completions. With agent mode in VS Code, developers gained an autonomous coding assistant that could plan, execute, and iterate on complex tasks. But as workflows grew more sophisticated, a single agent type wasn’t enough to cover every scenario, from quick interactive debugging to full autonomous issue resolution that runs while you sleep.

Today, GitHub Copilot in VS Code supports four distinct agent types, each optimized for different workflows, contexts, and levels of autonomy. Understanding when to use each one, and how they interact, is the difference between fighting your tools and having them work seamlessly for you.

The Four Agent Types at a Glance

Before diving deep into each agent type, here’s a high-level comparison:

💡 Key concept: The session type dropdown in VS Code’s Chat view is your primary control for switching between Local, Background (Copilot CLI), and Cloud agents. Sub-agents are invoked programmatically within any session.

1. Local Agent (Standard Agent Mode)

The Local Agent is the standard interactive agent in VS Code. It’s the workhorse of day-to-day Copilot interactions, running directly in your VS Code instance with full access to your workspace, tools, and extensions.

How It Works

You open it with Ctrl+Alt+I (or Cmd+Shift+I on macOS) and interact with it directly in the Chat view. It can read and edit files, run terminal commands, and leverage any tools you’ve configured.

Three Built-In Personas

Full Tool Access

The Local Agent’s biggest advantage is unrestricted access to tools: built-in tools (file editing, terminal, search, debugging), any configured MCP servers, extension-provided tools, and BYOK models. This makes it the most versatile of all four types.

Autopilot Mode

For tasks where you want minimal interruption, enable Autopilot mode. The agent automatically approves tool calls and continues executing without per-step confirmation.

⚠️ Use Autopilot mode with caution. The agent will execute terminal commands and edit files without asking for permission. Always review the results when it’s done.

When to Use the Local Agent

• Interactive coding sessions needing back-and-forth conversation
• Tasks requiring extension tools (test runners, debuggers, specialized MCP servers)
• BYOK model usage when you need a specific model not available in cloud mode
• Sensitive codebases where you don’t want code leaving your machine
• Quick fixes that take a few minutes

Limitations

• Tied to VS Code: closing VS Code stops the agent
• Not collaborative: other team members can’t see or interact with your session

2. Coding Agent (Cloud)

The Coding Agent is GitHub Copilot’s fully autonomous cloud-based agent. It runs on GitHub’s infrastructure using Actions runners and is designed for tasks that don’t require your active involvement. You can assign it an issue and walk away.

How It Works

You can invoke it from multiple entry points: VS Code Chat view (“Cloud” session type), GitHub Issues (assign to @copilot), Pull Request comments, GitHub CLI, or external integrations (Jira, Slack, Teams).

Once triggered, the Coding Agent analyzes the task, reads the repository code, creates a branch with a copilot/ prefix, implements changes autonomously, runs security checks (CodeQL, dependency scanning, secret scanning), and creates a draft Pull Request.

Fully Asynchronous

This is the defining characteristic: you can close your laptop. The agent runs entirely on GitHub’s cloud infrastructure, so it keeps working whether or not you’re at your desk.

🔒 Security by default. The Coding Agent enforces CodeQL analysis, dependency scanning, and secret scanning on every PR it creates. These checks are non-negotiable.

When to Use the Coding Agent

• Well-defined issues with clear acceptance criteria
• Tasks you want done while you’re away (overnight, during meetings)
• Bug fixes with clear reproduction steps
• Boilerplate or repetitive tasks (CRUD endpoints, adding tests, updating configs)
• Team workflows where any team member can assign issues to @copilot

Limitations

The Coding Agent is scoped to one repository per task, produces exactly one draft PR per task, cannot access your VS Code extensions, and uses both premium requests and Actions minutes (the most expensive agent type per task). You also can’t guide it mid-task, though you can comment on the PR afterward.

The Coding Agent is available on Pro, Pro+, Business, and Enterprise plans.

3. Background Agent (Copilot CLI)

The Background Agent bridges the gap between local and cloud agents. It runs on your local machine but outside the VS Code process, meaning it survives VS Code restarts and can run multiple sessions in parallel.

How It Works

The Background Agent is powered by the Copilot CLI agent harness. You can invoke it from the Chat view dropdown (“Copilot CLI”), Command Palette, or by typing copilot directly in your terminal.

Isolation Modes

💡 Worktree mode is the safer option. It creates a separate copy of your repository, so the Background Agent’s changes never interfere with your active work.

Parallel Sessions

Unlike the Local Agent, the Background Agent can run multiple sessions simultaneously, each with its own independent context window and worktree. This is powerful for parallelizing work across different tasks.

The /delegate Command and Hand-Off

The /delegate command hands off a task to the Coding Agent (Cloud), creating a smooth escalation path: start working locally, realize the task needs full autonomy and a PR, and delegate to the cloud without losing context.

⚡ Conversation history carries over during hand-offs. The new agent receives the full conversation context so it can continue where the previous agent left off.

Cost: Premium Requests Only

The Background Agent uses only Copilot premium requests, with no GitHub Actions minutes. This makes it significantly cheaper than the Coding Agent for comparable tasks.

When to Use the Background Agent

• Long-running tasks that you don’t want tied to your VS Code session
• Parallel work where you need multiple agents on different tasks simultaneously
• Tasks you want to start and check on later
• Exploratory work using Worktree mode to safely experiment
• Pipeline to cloud: start locally, get context, then /delegate for PR creation

Limitations

• No extension-provided tools: runs outside the VS Code process, so it can’t access extension tools
• Requires machine to stay running: survives VS Code restarts but not machine shutdown or sleep
• Local resources only: uses your machine’s CPU, memory, and network

4. Sub-Agents: The Context Management Secret Weapon

Sub-Agents are the most underappreciated feature in the Copilot agent ecosystem. They’re spawned as isolated subtasks within a parent agent session, and their primary superpower is context isolation.

The Context Problem

Every AI agent has a finite context window. As your conversation grows with file reads, search results, and intermediate reasoning, that window fills up and the agent starts losing important details. For complex multi-step tasks, the research phases can consume so much context that the agent forgets critical details by the time it begins implementation.

How Sub-Agents Solve This

Sub-Agents run in a completely isolated context window:

1. The parent agent spawns a sub-agent with a specific task prompt (the sub-agent does NOT inherit the parent’s conversation history)
2. The sub-agent performs all its work (file reads, searches, analysis) in its own isolated context
3. It returns only a final summary/result to the parent; all intermediate data is discarded

The result: the parent agent gets concise answers without its context window being polluted by hundreds of lines of intermediate data.

Visual: How the #runSubAgent Tool Works

The image above illustrates the sub-agent flow. When you invoke the #runSubAgent tool, it creates a separate context window for the sub-agent. The sub-agent performs all tool calls independently, and only the final result summary is passed back to the main agent. The sub-agent appears as a collapsible tool call in the Chat UI.

Invoking Sub-Agents

Sub-Agents can be triggered in two ways:

Agent-initiated: The main agent autonomously decides to spawn a sub-agent when it recognizes a task that would benefit from context isolation.

User-hinted: You can explicitly request one:

1
2

You: Run a subagent to research all the authentication patterns used in this 
     codebase and give me a summary of the approaches used.

Or reference the tool directly: #runSubAgent

The Cost Advantage

If you’re using a 1x premium model, sub-agent calls are effectively free. You can run dozens of sub-agents in parallel without worrying about your premium request budget.

Instead of using one expensive model call that reads 50 files and floods context, spawn multiple cheap sub-agents to research different aspects in parallel, collect their concise summaries, then use a single focused model call for the final implementation.

Orchestration Pattern: Coordinator & Worker

1
2
3
4
5

Main Agent (Coordinator):
├── SubAgent 1: "Analyze the authentication module and list all auth strategies used"
├── SubAgent 2: "Analyze the database layer and list all query patterns"  
├── SubAgent 3: "Analyze the API layer and list all middleware in the pipeline"
└── Main Agent: Uses summaries from all three to implement a new feature

What Sub-Agents CAN and CANNOT Do

When to Use Which Agent: Quick Decision Matrix

Best Practices

Key Takeaways

• Local Agent is your interactive coding partner: versatile, full-featured, but tied to your VS Code session
• Coding Agent (Cloud) is your autonomous worker: fire and forget, creates PRs, runs on GitHub’s infrastructure
• Background Agent (Copilot CLI) is the bridge: local execution with persistence, parallelism, and cloud delegation
• Sub-Agents are the context management secret weapon: keep your main agent’s context clean while parallelizing research and analysis
• Cost optimization is key: use 1x premium models for sub-agents and routine tasks, reserve premium models for complex work
• Hand-off between agents is seamless: start local, escalate to background, delegate to cloud as needed

The most effective Copilot users don’t just use one agent type. They orchestrate all four to match the right tool to the right task.

References

• GitHub Copilot Coding Agent
• GitHub Copilot Plans and Pricing

Image Credits:

• Main image generated by GPT-Image-1.5

🎯 TL;DR: Subject-Driven Image Generation on 12GB VRAM

Large AI models like FLUX.1-schnell typically require datacenter GPUs with 48GB+ VRAM. Problem: Most developers and hobbyists only have access to consumer RTX cards which vary from 6 - 12GB VRAM in most cases (with the exception of the expensive 4090/5090 cards which can go up to 32gb).

Solution: Using mmgp (Memory Management for GPU Poor) with Docker containerization enables FLUX.1 OmniControl to run on RTX 3060 12GB through 8-bit quantization, dynamic VRAM/RAM offloading, and selective layer loading. The implementation provides a Gradio web interface generating 512x512 images in ~10 seconds after initial model loading, with models persisting in system RAM to avoid reload overhead.

Technical Approach: Profile 3 configuration quantizes the T5 text encoder (8.8GB → ~4.4GB), pins the FLUX transformer (22.7GB) to reserved system RAM, and dynamically loads only active layers to VRAM during inference. Tested and validated on RTX 3060 12GB with 64GB system RAM running Windows 11 + WSL2 + Docker Desktop.

Complete Implementation: All code, Dockerfile, and setup instructions are available at github.com/Ricky-G/docker-ai-models/omnicontrol

Recently, I wanted to experiment with OmniControl, a subject-driven image generation model that extends FLUX.1-schnell with LoRA adapters for precise control over object placement. The challenge? The model requirements listed 48GB+ VRAM, and I only had an RTX 3060 with 12GB sitting in my workstation.

This is a common frustration in the AI development community. Research papers showcase impressive results on expensive datacenter hardware, but practical implementation on consumer GPUs requires significant engineering effort. Could I actually run this model locally without upgrading to an RTX 4090/5090 or pay for a VM in Azure with A100?

The answer turned out to be yes - with some clever memory management and containerization. This blog post walks through the complete process of dockerizing OmniControl to run efficiently on a 12GB consumer GPU.

What is FLUX.1 OmniControl?

Before diving into the technical implementation, let’s understand what we’re working with. FLUX.1 OmniControl is a subject-driven image generation model that extends the base FLUX.1-schnell diffusion model with LoRA (Low-Rank Adaptation) adapters for precise control over object placement and composition.

Unlike traditional text-to-image models where you only provide a text prompt, OmniControl allows you to:

• Subject Consistency: Provide a reference image of a specific object (like a toy, person, or product) and have it accurately reproduced in generated images
• Spatial Control: Specify exactly where in the scene you want objects placed
• Style Preservation: Maintain the visual characteristics of the reference object across different contexts and environments

Think of it as “Photoshop + AI” - you can place your specific objects into any scene you can describe with text. This makes it incredibly powerful for product visualization, creative content generation, and prototyping visual concepts.

The trade-off? The model is massive - requiring over 30GB of model weights to achieve this level of control and quality. This is where the engineering challenge begins.

The Challenge: Model Size vs Available VRAM

Let’s start with the hard numbers:

FLUX.1-schnell model components:

• Transformer: 22.7GB (torch.bfloat16)
• T5 Text Encoder: 8.8GB
• CLIP Text Encoder: 162MB
• VAE: ~1GB
• OminiControl LoRA: 200MB
• Total: ~32.8GB of model weights

Available hardware:
I am constrained by my existing workstation specs:

• RTX 3060: 12GB VRAM
• System RAM: 64GB DDR4
• Storage: 1TB NVMe SSD + 2TB HDD
• CPU: Intel i7-11700K
• OS: Windows 11 + WSL2 (Ubuntu 22.04)
• Docker Desktop with NVIDIA Container Toolkit

The gap is obvious - we need nearly 3x more VRAM than the GPU provides. Traditional approaches like FP16 precision or model pruning weren’t going to cut it. We needed something more aggressive.

Understanding mmgp: Memory Management for GPU Poor

The key enabler for this project is mmgp (Memory Management for GPU Poor), a Python library specifically designed to run large models on consumer hardware. Here’s how it works:

8-Bit Quantization

mmgp uses quanto to quantize large model components from 16-bit to 8-bit precision:

• T5 encoder: 8.8GB → ~4.4GB (50% reduction)
• Quality impact: Minimal for text encoding tasks
• Speed impact: Slight increase in encoding time (~10-15%)

Dynamic VRAM/RAM Offloading

Instead of keeping all model weights in VRAM, mmgp maintains a “working set”:

• Critical layers: Loaded to VRAM during active use
• Inactive layers: Offloaded to pinned system RAM
• Transfers: Handled automatically during forward passes

RAM Pinning Strategy

Models are loaded once from disk to system RAM (one-time cost), then:

• Pinned memory allocation: 75% of system RAM reserved (48GB in my case)
• Fast transfers: Pinned RAM → VRAM takes ~200ms for 1GB
• Persistent storage: Models stay in RAM across generations

Profile System

mmgp provides 5 preconfigured profiles:

For RTX 3060, Profile 3 provides the best balance between speed and stability.

Prerequisites: What You’ll Need

Before starting the implementation, ensure you have the following components set up:

Hardware Requirements

Minimum Configuration:

• NVIDIA GPU: 12GB VRAM (RTX 3060, 3060 Ti, or better)
• System RAM: 64GB DDR4/DDR5 (48GB will be pinned for model storage)
• Storage: 50GB free space (35GB for models + overhead)
• CPU: Any modern multi-core processor

Recommended Configuration:

• GPU: RTX 3060 12GB or RTX 4060 Ti 16GB
• RAM: 64GB or more
• Storage: NVMe SSD for faster startup times (HDD works but adds 2-3 min to load times)

Software Requirements

Windows Users:

• Windows 11 (Windows 10 with WSL2 also works)
• WSL2 installed and configured
• Docker Desktop for Windows (latest version)
• NVIDIA Container Toolkit (installed via Docker Desktop)

Linux Users:

• Ubuntu 22.04 or similar distribution
• Docker Engine (latest version)
• NVIDIA Container Toolkit
• NVIDIA drivers (version 525+)

Account Requirements

• HuggingFace Account: Required to download models
• HuggingFace Token: Generate a read-access token at huggingface.co/settings/tokens

Verification Steps

Before proceeding, verify your setup:

1
2
3
4
5
6
7
8

# Check GPU availability
nvidia-smi

# Verify Docker installation
docker --version

# Test GPU access in Docker
docker run --rm --gpus all nvidia/cuda:12.1.1-base-ubuntu22.04 nvidia-smi

If all commands execute successfully, you’re ready to begin!

Docker Architecture: Why Containerization?

With prerequisites confirmed, let’s talk about why Docker is the right choice for this project. Running large AI models involves complex dependency chains - specific versions of PyTorch, CUDA libraries, Python packages, and system libraries that can conflict with your existing environment.

💡 Want to Skip Ahead?

The complete Docker implementation, including the Dockerfile, all Python code, and deployment scripts, is available in my GitHub repository: docker-ai-models/omnicontrol

You can clone and run it immediately, or continue reading to understand how it works under the hood.

Containerization solves this by:

• Isolating dependencies from your host system
• Ensuring reproducibility across different machines
• Simplifying deployment - one command to run the entire stack
• Enabling version control of the entire environment

The containerization approach provides several additional benefits:

• Eliminates dependency conflicts
• Ensures reproducible builds
• Simplifies deployment across machines
• Isolates model storage from application code

Container Structure

1
2
3
4
5
6
7

nvidia/cuda:12.1.1-cudnn8-runtime-ubuntu22.04
├── Python 3.10 + CUDA libraries
├── PyTorch 2.0 with CUDA support
├── Diffusers + Transformers
├── mmgp for memory management
├── Gradio for web interface
└── Custom FLUX integration code

Volume Mounts

1

-v D:\_Models\omnicontrol:/app/models    # Persistent model storage (34GB)

Models download once to the host system and persist across container rebuilds. This is critical for development iteration - rebuilding the container doesn’t trigger 30-minute model downloads.

GPU Access

1

--gpus all    # Exposes all NVIDIA GPUs to container

Docker Desktop + NVIDIA Container Toolkit handles GPU passthrough automatically on Windows via WSL2.

Implementation Details: From Code to Running System

Now that we understand the architecture and tools, let’s dive into how everything works in practice. This section covers the actual startup sequence, performance characteristics, and a critical optimization that makes this entire approach viable.

⚡ Key Performance Insight Ahead

One of the biggest challenges in this implementation was preventing VRAM from being cleared after each generation, which would cause 80+ second reload times. The solution? A single line of code change that reduced subsequent generation times from 120s to 10s. We’ll cover this critical fix in detail below.

Startup Sequence

The container initialization follows this sequence:

1. GPU Detection (~1 second)

1
2

nvidia-smi --query-gpu=name,memory.total --format=csv
# Output: NVIDIA GeForce RTX 3060, 12288 MiB

2. Profile Selection (automatic)

1
2
3

vram = get_gpu_memory()
if vram >= 11000:
    profile = 3  # 12GB optimized

3. Model Loading (2-3 minutes from HDD)

• FLUX.1-schnell: Downloads from HuggingFace (~22.7GB)
• OminiControl LoRA: Downloads adapter weights (~200MB)
• Loads to CPU first, then applies mmgp profiling

4. mmgp Profiling (1-2 minutes)

• Quantizes T5 encoder to 8-bit
• Allocates 48GB pinned RAM (75% of 64GB)
• Hooks model layers for dynamic offloading

5. Gradio Launch (~5 seconds)

• Web interface starts on port 7860
• Ready to accept generation requests

Total first-run time: 5-10 minutes (mostly downloading models)
Subsequent runs: ~3 minutes (loading from disk to RAM)

Generation Performance

First generation after startup:

• Time: ~110 seconds
• Breakdown:
  • VRAM loading: 80 seconds (22.7GB from RAM → VRAM)
  • Actual inference: 30 seconds (8 steps @ 512x512)
• GPU memory: Climbs from 3GB → 10-12GB

Subsequent generations:

• Time: ~10 seconds (target achieved!)
• VRAM stays at 10-12GB between generations
• No reload overhead

Critical Fix: Preventing VRAM Clearing

🚨 This Section Contains The Key Optimization

Initial testing revealed a major performance bottleneck that would have made this entire approach impractical. Understanding and fixing this issue is critical for achieving acceptable performance.

The Problem:

During initial testing, the first image generation took about 110 seconds (expected), but every subsequent generation also took 110+ seconds. Monitoring GPU memory usage revealed the issue:

• After generation completes: VRAM drops from 10-12GB back to 3GB
• Next generation starts: 80 seconds spent reloading models from RAM to VRAM
• Inference runs: 30 seconds of actual generation
• Total: 110 seconds per image, no matter how many you generate

This made the system unusable for practical work - imagine waiting nearly 2 minutes for every single image!

The Root Cause:

Diagnosis revealed that FLUX’s generation code was calling maybe_free_model_hooks() after every inference pass. This function is designed to free memory for systems running multiple models or tight memory scenarios, but in our case where we want to generate multiple images in sequence, it was counterproductive.

The culprit was in src/flux/generate.py:

1
2
3
4
5
6
7
8
9
10

# BEFORE (problematic)
def generate():
    # ... generation code ...
    self.maybe_free_model_hooks()  # ❌ Unloads everything from VRAM!

# AFTER (fixed)
def generate():
    # ... generation code ...
    # DISABLED: Keep models in VRAM between generations
    # self.maybe_free_model_hooks()  # ✅ Models stay loaded

The Impact:

This single line change transformed the performance profile:

Suddenly, generating 10 images went from 18 minutes to just 2 minutes (110s + 9 × 10s). This made the difference between “technically possible but impractical” and “actually usable for real work.”

📂 See the Implementation:

The complete modified FLUX generation code with this optimization is available in the GitHub repository at src/flux/generate.py. You can see exactly how the model loading and generation pipeline is structured, along with all the mmgp integration code.

Real-World Testing Results

Test Configuration

Hardware: RTX 3060 12GB, Intel i7-11700K, 64GB DDR4, Micron NVMe main drive, 7200RPM HDD secondary

OS: Windows 11 + WSL2 (Ubuntu 22.04)

Docker: Desktop 4.28 with NVIDIA Container Toolkit

Model: FLUX.1-schnell + OminiControl subject_512.safetensors

Settings: 8 inference steps, 512x512 resolution

Generation Tests

Test 1: Cold start

1
2
3
4
5
6

Prompt: "A film photography shot. This item is placed on a wooden desk 
         in a cozy study room. Warm afternoon sunlight streams through 
         the window."
Subject: Toy robot figure
Time: 108 seconds
Quality: Excellent, subject preserved with accurate placement

Test 2: Immediate follow-up

1
2
3
4
5

Prompt: "On Christmas evening, on a crowded sidewalk, this item sits 
         covered in snow wearing a Santa hat."
Subject: Same toy robot
Time: 11 seconds
Quality: Excellent, consistent subject representation

Test 3: Third generation

1
2
3
4
5

Prompt: "Underwater photography. This item sits on a coral reef with 
         tropical fish swimming around it."
Subject: Same toy robot
Time: 10 seconds
Quality: Good, some water distortion artifacts (expected)

Resource Monitoring

During active generation:

• GPU Utilization: 95-100%
• VRAM Usage: 10.2GB / 12GB (85%)
• System RAM: 52GB / 64GB (model pinning)
• CPU Usage: 15-20% (mainly data preprocessing)
• Power Draw: 170W (RTX 3060 TDP)

Storage Impact

HDD vs SSD comparison (estimated):

• HDD: 2-3 minutes initial load from disk
• SSD: 30-45 seconds initial load (2.5x faster)
• During generation: No difference (models in RAM)

Recommendation: SSD for faster startup, but not required for generation performance.

Lessons Learned: What Works and What Doesn’t

After extensive testing and iteration, here are the key insights organized by category. These lessons can save you hours of troubleshooting if you’re implementing something similar.

Memory Management Insights

✅ Profile 3 is the Sweet Spot for 12GB Cards

Tested all five mmgp profiles extensively. Profile 3 provides the perfect balance:

• Stable VRAM usage at 85% capacity (10.2GB / 12GB)
• Fast inference times (10s per image)
• No OOM errors or crashes across 100+ test generations

Profiles 1-2 required more VRAM than available, Profiles 4-5 were unnecessarily slow.

✅ RAM Pinning Eliminates the Disk Bottleneck

The 75% RAM allocation strategy (48GB pinned) was crucial:

• First load: 2-3 minutes from HDD to RAM (one-time cost)
• Subsequent loads: <5 seconds from pinned RAM to VRAM
• Models persist across generations with zero disk I/O

Without pinning, every generation would require disk access - absolutely impractical.

⚠️ WSL2 Memory Limits Are Deceiving

Initial attempts with default WSL2 settings failed. The issue:

• Host system: 64GB RAM available
• WSL2 container: Only sees ~31GB (50% default limit)
• mmgp profile calculation: Incorrectly assumes full RAM available

Solution: Explicitly configure .wslconfig to allocate more memory to WSL2, or force mmgp to use perc_reserved_mem_max=0.75 parameter.

❌ Auto-Offloading Strategies Don’t Work Well

Tried mmgp’s offloadAfterEveryCall feature - it caused frequent crashes:

• Unpredictable VRAM usage patterns
• Race conditions between loading/offloading
• No performance benefit over persistent loading

Lesson: For sequential generation workloads, keep models loaded.

Storage and I/O Optimization

📊 HDD vs SSD Impact Analysis

Key Insight: SSD only matters for startup time. Once models are in RAM, storage speed is irrelevant. If you’re doing many generations in one session, HDD is perfectly acceptable.

💾 Volume Mount Strategy Was Critical

Storing models on the host filesystem (-v D:\_Models:/app/models) provided:

• Persistence across container rebuilds
• Ability to share models between different containers
• Easy backup and version management
• No re-downloading during development iterations

Without this, every code change would require re-downloading 35GB of models.

Configuration and Deployment

✅ Gradio Provided Zero-Effort UI

Using Gradio for the web interface was brilliant:

• 20 lines of Python for complete web UI
• Automatic file upload handling
• Built-in image preview and download
• No frontend development required

Alternative approaches (Flask, React frontend) would have taken days vs hours.

✅ Docker Isolated the Complexity

Containerization proved invaluable:

• No conflicts with host Python environment
• Reproducible across machines (tested on 3 different PCs)
• Easy version control of entire stack
• Simple deployment (docker run and done)

❌ Profile 1 Caused Out-of-Memory Errors

Attempted to use Profile 1 (full model in VRAM) for maximum speed:

• Required 16GB+ VRAM
• RTX 3060’s 12GB couldn’t handle it
• Resulted in CUDA OOM errors mid-generation

Lesson: Always profile your actual available memory, not theoretical specs.

Quality and Performance Trade-offs

✅ 8-Bit Quantization Had Minimal Quality Impact

Side-by-side comparison of T5 encoder outputs:

• FP16 (original): Baseline quality
• INT8 (quantized): <5% subjective quality difference
• Memory savings: 8.8GB → 4.4GB (50% reduction)

Conclusion: For text encoding tasks, 8-bit quantization is essentially free VRAM.

📈 Generation Speed Met Targets

The 10-second generation time makes this practical for real creative work.

Production Considerations

For Personal Use

This setup works great for:

• Hobbyist AI experimentation
• Content creation (social media, art projects)
• Proof-of-concept development
• Learning FLUX.1 architecture

For Commercial Use

Consider these factors:

• Generation time: 10s/image × 1000 images = 2.8 hours
• Scalability: Single GPU, no batch processing
• Reliability: Consumer GPU thermal throttling under sustained load
• Support: mmgp is community-maintained, not enterprise-supported

For production workloads, consider:

• Cloud GPUs (Azure N-Series VMs or Azure Container Apps with GPU nodes) - minimum A40/A100
• Local GPU upgrade to RTX 4090 or A6000
• Batch processing optimizations
• Multiple parallel containers

Docker Hub & Repository

The complete implementation is available:

• GitHub: docker-ai-models/omnicontrol
• README: Full setup instructions and troubleshooting
• Dockerfile: Production-ready container definition
• Source code: Custom FLUX integration with mmgp

Quick Start

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

# Clone repository
git clone https://github.com/Ricky-G/docker-ai-models.git
cd docker-ai-models/omnicontrol

# Build container
docker build -t omnicontrol .

# Run with HuggingFace token
docker run -d --gpus all --name omnicontrol \
  -p 7860:7860 \
  -v D:\_Models\omnicontrol:/app/models \
  -e HF_TOKEN=your_token_here \
  omnicontrol

# Access web interface
# http://localhost:7860

Conclusion

Running FLUX.1 OmniControl on a 12GB RTX 3060 is not only possible but practical. Through careful memory management with mmgp, strategic quantization, and container optimization, we achieved:

• ✅ 10-second generation times (after initial load)
• ✅ Stable VRAM usage across multiple generations
• ✅ No quality degradation from quantization
• ✅ Reproducible Docker deployment

The key insight: Memory management is more important than raw VRAM capacity. With the right tools and configuration, consumer GPUs can run models designed for datacenter hardware.

If you have an RTX 3060 (or similar 12GB card) collecting dust because you thought it couldn’t handle modern AI models, give this approach a try. The democratization of AI isn’t just about open-source models - it’s about making them runnable on hardware people actually own.

Hardware tested: RTX 3060 12GB, 64GB RAM, Windows 11 + WSL2

Software stack: Docker Desktop, NVIDIA Container Toolkit, mmgp 3.6.9

Model: FLUX.1-schnell + OminiControl LoRA

Performance: 10s per 512x512 image (8 steps)

References:

• Memory optimization via mmgp (Memory Management for GPU Poor)
• FLUX.1-schnell Model
• OminiControl LoRA
• Docker Implementation Repository

Image Credits:

• Main image generated by GPT-Image-1.5

🎯 TL;DR: Deploy Microsoft Foundry Cross-Region with Private Endpoints

Microsoft Foundry isn’t available in every Azure region, but data residency requirements often mandate that all data at rest stays within specific regions. This post demonstrates how to keep your data in your compliant region (e.g., New Zealand North) while leveraging Microsoft Foundry in another region (e.g., Australia East) purely for AI inferencing. Using cross-region Private Endpoints over Azure’s backbone network, applications securely access Foundry’s AI capabilities without data traversing the public internet—maintaining both regional compliance and zero-trust security posture.

The Solution: All data at rest, applications, and Private Endpoints remain in NZN. Microsoft Foundry deployed in AUE provides AI inferencing only. Private connectivity ensures secure, compliant architecture across regions.

When deploying Microsoft Foundry (formerly Azure AI Foundry) in enterprise environments, you’ll face a critical constraint: Microsoft Foundry isn’t available in every Azure region, yet data residency requirements mandate that all data at rest remains within specific regions.

Imagine this scenario: Your organization must keep all data in New Zealand North due to regulatory compliance, but Microsoft Foundry is only available in Australia East. You can’t move data to AUE, but you need Foundry’s AI capabilities. How do you maintain compliance while accessing AI inferencing services?

The solution is architectural: Keep all data at rest in your compliant region (NZN) and use Microsoft Foundry in the available region (AUE) purely for AI inferencing. By deploying cross-region Private Endpoints, applications in NZN securely access Foundry’s AI services over Azure’s backbone network—no public internet, no data residency violations, no compromises.

This guide walks through the complete architecture, DNS configuration, security considerations, and implementation steps for deploying this cross-region private endpoint pattern.

⚠️ Important: Foundry Agents Service Limitation

If you plan to use the Foundry Agents service specifically, there is a known limitation at the time of writing: all Foundry workspace resources (Cosmos DB, Storage Account, AI Search, Foundry Account, Project, Managed Identity, Azure OpenAI, or other Foundry resources used for model deployments) must be deployed in the same region as the VNet.

This means the cross-region pattern described in this post will not work for Foundry Agents deployments—you would need to deploy everything in the same region (e.g., all resources in Australia East where Foundry is available).

However, if you are NOT using the Foundry Agents service (i.e., you’re only using Foundry for AI inferencing via API calls—OpenAI models, Speech Services, Vision, etc.), then the cross-region private endpoint pattern works perfectly, and all your data can reside in your chosen compliant region as described in this post.

For more details, see Microsoft Learn - Virtual Networks with Foundry Agents - Known Limitations

flowchart TB    subgraph azure["☁️ Azure Backbone"]        direction TB        subgraph NZN["🌏 NZN - Data Residency Region"]            direction TB            subgraph vnet["VNet:  10.1.0.0/16"]                subgraph appsnet["Subnet: snet-apps • 10.1.1.0/24"]                    client[👤 Client App / VM
10.1.1.10]                    data[(💾 Data at Rest
Storage, SQL, etc.)]                end                subgraph pesnet["Subnet: snet • 10.1.2.0/24"]                    pe[🔒 Private Endpoint
10.1.2.4]                end            end            dns[🔐 Private DNS Zones
Resolves to Private IP]        end                subgraph AUE["🌏 AUE - AI Inferencing"]            foundry[[🤖 Microsoft Foundry
myFoundry. cognitiveservices.azure.com]]        end                pe ==>|"🔐 Private Link
"| foundry    end        internet[/"🌐 Public Internet
❌ Blocked"/]        client --> dns    dns -.->|10.1.2.4| pe    client -->|HTTPS| pe        foundry -.-x internet        style azure fill:#f5f5f5,stroke:#666,stroke-width:2px,stroke-dasharray: 5 5    style NZN fill:#e3f2fd,stroke:#1976d2,stroke-width:3px    style AUE fill:#e8f5e9,stroke:#388e3c,stroke-width:3px    style internet fill:#ffebee,stroke:#c62828,stroke-width:2px    style vnet fill:#e1f5fe,stroke:#0288d1,stroke-width: 2px    style dns fill:#f3e5f5,stroke:#7b1fa2,stroke-width:2px    style pe fill:#fff3e0,stroke:#ef6c00,stroke-width:3px    style data fill:#e8f5e9,stroke:#388e3c,stroke-width:2px

Understanding Microsoft Foundry

Microsoft Foundry is a unified platform-as-a-service for enterprise AI that brings together agents, models (including Azure OpenAI, Speech, Vision), and AI development tools under one managed resource. Think of it as a comprehensive AI studio environment where developers can build, evaluate, and deploy generative AI applications without the complexity of stitching together multiple Azure services manually.

Why Foundry Matters for Enterprise AI

Foundry provides several enterprise-grade capabilities out of the box:

Unified AI Platform: Instead of managing separate Azure OpenAI, Cognitive Services, and ML resources, Foundry consolidates these into a single managed environment with consistent APIs and management experiences.

Enterprise Security & Compliance: Built-in support for managed identities, RBAC, audit logging, and network isolation ensures your AI applications meet corporate security requirements.

Observability & Monitoring: Integrated tracing, logging, and monitoring capabilities help you understand how your AI applications are performing and troubleshoot issues quickly.

Development Productivity: A unified development experience through Azure AI Studio portal and SDKs accelerates AI application development by reducing integration overhead.

Because Foundry encapsulates many Azure AI services and provides such comprehensive capabilities, it has specific networking requirements when you want to integrate it into a secure, enterprise cloud environment. This is where Private Endpoints become critical.

Azure Private Endpoints: The Foundation of Secure Connectivity

An Azure Private Endpoint is essentially a network interface (NIC) with a private IP address from your Azure Virtual Network that connects privately to a supported Azure service. When you create a private endpoint for a service like Foundry, you’re bringing that service into your VNet’s address space, allowing VNet resources to reach the service via an internal IP address instead of traversing the public internet.

How Private Link Works Under the Hood

Azure Private Link is the technology that makes Private Endpoints possible. Here’s what happens when you enable private connectivity:

1. Private IP Assignment: A network interface is created in your VNet subnet with a private IP (e.g., 10.1.2.4)
2. DNS Resolution: The service’s FQDN (e.g., myFoundry.cognitiveservices.azure.com) is configured to resolve to the private IP instead of the public endpoint
3. Backbone Network Transit: Traffic between your VNet and the service travels entirely on Microsoft’s backbone network, never touching public internet routes
4. Regional Flexibility: The private endpoint can be in a different region than the target service, enabling cross-region private connectivity

Key Benefits of Private Endpoints

Enhanced Security: Traffic never leaves Azure’s internal network, eliminating internet-based attack vectors and data exfiltration risks.

Supported Across Azure PaaS: Many Azure services support private endpoints including Storage accounts, SQL databases, Cosmos DB, Cognitive Services (which Foundry uses), Key Vault, and more.

Cross-Region Capability: Critically, Azure allows the private endpoint’s NIC to reside in a different region than the service’s region. The private endpoint must be in the same region as the VNet you place it in, but the target resource can be in any Azure region. This is the key feature that enables our solution.

DNS Integration: Azure Private DNS Zones provide seamless name resolution, making private endpoint connectivity transparent to applications.

DNS Configuration Requirements

Using a private endpoint requires configuring DNS so that the service’s FQDN resolves to the private IP instead of the public IP. For Azure Cognitive Services (which Foundry is built on), this typically involves:

1. Creating a Private DNS Zone: privatelink.cognitiveservices.azure.com
2. Linking the zone to your VNets
3. Adding an A record mapping the Foundry hostname to the private endpoint IP

Without proper DNS configuration, clients will continue resolving the public IP address and bypass the private link entirely, defeating the purpose of the security setup.

The Cross-Region Challenge: When Foundry Isn’t Available Locally

Understanding the Regional Availability Problem

Microsoft Foundry is not yet available in every Azure region. At the time of writing, Foundry has limited regional availability as Microsoft continues rolling out the service globally. This creates a common scenario for many organizations:

The Scenario: Let’s imagine an organization with Azure infrastructure primarily deployed in New Zealand North (NZN) – perhaps due to data residency requirements, latency optimization for local users, or established governance policies. However, Microsoft Foundry is only available in Australia East (AUE), with New Zealand North not yet on the supported regions list.

The Challenge: The organization needs to leverage Foundry’s powerful AI capabilities while maintaining private network connectivity and keeping all traffic within the Azure environment. Simply deploying everything in Australia East isn’t feasible due to data residency requirements etc. So all data at rest needs to be in New Zealand North region. Microsoft Foundry will be used for AI inferencing only and all data at rest must remain in New Zealand North.

Why Standard Approaches Don’t Work

Let’s consider the typical alternatives and why they fall short:

Deploy Everything in AUE: This requires relocating or duplicating existing infrastructure, increasing costs and complexity. Data residency requirements may prohibit this approach entirely.

Use Public Endpoints: Exposing Foundry via public endpoints contradicts enterprise zero-trust security policies and increases attack surface unnecessarily.

Accept Regional Limitations: Waiting for Foundry to become available in your region delays AI initiatives and competitive advantages.

The Solution: Cross-Region Private Endpoints

The solution leverages Azure’s support for cross-region Private Endpoints. By deploying a Private Endpoint in your New Zealand North VNet that connects to the Foundry service in Australia East, you achieve several critical outcomes:

• Private Connectivity: All traffic between NZN and AUE travels on Azure’s backbone network
• Regional Compliance: Resources remain in their designated regions with private interconnection
• Transparent Access: Applications in NZN access Foundry as if it’s a local service via private IP
• Security Posture: Foundry’s public endpoints can be disabled, enforcing private-only access

Architecture Deep Dive: Cross-Region Foundry Deployment

High-Level Architecture Overview

The architecture deploys Microsoft Foundry in Australia East while enabling secure access from resources in New Zealand North. Here’s how the components fit together:

Foundry Service (Australia East): The actual Microsoft Foundry resource deployed in AUE, configured as a Cognitive Services account with AI capabilities enabled.

Private Endpoint (New Zealand North): A network interface in your NZN VNet with a private IP address (e.g., 10.1.2.4) that acts as the gateway to the AUE Foundry service.

Private DNS Zone: A DNS zone (privatelink.cognitiveservices.azure.com) linked to your VNets that resolves the Foundry FQDN to the private endpoint IP.

Client Applications (New Zealand North): VMs, App Services, containers, or other resources in NZN that consume the Foundry APIs.

sequenceDiagram    participant C as 👤 Client
10.1.1.10    participant D as 🔐 Private DNS    participant P as 🔒 Private EP
10.1.2.4    participant B as 🌐 Public Internet    participant F as 🤖 Foundry
(AUE)        rect rgb(227, 242, 253)        Note over C,D:  New Zealand North Region    end    rect rgb(232, 245, 232)        Note over F: Australia East Region    end        C->>D: 1. DNS Query:  myFoundry.cognitiveservices. azure.com    D->>C:  2. Returns Private IP: 10.1.2.4        Note over C,P: Traffic stays on Azure backbone    C->>P: 3. HTTPS Request to 10.1.2.4:443    P->>F: 4. Azure Private Link (backbone)    F->>P: 5. AI Response    P->>C: 6. Response delivered        rect rgb(255, 235, 238)        Note over B:  Public path blocked        C--xB: ❌ No public internet route        B--xF: ❌ Public access disabled    end

Traffic Flow Explanation

Let’s trace what happens when a client application in NZN makes a request to Foundry:

1. Application Request: The application initiates a connection to myFoundry.cognitiveservices.azure.com
2. DNS Resolution: The VNet’s DNS configuration queries the Private DNS Zone
3. Private IP Return: DNS returns 10.1.2.4 (the private endpoint IP) instead of the public IP
4. Private Endpoint Connection: The application connects to the private endpoint in NZN
5. Cross-Region Transit: Azure Private Link routes the traffic across the backbone network to AUE
6. Foundry Processing: The Foundry service in AUE receives and processes the request
7. Response Path: The response follows the same path back through the private endpoint

From the application’s perspective, it’s communicating with a local service in NZN. All the cross-region complexity is abstracted by Azure’s networking layer.

Critical Architectural Considerations

Azure Backbone Network Transit

The connection between NZN and AUE is entirely internal to Azure. No traffic exits to the public internet, even though the regions are geographically separated across the Tasman Sea. Azure’s global backbone network handles the routing, ensuring:

• Security: Traffic never traverses public networks
• Reliability: Azure’s backbone offers higher SLAs than internet routes
• Performance: Optimized routing between Azure datacenters

Private DNS Resolution Requirements

DNS configuration is the linchpin that makes private endpoints work correctly. Microsoft Foundry (AIServices kind) requires multiple Private DNS zones for full functionality:

Required Private DNS Zones:

1. privatelink.cognitiveservices.azure.com - Primary Cognitive Services endpoint
2. privatelink.openai.azure.com - Azure OpenAI service endpoints
3. privatelink.services.ai.azure.com - AI Foundry management endpoints

Configuration Steps:

DNS Zone Creation: Create all required Private DNS zones in your subscription

VNet Linking: Link the DNS zones to your NZN VNet (and any other VNets that need access)

A Record Mapping: Azure automatically creates the necessary A records when using DNS zone groups with private endpoints, or you can manually add A records for your Foundry resource pointing to the private endpoint IP

Multi-VNet Scenarios: If you have separate VNets (e.g., hub-and-spoke topology), ensure all VNets are linked to the DNS zones or use DNS forwarding

Without proper DNS configuration across all zones, clients will resolve public IPs and bypass your private endpoint entirely, rendering the security setup ineffective.

Performance and Latency Implications

While traffic stays private, the cross-region architecture introduces additional latency:

Typical Latency: NZN to AUE is a trans-Tasman hop, usually adding 30-60ms compared to in-region calls

Acceptable Use Cases: Most AI API calls (text generation, embeddings, etc.) can tolerate this latency

Latency-Sensitive Workloads: Real-time voice applications or ultra-low-latency requirements may need careful evaluation

Bandwidth Charges: Azure charges for cross-region data transfer. Egress from AUE to NZN incurs bandwidth costs. Budget accordingly for high-volume scenarios.

Security Posture Enhancement

Private Endpoints enable several security improvements:

Disable Public Access: Configure Foundry to reject all public network connections, forcing traffic through approved private endpoints

Network Security Groups (NSGs): Apply NSGs to the private endpoint subnet to control which source IPs can reach the Foundry service

Azure Firewall Integration: Route private endpoint traffic through Azure Firewall for additional inspection and logging

On-Premises Connectivity: If you have ExpressRoute or VPN connecting on-premises to Azure, those networks can also access Foundry privately through the NZN VNet

Step-by-Step Implementation Overview

⚠️ Important: Choose Your Own Unique Name

The name myFoundryDemo used throughout these examples is already taken in Azure’s global namespace. Cognitive Services accounts require globally unique names across all Azure subscriptions worldwide.

Before running any commands below: Replace myFoundryDemo with your own unique name (e.g., myFoundryProd2025, contoso-ai-foundry, etc.) in all the Azure CLI commands. This name will become part of your endpoint URL: <your-unique-name>.cognitiveservices.azure.com

The following sections walk through the high-level steps to implement this architecture. Part 2 will provide detailed scripts and Infrastructure-as-Code templates.

Step 1: Deploy Foundry in Australia East

Start by creating your Microsoft Foundry resource in the Australia East region:

1
2
3
4
5
6
7
8
9
10
11

# Create resource group in Australia East
az group create --name rg-foundry-aue --location australiaeast

# Create Foundry resource (Cognitive Services account)
az cognitiveservices account create \
  --name myFoundryDemo \
  --resource-group rg-foundry-aue \
  --kind AIServices \
  --sku S0 \
  --location australiaeast \
  --custom-domain myFoundryDemo

Important Notes:

• Initially allow public network access during setup
• Note the resource name and ID for the next steps
• Foundry appears as a Cognitive Services account of kind AIServices

Step 2: Create Virtual Network in New Zealand North

Set up a VNet in NZN where the private endpoint will reside:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

# Create resource group in New Zealand North
az group create --name rg-networking-nzn --location newzealandnorth

# Create virtual network
az network vnet create \
  --name vnet-nzn-main \
  --resource-group rg-networking-nzn \
  --location newzealandnorth \
  --address-prefix 10.1.0.0/16 \
  --subnet-name snet-apps \
  --subnet-prefix 10.1.1.0/24

# Create subnet for private endpoints
az network vnet subnet create \
  --name snet-private-endpoints \
  --resource-group rg-networking-nzn \
  --vnet-name vnet-nzn-main \
  --address-prefix 10.1.2.0/24 \
  --disable-private-endpoint-network-policies true

Network Planning Considerations:

• Ensure address space doesn’t conflict with other VNets you’ll peer
• Create a dedicated subnet for private endpoints
• Disable network policies on the private endpoint subnet

Step 3: Create the Cross-Region Private Endpoint

Now create the private endpoint in NZN that connects to the AUE Foundry resource:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

# Get Foundry resource ID
FOUNDRY_ID=$(az cognitiveservices account show \
  --name myFoundryDemo \
  --resource-group rg-foundry-aue \
  --query id -o tsv)

# Create private endpoint
az network private-endpoint create \
  --name pe-foundry-nzn \
  --resource-group rg-networking-nzn \
  --location newzealandnorth \
  --vnet-name vnet-nzn-main \
  --subnet snet-private-endpoints \
  --private-connection-resource-id $FOUNDRY_ID \
  --group-id account \
  --connection-name foundry-connection

Key Parameters:

• --location must match the VNet’s region (newzealandnorth)
• --private-connection-resource-id points to the AUE Foundry resource
• --group-id account specifies the Cognitive Services sub-resource type
• Connection auto-approves since you own both resources

Step 4: Configure Private DNS Integration

Set up DNS to resolve the Foundry FQDN to the private endpoint IP. Microsoft Foundry requires multiple Private DNS zones:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

# Create all required Private DNS Zones for Microsoft Foundry
az network private-dns zone create \
  --resource-group rg-networking-nzn \
  --name privatelink.cognitiveservices.azure.com

az network private-dns zone create \
  --resource-group rg-networking-nzn \
  --name privatelink.openai.azure.com

az network private-dns zone create \
  --resource-group rg-networking-nzn \
  --name privatelink.services.ai.azure.com

# Link DNS zones to VNet
for zone in "privatelink.cognitiveservices.azure.com" "privatelink.openai.azure.com" "privatelink.services.ai.azure.com"
do
  az network private-dns link vnet create \
    --resource-group rg-networking-nzn \
    --zone-name $zone \
    --name "link-vnet-nzn-main-${zone%%.azure.com}" \
    --virtual-network vnet-nzn-main \
    --registration-enabled false
done

# Alternative: Use DNS Zone Group for automatic configuration
# This automatically creates A records in all required zones
az network private-endpoint dns-zone-group create \
  --resource-group rg-networking-nzn \
  --endpoint-name pe-foundry-nzn \
  --name default \
  --private-dns-zone privatelink.cognitiveservices.azure.com \
  --zone-name cognitiveservices

az network private-endpoint dns-zone-group add \
  --resource-group rg-networking-nzn \
  --endpoint-name pe-foundry-nzn \
  --name default \
  --private-dns-zone privatelink.openai.azure.com \
  --zone-name openai

az network private-endpoint dns-zone-group add \
  --resource-group rg-networking-nzn \
  --endpoint-name pe-foundry-nzn \
  --name default \
  --private-dns-zone privatelink.services.ai.azure.com \
  --zone-name aiservices

DNS Configuration Options:

Option 1 - DNS Zone Groups (Recommended): Using DNS zone groups (shown above) automatically creates and maintains A records in all zones. This is the preferred approach as Azure manages the DNS records lifecycle.

Option 2 - Manual A Records: If you need manual control, get the private endpoint IP and create A records manually:

1
2
3
4
5
6
7
8
9
10
11
12

# Get private endpoint IP
PE_IP=$(az network private-endpoint show \
  --name pe-foundry-nzn \
  --resource-group rg-networking-nzn \
  --query 'customDnsConfigs[0].ipAddresses[0]' -o tsv)

# Create DNS A record in the primary zone
az network private-dns record-set a add-record \
  --resource-group rg-networking-nzn \
  --zone-name privatelink.cognitiveservices.azure.com \
  --record-set-name myFoundryDemo \
  --ipv4-address $PE_IP

DNS Verification:

• All DNS zones must be created and linked to VNets needing access
• DNS zone groups automatically handle A record creation and updates
• For manual records, ensure the record name matches your Foundry resource name
• Multi-VNet deployments require linking all zones to each VNet

Step 5: Test Private Connectivity

Deploy a test VM in the NZN VNet and verify connectivity:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

# Create test VM
az vm create \
  --resource-group rg-networking-nzn \
  --name vm-test-nzn \
  --location newzealandnorth \
  --vnet-name vnet-nzn-main \
  --subnet snet-apps \
  --image Ubuntu2204 \
  --admin-username azureuser \
  --generate-ssh-keys

# SSH to VM and test DNS resolution
ssh azureuser@<vm-public-ip>

# Verify DNS resolves to private IP
nslookup myFoundryDemo.cognitiveservices.azure.com

Expected DNS Output:

1
2
3
4
5
6

Server:  10.1.0.4
Address: 10.1.0.4#53

Name:    myFoundryDemo.privatelink.cognitiveservices.azure.com
Address: 10.1.2.4
Aliases: myFoundryDemo.cognitiveservices.azure.com

The resolution to 10.1.2.4 (private IP) confirms DNS is working correctly.

Test API Connectivity:

1
2
3
4
5
6

# Get Foundry API key (from Azure portal or CLI)
API_KEY="your-foundry-api-key"

# Test API call
curl -X GET "https://myFoundryDemo.cognitiveservices.azure.com/openai/deployments?api-version=2024-02-01" \
  -H "api-key: $API_KEY"

If you receive a valid response (list of deployments or empty array), private connectivity is working. If you disabled public access on Foundry, this same curl command from your local machine should fail with a network error.

Step 6: Lock Down Public Access

With private connectivity confirmed, enhance security by disabling public access:

1
2
3
4
5

# Disable public network access
az cognitiveservices account update \
  --name myFoundryDemo \
  --resource-group rg-foundry-aue \
  --public-network-access Disabled

Additional Security Hardening:

• Apply NSGs to the private endpoint subnet restricting source IPs
• Enable Azure Firewall for additional traffic inspection
• Configure diagnostic logs to monitor access patterns
• Use managed identities instead of API keys for authentication

Real-World Considerations and Trade-Offs

Cost Implications

Cross-Region Data Transfer: Azure charges for data egress between regions. For NZN ↔ AUE, expect approximately $0.02-0.05 per GB transferred (approx pricing at the time of this writing). Monitor your Foundry usage patterns and budget accordingly.

Private Endpoint Costs: Private endpoints incur a small hourly charge plus data processing charges per GB.

Foundry Service Costs: The Foundry service itself has per-transaction or per-token costs depending on the AI services used (OpenAI, Speech, etc.).

Latency Considerations

Baseline Latency: Expect 30-60ms additional latency for cross-region calls compared to in-region

Impact Assessment: For most AI workloads (text generation, embeddings, document analysis), this latency is acceptable

Optimization Strategies:

• Use async/await patterns to parallelize independent API calls
• Implement request batching where applicable
• Cache frequently-requested results
• Consider regional caching layers for static content

On-Premises Integration

If your organization has on-premises datacenters connected to Azure via ExpressRoute or Site-to-Site VPN, you can extend private connectivity to those environments:

DNS Forwarding

Configure on-premises DNS servers to forward queries for *.cognitiveservices.azure.com to Azure’s DNS resolvers:

1
2
3
4
5

# Example BIND configuration
zone "cognitiveservices.azure.com" {
    type forward;
    forwarders { 10.1.0.4; };  # Azure VNet DNS server
};

Network Connectivity

Ensure your on-premises network has routes to the NZN VNet where the private endpoint resides:

• ExpressRoute: Private peering enables private IP connectivity
• Site-to-Site VPN: VPN gateway provides encrypted connectivity
• Route Tables: Verify routes exist for the private endpoint subnet (10.1.2.0/24)

With proper DNS and routing configuration, on-premises applications can access Foundry privately through the Azure backbone network.

Cleanup Resources

If you’re finished testing and want to remove all resources created in this guide, you can delete the resource groups:

1
2
3
4
5
6
7
8
9
10
11
12
13

# Delete the networking resource group in New Zealand North
# This removes the VNet, subnets, private endpoint, DNS zones, and test VM
az group delete \
  --name rg-networking-nzn \
  --yes \
  --no-wait

# Delete the Foundry resource group in Australia East
# This removes the Microsoft Foundry (Cognitive Services) account
az group delete \
  --name rg-foundry-aue \
  --yes \
  --no-wait

Important Notes:

• The --yes flag skips confirmation prompts
• The --no-wait flag allows the command to return immediately without waiting for deletion to complete
• Resource group deletion is permanent and cannot be undone
• Deletion can take several minutes; you can check status in the Azure portal or with az group show

Wrapping Up Part 1

This architecture solves a critical challenge: Microsoft Foundry isn’t available in all Azure regions, but data residency requirements often mandate that data at rest remains within specific regions.

The solution demonstrated here keeps all data at rest in New Zealand North (your region of choice), while leveraging Microsoft Foundry in Australia East purely for AI inferencing capabilities. By deploying Private Endpoints in the NZN region, applications access Foundry’s AI services securely over Azure’s backbone network without any data ever traversing the public internet.

Key Takeaway: Data stays in your compliant region (NZN), AI inferencing happens in the Foundry region (AUE), and all communication flows privately through Azure’s backbone. This pattern works for any region pair where Foundry availability doesn’t align with your data residency requirements.

In Part 2, we’ll look at Infrastructure-as-Code templates using Bicep and Terraform to automate this deployment.

References

• Microsoft Docs – What is Azure AI Foundry?
• Microsoft Docs – What is a private endpoint?
• Azure Architecture Center – Baseline Azure AI Foundry Architecture
• Microsoft Q&A – VNet-Integrated AI Deployment
• Main image generated by GPT-Image-1.5

🎯 TL;DR: Automated Oh My Posh Terminal Setup for Cloud Native Development

Every new machine or fresh Windows install means reconfiguring your terminal environment from scratch. Problem: Manually setting up Oh My Posh, installing Nerd Fonts, and configuring custom themes is tedious and error-prone across multiple machines.

Solution: (A single PowerShell script available on GitHub https://github.com/Ricky-G/script-library/blob/main/pimp-my-terminal.ps1) that automates the entire process - installing Oh My Posh via winget, deploying a Nerd Font, Terminal-Icons module, creating a custom “Cloud Native Azure” theme optimized for Kubernetes and Azure workflows, and configuring your PowerShell profile with PSReadLine enhancements.

Prerequisites: Enable script execution with Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser before running. This approach transforms the multi-hour setup process into a one-command operation, providing immediate visual context for Git branches, Kubernetes clusters, Azure subscriptions, and command execution times - critical information for modern cloud native development.

Recently, I found myself setting up yet another development machine, and as I stared at the blank PowerShell terminal, I realized I’d reached my limit with manual terminal configuration. Every new machine or clean install meant the same tedious process: download Oh My Posh, find a Nerd Font installer, copy configuration files, edit PowerShell profiles, and spend 30 minutes getting everything just right.

The frustration wasn’t just about aesthetics - a properly configured terminal is a productivity multiplier. When you’re constantly switching between multiple Git repositories, Kubernetes clusters, and Azure subscriptions throughout the day, having that contextual information immediately visible saves countless keystrokes and eliminates mental overhead.

This blog post shares my automated solution: a single PowerShell script that takes a bare Windows terminal and transforms it into a fully-configured, cloud native-ready development environment in under 5 minutes. Whether you’re setting up a new machine, rebuilding after a Windows update disaster, or just want to standardize terminal configuration across your team, this automation eliminates the manual work.

Quick Start - Get Up and Running in 5 Minutes

Want to skip the details and just get started? Here’s everything you need to run the automation script:

Step 1: Enable Script Execution

Open PowerShell as Administrator and run:

1

Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser

When prompted, type Y and press Enter.

Step 2: Download and Run the Script

1
2
3

# Download and run the automation script
Invoke-WebRequest -Uri "https://raw.githubusercontent.com/Ricky-G/script-library/main/pimp-my-terminal.ps1" -OutFile "$env:TEMP\pimp-my-terminal.ps1"
& "$env:TEMP\pimp-my-terminal.ps1"

The script will automatically install:

• ✅ Oh My Posh via winget
• ✅ MesloLGM Nerd Font
• ✅ Terminal-Icons PowerShell module
• ✅ Cloud Native Azure theme
• ✅ PSReadLine enhancements
• ✅ Custom keyboard shortcuts

Step 3: Configure Your Terminal Font

After the script completes, configure your terminal font:

Windows Terminal:

1. Open Settings (Ctrl + ,)
2. Go to Profiles → Defaults → Appearance
3. Set Font face to: MesloLGM Nerd Font
4. Save and restart terminal

VS Code:

1. Open Settings (Ctrl + ,)
2. Search for “terminal font”
3. Set Terminal › Integrated: Font Family to: MesloLGM Nerd Font

Done! Open a new terminal and enjoy your beautiful, cloud native-ready prompt.

Understanding Oh My Posh: The Modern Prompt Engine

Before diving into the automation, it’s worth understanding what Oh My Posh brings to the table and why it’s become the de facto standard for PowerShell prompt customization.

What Is Oh My Posh?

Oh My Posh is a custom prompt theme engine that works across multiple shells including PowerShell, Bash, Zsh, Fish, and more. Originally inspired by the popular Oh My Zsh project for Linux/macOS, Oh My Posh brings the same level of customization and visual polish to Windows terminals while maintaining cross-platform compatibility.

The key differentiator of Oh My Posh is its segment-based architecture. Rather than being a monolithic prompt generator, it provides a framework for composing different “segments” of information into your prompt. Each segment represents a different piece of contextual information:

Version Control Segments: Display Git branch names, ahead/behind status, working directory changes, stash counts, and more. Supports multiple version control systems.

Cloud Context Segments: Show your active Kubernetes cluster and namespace, AWS profile, Azure subscription, or Google Cloud project. Essential for multi-cloud development.

Development Environment Segments: Display programming language versions (Python, Node.js, Go, .NET), virtual environment status, package manager context, and more.

System Information Segments: Show current directory, execution time of the last command, error status, battery level, and system load.

The Nerd Fonts Requirement

One aspect of Oh My Posh that initially confuses newcomers is the requirement for Nerd Fonts. Standard fonts don’t include the special glyphs and icons that Oh My Posh uses to display information compactly and beautifully.

Nerd Fonts are patches of popular programming fonts that add thousands of additional glyphs from various icon sets including:

• Font Awesome icons
• Devicons for programming languages
• Octicons from GitHub
• Material Design icons
• Weather icons
• Powerline extra symbols

Without a Nerd Font installed, you’ll see blank squares or question marks instead of the beautiful icons that make Oh My Posh themes shine. This is why the installation script specifically handles font installation as a critical step.

The Solution: Automated Setup Script

The core idea behind this automation is simple: replicate exactly what you’d do manually, but in a repeatable, version-controlled script that can be run on any Windows machine. The script handles five critical steps in sequence.

What The Script Installs and Configures

The PowerShell script automates the installation and configuration of several components:

PowerShell Profile Enhancements

The script creates a comprehensive PowerShell profile ($PROFILE) that loads automatically every time you open a terminal:

PSReadLine: Enhanced Command-Line Editing

PSReadLine is a PowerShell module that dramatically improves command-line editing. The script configures these productivity features:

These features work together to minimize typing and maximize efficiency. For example, if you previously ran kubectl get pods -n production, you can start typing kub and press ↑ to find it immediately, or wait for the inline prediction to appear and press → to accept it.

Exploring Oh My Posh Themes

Before we dive into my custom theme, it’s worth exploring the extensive theme library that Oh My Posh provides out of the box. The project includes over 200 pre-built themes ranging from minimal single-line prompts to elaborate multi-line displays with extensive system information.

Browsing the Theme Gallery

Oh My Posh provides an excellent visual gallery where you can see screenshots of all available themes:

👉 https://ohmyposh.dev/docs/themes

The gallery is searchable and filterable, making it easy to find themes that match your preferences:

Minimal Themes: Clean, single-line prompts with just essential information (agnoster, paradox, clean-detailed)

Powerline Themes: Classic powerline-style prompts with angled segments and rich colors (powerlevel10k_rainbow, hotstick.minimal, jandedobbeleer)

Cloud-Native Themes: Themes specifically designed for cloud development with Kubernetes and Azure context (cloud-native-azure, night-owl, atomic)

Language-Specific Themes: Optimized for specific programming languages or frameworks (kushal, amro, lambda)

Fun and Whimsical: Themes with unique character or personality (di4am0nd, emodipt-extend, iterm2)

Testing Themes Before Committing

Oh My Posh makes it easy to preview themes before making them permanent:

1
2
3
4
5

# Preview a specific theme temporarily
oh-my-posh init pwsh --config "$env:POSH_THEMES_PATH\jandedobbeleer.omp.json" | Invoke-Expression

# View all available themes with Get-PoshThemes
Get-PoshThemes

This is particularly useful when setting up a new machine - you can quickly cycle through themes to find one that resonates with your workflow before committing to a permanent configuration.

The Cloud Native Azure Theme: Design Philosophy

The script uses the official Cloud Native Azure theme from Oh My Posh. It’s specifically designed for developers working in the Azure and Kubernetes ecosystem.

What The Theme Displays

At a glance, your prompt shows comprehensive contextual information:

Theme Color Scheme

The theme uses carefully chosen colors for instant visual recognition:

While Oh My Posh ships with many excellent themes, this one is specifically optimized for cloud native development workflows. The design philosophy centers on three core principles:

Priority Information First

The most critical contextual information - path, Git status, Kubernetes context, and Azure subscription - is always visible without requiring any additional commands. This eliminates the “where am I?” moment that costs seconds of mental processing dozens of times per day.

Visual Hierarchy Through Color

The theme uses the official Azure brand colors strategically:

• Azure Blue (#0078D4) for operating system and Azure context
• Cyan (#00A4EF) for file path navigation
• Green (#7FBA00) for Git status with dynamic background colors indicating repository state
• Kubernetes Blue (#326CE5) for cluster context
• Gray (#505050) for secondary information like execution time

This color-coding creates instant visual recognition - your eyes learn to find specific information based on color alone.

Cloud Native Workflow Optimization

The theme specifically addresses common scenarios in cloud native development:

Multi-Cluster Scenarios: When working with development, staging, and production Kubernetes clusters, the prominent cluster name prevents accidentally running destructive commands in production.

Multi-Subscription Development: Azure developers often switch between client subscriptions, personal subscriptions, or different environments. The Azure segment shows which subscription is active to prevent resource creation in the wrong tenant.

Git-Heavy Workflows: With branch name, ahead/behind indicators, working directory changes, staging status, and stash count all visible, you have complete Git situational awareness without running git status.

Theme Anatomy: Understanding the Configuration

The Cloud Native Azure theme is defined as a JSON configuration file that Oh My Posh parses to render your prompt. Understanding this structure allows you to customize the theme to your specific needs.

Block Structure

The theme uses Oh My Posh’s block system to organize segments into logical groupings:

Block 1 (Left-Aligned): Contains the primary context segments that appear on the main prompt line:

• Operating System indicator
• Current path with folder-style display
• Git repository information with status

Block 2 (Right-Aligned): Displays cloud and infrastructure context:

• Kubernetes cluster and namespace
• Azure subscription name
• Command execution time

Block 3 (Left-Aligned, New Line): Simple prompt character for command entry

This three-block structure creates a balanced layout where essential information doesn’t crowd the area where you’re typing commands, while cloud context is visible but not intrusive.

Segment Configuration Details

Each segment in the theme has specific configuration properties that control its behavior and appearance:

1
2
3
4
5
6
7
8
9
10
11
12

{
  "type": "kubectl",
  "style": "powerline",
  "powerline_symbol": "\ue0b2",
  "invert_powerline": true,
  "foreground": "#ffffff",
  "background": "#326CE5",
  "template": " \ufd31 {{ .Context }}{{ if .Namespace }} :: {{ .Namespace }}{{ end }} ",
  "properties": {
    "parse_kubeconfig": true
  }
}

Breaking down this Kubernetes segment:

Type Property: Specifies which Oh My Posh segment provider to use (kubectl for Kubernetes context)

Style and Powerline Symbol: Creates the angled transitions between segments that give the prompt its distinctive look

Color Configuration: Foreground and background colors using hex codes for precise brand matching

Template String: Defines what information to display using Go template syntax with conditional logic

Properties: Segment-specific settings like parse_kubeconfig which tells the segment to read from your kubectl config file

Dynamic Git Status Indicators

The Git segment includes sophisticated logic for changing appearance based on repository state:

1
2
3
4
5
6

"background_templates": [
  "{{ if or (.Working.Changed) (.Staging.Changed) }}#FFB900{{ end }}",
  "{{ if and (gt .Ahead 0) (gt .Behind 0) }}#F25022{{ end }}",
  "{{ if gt .Ahead 0 }}#B4009E{{ end }}",
  "{{ if gt .Behind 0 }}#F25022{{ end }}"
]

These background templates create visual warnings:

• Yellow (#FFB900): Uncommitted changes in working directory or staging area
• Orange/Red (#F25022): Branch is behind the remote (need to pull)
• Purple (#B4009E): Branch is ahead of remote (need to push)

This color-coding provides instant feedback about repository state without reading the status text.

The Complete Automation Script

The full automation script is maintained in a GitHub repository for easy access and version control. You can find the complete, up-to-date script here:

📜 Script Location: https://github.com/Ricky-G/script-library/blob/main/pimp-my-terminal.ps1

The script handles the entire setup process including:

• Oh My Posh installation via winget
• Nerd Font (MesloLGM) installation
• Terminal-Icons PowerShell module installation
• Cloud Native Azure theme configuration
• PowerShell profile setup with PSReadLine enhancements
• Custom keyboard shortcuts for dotnet commands
• Intelligent tab completion for winget and dotnet CLI

Quick Start

To run the script, open PowerShell as Administrator and execute:

1
2
3

# Download and run the script directly
Invoke-WebRequest -Uri "https://raw.githubusercontent.com/Ricky-G/script-library/main/pimp-my-terminal.ps1" -OutFile "$env:TEMP\pimp-my-terminal.ps1"
& "$env:TEMP\pimp-my-terminal.ps1"

Or if you’ve cloned the repository locally:

1
2

# Run from local clone
.\pimp-my-terminal.ps1

Script Breakdown: Key Components

The automation script includes several critical sections that ensure a smooth, repeatable setup process. Let’s examine the key components:

Administrative Privileges Requirement

1

#Requires -RunAsAdministrator

The script requires administrative privileges for two reasons:

1. Font Installation: Installing system-wide fonts requires elevated permissions to write to C:\Windows\Fonts
2. Winget Operations: While winget can run without admin rights, some installations require elevation for proper PATH configuration

If you run the script without elevation, PowerShell will automatically prompt for admin credentials before execution.

Winget Installation with Accept Flags

1

winget install JanDeDobbeleer.OhMyPosh -s winget --accept-package-agreements --accept-source-agreements

The --accept-package-agreements and --accept-source-agreements flags automate the acceptance of license terms, enabling the script to run non-interactively. This is crucial for CI/CD scenarios or remote machine setup where manual interaction isn’t possible.

The -s winget parameter explicitly specifies the winget repository, preventing potential conflicts if you have multiple package sources configured.

Environment Path Refresh

1

$env:Path = [System.Environment]::GetEnvironmentVariable("Path","Machine") + ";" + [System.Environment]::GetEnvironmentVariable("Path","User")

This is a critical step that’s often overlooked in automation scripts. When winget installs Oh My Posh, it modifies the system PATH variable, but PowerShell sessions don’t automatically refresh their environment. This line explicitly reloads the PATH, making the oh-my-posh command immediately available without requiring a terminal restart.

Without this refresh, the subsequent oh-my-posh font install command would fail with a “command not found” error.

Terminal-Icons Module Installation

1

Install-Module -Name Terminal-Icons -Repository PSGallery -Force

The script also installs the Terminal-Icons module, which adds beautiful file type icons to your directory listings. When you run ls or Get-ChildItem, you’ll see:

• 📁 Folder icons for directories
• 🐍 Python icon for .py files
• 📄 Document icons for text files
• ⚙️ Config icons for .json, .xml, .yaml files
• And many more language-specific icons

Nerd Font Installation

1

oh-my-posh font install Meslo

Oh My Posh includes a built-in font installer that downloads and installs Nerd Fonts from their GitHub releases. The Meslo font (MesloLGM Nerd Font) is recommended because:

• Excellent readability at small and large sizes
• Wide character coverage including all necessary glyphs
• Good distinction between similar characters (1, l, I, 0, O)
• Proper line height and spacing for terminal use

Alternative Nerd Fonts you might consider:

• CascadiaCode - Microsoft’s open-source programming font
• FiraCode - Popular font with extensive ligature support
• JetBrainsMono - Optimized for long coding sessions
• Hack - Minimal, clean appearance

Profile Configuration with Idempotency

The script checks if Oh My Posh is already configured before making changes, allowing safe repeated execution:

1
2
3
4
5
6

if (Test-Path $PROFILE) {
    $existingProfile = Get-Content $PROFILE -Raw
    if ($existingProfile -notmatch "oh-my-posh") {
        Add-Content -Path $PROFILE -Value $profileContent
    }
}

This idempotency is essential for:

• Updating the theme configuration without duplicating profile entries
• Re-running the script after failed installations
• Using the script in configuration management systems

Step-by-Step Implementation Guide

Prerequisites and Preparation

Before running the script, ensure you have:
5. Click Appearance in the sub-menu
6. Under Font face, select MesloLGM Nerd Font
7. Optionally adjust Font size (I recommend 10-11pt for readability)
8. Click Save

Pro Tip: You can also configure this per-profile (PowerShell, Command Prompt, Ubuntu, etc.) if you want different fonts for different shells. However, setting it in Defaults applies to all profiles consistently.

Visual Studio Code Terminal Configuration

If you spend significant time in VS Code’s integrated terminal, you’ll want the same beautiful prompt there:

1. Open VS Code
2. Press Ctrl + , to open Settings
3. Search for terminal font
4. Find Terminal › Integrated: Font Family
5. Enter: MesloLGM Nerd Font
6. Restart any open terminal instances

Alternatively, edit your settings.json directly:

1
2
3
4

{
  "terminal.integrated.fontFamily": "MesloLGM Nerd Font",
  "terminal.integrated.fontSize": 11
}

Verifying Font Installation

After configuring your terminal, restart it and open a new PowerShell session. You should see the themed prompt with proper icons. If you see squares, question marks, or missing characters, the font isn’t properly applied.

Common fixes:

• Ensure you spelled the font name exactly: MesloLGM Nerd Font (not MesloLGM NF)
• Try restarting Windows Terminal completely (close all windows)
• Verify the font appears in Windows Settings → Personalization → Fonts

Advanced Customization Options

Modifying the Theme

The beauty of this automated setup is that your theme is now stored as a JSON file at $HOME\.config\ohmyposh\cloud-native-azure.omp.json. You can modify this file to customize the prompt to your preferences.

Adding Additional Segments

Want to display your Python virtual environment or Node.js version? Add segments to the appropriate block:

1
2
3
4
5
6
7
8

{
  "type": "python",
  "style": "powerline",
  "powerline_symbol": "\ue0b0",
  "foreground": "#ffffff",
  "background": "#306998",
  "template": " \ue235 {{ if .Venv }}{{ .Venv }} {{ end }}{{ .Full }} "
}

Available segment types include: aws, battery, cmake, dart, docker, dotnet, elixir, flutter, go, java, julia, kotlin, lua, node, perl, php, python, ruby, rust, scala, swift, terraform, and many more.

Changing Colors

Modify the foreground and background properties to use your preferred color scheme:

1
2

"foreground": "#ffffff",
"background": "#your-hex-color"

You can also use color definitions for dark/light terminal themes:

1
2

"foreground": "p:white",
"background": "p:blue"

Adjusting Git Status Indicators

Customize which Git information displays by modifying the template:

1

"template": " {{ .HEAD }}{{ if .BranchStatus }} {{ .BranchStatus }}{{ end }}{{ if .Working.Changed }} \uf044 {{ .Working.String }}{{ end }} "

Remove sections you don’t need or add additional information like:

• {{ .UpstreamIcon }} - Shows if branch has an upstream
• {{ .StashCount }} - Number of stashed changes
• {{ .WorktreeCount }} - Number of worktrees

Creating Multiple Theme Configurations

You might want different themes for different scenarios - perhaps a minimal theme for screen recordings or presentations, and a detailed theme for daily work.

Create multiple theme files:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

# Create a minimal theme
$minimalTheme = @'
{
  "$schema": "https://raw.githubusercontent.com/JanDeDobbeleer/oh-my-posh/main/themes/schema.json",
  "version": 2,
  "final_space": true,
  "blocks": [
    {
      "type": "prompt",
      "alignment": "left",
      "segments": [
        {
          "type": "path",
          "style": "plain",
          "foreground": "#00A4EF",
          "template": "{{ .Path }} "
        },
        {
          "type": "git",
          "style": "plain",
          "foreground": "#7FBA00",
          "template": "{{ .HEAD }} "
        },
        {
          "type": "text",
          "style": "plain",
          "foreground": "#0078D4",
          "template": "\u276f "
        }
      ]
    }
  ]
}
'@

$minimalTheme | Out-File -FilePath "$HOME\.config\ohmyposh\minimal.omp.json" -Encoding utf8

Switch themes by modifying your profile or creating PowerShell functions:

1
2
3
4
5
6
7
8

# Add to your PowerShell profile
function Set-CloudNativeTheme {
    oh-my-posh init pwsh --config "$HOME\.config\ohmyposh\cloud-native-azure.omp.json" | Invoke-Expression
}

function Set-MinimalTheme {
    oh-my-posh init pwsh --config "$HOME\.config\ohmyposh\minimal.omp.json" | Invoke-Expression
}

Conditional Theme Loading

Load different themes based on context - for example, use a detailed theme on your workstation but a minimal theme when SSH’d into servers:

1
2
3
4
5
6

# In your PowerShell profile
if ($env:SSH_CONNECTION) {
    oh-my-posh init pwsh --config "$HOME\.config\ohmyposh\minimal.omp.json" | Invoke-Expression
} else {
    oh-my-posh init pwsh --config "$HOME\.config\ohmyposh\cloud-native-azure.omp.json" | Invoke-Expression
}

Keyboard Shortcuts Reference

After setup, you’ll have enhanced keyboard shortcuts available in PowerShell thanks to PSReadLine configuration:

Using History Search Effectively

The history search feature is particularly powerful for repetitive commands:

Example 1 - Git Commands:

• Type git and press ↑
• Cycles through all previous git commands
• Much faster than retyping or searching entire history

Example 2 - Kubectl Commands:

• Type kubectl get pods -n and press ↑
• Finds all previous kubectl commands starting with that pattern
• Quickly switch between namespaces you’ve used before

Example 3 - Complex Commands:

• Type the first few characters of a long command
• Press ↑ to find it immediately
• No need to remember the entire command syntax

PSReadLine Prediction Modes

The script configures PSReadLine to show predictions as you type:

1
2
3
4

# Predictions appear grayed out based on your command history
PS> git checkout ma|ain     # Gray text shows prediction
                  ↑
            Press → to accept

This feature learns from your command patterns and becomes more useful over time as you build up command history.

Customization and Theme Switching

Switching to a Different Theme

To use a different Oh My Posh theme after installation, edit your PowerShell profile:

1

notepad $PROFILE

Replace the theme URL with any theme from the themes gallery:

1
2

# Example: Switch to 'agnoster' theme
oh-my-posh init pwsh --config 'https://raw.githubusercontent.com/JanDeDobbeleer/oh-my-posh/main/themes/agnoster.omp.json' | Invoke-Expression

Adding Custom Keyboard Shortcuts

You can add more PSReadLine shortcuts to your profile:

1
2
3
4
5
6
7
8
9
10
11
12
13

# Example: Ctrl+Shift+R for dotnet run
Set-PSReadLineKeyHandler -Key Ctrl+Shift+r -ScriptBlock {
    [Microsoft.PowerShell.PSConsoleReadLine]::RevertLine()
    [Microsoft.PowerShell.PSConsoleReadLine]::Insert("dotnet run")
    [Microsoft.PowerShell.PSConsoleReadLine]::AcceptLine()
}

# Example: Ctrl+Shift+G for git status
Set-PSReadLineKeyHandler -Key Ctrl+Shift+g -ScriptBlock {
    [Microsoft.PowerShell.PSConsoleReadLine]::RevertLine()
    [Microsoft.PowerShell.PSConsoleReadLine]::Insert("git status")
    [Microsoft.PowerShell.PSConsoleReadLine]::AcceptLine()
}

Troubleshooting Common Issues

Oh My Posh Command Not Found

Symptom: After running the script, oh-my-posh command isn’t recognized.

Causes and Fixes:

1. PATH not refreshed: Close and reopen your terminal, or run:

   1	$env:Path = [System.Environment]::GetEnvironmentVariable("Path","Machine") + ";" + [System.Environment]::GetEnvironmentVariable("Path","User")

2. Winget installation failed: Check if Oh My Posh was actually installed:

   1	winget list | Select-String "OhMyPosh"

   If not present, manually install:

   1	winget install JanDeDobbeleer.OhMyPosh

3. Installation location issues: Verify the executable location:

   1	Get-Command oh-my-posh

Script Execution Is Disabled

Symptom: Error message “running scripts is disabled on this system”

Fix:

1

Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser

When prompted, type Y and press Enter. This policy allows locally-created scripts to run while maintaining security for downloaded scripts.

Font Rendering Issues

Symptom: Boxes, question marks, or missing icons in your prompt.

Fixes:

1. Verify font installation:

   • Open Windows Settings → Personalization → Fonts
   • Search for “MesloLGM”
   • If not present, manually run: oh-my-posh font install Meslo

2. Correct font name in terminal:

   • The exact font name is MesloLGM Nerd Font
   • Some terminals are case-sensitive
   • Don’t use abbreviations like “MesloLGM NF”

3. Terminal compatibility:

   • Windows Terminal, VS Code, and modern terminals support Nerd Fonts well
   • Legacy terminals (cmd.exe window) may have poor support
   • Consider upgrading to Windows Terminal

4. Manual font installation:
   If automatic installation fails, download manually:

   • Go to https://github.com/ryanoasis/nerd-fonts/releases
   • Download Meslo.zip
   • Extract and right-click the .ttf files
   • Select Install for all users

PSReadLine Version Error

Symptom: Error about -PredictionSource parameter

Fix:
The script includes version checking, but if you manually edited your profile, ensure PSReadLine 2.1+ is installed:

1
2
3
4
5

# Check version
(Get-Module PSReadLine).Version

# Update if needed
Install-Module -Name PSReadLine -Force -SkipPublisherCheck

Kubernetes Segment Not Appearing

Symptom: The kubectl segment doesn’t show even though you have kubectl configured.

Causes:

1. No current context: Run kubectl config current-context to verify you have an active context

2. Kubeconfig not in default location: Oh My Posh looks for ~/.kube/config. If you use KUBECONFIG environment variable with custom paths, the segment may not find it.

3. Performance optimization: Oh My Posh might disable slow segments. Check if kubectl commands are responsive:

   1	Measure-Command { kubectl config current-context }

   If this takes more than 500ms, Oh My Posh may timeout the segment.

Azure Segment Not Showing

Symptom: No Azure subscription information in your prompt.

Fixes:

1. Azure CLI not installed: The segment requires Azure CLI:

   1	winget install Microsoft.AzureCLI

2. Not logged in: Authenticate with Azure:

   1

   az login

3. No active subscription: Set a default subscription:

   1	az account set --subscription "Your Subscription Name"

Slow Prompt Performance

Symptom: Noticeable delay before prompt appears, especially after pressing Enter.

Common Causes:

1. Slow Git operations: Large repositories or network-based Git remotes can slow the Git segment. Disable fetch_status for specific repositories:

   1 2 3

   "properties": { "fetch_status": false }

2. Multiple cloud segments: Each cloud segment (kubectl, az, aws) makes system calls. Remove segments you don’t actively use.

3. Network timeouts: If segments query network resources (like Kubernetes API servers), timeouts can cause delays. Consider adjusting timeout settings or removing problematic segments.

Performance Considerations and Optimizations

Segment Caching

Oh My Posh caches segment results to improve performance. You can adjust cache durations in the segment properties:

1
2
3

"properties": {
  "cache_timeout": 5
}

This tells the segment to cache results for 5 minutes, reducing repeated system calls.

Selective Segment Enablement

Not every developer needs every segment. Consider creating role-specific variants:

Backend Developers: Focus on Git, Azure, and database context
Frontend Developers: Emphasize Node.js version, Git, and build tool status
DevOps Engineers: Full cloud context with Kubernetes, Azure, and AWS
Full Stack: Balanced approach with programming language versions and cloud context

Async Segment Updates

For segments that make network calls, consider using Oh My Posh’s background update feature to prevent blocking the prompt:

1
2
3

"properties": {
  "async": true
}

Wrapping Up

What started as frustration with repeatedly configuring terminals across new machines evolved into a robust automation solution that eliminates setup friction entirely. One PowerShell script, a few minutes of execution time, and your terminal transforms from a blank slate into a fully-configured, cloud native development environment.

The script and theme are opinionated - they reflect my specific workflow and preferences. But that’s the beauty of Oh My Posh’s flexibility. Take this automation as a starting point, customize the theme to match your daily tasks, add or remove segments based on your tech stack, and create your perfect terminal environment.

No more hunting for font installers. No more manually editing profile files. No more copying configuration snippets from old machines. Just run the script, configure your font settings, and get back to building amazing things.

Happy terminal pimping! 🎉

References

• Oh My Posh Official Website
• Oh My Posh GitHub Repository
• Windows Package Manager (winget)
• Nerd Fonts Project
• Windows Terminal Documentation
• PowerShell Profile Documentation
• Main image generated by GPT-Image-1.5

🎯 TL;DR: Bulk Configure Searchable Branches in Azure DevOps via Hidden Policy API

Azure DevOps code search only indexes the default branch (master/main) by default, causing issues when teams use develop branches for JFrog Artifactory detection scripts. Problem: No documented API exists for bulk updating searchable branches across thousands of repositories. Solution: Use the undocumented Policy Configuration API with policy type 0517f88d-4ec5-4343-9d26-9930ebd53069 to programmatically add branches to the searchable list. This approach leverages the same API calls the Azure DevOps UI uses internally, enabling automation of what would otherwise require manual configuration across massive repository collections.

Recently, I encountered an interesting challenge while working on a JFrog Artifactory adoption tracking project across a large Azure DevOps organization. The requirement was to scan repositories for JFrog URL references to determine which teams had successfully onboarded to their new artifact management system. The problem? Some development teams exclusively work in develop branches instead of master or main, and Azure DevOps code search only indexes the default branch by default.

This seemingly simple requirement - adding develop to the searchable branches for thousands of repositories - turned into a fascinating exploration of Azure DevOps’ undocumented APIs. While there’s no official documentation for bulk updating searchable branches, I discovered that the Azure DevOps UI uses a specific Policy Configuration API under the hood that we can leverage for automation.

This blog post shares a practical approach to programmatically configure searchable branches across large Azure DevOps organizations using REST APIs that Microsoft doesn’t officially document but absolutely supports.

The Challenge: Azure DevOps Code Search Limitations

Azure DevOps code search is a powerful feature, but it comes with a significant limitation that affects many organizations: by default, only the repository’s default branch (typically master or main) is indexed for search operations.

This creates problems in several scenarios:

JFrog Adoption Tracking: Organizations implementing JFrog Artifactory need to scan all repositories for configuration files and dependency references, but teams using feature branches or develop as their primary branch won’t be detected.

Multi-Branch Development: Teams practicing GitFlow or similar branching strategies may have critical code in develop, release/*, or feature branches that needs to be searchable.

Compliance and Security Scanning: Security tools and compliance scripts that rely on code search may miss important files if they’re not in the default branch.

Understanding Azure DevOps Searchable Branches

In Azure DevOps, searchable branches are configured at the repository level through the UI:

Manual Configuration Path:

1. Navigate to Project Settings → Repositories
2. Select your repository
3. Go to Settings → Searchable Branches
4. Add additional branches (maximum of 5 extra branches)

What Actually Happens:
When you configure searchable branches through the UI, Azure DevOps creates or updates a policy configuration with a specific policy type. This policy type - 0517f88d-4ec5-4343-9d26-9930ebd53069 - controls which branches are indexed for code search.

The Discovery: Hidden Policy Configuration API

After extensive research and reverse engineering the Azure DevOps UI network traffic, I discovered that searchable branch configuration is managed through the Policy Configuration API with a specific, undocumented policy type.

Key Insights:

• Azure DevOps doesn’t expose a direct “searchable branches” API endpoint
• The UI uses the Policy Configuration API with policy type 0517f88d-4ec5-4343-9d26-9930ebd53069
• This API is officially supported but not documented for this specific use case
• The same approach works for both Azure DevOps Server and Azure DevOps Cloud

The Solution: Automated Policy Configuration

The approach involves three main steps:

Step 1: Retrieve Existing Policy Configuration

First, we need to check if a searchable branches policy already exists for the repository:

1
2
3
4
5
6
7
8
9

# Get existing searchable branch policy for a repository
$policyUrl = "https://dev.azure.com/{organization}/{project}/_apis/policy/configurations"
$params = @{
    'repositoryId' = $repositoryId
    'policyType' = '0517f88d-4ec5-4343-9d26-9930ebd53069'
    'api-version' = '7.1-preview.1'
}

$existingPolicy = Invoke-RestMethod -Uri $policyUrl -Headers $headers -Method Get -Body $params

Step 2: Create or Update Policy Configuration

If a policy exists, we update it. If not, we create a new one:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

# Policy configuration for searchable branches
$policySettings = @{
    'type' = @{
        'id' = '0517f88d-4ec5-4343-9d26-9930ebd53069'
    }
    'isEnabled' = $true
    'isBlocking' = $false
    'settings' = @{
        'searchBranches' = @(
            'refs/heads/master',
            'refs/heads/main', 
            'refs/heads/develop'
        )
    }
    'scope' = @(
        @{
            'repositoryId' = $repositoryId
            'refName' = 'refs/heads/master'
            'matchKind' = 'exact'
        }
    )
}

Step 3: Apply the Configuration

Send the policy configuration to Azure DevOps:

1
2
3
4
5
6
7
8
9
10

if ($existingPolicy.count -gt 0) {
    # Update existing policy
    $configId = $existingPolicy.value[0].id
    $updateUrl = "https://dev.azure.com/{organization}/{project}/_apis/policy/configurations/$configId"
    Invoke-RestMethod -Uri $updateUrl -Headers $headers -Method Put -Body ($policySettings | ConvertTo-Json -Depth 10) -ContentType "application/json"
} else {
    # Create new policy
    $createUrl = "https://dev.azure.com/{organization}/{project}/_apis/policy/configurations"
    Invoke-RestMethod -Uri $createUrl -Headers $headers -Method Post -Body ($policySettings | ConvertTo-Json -Depth 10) -ContentType "application/json"
}

Complete PowerShell Implementation

Here’s a complete script that implements this solution for bulk updating searchable branches across multiple repositories:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98

# Azure DevOps configuration
$organization = "your-org"
$project = "your-project" 
$pat = "your-personal-access-token"

# Create authentication headers
$headers = @{
    'Authorization' = "Basic " + [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(":$pat"))
    'Content-Type' = 'application/json'
}

# Function to update searchable branches for a repository
function Update-SearchableBranches {
    param(
        [string]$organizationName,
        [string]$projectName,
        [string]$repositoryId,
        [array]$branches,
        [hashtable]$authHeaders
    )
    
    $policyTypeId = '0517f88d-4ec5-4343-9d26-9930ebd53069'
    $baseUrl = "https://dev.azure.com/$organizationName/$projectName/_apis/policy/configurations"
    
    try {
        # Check for existing policy
        $getUrl = "$baseUrl" + "?repositoryId=$repositoryId&policyType=$policyTypeId&api-version=7.1-preview.1"
        $existingPolicy = Invoke-RestMethod -Uri $getUrl -Headers $authHeaders -Method Get
        
        # Format branches with refs/heads/ prefix
        $searchBranches = $branches | ForEach-Object { 
            if ($_ -like "refs/heads/*") { $_ } else { "refs/heads/$_" }
        }
        
        # Create policy configuration
        $policyConfig = @{
            'type' = @{ 'id' = $policyTypeId }
            'isEnabled' = $true
            'isBlocking' = $false
            'settings' = @{
                'searchBranches' = $searchBranches
            }
            'scope' = @(
                @{
                    'repositoryId' = $repositoryId
                    'refName' = $searchBranches[0]
                    'matchKind' = 'exact'
                }
            )
        }
        
        $jsonBody = $policyConfig | ConvertTo-Json -Depth 10
        
        if ($existingPolicy.count -gt 0) {
            # Update existing policy
            $configId = $existingPolicy.value[0].id
            $updateUrl = "$baseUrl/$configId" + "?api-version=7.1-preview.1"
            $result = Invoke-RestMethod -Uri $updateUrl -Headers $authHeaders -Method Put -Body $jsonBody
            Write-Host "Updated searchable branches for repository $repositoryId" -ForegroundColor Green
        } else {
            # Create new policy
            $createUrl = "$baseUrl" + "?api-version=7.1-preview.1"
            $result = Invoke-RestMethod -Uri $createUrl -Headers $authHeaders -Method Post -Body $jsonBody
            Write-Host "Created searchable branches policy for repository $repositoryId" -ForegroundColor Green
        }
        
        return $result
    }
    catch {
        Write-Error "Failed to update searchable branches for repository $repositoryId`: $($_.Exception.Message)"
        return $null
    }
}

# Get all repositories in the project
$reposUrl = "https://dev.azure.com/$organization/$project/_apis/git/repositories?api-version=7.1"
$repositories = Invoke-RestMethod -Uri $reposUrl -Headers $headers

# Define branches to make searchable
$searchableBranches = @('master', 'main', 'develop')

# Update searchable branches for each repository
foreach ($repo in $repositories.value) {
    Write-Host "Processing repository: $($repo.name)" -ForegroundColor Yellow
    
    $result = Update-SearchableBranches -organizationName $organization -projectName $project -repositoryId $repo.id -branches $searchableBranches -authHeaders $headers
    
    if ($result) {
        Write-Host "Successfully configured searchable branches for $($repo.name)" -ForegroundColor Green
    } else {
        Write-Host "Failed to configure searchable branches for $($repo.name)" -ForegroundColor Red
    }
    
    # Add delay to avoid rate limiting
    Start-Sleep -Milliseconds 500
}

Write-Host "Searchable branches configuration complete!" -ForegroundColor Cyan

Important Considerations and Limitations

API Version and Stability

Preview API: The Policy Configuration API is in preview (7.1-preview.1), which means:

• The API may change without notice
• Microsoft doesn’t guarantee backward compatibility
• Monitor for API updates and test thoroughly before production use

Repository Limitations

Branch Limits: Azure DevOps allows a maximum of 6 searchable branches per repository (including the default branch).

Indexing Delays: After updating searchable branches, Azure DevOps may take several hours to index the new branches. Search results won’t be immediately available.

Performance Considerations

Rate Limiting: Implement appropriate delays between API calls to avoid hitting rate limits, especially when processing thousands of repositories.

Batch Processing: For large organizations, consider processing repositories in batches and implementing retry logic for failed requests.

Error Handling

Repository States: Some repositories may not have branches configured or may be in archived states. Implement proper error handling for these scenarios.

Permission Issues: Ensure your Personal Access Token has sufficient permissions for policy configuration (Project Settings, Read & Manage).

Authentication and Security Setup

Personal Access Token Configuration

Create a Personal Access Token with the following permissions:

• Code (Read) - To access repository information
• Project and Team (Read) - To list projects and repositories
• Policy Configuration (Read & Manage) - To update searchable branch policies

Security Best Practices

Token Storage: Store Personal Access Tokens securely and rotate them regularly according to your organization’s security policies.

Least Privilege: Use dedicated service accounts with minimal required permissions for automation scripts.

Audit Logging: Log all policy changes for compliance and troubleshooting purposes.

Advanced Usage Scenarios

Conditional Branch Configuration

Configure different searchable branches based on repository naming conventions or team requirements:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

# Example: Configure different branches based on repository name patterns
$branchConfig = @{
    'web-*' = @('master', 'develop', 'staging')
    'api-*' = @('master', 'develop', 'release')
    'mobile-*' = @('master', 'develop', 'feature/*')
}

foreach ($repo in $repositories.value) {
    $branches = @('master', 'main') # Default branches
    
    # Add specific branches based on repository name
    foreach ($pattern in $branchConfig.Keys) {
        if ($repo.name -like $pattern) {
            $branches += $branchConfig[$pattern]
            break
        }
    }
    
    # Remove duplicates and configure
    $uniqueBranches = $branches | Select-Object -Unique
    Update-SearchableBranches -repositoryId $repo.id -branches $uniqueBranches
}

Integration with CI/CD Pipelines

Integrate searchable branch configuration into your DevOps workflows:

1
2
3
4
5
6
7
8

# Azure DevOps Pipeline example
- task: PowerShell@2
  displayName: 'Configure Searchable Branches'
  inputs:
    targetType: 'inline'
    script: |
      # Your PowerShell script here
      # Use pipeline variables for organization, project, and PAT

Troubleshooting Common Issues

Policy Configuration Not Found

If the GET request returns empty results, the repository may not have searchable branches configured yet. Create a new policy instead of updating an existing one.

Invalid Branch References

Ensure branch names use the correct format:

• Correct: refs/heads/develop
• Incorrect: develop or origin/develop

Permission Denied Errors

Verify that your Personal Access Token has the required permissions and that you have administrative access to the project.

Indexing Delays

After configuration, search indexing may take several hours. Test with small repositories first to verify the configuration is working before processing large numbers of repositories.

Real-World Use Cases

JFrog Artifactory Adoption Tracking

The original use case that sparked this solution:

1
2
3
4
5
6
7
8
9
10
11
12
13

# Search for JFrog references across all configured branches
$searchQuery = "jfrog.yourcompany.com"
$searchUrl = "https://dev.azure.com/$organization/$project/_apis/search/codesearchresults?api-version=7.1-preview.1"

$searchBody = @{
    searchText = $searchQuery
    includeFacets = $true
    top = 1000
} | ConvertTo-Json

$searchResults = Invoke-RestMethod -Uri $searchUrl -Headers $headers -Method Post -Body $searchBody -ContentType "application/json"

# Process results to identify repositories using JFrog

Security Compliance Scanning

Scan for security-sensitive patterns across multiple branches:

1
2
3
4
5
6
7
8
9
10
11

# Search for potential security issues across all searchable branches
$securityPatterns = @(
    "password\s*=",
    "api[_-]?key\s*[:=]",
    "secret[_-]?key\s*[:=]"
)

foreach ($pattern in $securityPatterns) {
    # Search across all configured branches
    # Generate compliance reports
}

Code Quality Assessment

Analyze code patterns and best practices across development branches:

1
2
3
4
5
6
7
8

# Search for deprecated patterns across main and develop branches
$deprecatedPatterns = @(
    "System.Web.HttpContext",
    "ConfigurationManager.AppSettings",
    "HttpResponse.Write"
)

# Generate modernization reports

Future Considerations and Alternatives

Azure DevOps CLI Integration

While the Azure DevOps CLI doesn’t have native support for searchable branches, you can use az devops invoke to call the REST APIs:

1

az devops invoke --area policy --resource configurations --route-parameters project=YourProject --http-method GET --api-version 7.1-preview.1

PowerShell Module Development

Consider creating a dedicated PowerShell module for searchable branch management:

1
2
3
4
5

# Future PowerShell module structure
Import-Module AzureDevOpsSearchableBranches

Get-AdoSearchableBranches -Organization "YourOrg" -Project "YourProject" -Repository "YourRepo"
Set-AdoSearchableBranches -Organization "YourOrg" -Project "YourProject" -Repository "YourRepo" -Branches @("master", "develop")

Microsoft Graph Integration

For organizations using Microsoft Graph, consider integrating searchable branch configuration with broader DevOps governance workflows.

Key Takeaways

Working with Azure DevOps searchable branches requires understanding the underlying Policy Configuration API that Microsoft uses internally but doesn’t officially document for this purpose. The key insights from this solution are:

• Hidden APIs Exist: Azure DevOps has powerful APIs that aren’t always documented for every use case
• UI Reverse Engineering: Network traffic analysis can reveal API patterns for automation
• Policy-Based Configuration: Many Azure DevOps features use the Policy Configuration API under the hood
• Batch Processing Considerations: Large-scale automation requires careful attention to rate limiting and error handling

This solution provides a robust foundation for any Azure DevOps organization needing to manage searchable branches at scale. Whether you’re tracking technology adoption, performing security scanning, or implementing code quality initiatives, this approach enables the automation that makes these tasks feasible across large repository collections.

The undocumented nature of this API means it should be used with appropriate caution and monitoring, but for organizations with thousands of repositories, it’s currently the only viable approach for bulk searchable branch configuration.

References

• Azure DevOps REST API - Policy Configurations
• Azure DevOps REST API - Git Repositories
• Stack Overflow: Cross Search in all repositories and branches in Azure DevOps Repos
• Azure DevOps Code Search Documentation
• Main image generated by DALL-E

🎯 TL;DR: Real-time Voice Agent Implementation

This post walks through building a voice agent that connects traditional phone calls to Azure’s AI services. The system intercepts incoming calls via Azure Communication Services, streams audio in real-time to the Voice Live API, and processes conversations through pre-configured AI agents in Azure AI Studio. The implementation uses FastAPI for webhook handling, WebSocket connections for bidirectional audio streaming, and Azure Managed Identity for authentication (no API keys to manage). The architecture handles multiple concurrent calls on a single Python thread using asyncio.

Implementation details: Audio resampling between 16kHz (ACS requirement) and 24kHz (Voice Live requirement), connection resilience for preview services, and production deployment considerations. Full source code and documentation available here

Recently, I found myself co-leading an innovation project that pushed me into uncharted territory. The challenge? Developing a voice-based agentic solution with an ambitious goal - routing at least 25% of current contact center calls to AI voice agents. This was bleeding-edge stuff, with both the Azure Voice Live API and Azure AI Agent Service voice agents still in preview at the time of writing.

When you’re working with preview services, documentation is often sparse, and you quickly learn that reverse engineering network calls and maintaining close relationships with product teams becomes part of your daily routine. This blog post shares the practical lessons learned and the working solution we built to integrate these cutting-edge services.

The Innovation Challenge

Building a voice agent system that could handle real customer interactions meant tackling several complex requirements:

• Real-time voice processing with minimal latency
• Natural conversation flow without awkward pauses
• Integration with existing contact center infrastructure
• Scalability to handle multiple concurrent calls
• Reliability for production use cases

With both Azure Voice Live API and Azure AI Voice Agent Service in preview, we were essentially building on shifting sands. But that’s what innovation is about - pushing boundaries and finding solutions where documentation doesn’t yet exist.

Understanding the Architecture

Our solution bridges Azure Communication Services (ACS) with Azure AI services to create an intelligent voice agent. Here’s how the pieces fit together:

graph TB    subgraph "Phone Network"        PSTN[📞 PSTN Number
+1-555-123-4567]    end        subgraph "Azure Communication Services"        ACS[🔗 ACS Call Automation
Event Grid Webhooks]        MEDIA[🎵 Media Streaming
WebSocket Audio]    end        subgraph "Python FastAPI App"        API[🐍 FastAPI Server
localhost:49412]        WS[🔌 WebSocket Handler
Audio Processing]        HANDLER[⚡ Media Handler
Audio Resampling]    end        subgraph "Azure OpenAI"        VOICE[🤖 Voice Live API
Agent Mode
gpt-4o Realtime]        AGENT[👤 Pre-configured Agent
Azure AI Studio]    end        subgraph "Dev Infrastructure"        TUNNEL[🚇 Dev Tunnel
Public HTTPS Endpoint]    end        PSTN -->|Incoming Call| ACS    ACS -->|Webhook Events| TUNNEL    TUNNEL -->|HTTPS| API    ACS -->|WebSocket Audio| WS    WS -->|PCM 16kHz| HANDLER    HANDLER -->|PCM 24kHz| VOICE    VOICE -->|Agent Processing| AGENT    AGENT -->|AI Response| VOICE    VOICE -->|AI Response| HANDLER    HANDLER -->|PCM 16kHz| WS    WS -->|Audio Stream| ACS    ACS -->|Audio| PSTN        style PSTN fill:#ff9999    style ACS fill:#87CEEB    style API fill:#90EE90    style VOICE fill:#DDA0DD    style TUNNEL fill:#F0E68C

Core Components

1. Azure Communication Services: Handles the telephony infrastructure, providing phone numbers and call routing
2. Voice Live API: Enables real-time speech recognition and synthesis with WebRTC streaming
3. Azure AI Agent Service: Provides the intelligence layer for understanding and responding to customer queries
4. WebSocket Bridge: Our custom Python application that connects these services

The Flow

When a customer calls, here’s what happens behind the scenes:

1
2
3

Customer Call → ACS Phone Number → Webhook to Our Service → 
WebSocket Connection → Voice Live API ↔ AI Agent Service → 
Real-time Voice Response → Customer

Setting Up the Foundation

Let’s walk through the practical implementation. You can find the complete code in my GitHub repository.

Prerequisites

First, you’ll need to set up several Azure services. Here’s what we discovered through trial and error:

1
2
3
4
5

# Required Azure services
- Azure Communication Services (with phone number provisioning)
- Azure AI Services (Speech Service enabled)
- Azure AI Agent Service (with voice capabilities)
- Azure App Service or Container Instance (for hosting)

Environment Configuration

One of the first challenges was figuring out all the required configuration parameters. Here’s what you’ll need:

1
2
3
4
5
6

# Essential environment variables (Using Azure Managed Identity - No API Keys!)
ACS_CONNECTION_STRING = "endpoint=https://your-acs.communication.azure.com/;accesskey=your-key"
AZURE_VOICE_LIVE_ENDPOINT = "https://your-aoai.cognitiveservices.azure.com/"
AGENT_ID = "your_agent_id_from_azure_ai_studio"
AGENT_PROJECT_NAME = "your_project_name"
BASE_URL = "https://your-tunnel-url.asse.devtunnels.ms"  # Dev Tunnel URL

Building the WebSocket Bridge

The heart of our solution is a Python application that acts as a bridge between ACS and the Voice Live API. This wasn’t documented anywhere - we had to figure it out by analyzing network traffic and experimenting.

Handling Incoming Calls

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

from fastapi import FastAPI, WebSocket
from azure.communication.callautomation import CallAutomationClient
from azure.identity import DefaultAzureCredential
import asyncio
import websockets

app = FastAPI()
call_automation_client = CallAutomationClient.from_connection_string(
    ACS_CONNECTION_STRING
)

@app.post("/api/incomingCall")
async def incoming_call(request: dict):
    """Handle incoming call webhook from ACS"""
    try:
        # Parse the incoming call context
        incoming_call_context = request.get("incomingCallContext")
        
        # Answer the call
        call_connection = call_automation_client.answer_call(
            incoming_call_context=incoming_call_context,
            callback_url=f"{CALLBACK_URI}/api/callbacks/{call_id}",
        )
        
        # Start WebSocket connection to Voice Live API
        asyncio.create_task(
            establish_voice_connection(call_connection.call_connection_id)
        )
        
        return {"status": "success"}
        
    except Exception as e:
        logger.error(f"Error handling incoming call: {e}")
        return {"error": str(e)}

Establishing the Voice Connection

This is where things got interesting. The Voice Live API uses WebRTC for real-time audio streaming, but the documentation was minimal. Here’s what we discovered:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

async def establish_voice_connection(call_connection_id):
    """Establish WebSocket connection to Voice Live API using Azure Managed Identity"""
    
    # Get access token using managed identity
    from azure.identity import DefaultAzureCredential
    credential = DefaultAzureCredential()
    token = credential.get_token("https://cognitiveservices.azure.com/.default")
    
    # Construct the WebSocket URL for Voice Live API
    ws_url = f"wss://your-region.cognitiveservices.azure.com/openai/realtime?api-version=2024-10-01-preview"
    
    headers = {
        "Authorization": f"Bearer {token.token}",
        "OpenAI-Beta": "realtime=v1"
    }
    
    async with websockets.connect(ws_url, extra_headers=headers) as websocket:
        # Initialize session with Agent ID
        await websocket.send(json.dumps({
            "type": "session.update",
            "session": {
                "agent": {
                    "agent_id": AGENT_ID,
                    "project_name": AGENT_PROJECT_NAME
                }
            }
        }))
        
        # Handle bidirectional audio streaming
        await asyncio.gather(
            receive_audio_from_caller(websocket, call_connection_id),
            send_audio_to_caller(websocket, call_connection_id)
        )

Integrating with Azure AI Agent Service

The AI Agent Service provides the intelligence for our voice agent. Here’s how we connected it:

Processing Voice Input

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

async def process_voice_with_agent(audio_data, session_id):
    """Send audio directly to Voice Live API in Agent Mode"""
    
    # Using Azure Managed Identity - no API keys needed
    from azure.identity import DefaultAzureCredential
    credential = DefaultAzureCredential()
    token = credential.get_token("https://cognitiveservices.azure.com/.default")
    
    # Send audio input event to Voice Live API
    audio_event = {
        "type": "input_audio_buffer.append",
        "audio": base64.b64encode(audio_data).decode()
    }
    
    # Voice Live API will handle agent processing automatically
    # when configured with agent_id in session.update
    return audio_event

Handling Real-World Challenges

Working with preview services meant encountering numerous undocumented behaviors. Here are some key challenges we solved:

1. Audio Format Compatibility

The Voice Live API expects specific audio formats. We discovered through trial and error:

1
2
3
4
5
6
7
8
9
10
11
12
13

# Audio configuration that actually works (Voice Live API format)
AUDIO_CONFIG = {
    "format": "pcm16",  # 16-bit PCM for Voice Live API
    "sample_rate": 24000,  # 24kHz required by Voice Live
    "channels": 1  # Mono
}

# ACS requires 16kHz, so we need resampling
ACS_AUDIO_CONFIG = {
    "format": "pcm16",
    "sample_rate": 16000,  # ACS requirement
    "channels": 1
}

2. Latency Optimization

To achieve natural conversation flow, we implemented several optimizations:

1
2
3
4
5
6
7
8
9
10
11
12

# Start voice synthesis before full response is ready
async def stream_synthesize_speech(text_stream):
    """Synthesize speech in chunks for lower latency"""
    
    buffer = ""
    async for chunk in text_stream:
        buffer += chunk
        
        # Send to synthesis when we have a complete sentence
        if any(punct in buffer for punct in ['.', '!', '?']):
            await synthesize_and_send(buffer)
            buffer = ""

3. Connection Resilience

Preview services can be unstable. We added robust error handling:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

async def maintain_connection(websocket, call_id):
    """Maintain WebSocket connection with automatic reconnection"""
    
    retry_count = 0
    max_retries = 3
    
    while retry_count < max_retries:
        try:
            await websocket.ping()
            await asyncio.sleep(30)  # Ping every 30 seconds
            
        except websockets.ConnectionClosed:
            logger.warning(f"Connection lost for call {call_id}")
            retry_count += 1
            await asyncio.sleep(2 ** retry_count)  # Exponential backoff
            
            # Attempt reconnection
            websocket = await reconnect_websocket(call_id)

Deployment Considerations

When deploying this solution, we learned several important lessons:

Container Deployment

We packaged our Python application as a container for easier deployment:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

FROM python:3.11-slim

WORKDIR /app

# Install system dependencies for audio processing
RUN apt-get update && apt-get install -y \
    libopus0 \
    libopus-dev \
    && rm -rf /var/lib/apt/lists/*

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

CMD ["python", "start.py"]

Scaling Considerations

For handling multiple concurrent calls:

1. Use Azure Container Instances or App Service with autoscaling
2. Implement connection pooling for WebSocket connections
3. Monitor memory usage - audio processing can be memory-intensive

Monitoring and Debugging

Working with preview services means extensive logging is crucial:

1
2
3
4
5
6
7
8
9
10

import logging
from azure.monitor.opentelemetry import configure_azure_monitor

# Configure Azure Monitor for production debugging
configure_azure_monitor(
    connection_string=APPLICATIONINSIGHTS_CONNECTION_STRING
)

# Log all WebSocket events
logging.getLogger('websockets').setLevel(logging.DEBUG)

Lessons Learned

After weeks of development and close collaboration with Azure product teams, here are our key takeaways:

1. Preview Services Require Patience: Be prepared for undocumented features and changing APIs
2. Network Analysis is Your Friend: Tools like Wireshark helped us understand the protocol
3. Build in Resilience: Assume connections will drop and services will be intermittently unavailable
4. Start Simple: Get basic voice working before adding complex AI interactions
5. Monitor Everything: You’ll need extensive logging to debug issues in production

Get Started

Ready to build your own voice agent? Check out the complete implementation in my GitHub repository. The repository includes:

• Complete Python application code
• Deployment scripts and Docker configuration
• Environment setup instructions
• Troubleshooting guide

Remember, innovation often means venturing into undocumented territory. Don’t be afraid to experiment, reverse-engineer, and collaborate with product teams. The future of voice-based AI agents is being written right now, and you can be part of it.

References

• Azure Voice Live API Documentation
• Azure AI Agent Service Overview
• Complete Code Repository
• Azure Communication Services Documentation
• Main image generated by DALL-E

🎯 TL;DR: Retrieving TFVC Repository Structure via REST API

This post demonstrates how to programmatically enumerate TFVC repository folders using Azure DevOps Server REST APIs. Unlike Git repositories, TFVC follows a one-repository-per-project model with hierarchical folder structures starting at $/ProjectName. The solution uses the TFVC Items API with specific parameters: scopePath=$/ProjectName to target the project root, and recursionLevel=OneLevel to retrieve immediate children. The implementation handles authentication via Personal Access Tokens, filters results to show only folders (excluding the root), and includes error handling for projects without TFVC repositories or insufficient permissions.

Key technical details: PowerShell script implementation, proper API parameter usage, authentication setup, and handling edge cases like empty repositories and access permissions. Complete PowerShell script and utilities available here

Recently, I was asked an interesting question by a developer who was struggling with Azure DevOps Server APIs around fetching repository metadata for legacy TFVC structures as part of a GitHub migration from ADO Server. This was a nice little problem to solve because, let’s be honest, we don’t really deal with these legacy TFVC repositories much anymore. Most teams have migrated to Git, and the documentation around TFVC API interactions has become somewhat sparse over the years.

The challenge was straightforward but frustrating: they could retrieve project information just fine, but getting the actual TFVC folder structure within each project? That’s where things got tricky. After doing a bit of digging through the API documentation and testing different approaches, I’m happy to say that yes, it is absolutely possible to enumerate all TFVC repositories and their folder structures programmatically.

This blog post shares the solution I put together - a practical approach to retrieve TFVC repository structure using the Azure DevOps Server REST APIs. If you’re working with legacy TFVC repositories and need to interact with them programmatically, this one’s for you.

The Challenge: Understanding TFVC API Limitations

Unlike Git repositories where each project can contain multiple repos, TFVC follows a different model where each project contains exactly one TFVC repository. This fundamental difference affects how you interact with the API and retrieve repository information.

The main challenge developers face is distinguishing between project metadata and actual TFVC repository structure. When calling the standard Projects API, you receive project information but not the folder structure within the TFVC repository itself.

Common Misconceptions About TFVC APIs

Many developers make the mistake of thinking that the Projects API or the Items API with default parameters will return the TFVC folder structure. Here’s what typically happens:

What doesn’t work:

• Using only the Projects API - returns project metadata, not TFVC structure
• Calling Items API without proper scopePath parameter - returns all items across the organization
• Using the Branches API - doesn’t apply to TFVC repositories the same way as Git

The root cause: The API requires specific parameters to traverse the TFVC hierarchy correctly.

Understanding TFVC Repository Structure

In TFVC, the repository structure follows this pattern:

• Each project has one TFVC repository
• The repository root is always $/ProjectName
• Folders are organized hierarchically under this root
• You need to specify recursion levels to control how deep the API traverses

The Solution: Targeted API Calls with Proper Parameters

The key to retrieving TFVC folder structure lies in using the correct combination of API endpoints and parameters. Here’s the step-by-step approach:

Step 1: Get All Projects

First, retrieve all projects in your Azure DevOps Server instance using the Projects API.

Step 2: Query TFVC Items for Each Project

For each project, call the TFVC Items API with these specific parameters:

• scopePath=$/ProjectName - Sets the starting point to the project’s TFVC root
• recursionLevel=OneLevel - Returns immediate children only (not recursive)
• Filter results to show only folders, excluding the root itself

PowerShell Script Implementation

Here’s a complete PowerShell script that implements this solution. You can find the full script and additional TFVC utilities in my TFS/TFVC Scripts collection:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

# Azure DevOps Server configuration
$tfsUrl = "https://mytfs.com"
$PAT = "your-personal-access-token"

# Create authentication headers
$headers = @{
    Authorization = "Basic " + [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(":$PAT"))
}

# Get all projects
$projects = Invoke-RestMethod -Uri "$tfsUrl/DefaultCollection/_apis/projects?api-version=5.0" -Headers $headers

foreach ($project in $projects.value) {
    Write-Host "Project: $($project.name)"
    
    # Construct TFVC path for this project
    $tfvcPath = "`$/$($project.name)"
    $itemsUrl = "$tfsUrl/DefaultCollection/$($project.id)/_apis/tfvc/items?scopePath=$tfvcPath&recursionLevel=OneLevel&api-version=5.0"
    
    try {
        # Get TFVC items for this project
        $items = Invoke-RestMethod -Uri $itemsUrl -Headers $headers
        
        # Filter to get only folders (excluding the root itself)
        $folders = $items.value | Where-Object {
            $_.isFolder -eq $true -and
            $_.path -ne $tfvcPath
        }
        
        foreach ($folder in $folders) {
            Write-Host "  - $($folder.path)"
        }
        
        if ($folders.Count -eq 0) {
            Write-Host "  No top-level folders found"
        }
        
    } catch {
        Write-Host "  No TFVC content or access denied"
    }
}

Script Configuration and Security

Personal Access Token Setup

To use this script, you’ll need to create a Personal Access Token (PAT) with appropriate permissions:

1. Navigate to your Azure DevOps Server user settings
2. Create a new Personal Access Token
3. Grant Code (read) permissions at minimum
4. Copy the token and replace your-personal-access-token in the script

URL Configuration

Replace https://mytfs.com with your actual Azure DevOps Server URL. The format should be:

• On-premises TFS: https://your-tfs-server
• Azure DevOps Server: https://your-server-name

Understanding the API Response

The TFVC Items API returns objects with these key properties:

1
2
3
4
5
6

{
    "path": "$/ProjectName/FolderName",
    "isFolder": true,
    "version": 12345,
    "size": 0
}

Important properties:

• path: The full TFVC path to the item
• isFolder: Boolean indicating if the item is a folder
• version: The changeset number when this item was last modified

Error Handling and Edge Cases

The script includes error handling for common scenarios:

No TFVC Repository: Some projects might not have TFVC repositories initialized. The script catches these cases and displays an appropriate message.

Access Permissions: If your PAT doesn’t have sufficient permissions for a project, the API call will fail gracefully.

Empty Repositories: Projects with TFVC repositories but no folders will display “No top-level folders found.”

Advanced Customizations

Filtering Specific Projects

To target specific projects, you can filter the projects array:

1

$projects = $projects.value | Where-Object { $_.name -like "*YourFilter*" }

Deeper Recursion

To get more than just top-level folders, change the recursionLevel parameter:

• OneLevel: Immediate children only
• Full: Complete hierarchy (use with caution on large repositories)

Output Formatting

You can modify the output format to suit your needs:

1
2
3
4
5
6
7
8
9
10

# Export to CSV
$results = @()
foreach ($folder in $folders) {
    $results += [PSCustomObject]@{
        Project = $project.name
        FolderPath = $folder.path
        LastModified = $folder.version
    }
}
$results | Export-Csv -Path "tfvc-folders.csv" -NoTypeInformation

Performance Considerations

For organizations with many projects, consider implementing: