Building an AI Agent for Azure Policy Assistant - Part 1

Hello Friend,

Today, I’m going to build a quick-win solution called the AI Agent for Azure Policy Governance Assistant. This AI Agent is meant to help you rapidly identify non-compliant policies and improve governance across your Azure subscriptions or resource groups. Imagine that being able to ask the AI Agent questions like, "Which resources are non-compliant?" or "Which exemptions are about to expire within my subscription scope?" In this article, I’ll walk you through setting up the necessary resources using PowerShell scripts, and then show you how to integrate them with an AI Agent for actionable governance insights.

Prerequisites

Before you begin, please ensure you have the following prerequisites in place:

• An active Azure Subscription that you can use to create resources for Azure AI Foundry and quota for Generative AI models
• Azure CLI and/or Azure PowerShell
• A basic understanding of PowerShell scripting and Azure Resource Graph queries
• Interacting with Azure Congnitive Services APIs using PowerShell
• Azure Policy compliance data will be fetched using Azure Resource Graph queries, so you should have a basic understanding of Azure Policy and Resource Graph

Create an Azure AI Foundry Project

Let’s start by creating a new project in the Azure AI Foundry portal. If you're wondering Azure AI Foundry Hub and Project, I think there are lots of resources available on the internet. But one of the videos I can recommend is from John Savill. You can watch the video here.

Step 1. Open the Portal: Open your web browser and navigate to Azure AI Foundry. Sign in using your Azure credentials.

Step 2. Create a New Project: On the home page, click the + Create project button. In the project creation wizard, enter a suitable name for your project (for example, governance-ai-project). Review the Azure resources that will be automatically set up to support your project.

Step 3. Customize Your Hub Settings:

• Hub Name: Choose a unique name, such as governance-ai-hub.
• Subscription: Select your Azure subscription.
• Resource Group: Create a new resource group with a unique name (e.g., governance-ai-resources) or choose an existing one.
• Location: Choose a region from the following options:
  • australiaeast
  • eastus
  • eastus2
  • francecentral
  • swedencentral - I will be relying on the swedencentral region for this example.

Step 4. Connect Services:

• Azure AI Services / Azure OpenAI: Either create a new AI Services resource (e.g., governance-ai-services) or select an existing one.
• Azure AI Search: You can skip this step for now.

Step 5. Review and Create:

Click Next to review your configuration, then click Create and wait for the project to be set up. Once your project is ready, dismiss any tips and review the project details on the portal.

Deploy a Generative AI Model

When the project is set up, we can start deploying a generative AI model to help answer Azure Policy compliance questions part of the governance assistant. In this example, we will deploy the GPT-4 model.We will use the Azure AI Foundry portal to deploy the model.

1. In the left pane under My assets, select the Models + endpoints page.
2. In the Model deployments tab, click + Deploy model and choose Deploy base model.
3. Search for the gpt-4 model, select it, and confirm your choice.
4. Click Customize to set up the deployment with these settings:
   • Deployment Name: Provide a unique name (for example, pol-gpt-4o-mini).
   • Deployment Type: Standard.
   • Model Version: 0613 (or the default version).
   • Connected AI Resource: Select your Azure OpenAI resource connection.
   • Tokens per Minute Rate Limit: Set to 5K to avoid exceeding your quota.
   • Content Filter: DefaultV2.
   • Enable Dynamic Quota: Disabled.

Wait until the deployment status shows Completed.

Alternatively, you can use the following PowerShell script to deploy the model. The output will appear as shown in the screenshot from the Project page on AI Foundry. With these resources in place, we now have everything required to create the agent. Here is the PowerShell script to deploy the model:

$subscriptionId    = "SUBSCRIPTION_ID"         # It will be the subscription ID
$resourceGroupName = "RESOURCE_GROUP_NAME"     # It will be the name of the resource group
$accountName       = "COGNITIVE_SERVICE_NAME"  # It will be the name of the Cognitive Service account
$deploymentName    = "DEPLOYMENT_NAME"         # It will be the name of the deployment
$apiVersion        = "2021-04-30"              # It will be the API version


# Construct the REST API URL
$ApiUrl = "https://management.azure.com/subscriptions/$SubscriptionId/resourceGroups/$ResourceGroupName/providers/Microsoft.CognitiveServices/accounts/$AccountName/deployments/$DeploymentName`?api-version=$ApiVersion"

# Define the deployment configuration
$deploymentConfig  = @{
    sku = @{
        name     = "Standard"
        capacity = 1
    }
    properties = @{
        model = @{
            format  = "OpenAI"
            name    = "pol-gpt-4o-mini"
            version = "2024-07-18"
        }
    }
} | ConvertTo-Json -Depth 3

$AccessToken = (Get-AzAccessToken -ResourceUrl "https://management.azure.com").Token
Write-Host "Creating AI Deployment in Azure..." -ForegroundColor Cyan
$Response = Invoke-AzRestMethod -Uri $ApiUrl -Method PUT -Payload $deploymentConfig -WarningAction SilentlyContinue

# Output response
Write-Host "Deployment Created Successfully!" -ForegroundColor Green
$Response.Content | ConvertFrom-Json

tip

Don't forget to replace the placeholders with your actual values before running the script. You're free to customize the script as needed.

So far, we have set up the necessary resources to create an Azure AI Foundry Hub, Project and deployed a generative AI model to assist in answering Azure Policy compliance questions. Now, we move on to the next step—creating the AI Agent, dynamically gathering knowledge data using PowerShell scripts, and ensuring the AI Agent stays up to date with the latest Azure Policy compliance data.

Stay tuned for the next part of this series, where we will dive into the details of creating AI Agent and analyzing Azure Policy compliance data using PowerShell scripts.