We’re bringing together a comprehensive suite of content designed to guide you through the latest in AI powered application development. Our goal is to empower developers, data scientists, and business leaders with the knowledge and tools needed to leverage AI effectively. Follow along for a sneak peek at what to expect.

What We’ll Cover​

• What is Build Intelligent Apps?
• How can you participate?
• How can you skill up? (in just 30 Days)

Get Ready to Build #IntelligentApps starting September 16!​

Today, we kick-off with new content and activities for you to skill up on all things Intelligent Apps on Azure with “how-to” blogs, events, and community interactions! Read on to learn about what is coming!

Explore Our Initiatives​

We have a number of initiatives exclusively curated to build an engaging learning journey for you to skill up on relevant technologies for building intelligent apps.

• #30Days of IA - 4 themed weeks of blogs on AI led application development in Java or Python using Azure Application Platform, Azure Data Platform and Azure AI.
• Learn Live Series – 5 weekly livestream episodes on AI app development in Python or .NET.
• Demo Bytes – snackable demo playlist on building intelligent apps in Java, Python, .NET and using GitHub Copilot.
• AI Skills Challenge – show your skills by competing with peers to complete learn modules and earn a badge.

#30Days Of Intelligent Apps​

#30Days of IA is a series of “how-to” blog posts grouped into themed weeks - taking you from core concepts to end-to-end solution examples in 30 days. Each blog will provide conceptual lessons paired with exercises and resources to help you reinforce learnings and take next steps.

This series takes you through learning journey in four stages, each building on the previous week to help you skill up in a beginner-friendly way:

• Week 1: Build AI powered content generation intelligent app in Java on Azure Kubernetes Service (AKS)
• Week 2: Build AI powered content generation intelligent app in Java on Azure App Service
• Week 3: Build your own copilot with Azure AI Studio, Prompty and AKS
• Week 4: Infuse AI-assisted development tools to enhance your efforts and experience.

We will start with defining intelligent apps and then expand on how to build with cloud technologies like Azure Kubernetes Service (AKS), Azure App Service, Secure API Management, and GitHub Copilot as well as integrate AI and cloud-scale data. You will learn how to build end-to-end scenarios for real world application development based on reference architectures. Before we dive deep on intelligent apps, here is a high-level overview of the Intelligent Apps landscape on Azure for you to leverage the most comprehensive, trusted cloud to prime the customer and employee experiences.

The Azure application platform is designed to help you deliver on the promise of intelligent applications. It’s an integrated platform that encompasses developer services, application services, databases, and AI services. Leveraging Azure’s robust security capabilities and principles around governance and responsible AI, the platform supports what you already use to help you make your apps intelligent. The developer stack on Azure also integrates AI, enhancing productivity for developers’ efficiency.

There are eight key areas in which you can reinvent experiences for external customers as well as reshape internal processes to derive high productivity.

Bring your applications to a modern application platform in the cloud for AI, which leverages a cloud data platform at scale, and agile development methods with DevOps. Embrace the most effective approaches to enhance both customer and employee experiences. Azure offers the latest apps, data, AI and is the most comprehensive, trusted cloud.

To start with the basics for understanding AI app development concepts, explore #30Days of IA (2023) and #60Days of IA.

Let’s Get Started​

Now you know everything! We hope you are as excited as we are to dive into a full month of active learning and doing! Don't forget to subscribe for updates in your favorite feed reader. And, look out for our first Intelligent Apps blog.

What We’ll Cover​

• Why generative AI?
• A high-level overview of our example scenario
• An overview of the deployment options 

Why generative AI?​

Unlike traditional AI systems that follow predefined rules, generative AI models use deep learning techniques to generate outputs such as text, images, music, and even code in a way that mimics human creativity. If you’re reading this, you’re likely interested in how you can harness these capabilities to build new and innovative solutions for your organization or your customers. But what benefits does this new technology bring, and when would you use it? 

For businesses, one of the main benefits of generative AI is enhanced productivity. By automating tasks such as content creation, customer service responses, and data analysis, you can significantly reduce tasks that your teams previously had to do manually, and this automation can result in radically increased efficiency. Marketing teams, for instance, can use generative AI to draft personalized content or generate ideas for campaigns, accelerating time to market. Similarly, customer support teams can make the most of AI-powered chatbots that provide accurate responses based on large datasets, supporting faster and more consistent handling of customer requests or issues. 

In addition to the productivity benefits, intelligent apps built by using generative AI can scale to meet the needs of growing businesses, allowing these businesses to meet increased demand for personalized solutions and messaging. AI helps your business to do this without sacrificing quality or capabilities. 

In this series, we show you how you can combine multiple Azure products and services to effectively build an example generative AI application that supports marketing teams. 

The example application​

In our example scenario, we’re building a solution for the Best For You Organics Company, which is releasing an innovative, new skin-care product designed to enhance natural beauty and promote skin health. In a competitive market, it’s crucial for marketing efforts to create engaging content that captures the attention of potential customers and aids the marketing team in market penetration. 

Through this blog series, you can learn to use Azure services to build an application that supports the Best For You Organics Company marketing team, making the most of the capabilities of Azure to create exceptional customer experiences and generate engaging marketing content.  

Our solution generates the following types of content to assist the marketing team: 

• Social media posts. Announcements, product highlights, customer testimonials, and promotional offers, with the hashtags #Skincare #NaturalBeauty #BestForYouOrganics 

• Blog articles. In-depth articles on skin-care tips, the benefits of natural ingredients, and product usage guides 

• Email content. Product launches, exclusive offers, skin-care routines, and collateral 

The application is built using a three-part architecture consisting of a back end, middleware, and front end, deployed on either Azure Kubernetes Service (AKS) or Azure App Service. This architecture forms the foundation for integrating modern cloud and AI technologies, which is our focus throughout this case study. 

Application flow​

The content generation process in the application is simple yet powerful. Here's how the flow works, from the user's perspective: 

1. User input. The user provides details, such as the type of content they need—whether it's an email campaign, social media post, or blog article (steps 1-4 in the diagram below). 

2. Back-end processing. The system retrieves necessary data, such as customer demographics, product details, and relevant information stored in the back end (steps 5-6 in the diagram). 

3. Content generation. The middleware communicates with Azure OpenAI Service to generate content based on the provided input and back-end data, helping to ensure that the content is contextually relevant (steps 7-9 in the diagram). 

4. Result display. The generated content is then displayed on the ReactJS front end, where the user can review, copy, and implement it directly into their marketing campaigns (step 10 in diagram). 

Let’s now look at the data flow for this application flow to understand how Azure OpenAI can be used in a Retrieval-Augmented Generation (RAG) architecture pattern. 

This flow allows for fast and efficient content creation while tailoring the output to specific campaign needs, all powered by AI. 

GitHub repository​

The project’s full source code can be found in the GitHub repository for the Java AI-based content generator.

Technologies used ​

Our application makes the most of several cutting-edge technologies, each serving a critical role in the system:   

• Azure OpenAI Service. At the core of content generation is Azure OpenAI, which uses GPT models to create coherent and relevant content based on user input. The Chat Completion API (GPT-4o) generates the text, while the embedding model helps ensure that data retrieved for content is semantically relevant to the user's needs. 
• Azure Key Vault. For secure storage of secrets such as API keys, credentials, and sensitive configuration data, Key Vault is integrated into the system. This helps ensure that all sensitive data is stored securely and retrieved only when needed. 
• Microsoft Entra ID (formerly Azure Active Directory). Microsoft Entra ID is used for authentication and identity management, supporting secure access to the back end and to APIs. It helps effectively manage user roles and access permissions. 
• Managed identity. The application uses managed identity to securely access Azure resources without hard-coding credentials. This simplifies the authentication process when interacting with services like Key Vault and Azure Cosmos DB. 
• Azure App Service and Azure Kubernetes Service (AKS).The application supports two deployment models: App Service and AKS. Both provide scalable infrastructure but differ in flexibility and complexity (more on this in the "Choosing the right deployment technology for you" section). 
• Azure API Management. API Management manages the API gateway, controlling access between the front end, middleware, and back end. With features like Cross-Origin Resource Sharing (CORS), throttling, rate limiting, and authentication, API Management supports efficient communication and security across the system.

Choosing the right deployment technology for you​

The application can be deployed by using either Azure App Service or Azure Kubernetes Service (AKS), depending on your business needs. Both options provide advantages, but the choice often depends on the application’s complexity and scale.

Azure App Service​

• Simplicity. App Service is a fully managed platform as a service (PaaS) that abstracts the underlying infrastructure. This allows for quick deployments with less overhead. 

• Cost-effective. For small to medium-sized applications, App Service can reduce operational costs by automating much of the infrastructure management. 

• Use case. The service is ideal for businesses looking for rapid deployment without the need for extensive control over infrastructure. It works well for straightforward applications with predictable workloads.

Azure Kubernetes Service (AKS)​

• Flexibility. AKS offers more control over infrastructure, allowing for fine-tuned customization of containerized microservices. 

• Scalability. The service is ideal for applications with large-scale operations or high customization requirements. It allows for horizontal scaling and integration with multiple cloud-native tools. 

• Use case. AKS is best suited for enterprise-level applications or businesses requiring multi-container orchestration and complex workloads.

Both options are scalable and robust, and businesses can select the one that aligns best with their operational needs and scale of application. 

Summary​

In today’s post, we introduced you to the example app that we’ll build in this series—the Best For You Organics content generation app. We discussed our goals, overall approach, and the architecture that we’re using to create this app.

In our next post, we cover the environment you’ll set up to prepare for the development of the application. Learn how to configure your local environment; set up Java, Spring Boot, and ReactJS; and integrate with Azure services, like Azure OpenAI, Key Vault, and more.

Each subsequent post will dive deeper into the specific technologies that power this AI-driven application. Stay tuned as we explore everything from back-end data management to deploying on AKS or App Service.

Additional resources​

• To help you build the skills you need to create intelligent apps by using Azure OpenAI, complete the Get started with Azure OpenAI Service module. 

• Register for live learning or a demo playlist on Build Intelligent Apps on Azure.

What we cover:​

• Installing required development tools and services on your local machine
• Configuring the Azure Cosmos DB that the back-end application uses
• Setting up the Azure Key Vault that will store application secrets
• Creating a Microsoft Entra ID app registration to enable Microsoft Account sign-in for our app

Introduction​

As we introduced in the previous post, this series walks you through building an example AI-powered content generation app for Best For You Organics Company. This app will help our fictitious company support a new skin-care product by generating a variety of marketing materials, including social media posts, blog articles, and email campaigns. Tailored specifically for the skin-care industry, the app will allow them to efficiently create and customize content for different platforms, enabling them to more effectively reach their target audience.

In this blog, we take you through setting up your local development environment and configuring several essential services in the Azure portal. You install tools, such as Git, Java, and Node.js, and set up Azure resources, such as Azure Cosmos DB and Azure Key Vault.

Local machine setup​

Before diving into the Azure components, let’s ensure that your local development machine is equipped with all the tools you need to build the app.

Step 1. Install Git.​

Git is essential for version control and managing your project’s source code.

1. Download Git. Go to Install and set up Git, and download the installer for your operating system.
2. Installation. Run the installer. Unless you have specific preferences, use the default settings.
3. Verify installation. Open your terminal and run:

   git --version

Step 2. Install Java (17 and 21).​

Your Spring Boot back end requires both Java 17 and 21.

1. Download. Obtain both versions from Download the Microsoft Build of OpenJDK.
2. Configure. You can manage multiple Java versions using update alternatives (Linux) or SDKMAN (for Linux/Mac). For Windows, use Java Development Kit (JDK) tools, like JDKTool.
3. Verify installation. Open your terminal and run:

   java -version

Step 3. Install the Azure CLI.​

The Azure CLI is essential for managing your Azure resources from the terminal.

1. Download the Azure CLI. Follow the instructions in How to install the Azure CLI.
2. Sign in to Azure. After installing the Azure CLI, sign in. Open your terminal and run:

   az login

Step 4. Install Node.js and set up React.​

We use Node.js for managing front-end dependencies and running the React development server.

1. Install Node.js. Go to Node.js, and download the appropriate version for your system.
2. Verify installation. Confirm that Node.js is correctly installed. Open your terminal and run:

   node -v && npm -v

Step 5. Install an integrated development environment.​

You can use your preferred integrated development environment (IDE), but we recommend Visual Studio Code and IntelliJ IDEA for Java and JavaScript development.

• Install Visual Studio Code.
  • or
• Install IntelliJ IDEA.

Both IDEs support Azure integrations through plugins for managing Azure resources and services.

Azure setup​

Now that your local environment is ready, let’s configure the Azure resources you need to build and run the app.

Step 1. Set up Azure basics.​

Before you start configuring resources, choose the Azure account you want to use. If you want to use the Azure CLI to configure Azure resources, perform a few configuration steps:

• Create a new account—unless you already have one (optional).
  If you don’t have an existing Azure account that you want to use when creating Azure resources, create a new one. Go to the Azure portal and sign up. Choose the subscription plan that fits your business needs.
• Configure the Azure CLI, if you want use it (optional).
  The steps throughout this series include instructions for using either the browser-based Azure portal or the command-line Azure CLI tool. If you want to use the Azure CLI but don’t already have it installed, follow these steps:
  • Follow the instructions provided at Install Azure CLI.
  • Verify the installation. Open your terminal or command prompt and run:

az --version

After installing and verifying the Azure CLI, follow these steps to sign in, list your subscriptions, and select a subscription:

1. Sign in to Azure. Run the following command:

az login

This opens a web browser for you to authenticate your Azure account. After successful authentication, the CLI automatically lists your subscriptions and asks you to select one by number.

2. Select a subscription. After you sign in, a prompt is displayed.

   

3. Type the number corresponding to the subscription you want to select, and press Enter. After you select the subscription, it is set as the default for your current session.

Step 2. Create Azure Cosmos DB resource (MongoDB API).​

Cosmos DB will serve as the database for storing your application data, such as customer information and product details.

Azure portal instructions​

1. Go to the Azure Portal and search for "Cosmos DB".

2. Click on "Create" and select MongoDB API.

3. You will see two options: Request unit (RU) database account and vCore cluster (Recommended). Choose Request unit (RU) database account.

4. Configure the Request Units per second (RU/s) based on your application's scale (e.g., 400 RU/s as a starting point for small applications). The RU/s determines how many operations Cosmos DB can handle per second, so adjust it according to your expected load and usage.

5. Proceed with the rest of the setup, such as specifying the resource group, region, and database name.

6. Once the Cosmos DB account is created, you can add a database and collections to organize your data.

   

CLI instructions​

1. Create a resource group (if you don’t already have one) by running the following command:

   az group create --name <resource-group-name> --location <location>

2. Create an Azure Cosmos DB account with MongoDB API.

   Use the following command to create an Azure Cosmos DB account with the MongoDB API, and specify throughput in Request Units per second (RU/s):

   az cosmosdb create \
     --name <cosmos-account-name> \
     --resource-group <resource-group-name> \
     --kind MongoDB \
     --locations regionName=<location> failoverPriority=0 isZoneRedundant=False \
     --default-consistency-level Eventual \
     --enable-automatic-failover false

3. Create a MongoDB database with a specific throughput.

   After creating the Azure Cosmos DB account, create a MongoDB database and set the throughput (RU/s):

   az cosmosdb mongodb database create \
     --account-name <cosmos-account-name> \
     --name <database-name> \
     --resource-group <resource-group-name>

Step 3. Create Azure Storage account.​

Our Azure Storage account is used to store any static files (for example, product brochures or images).

Azure portal instructions​

1. In the Azure portal, search for storage account and create a new one.
2. Choose Blob Storage for storing documents and media.
3. Complete the resource creation wizard, and select Create to deploy the storage account.

CLI instructions​

1. Determine whether resource group exists (optional).

   Use the following command:

   az group show --name <resource-group-name>

• If the resource group exists, it returns the details.
• If it doesn't exist, it throws an error.

2. Create the resource group (optional).

   If the resource group doesn't exist, create it using the following command, replacing <resource-group-name> with your desired name for the resource group and <location> with the Azure region where you want to deploy the resource, (for example, eastus, westus):

   az group create --name <resource-group-name> --location <location>

3. Create the storage account.

   Use the following command to create the Azure Storage account, replacing <storage-account-name> with a globally unique name for your storage account; replacing <resource-group-name> with the name of the resource group you created or chose; and replacing <location> with the Azure region where you want to deploy the storage account (for example, eastus, westeurope):

   az storage account create \
     --name <storage-account-name> \
     --resource-group <resource-group-name> \
     --location <location> \
     --sku Standard_LRS \
     --kind StorageV2

4. Verify storage account creation.

   To ensure that the storage account was created successfully, list all storage accounts in the resource group:

   az storage account list --resource-group <resource-group-name> --output table

Step 4. Set up Azure Key Vault.​

To securely store our secrets, like API keys and connection strings, we use Azure Key Vault.

Azure portal instructions​

1. Create a key vault. In the portal, search for key vault and create one.

2. Register an application. In Microsoft Entra ID, register your application to grant it access to the key vault.

3. Assign managed identity. Enable Managed Identity to allow secure access from your application.

4. Complete the resource creation wizard, and select Create to deploy the key vault.

   

CLI instructions​

1. Create a resource group (optional).

   If you need to create a resource group (like we did in the Azure Cosmos DB steps), use the following command, replacing <resource-group-name> with your desired name and <location> with the region (for example, eastus, westeurope):

   az group create --name <resource-group-name> --location <location>

2. Create the Azure key vault.

   Now, use the following command to create the Azure key vault, replacing <key-vault-name> with a globally unique name for the key vault; replacing <resource-group-name> with the name of the resource group created or selected in the previous step; and replacing <location> with your preferred Azure region. Note that “--sku standard” is a common pricing tier. You can also use premium if you need additional features like Hardware Security Module (HSM) support.

   az keyvault create \
     --name <key-vault-name> \
     --resource-group <resource-group-name> \
     --location <location> \
     --sku standard

3. Verify key vault creation.

   To verify that the key vault was created, you can list all key vaults in the specified resource group:

   az keyvault list --resource-group <resource-group-name> --output table

Step 5. Use Microsoft Entra ID to create an app registration for Microsoft account sign-in.​

To support multi-user access, including sign-in from any Microsoft account, register your application in Microsoft Entra ID.

Azure portal instructions​

1. Navigate to Microsoft Entra ID.

   • In the Azure portal, search for Microsoft Entra ID, and select Manage → App registrations.

2. Register a new application.

   1. Select New registration.
   2. Name. Enter a name for your application (for example, ContentGenApp).
   3. Supported account types. Select Accounts in any organizational directory (Any Microsoft Entra ID tenant - Multitenant) and personal Microsoft accounts (e.g. Skype, Xbox).
   4. Redirect URI. Set this to the front end's callback URL.
      • For local development (React app running locally), use http://localhost:3000.
      • For production (when deployed on Azure Kubernetes Service (AKS) or Azure App Service), add the URL for the App Service or AKS deployment you plan to create for this app (We’ll create these in our next post). For example, https://your-app.azurewebsites.net for App Service or https://your-aks-domain.com for AKS.
      • If you plan to use a custom domain for your application, add that domain here (for example, https://your-custom-domain.com).

   Important: Ensure that all potential URLs (local, deployed, and custom domain) are listed as redirect URIs.

   

3. Configure API permissions.

   1. Go to the API permissions tab, and select Add a permission.
   2. Choose Microsoft Graph → Delegated permissions, and then select the following permissions:
      • User.Read. Allows users to sign in and read their profile.
   3. Grant admin consent. After permissions are added, select Grant admin consent for your organization.

   

4. Configure authentication settings.

   1. Go to the Authentication tab, and add your redirect URIs (both local and production URLs).
   2. Enable ID tokens to authenticate users.

   

This revision integrates Microsoft Entra ID and highlights the navigation flow for app registration through the Manage section. It also ensures that you're set up for both local development and production environments using AKS, App Service, or a custom domain.

CLI instructions​

You’ll use the following script when performing these instructions.

Shell script: create-azure-ad-app.sh

#!/bin/bash

# Function to display help
function display_help() {
  echo "Usage: $0 --name <App Name> --redirect-uris <Redirect URI 1> <Redirect URI 2> ... [options]"
  echo
  echo "Options:"
  echo "  --name              The display name of the Azure AD App."
  echo "  --redirect-uris     Space-separated list of redirect URIs to configure."
  echo "  --help              Display this help message and exit."
  echo
  exit 0
}

# Function to check if the user is logged in
function check_azure_login() {
  echo "Checking Azure CLI login status..."
  az account show &> /dev/null
  if [ $? -ne 0 ]; then
    echo "You are not logged in. Logging in now..."
    az login
    if [ $? -ne 0 ]; then
      echo "Azure login failed. Exiting script."
      exit 1
    fi
  else
    echo "You are already logged in."
  fi
}

# Parse command line arguments
APP_NAME=""
REDIRECT_URIS=()

while [[ $# -gt 0 ]]; do
  case "$1" in
    --name)
      APP_NAME="$2"
      shift 2
      ;;
    --redirect-uris)
      shift
      while [[ $# -gt 0 && ! "$1" =~ ^-- ]]; do
        REDIRECT_URIS+=("$1")
        shift
      done
      ;;
    --help)
      display_help
      ;;
    *)
      echo "Unknown option: $1"
      display_help
      ;;
  esac
done

# Ensure required arguments are provided
if [[ -z "$APP_NAME" || ${#REDIRECT_URIS[@]} -eq 0 ]]; then
  echo "Error: Both --name and --redirect-uris are required."
  display_help
fi

# Step 0: Check if logged in
check_azure_login

API_PERMISSION="e1fe6dd8-ba31-4d61-89e7-88639da4683d"  # User.Read permission ID
GRAPH_API="00000003-0000-0000-c000-000000000000"        # Microsoft Graph API ID
SIGN_IN_AUDIENCE="AzureADandPersonalMicrosoftAccount"    # Supported account types

# Define the required resource accesses in JSON format
REQUIRED_RESOURCE_ACCESS=$(cat <<EOF
[
  {
    "resourceAppId": "$GRAPH_API",
    "resourceAccess": [
      {
        "id": "$API_PERMISSION",
        "type": "Scope"
      }
    ]
  }
]
EOF
)

# Step 1: Check if the app already exists
APP_ID=$(az ad app list --filter "displayName eq '$APP_NAME'" --query "[0].appId" -o tsv)

if [ -z "$APP_ID" ]; then
  echo "App doesn't exist. Creating the app registration..."

  # Step 2: Create the app registration
  APP_ID=$(az ad app create \
    --display-name "$APP_NAME" \
    --required-resource-accesses "$REQUIRED_RESOURCE_ACCESS" \
    --web-redirect-uris "${REDIRECT_URIS[@]}" \
    --enable-id-token-issuance true \
    --enable-access-token-issuance true \
    --sign-in-audience "$SIGN_IN_AUDIENCE" \
    --query appId -o tsv)

  if [ -z "$APP_ID" ]; then
    echo "App creation failed. Exiting script."
    exit 1
  fi

  echo "App created with ID: $APP_ID"

else
  echo "App already exists with ID: $APP_ID. Updating the redirect URIs..."

  # Step 3: Update the redirect URIs
  az ad app update --id "$APP_ID" \
    --web-redirect-uris "${REDIRECT_URIS[@]}"

  if [ $? -ne 0 ]; then
    echo "Failed to update redirect URIs. Exiting script."
    exit 1
  fi

  echo "Redirect URIs updated for app ID: $APP_ID"
fi

# Display the app details
echo "Here are the details of the app:"
az ad app show --id "$APP_ID"

Run the script:

1. Save the script. Save the preceding script as create-azure-ad-app.sh.
2. Make the script executable:

chmod +x create-azure-ad-app.sh

3. Run the script.

./create-azure-ad-app.sh --name <YOUR_APP_NAME> --redirect-uris http://localhost:3000 https://your-custom-domain.com

Summary​

In this blog, we’ve covered setting up your local development environment and creating the Azure-based database, storage, and secret management services you’ll use when running an AI powered content generation sample application in Java. However, there’s still one critical piece of infrastructure to provision before you can start building and deploying the app.

In the next blog, we’ll cover how to set up and configure Azure OpenAI Service to power this sample app.

Additional resources​

• Register for Demo Bytes, a snackable playlist of demos designed to help you skill up on AI powered app development.

Part 2. Preparing the Azure OpenAI Service resource​

Learn how to create and configure the Azure OpenAI Service resource that we’ll use in our example app.

What we cover:​

• Creating an Azure OpenAI instance
• Setting up Azure API Management
• Exploring the options for creating our app hosting environments on Azure

Introduction​

In our previous post, we discussed setting up your local development environment and creating the Azure-based database, storage, and secret management services that you’ll use when running our application. In this post, we set up the Azure OpenAI instance that provides the core of our content generation capabilities. After you’ve created the Azure OpenAI instance, we’ll finish our preparation tasks by setting up Azure API Management and then finally creating our preferred app hosting resources (Azure Kubernetes Service [AKS] or Azure App Service).

Step 1. Set up Azure OpenAI​

Create two Azure OpenAI deployments—one for GPT-4o (chat completion) and another for text embedding.

1. Create a new Azure OpenAI resource. Go to the Azure portal, and search for Azure OpenAI.
2. Deploy two separate instances:
   • Chat completion (GPT-4o) for generating content.
   • Embedding model for semantic search functionality.

Create the Azure OpenAI resource​

Azure portal instructions​

1. Use the Azure portal to deploy Azure OpenAI.

   

2. After provisioning Azure, open Azure OpenAI and go to Azure OpenAI Studio.

   

3. Go to Deployments, and then deploy the models.

   

4. Search GPT-4o for chat completion. Set tokens-per-minute rate limit (depending on response length), and deploy.

   

   Make sure to save the Target URI and Key for future use.

   

   Deploy the text-embedding-3-small model to embed the data in the same way.

   

CLI instructions​

Use the following script when performing these instructions:

Shell script: create-openai-resource.sh

#!/bin/bash

# Function to display help
function display_help() {
  echo "Usage: $0 --name <OpenAI Account Name> --resource-group <Resource Group Name> --location <Location> [--help]"
  echo
  echo "Options:"
  echo "  --name              The display name of the Azure OpenAI account."
  echo "  --resource-group    The name of the Azure resource group."
  echo "  --location          The Azure region/location for the resource. (e.g., eastus, westus)"
  echo "  --help              Display this help message and exit."
  echo
  exit 0
}

# Function to check if the user is logged in
function check_azure_login() {
  echo "Checking Azure CLI login status..."
  az account show &> /dev/null
  if [ $? -ne 0 ]; then
    echo "You are not logged in. Logging in now..."
    az login
    if [ $? -ne 0 ]; then
      echo "Azure login failed. Exiting script."
      exit 1
    fi
  else
    echo "You are already logged in."
  fi
}

# Parse command line arguments
OPENAI_ACCOUNT_NAME=""
RESOURCE_GROUP=""
LOCATION=""

while [[ $# -gt 0 ]]; do
  case "$1" in
    --name)
      OPENAI_ACCOUNT_NAME="$2"
      shift 2
      ;;
    --resource-group)
      RESOURCE_GROUP="$2"
      shift 2
      ;;
    --location)
      LOCATION="$2"
      shift 2
      ;;
    --help)
      display_help
      ;;
    *)
      echo "Unknown option: $1"
      display_help
      ;;
  esac
done

# Ensure required arguments are provided
if [[ -z "$OPENAI_ACCOUNT_NAME" || -z "$RESOURCE_GROUP" || -z "$LOCATION" ]]; then
  echo "Error: --name, --resource-group, and --location are required."
  display_help
fi

# Step 0: Check if logged in
check_azure_login

# Step 1: Create resource group (if not exists)
echo "Creating resource group $RESOURCE_GROUP (if it doesn't already exist)..."
az group create --name "$RESOURCE_GROUP" --location "$LOCATION"

# Step 2: Create OpenAI resource
echo "Creating Azure OpenAI resource..."
az cognitiveservices account create \
  --name "$OPENAI_ACCOUNT_NAME" \
  --resource-group "$RESOURCE_GROUP" \
  --kind OpenAI \
  --sku S0 \
  --location "$LOCATION" \
  --yes

# Step 3: Deploy GPT-4o model with increased token limit (10K capacity)
echo "Deploying GPT-4o model..."
az cognitiveservices account deployment create \
  --name "$OPENAI_ACCOUNT_NAME" \
  --resource-group "$RESOURCE_GROUP" \
  --deployment-name "gpt-4o-deployment" \
  --model-name "gpt-4o" \
  --model-version "2024-05-13" \
  --model-format OpenAI \
  --sku-capacity 10 \
  --sku-name "Standard"

# Step 4: Deploy text-embedding-3-small model
echo "Deploying text-embedding-3-small model..."
az cognitiveservices account deployment create \
  --name "$OPENAI_ACCOUNT_NAME" \
  --resource-group "$RESOURCE_GROUP" \
  --deployment-name "text-embedding-3-small-deployment" \
  --model-name "text-embedding-3-small" \
  --model-version "1" \
  --model-format OpenAI \
  --sku-name "Standard" \
  --sku-capacity 120 \

# Verify deployments
echo "Listing deployed models..."
az cognitiveservices account deployment list \
  --name "$OPENAI_ACCOUNT_NAME" \
  --resource-group "$RESOURCE_GROUP"

Run the script.

Save the script. Save the script as create-openai-resource.sh.

Make the script executable:

chmod +x create-openai-resource.sh 

Run the script:

./create-openai-resource.sh --name <NAME> --resource-group <RESOURCE_GROUP_NAME> --location <LOCATION>

Step 2. Set up Azure API Management​

API Management serves as the gateway between your front-end, middleware, and back-end services.

1. Create an API Management instance. Search for API Management, and create a new instance.
2. Configure two services (back-end service API Management and middleware service API Management):
   • Back end for data management (Azure Cosmos DB, storage account)
   • Middleware for Azure OpenAI Service

Azure portal instructions​

1. In the Azure portal, search for API Management.
2. Choose Create, and complete the resource creation wizard.
3. Select Create to deploy the API Management service.

CLI instructions​

1. Create a resource group (optional).

   If you need to create a resource group, use the following command, replacing <resource-group-name> with your desired name and <location> with the region (for example, eastus, westeurope):

az group create --name <resource-group-name> --location <location>

2. Create the API Management service using the following command:

az apim create \
  --resource-group <your-resource-group-name> \
  --name <your-apim-name> \
  --publisher-email <your-email-address> \
  --publisher-name <your-publisher-name> \
  --location <your-location> \
  --sku-name <sku-type>

Parameters

• --resource-group: The name of the resource group where the API Management instance will be created.
• --name: The name of your API Management instance.
• --publisher-email: The email address of the API publisher.
• --publisher-name: The name of the API publisher.
• --location: The region where your API Management service will be deployed.
• --sku-name: The pricing tier for the API Management instance (Developer, Basic, Standard, or Premium). Default value is Developer.

SKUs

• Developer: Best for non-production use.
• Basic: Entry-level production tier.
• Standard: Mid-range production tier.
• Premium: Enterprise-level tier with additional features like multiregion support.

Step 3. Prepare your app hosting environments—AKS or App Service​

With the Azure OpenAI instance in place and our API Management configuration created, we have one more piece to set up before we can start building and deploying our app. We need to determine what type of service we’re going to use in Azure for hosting the front-end, middleware, and back-end app code.

Choose your deployment technology​

Depending on your preferences or business requirements, you can choose either Azure Kubernetes Service (AKS) or Azure App Service when deploying your app to Azure.

To recap, here’s a quick high-level overview of the AKS versus App Service.

Both are robust options, and you can select the one that best fits your business needs.

Use the Azure App Service path​

Create a separate App Service instance for the front-end, middleware, and back-end services.

Azure portal instructions​

Create a web app. In the Azure portal, search for App Service and create a new app.

In the first section of the resource creation wizard, select the correct Runtime stack:

• For the back-end service, select Java 21.
• For the middleware service, select Java 17.
• For the front-end service, select Node 20 LTS.

Complete the resource creation wizard, and select Create to deploy the App Service instance.

CLI instructions​

1. Create a resource group (optional).

If you need to create a resource group, use the following command, replacing <resource-group-name> with your desired name and <location> with the region (for example, eastus, westeurope):

az group create --name <resource-group-name> --location <location>

2. Create an App Service plan.

First, create an App Service plan, which defines the region, pricing tier, and scaling options of your App Service instance:

az appservice plan create \
  --name <your-app-service-plan-name> \
  --resource-group <your-resource-group-name> \
  --sku B1 \
  --is-linux 

• --sku B1: Specifies the pricing tier (Basic B1).
• --is-linux: Specifies that the app will run on Linux.

3. Create a web app.

   To create a web app, use the following command:

az webapp create \
  --resource-group <your-resource-group-name> \
  --plan <your-app-service-plan-name> \
  --name <your-backend-app-service-name> \
  --runtime "JAVA:21-java21"

Runtime specifications

• Back end: --runtime "JAVA:21-java21": Specifies the Java 21 runtime.
• Middleware: --runtime "JAVA:17-java17": Specifies the Java 17 runtime.
• Front end: --runtime "NODE:22-lts": Specifies the Node.js 22 runtime.

Use these runtime values in your az webapp create commands when setting up the respective services for back-end, middleware, and front-end applications.

Use the Azure Kubernetes Service (AKS) path​

To deploy your Kubernetes-based resources, you need to perform two actions:

1. Set up a container registry to host your application’s container images.
2. Create a Kubernetes cluster.

Azure portal instructions​

Create a container registry.

1. In the Azure portal, search for Kubernetes Service and create a new AKS cluster.
2. Complete the resource creation wizard, and select Create to deploy the container registry.

Create a Kubernetes cluster.

1. In the Azure portal, search for Kubernetes Service and create a new Kubernetes cluster.
2. Complete the resource creation wizard, and select Create to deploy the cluster.

CLI instructions​

1. Create a resource group (optional).

   If you need to create a resource group, use the following command, replacing <resource-group-name> with your desired name and <location> with the region (for example, eastus, westeurope):

az group create --name <resource-group-name> --location <location> 

2. Create an Azure container registry.

  az acr create \
  --resource-group <your-resource-group-name> \
  --name <your-acr-name> \
  --sku Standard \
  --admin-enabled true 

Replace <your-acr-name> with the name you want for your container registry. The --sku Standard specifies the Azure Container Registry tier, and --admin-enabled true enables admin access.

Command parameters

• --sku Standard: Specifies the pricing tier for the Azure container registry. (Standard is a good default option.)

• --admin-enabled true: Enables admin access for easier management of the Azure container registry.

  

3. Create an AKS cluster:

az aks create \
  --resource-group <your-resource-group-name> \
  --name <your-frontend-aks-cluster-name> \
  --node-count 3 \
  --enable-addons monitoring \
  --generate-ssh-keys \
  --node-vm-size Standard_DS2_v2

Summary​

In this post, we walked through creating the Azure OpenAI instance that will serve as the core of our app’s content generation capabilities. We finished up our setup tasks by configuring Azure API Management, and then we created the hosting resources we need to deploy our app to Azure.

In our next post, we’ll discuss how you can start building and running the app on your local dev machine.

Additional resources​

• Register for Demo Bytes, a snackable playlist of demos designed to help you skill up on AI powered app development.

Learn how to download our example app code, and prepare to run it in your local dev environment.

What we cover:​

• Downloading the app code from GitHub to your local machine
• Setting up app secrets and connecting them to Azure Key Vault

Introduction​

In the previous topic, we finished setting up the local tools and Azure services we need to run our app in your local development environment. In this blog, we walk you through getting the app code set up on your dev machine. This involves cloning the app code’s GitHub repository to your local machine. In addition, we’ll securely configure the app’s connection strings and other secrets using a key vault that will be used in both your local environment and in the Azure Kubernetes Service (AKS) or Azure App Service hosting environments that you’ll use when deploying to Azure.

System Architecture Diagram – Local Setup​

Before diving into the setup and configuration details, let's take a look at the overall architecture of the Content Generation System. The architecture diagram below illustrates the key components and their interactions, including the backend, middleware, and frontend services, as well as how they integrate with Azure Key Vault and other Azure services.

In the diagram: 

• Frontend Service: Users interact with this component through the web interface. It communicates with the backend and middleware services.
• Middleware Service: Acts as an intermediary, handling requests between the frontend and backend services.
• Backend Service: Contains the core logic and data access layers for the content generation functionality.
• Azure Key Vault: Stores and manages secrets used by the services for secure access to configuration and credentials.
• Azure Cosmos DB: Database service used for storing and retrieving content data.
• Azure Storage: Manages storage of files and other data used by the application.
• Azure OpenAI: Provides AI-based services for content generation and embedding.

Now that you have a high-level understanding of the architecture, let’s proceed with the steps to set up and run the application locally.

Step 1. Clone the GitHub repository​

Begin by cloning the repository for the Java AI-Based Content Generator.

git clone https://github.com/Azure-Samples/Java-AI-Based-Content-Generator

Step 2: Set Up Azure Key Vault Secrets and User Identity (Local Setup)​

1. Set Key Vault URL and Authentication Environment Variables​

Before running the services, you must set the environment variables required to access Azure Key Vault securely.

• Azure Key Vault URL:
  • Set this in your environment as AZURE_KEYVAULT_URI using the URL of your Key Vault.
• Configure Spring Cloud Azure:
  • Ensure that you have the Spring Cloud Azure dependencies added to your project. Spring Cloud Azure provides integration with Azure services and supports using different credential types.
  • By default, Spring Cloud Azure will attempt to use multiple credential builders to authenticate, including:
    • EnvironmentCredential
    • WorkloadIdentityCredential
    • ManagedIdentityCredential
    • SharedTokenCacheCredential
    • IntelliJCredential
    • VSCodeCredential
    • AzureCliCredential
  • For local development, the application will use AzureCliCredential if no other credentials are configured. This ensures that your local environment can authenticate and interact with Azure services seamlessly.

For more details on configuring and using Spring Cloud Azure with Key Vault, refer to the Spring Cloud Azure authentication documentation.

2. Assign Key Vault Administrator Role​

• Go to the Azure Portal.
• In the left-hand menu, select Resource groups and choose the resource group that contains your Key Vault.
• Select your Key Vault from the list of resources.
• In the Key Vault menu, navigate to Access control (IAM).
• Click on the + Add button at the top and select Add role assignment.
• In the Role dropdown, search for and select Key Vault Administrator.
• In the Assign access to field, choose User, group, or service principal.
• Search for your user (or the user who will manage secrets) and click Select.
• Finally, click Review + assign.

This role gives the user permission to manage secrets in the Key Vault.

Configure User Identity for Local Development​

For local development, you can use Azure CLI credentials to authenticate with Azure Key Vault. Follow these steps:

1. Install Azure CLI: Ensure that the Azure CLI is installed on your local machine.

2. Open your command-line interface (CLI) and run the following command to login:

   az login
   

3. Follow the prompts to complete the authentication process. This command authenticates your local CLI session and stores the necessary credentials.

4. Access Key Vault Secrets: After running az login, your application will use the Azure CLI session to authenticate with Azure Key Vault and retrieve secrets during local development.

Summary​

In this blog, we cloned our example app code onto your local development machine, and we set up the required environment variables needed to run the app. In addition, we configured the Azure Key Vault settings to securely store the secrets, such as connection strings and keys, that you’ll use when running the app.  

In our next blog, we’ll walk through adding those secrets to your key vault and then we’ll build and run the app for the first time in your local development environment.

Learn how to make the final configuration updates and run our example application for the first time.

What we cover:​

• Adding required connection strings and other secrets to Azure Key Vault
• Starting the back-end service locally
• Starting the middleware service locally
• Starting the front-end service locally
• Managing application secrets when transitioning from local to Azure Kubernetes Service (AKS) or Azure App Service deployments 

Introduction​

Throughout the last three days of topics, we’ve been working on getting your local dev environment prepared to run our example app and on creating and configuring the Azure services that the app depends on. In our previous topic, we completed this prep work by cloning our example app code from GitHub and then getting our Azure Key Vault configured.

Now we’re ready to run the application for the first time in your local development environment. In today’s post, we walk through adding your app connection secrets to the key vault and finally running the app’s back-end, middleware, and front-end service components.

Step 1. Add key vault secrets.​

After access is granted, add the following secrets into Azure Key Vault to securely configure the back-end and middleware services.

Back-end service secret keys:

• AzureCosmosConnectionString
• MongoDBDatabaseName
• AzureStorageConnectionString
• StorageContainerName
• MiddlewareServiceBaseUrl
• MiddlewareServiceProductEmbeddingEndpoint
• MiddlewareServiceAccessKey

Middleware service secret keys:

• BackendServiceBaseUrl
• BackendServiceProductEndpoint
• BackendServiceSimilarProductEndpoint
• BackendServiceAccessKey
• AzureOpenAiEndpointUrl
• AzureOpenAiAccessKey
• AzureOpenAiEmbeddingEndpointUrl

CLI instructions ​

Use the following script when performing these instructions: 

Shell script: add-secrets-to-keyvault.sh

#!/bin/bash

# Function to display usage information
usage() {
  echo "Usage: $0 --keyvault-name <keyvault-name> --resource-group <resource-group> --cosmosdb-name <cosmos-mongo> --mongodb-name <dbname> --storage <sa-name> --storage-container <container-name> --openai-name <openai-details> --openai-completion-model <ai-service-completion-deployment-name> --openai-embedding-model <embedding-deployment-name> [options]"
  echo
  echo "Options:"
  echo "  --keyvault-name                     Azure Key Vault name (required)"
  echo "  --resource-group                    Azure resource group name (required)"
  echo "  --cosmosdb-name                     CosmosDB name (required)"
  echo "  --mongodb-name                      MongoDB database name (default: ContentGenerator)"
  echo "  --storage                           Azure storage account name (required)"
  echo "  --storage-container                 Storage container name (default: contentgen)"
  echo "  --openai-name                       Azure OpenAI service name (required)"
  echo "  --openai-completion-model           OpenAI chat completion deployment name (default: gpt-4o)"
  echo "  --openai-embedding-model            OpenAI embedding deployment name (default: text-embedding-3-small)"
  echo "  --middleware-base-url               Middleware service base URL (default: http://localhost:8081)"
  echo "  --middleware-product-embedding-endpoint Middleware product embedding endpoint (default: /api/v1/generate/embeddings)"
  echo "  --backend-base-url                  Backend service base URL (default: http://localhost:8080)"
  echo "  --backend-product-endpoint          Backend product endpoint (default: /api/v1/products)"
  echo "  --backend-similar-product-endpoint  Backend similar product endpoint (default: /api/v1/products/similar)"
  echo "  -h, --help                          Show this help message and exit"
  exit 1
}

# Check for help flag
if [[ "$1" == "-h" || "$1" == "--help" ]]; then
  usage
fi

# Function to set or update secrets in Azure Key Vault
set_or_update_secret() {
  local secret_name="$1"
  local secret_value="$2"

  echo "Checking secret: $secret_name"

  # Check if the secret already exists in the Key Vault
  secret_exists=$(az keyvault secret show --vault-name "$KEYVAULT_NAME" --name "$secret_name" --query "id" --output tsv 2>/dev/null)

  if [[ -n "$secret_exists" ]]; then
    echo "Secret $secret_name already exists. Updating it..."
  else
    echo "Secret $secret_name does not exist. Creating it..."
  fi

  # Add the Azure CLI command to set or update the secret
  az keyvault secret set --vault-name "$KEYVAULT_NAME" --name "$secret_name" --value "$secret_value" >/dev/null
}

# Default values
MONGO_DB_DATABASE_NAME="ContentGenerator"
STORAGE_CONTAINER_NAME="contentgen"
OPENAI_COMPLETION_MODEL="gpt-4o"
OPENAI_EMBEDDING_MODEL="text-embedding-3-small"
BACKEND_BASE_URL="http://localhost:8080"
BACKEND_PRODUCT_ENDPOINT="/api/v1/products"
BACKEND_SIMILAR_PRODUCT_ENDPOINT="/api/v1/products/similar"
MIDDLEWARE_BASE_URL="http://localhost:8081"
MIDDLEWARE_PRODUCT_EMBEDDING_ENDPOINT="/api/v1/generate/embeddings"

# Parse arguments
while [[ $# -gt 0 ]]; do
  case "$1" in
    --keyvault-name)
      KEYVAULT_NAME="$2"
      shift 2
      ;;
    --resource-group)
      RESOURCE_GROUP="$2"
      shift 2
      ;;
    --cosmosdb-name)
      COSMOSDB_NAME="$2"
      shift 2
      ;;
    --mongodb-name)
      MONGO_DB_DATABASE_NAME="$2"
      shift 2
      ;;
    --storage)
      STORAGE_ACCOUNT_NAME="$2"
      shift 2
      ;;
    --storage-container)
      STORAGE_CONTAINER_NAME="$2"
      shift 2
      ;;
    --openai-name)
      OPENAI_NAME="$2"
      shift 2
      ;;
    --openai-completion-model)
      OPENAI_COMPLETION_MODEL="$2"
      shift 2
      ;;
    --openai-embedding-model)
      OPENAI_EMBEDDING_MODEL="$2"
      shift 2
      ;;
    --middleware-base-url)
      MIDDLEWARE_BASE_URL="$2"
      shift 2
      ;;
    --middleware-product-embedding-endpoint)
      MIDDLEWARE_PRODUCT_EMBEDDING_ENDPOINT="$2"
      shift 2
      ;;
    --backend-base-url)
      BACKEND_BASE_URL="$2"
      shift 2
      ;;
    --backend-product-endpoint)
      BACKEND_PRODUCT_ENDPOINT="$2"
      shift 2
      ;;
    --backend-similar-product-endpoint)
      BACKEND_SIMILAR_PRODUCT_ENDPOINT="$2"
      shift 2
      ;;
    *)
      echo "Unknown option: $1"
      usage
      ;;
  esac
done

# Validate required arguments
if [[ -z "$KEYVAULT_NAME" || -z "$RESOURCE_GROUP" || -z "$COSMOSDB_NAME" || -z "$STORAGE_ACCOUNT_NAME" || -z "$OPENAI_NAME" || -z "$OPENAI_EMBEDDING_MODEL" ]]; then
  echo "Error: Missing required arguments."
  usage
fi

# Generate UUIDs for access keys
BACKEND_SERVICE_ACCESS_KEY=$(uuidgen)
MIDDLEWARE_SERVICE_ACCESS_KEY=$(uuidgen)

# Fetch Azure OpenAI service endpoint and keys using az cli
AZURE_OPENAI_ENDPOINT_URL=$(az cognitiveservices account show --name "$OPENAI_NAME" --resource-group "$RESOURCE_GROUP" --query "properties.endpoint" --output tsv)
AZURE_OPENAI_ACCESS_KEY=$(az cognitiveservices account keys list --name "$OPENAI_NAME" --resource-group "$RESOURCE_GROUP" --query "key1" --output tsv)

# Construct the completion and embedding endpoints dynamically
AZURE_OPENAI_COMPLETION_ENDPOINT_URL="$AZURE_OPENAI_ENDPOINT_URL/openai/deployments/$OPENAI_COMPLETION_MODEL/chat/completions?api-version=2023-03-15-preview"
AZURE_OPENAI_EMBEDDING_ENDPOINT_URL="$AZURE_OPENAI_ENDPOINT_URL/openai/deployments/$OPENAI_EMBEDDING_MODEL/embeddings?api-version=2023-05-15"

# Retrieve other secrets using Azure CLI
AZURE_COSMOS_CONNECTION_STRING=$(az cosmosdb keys list --resource-group "$RESOURCE_GROUP" --name "$COSMOSDB_NAME" --type connection-strings --query "connectionStrings[0].connectionString" --output tsv)
AZURE_STORAGE_CONNECTION_STRING=$(az storage account show-connection-string --resource-group "$RESOURCE_GROUP" --name "$STORAGE_ACCOUNT_NAME" --query connectionString --output tsv)

# List of secrets to set
secrets=(
  "BackendServiceBaseUrl" "$BACKEND_BASE_URL"
  "BackendServiceProductEndpoint" "$BACKEND_PRODUCT_ENDPOINT"
  "BackendServiceSimilarProductEndpoint" "$BACKEND_SIMILAR_PRODUCT_ENDPOINT"
  "BackendServiceAccessKey" "$BACKEND_SERVICE_ACCESS_KEY"
  "AzureOpenAiEndpointUrl" "$AZURE_OPENAI_ENDPOINT_URL"
  "AzureOpenAiAccessKey" "$AZURE_OPENAI_ACCESS_KEY"
  "AzureOpenAiCompletionEndpointUrl" "$AZURE_OPENAI_COMPLETION_ENDPOINT_URL"
  "AzureOpenAiEmbeddingEndpointUrl" "$AZURE_OPENAI_EMBEDDING_ENDPOINT_URL"
  "MiddlewareServiceBaseUrl" "$MIDDLEWARE_BASE_URL"
  "MiddlewareServiceProductEmbeddingEndpoint" "$MIDDLEWARE_PRODUCT_EMBEDDING_ENDPOINT"
  "MiddlewareServiceAccessKey" "$MIDDLEWARE_SERVICE_ACCESS_KEY"
  "AzureCosmosConnectionString" "$AZURE_COSMOS_CONNECTION_STRING"
  "MongoDBDatabaseName" "$MONGO_DB_DATABASE_NAME"
  "AzureStorageConnectionString" "$AZURE_STORAGE_CONNECTION_STRING"
  "StorageContainerName" "$STORAGE_CONTAINER_NAME"
)

# Loop through the array and set or update each secret
for ((i=0; i<${#secrets[@]}; i+=2)); do
  secret_name="${secrets[$i]}"
  secret_value="${secrets[$i+1]}"

  # Call the set_or_update_secret function
  set_or_update_secret "$secret_name" "$secret_value"
done

echo "Secrets have been set or updated successfully." 

Script overview ​

Required arguments​

These arguments are mandatory for the script to run successfully:

• --keyvault-name <keyvault-name>: The name of the Azure key vault where secrets will be set or updated.

• --resource-group <resource-group>: The Azure resource group that contains the resources (for example, Key Vault, Azure Cosmos DB) to be used.

• --cosmosdb-name <cosmos-mongo>: The name of the Azure Cosmos DB instance (MongoDB API).

• --storage <sa-name>: The name of the Azure Storage account used for storing data.

• --openai-name <openai-details>: The name of the Azure OpenAI Service instance.

• --openai-embedding-model <embedding-deployment-name>: The deployment name for the Azure OpenAI embedding model.

Optional arguments​

These arguments have default values and can be overridden if needed:

• --mongodb-name <dbname>: Name of the MongoDB database.
  Default value: ContentGenerator

• --storage-container <container-name>: Name of the storage container within the Azure Storage account.
  Default value: contentgen

• --openai-completion-model <ai-service-completion-deployment-name>: Deployment name for the Azure OpenAI chat completion model.
  Default value: gpt-4o

• --middleware-base-url <url>: Base URL for the middleware service.
  Default value: http://localhost:8081

• --middleware-product-embedding-endpoint <endpoint>: Endpoint for the product embedding service in the middleware.
  Default value: /api/v1/generate/embeddings

• --backend-base-url <url>: Base URL for the back-end service.
  Default value: http://localhost:8080

• --backend-product-endpoint <endpoint>: Endpoint for accessing product data in the back-end service.
  Default value: /api/v1/products

• --backend-similar-product-endpoint <endpoint>: Endpoint for accessing similar product data in the back-end service.
  Default value: /api/v1/products/similar

Save this script, and then run it using the following command:

./add-secrets-to-keyvault.sh \
    --keyvault-name <keyvault-name> \
    --resource-group <resource-group> \
    --cosmosdb-name <cosmos-mongo> \
    --mongodb-name <dbname> \
    --storage <sa-name> \
    --storage-container <container-name> \
    --openai-name <openai-details> \
    --openai-completion-model <ai-service-completion-deployment-name> \
    --openai-embedding-model <embedding-deployment-name>

Step 2. Run the back-end service locally​

1. Navigate to the back-end folder. In your terminal, navigate to the back-end folder in the cloned repo.

2. Open the folder in your IDE. Use Visual Studio Code or IntelliJ IDEA to open the back-end folder.

3. Add Key Vault environment variables. Be sure that the following variable is set in your local environment:

   • AZURE_KEYVAULT_URI: The URL of your Key Vault.

4. Run the back-end service. To start the back-end service, run the following command:

   ./mvnw spring-boot:run
   

   

   The back-end service will run on port 8080.

Step 3: Run the middleware service locally​

1. Navigate to the middleware folder. Open another terminal window, and navigate to the middleware folder.

2. Change the port for middleware. Open the application.properties file in the middleware folder, and change the default port to 8081 by adding:

   

3. Add key vault environment variables. Set the environment variables as itemized (AZURE_KEYVAULT_URI). 

4. Run the middleware service. Use the following command:

   ./mvnw spring-boot:run  
   

   

The middleware will run on port 8081.

Step 4: Run the front-end service locally​

1. Navigate to the front-end folder. Go to the front-end folder in your cloned repo.

2. Copy the .env.example file. Copy the .env.example file, and rename it to .env:

   cp .env.example .env
   

3. Configure the .env file. Update the following variables in the .env file:

   

4. Install dependencies. Run npm install or yarn install to install the necessary front-end dependencies. 

5. Run the front-end service. Start the front-end service using one of the following commands:

   npm start  
   

   or

   yarn start  
   

Note: At this point, you’re prompted to sign in with your Microsoft Entra ID credentials. The front end authenticates the user with Microsoft Entra ID (user identity). After you sign in, the following screen appears.

The front end runs on http://localhost:3000.

Local vs. deployment environments​

For local development, you can set the endpoint URLs directly in .env or application.properties files. However, after deployment to AKS or App Service, you need to update the Key Vault secrets to reflect the public-facing URLs and credentials:

• Back-end and middleware URLs: Replace localhost with the public URL of your deployed services.

• API Management: When deployed, make sure that API keys and exposed API URLs are managed through API Management.

We’ll cover the details of deploying to AKS or App Service and using API Management in subsequent topics.

Summary​

In today’s topic, we finished configuring the app’s connection strings and other secrets in Azure Key Vault. Then we got the back-end, middleware, and front-end services running on your local machine, allowing you to test the app in your browser.

Now that you’ve got a running app, our next topic will cover deploying it to Azure using either Azure Kubernetes Service (AKS) or Azure App Service, along with configuring Azure API Management to help ensure that your APIs are secured using key-based authentication.

In this section, we will configure Azure API Management (APIM) to define and secure our APIs, and then use Azure Key Vault to securely store and access the required secrets for these APIs. This setup ensures that all API endpoints are protected and that sensitive information is securely managed.

What we will cover:​

1. Defining and Adding APIs to Azure API Management
2. Configuring Azure Key Vault and Granting Access 

Introduction​

In our previous blog post, we ran the application locally to verify its functionality. Now, we will focus on securing the APIs and managing sensitive data before deploying the application. First, we will define and add the necessary APIs for the back-end and middleware services using Azure API Management (APIM). After configuring APIM, we will set up Azure Key Vault to securely store API keys and other sensitive information and grant secure access to these secrets using Managed Identities.

Step 1: Defining and Adding APIs to Azure API Management (APIM)​

We will start by creating and configuring the APIs for both back-end and middleware services in Azure API Management (APIM). Each service will have its own set of operations that clients can interact with. 

1.1 Define and Add APIs to APIM ​

To add and configure the APIs for both back-end and middleware services, follow these steps: 

1. Navigate to the Azure API Management Service:
   • Go to the Azure portal.
   • Select your API Management instance.
   • Click on APIs and then select + Add API.

The screenshot shows the navigation to the "APIs" section in Azure API Management.

2. Select the API Type:

   • Choose HTTP as the API type since we are exposing HTTP endpoints.

The screenshot shows the "Add API" button and the selection of the HTTP type. 

3. Configure the API Details:
   • For each API, configure the following settings:
     • Display Name: Name of the API (e.g., Product API for back-end, Generate Content API for middleware).
     • Name: A unique identifier for the API (e.g., products, generate-content).
     • URL: The relative path for the API endpoint (e.g., /api/v1/products for Product List, /api/v1/generate/content for content generation).
     • Method: Choose the HTTP method (e.g., GET, POST).
   • Click Create to add the API.

4. Add Operations for Each API:

   • After the API is created, add operations that define specific actions for the API. Follow these steps for both back-end and middleware services:
     • Back-end Service Operations:
       • GET /api/v1/products: Retrieves a list of products.
       • POST /api/v1/products/similar: Returns similar products based on the request body.
     • Middleware Service Operations:
       • POST /api/v1/generate/content: Generates content based on input data.
       • POST /api/v1/generate/embeddings: Generates embeddings based on input data.
   • For each operation, configure the following:
     • Operation Name: A descriptive name for the operation (e.g., "Get Product List", "Generate Content").
     • Method: HTTP method (GET or POST).
     • URL: Relative URL path.
     • Request Body/Parameters: Define any request body or parameters required for the operation.
   • Click Save to add the operation.

5. Configure Security Policies for the APIs:
   • After adding the APIs, go to the Design tab for each API and add security policies.
     • Header Check Policy: Validate the presence of an API key.
     • CORS Policy: Allow only specific origins to access the API endpoints.
     • Rate Limit Policy: Limit the number of requests a client can make to the API within a specified time frame 

Header Check Policy:​

<check-header name="api-key"
failed-check-httpcode="401"
failed-check-error-message="API Key Invalid or Not Found" ignore-case="true">
    <value>API_ACCESS_KEY</value>
</check-header>

CORS Policy:​

<cors allow-credentials="false">
    <allowed-origins>
        <origin>APP_SERVICE_URL</origin>
    </allowed-origins>
    <allowed-methods>
        <method>GET</method>
        <method>POST</method>
    </allowed-methods>
</cors>

Rate Limit Policy:​

<rate-limit calls="100" renewal-period="60" />

• This policy limits each client to 100 requests per minute.

6. Verify the API Configuration:
   • Once the APIs and operations are defined and secured, verify the configuration by using the Test feature in Azure API Management. 
   • Make a sample request to each endpoint to ensure that they respond correctly.This testing will occur after the application has been deployed for both the back-end and middleware services.

Azure CLI instructions​

You can create and configure all of the APIs used by our back-end and middleware services by saving and running the following script:

Shell script: Create API and add policy (create-api.sh)

#!/bin/bash  

# Function to display help  
function display_help() {  
  echo "Usage: $0 -g <resource-group> -b <backend-url> -m <middleware-url> -f <frontend-url> -ba <backend-apim-name> -ma <middleware-apim-name> [--backend-api-key <backend-api-key>] [--middleware-api-key <middleware-api-key>]"  
  echo  
  echo "Options:"  
  echo "  -g, --resource-group              Azure Resource Group"  
  echo "  -b, --backend-url                 Backend service URL"  
  echo "  -m, --middleware-url              Middleware service URL"  
  echo "  -f, --frontend-url                Frontend service URL"  
  echo "  -ba, --backend-apim-name          Backend API Management name"  
  echo "  -ma, --middleware-apim-name       Middleware API Management name"  
  echo "  --backend-api-key                 (Optional) API Key for Backend service"  
  echo "  --middleware-api-key              (Optional) API Key for Middleware service"  
  echo "  -h, --help                        Display this help message"  
  exit 0  
}  

# Function to generate API key if not provided  
function generate_api_key() {  
  echo $(uuidgen)  
}  

# Parse arguments  
while [[ "$#" -gt 0 ]]; do  
  case $1 in  
    -g|--resource-group) RESOURCE_GROUP="$2"; shift ;;  
    -b|--backend-url) BACKEND_URL="$2"; shift ;;  
    -m|--middleware-url) MIDDLEWARE_URL="$2"; shift ;;  
    -f|--frontend-url) FRONTEND_URL="$2"; shift ;;  
    -ba|--backend-apim-name) BACKEND_APIM_NAME="$2"; shift ;;  
    -ma|--middleware-apim-name) MIDDLEWARE_APIM_NAME="$2"; shift ;;  
    --backend-api-key) BACKEND_API_KEY="$2"; shift ;;  
    --middleware-api-key) MIDDLEWARE_API_KEY="$2"; shift ;;  
    -h|--help) display_help ;;  
    *) echo "Unknown option: $1"; display_help ;;  
  esac  
  shift  
done  

# Check for required arguments  
if [[ -z "$RESOURCE_GROUP" || -z "$BACKEND_URL" || -z "$MIDDLEWARE_URL" || -z "$FRONTEND_URL" || -z "$BACKEND_APIM_NAME" || -z "$MIDDLEWARE_APIM_NAME" ]]; then  
  echo "Error: Missing required arguments."  
  display_help  
fi  

# Generate API keys if not provided  
BACKEND_API_KEY=${BACKEND_API_KEY:-$(generate_api_key)}  
MIDDLEWARE_API_KEY=${MIDDLEWARE_API_KEY:-$(generate_api_key)}  

echo "Using Backend API Key: $BACKEND_API_KEY"  
echo "Using Middleware API Key: $MIDDLEWARE_API_KEY"  

# Create Backend API in APIM  
az apim api create \  
  --resource-group "$RESOURCE_GROUP" \  
  --service-name "$BACKEND_APIM_NAME" \  
  --api-id backend-api \  
  --path /api/v1/products \  
  --display-name "Backend Product API" \  
  --service-url "$BACKEND_URL" \  
  --protocols https  

# Add Backend API Operations  
az apim api operation create \  
  --resource-group "$RESOURCE_GROUP" \  
  --service-name "$BACKEND_APIM_NAME" \  
  --api-id backend-api \  
  --url-template "/" \  
  --method GET \  
  --display-name "Get Product List"  

az apim api operation create \  
  --resource-group "$RESOURCE_GROUP" \  
  --service-name "$BACKEND_APIM_NAME" \  
  --api-id backend-api \  
  --url-template "/similar" \  
  --method POST \  
  --display-name "Get Similar Product List"  

# Apply Backend API Policy (CORS, API Key Validation)  
az apim api policy create \  
  --resource-group "$RESOURCE_GROUP" \  
  --service-name "$BACKEND_APIM_NAME" \  
  --api-id backend-api \  
  --xml-policy "  
<inbound>  
  <base />  
  <cors allow-credentials='true'>  
    <allowed-origins>  
      <origin>$MIDDLEWARE_URL</origin>  
    </allowed-origins>  
    <allowed-methods>  
      <method>GET</method>  
      <method>POST</method>  
    </allowed-methods>  
    <allowed-headers>  
      <header>*</header>  
    </allowed-headers>  
  </cors>  
  <check-header name='api-key' failed-check-httpcode='401' failed-check-error-message='API Key Invalid or Not Found' ignore-case='true'>  
    <value>$BACKEND_API_KEY</value>  
  </check-header>  
</inbound>"  

# Create Middleware API in APIM  
az apim api create \  
  --resource-group "$RESOURCE_GROUP" \  
  --service-name "$MIDDLEWARE_APIM_NAME" \  
  --api-id middleware-api \  
  --path /api/v1/generate \  
  --display-name "Middleware Generate API" \  
  --service-url "$MIDDLEWARE_URL" \  
  --protocols https  

# Add Middleware API Operations  
az apim api operation create \  
  --resource-group "$RESOURCE_GROUP" \  
  --service-name "$MIDDLEWARE_APIM_NAME" \  
  --api-id middleware-api \  
  --url-template "/content" \  
  --method POST \  
  --display-name "Generate Content"  

az apim api operation create \  
  --resource-group "$RESOURCE_GROUP" \  
  --service-name "$MIDDLEWARE_APIM_NAME" \  
  --api-id middleware-api \  
  --url-template "/embeddings" \  
  --method POST \  
  --display-name "Generate Embeddings"  

# Apply Middleware API Policy (CORS, API Key Validation)  
az apim api policy create \  
  --resource-group "$RESOURCE_GROUP" \  
  --service-name "$MIDDLEWARE_APIM_NAME" \  
  --api-id middleware-api \  
  --xml-policy "  
<inbound>  
  <base />  
  <cors allow-credentials='true'>  
    <allowed-origins>  
      <origin>$BACKEND_URL</origin>  
      <origin>$FRONTEND_URL</origin>  
    </allowed-origins>  
    <allowed-methods>  
      <method>POST</method>  
    </allowed-methods>  
    <allowed-headers>  
      <header>*</header>  
    </allowed-headers>  
  </cors>  
  <check-header name='api-key' failed-check-httpcode='401' failed-check-error-message='API Key Invalid or Not Found' ignore-case='true'>  
    <value>$MIDDLEWARE_API_KEY</value>  
  </check-header>  
</inbound>"  

echo "APIs created and policies applied successfully."  

What does the script do?​

The script configures the following API rules:

• Allowed origins in CORS:
  • The back-end API allows only the middleware URL as the allowed origin.
  • The middleware API allows both back-end URL and front-end URL as allowed origins.
• Updated check-header validation:
  • Replaces validate-header with check-header for API key validation.
  • Includes failed-check-httpcode="401", failed-check-error-message="API Key Invalid or Not Found", and ignore-case="true".

After running the script, we’ll have created the APIs to use and will have applied the CORS and check-header policies for both back-end and middleware APIs. The script will also generate the required API keys, if you didn’t already provide them.

Step 2: Configuring Azure Key Vault and Granting Access​

Once the APIs are defined and secured, we need to securely store the API keys and other sensitive information in Azure Key Vault and grant the application access to these secrets.

2.1 Storing API Keys in Key Vault​

1. Go to your Azure Key Vault in the Azure portal.
2. Select Secrets and create a new secret for each API key (e.g., BackendServiceAccessKey, MiddlewareServiceAccessKey).
3. Update the values with the API keys configured in APIM.

2.2 Granting Access to Key Vault via Managed Identities​

1. To allow your App Service to access the secrets in Key Vault, enable the System-assigned Managed Identity:
   • Navigate to your Azure App Service.
   • Under the Identity section, enable "System-assigned Managed Identity".

2. In the Azure Key Vault, navigate to Access policies.
3. Create a new access policy using the "Secret Management" template.
4. Select the Managed Identity of your App Service as the principal and grant it Get and List permissions for secrets.

Azure CLI instructions​

Run the following commands:

# Get the Managed Identity's client ID
IDENTITY_CLIENT_ID=$(az webapp identity show --resource-group <resource-group-name> --name <app-service-name> --query principalId --output tsv)

# Grant access to the Key Vault for the Managed Identity
az keyvault set-policy --name <key-vault-name> --secret-permissions get list --object-id $IDENTITY_CLIENT_ID

2.3 Configure Environment Variables in App Service​

1. Navigate to your App Service in the Azure portal:
   • Go to the Azure portal and select your App Service.
2. Open Configuration Settings:
   • In the left-hand menu, select Configuration under the Settings section.

3. Add a New Application Setting

• In the Application settings tab, click on + New application setting.
• Enter the following details:
  • Name: AZURE_KEYVAULT_URL
  • Value: https://<your-key-vault-name>.vault.azure.net/

4. Save the Changes:
   • After adding the setting, click Save at the top to apply the changes.

5. Restart the App Service (if necessary):
   • You may need to restart your App Service for the changes to take effect. You can do this by going to the Overview section and clicking on the Restart button.

Azure CLI instructions​

Run the following command:

# Set the environment variable in the App Service

az webapp config appsettings set --resource-group <resource-group-name> --name <app-service-name> --settings AZURE_KEYVAULT_URL=https://<key-vault-name>.vault.azure.net/

Conclusion​

In this section, we created and configured the APIs for both back-end and middleware services using Azure API Management (APIM). We secured the APIs using CORS policies, header checks, and rate limits. After configuring APIM, we securely stored the API keys and other sensitive data in Azure Key Vault and granted access using Managed Identity. This setup ensures that all components interact securely and that sensitive information is managed properly.

In the next section, we will deploy the application using Azure App Service, ensuring it utilizes the secure configurations established here.

Additional Resources​

• Understanding Azure API Management
• Getting Started with Azure Key Vault
• Configuring Managed Identities in Azure

In the previous blog post, we configured Azure API Management (APIM) and Azure Key Vault to secure the application and manage secrets. In this section, we will deploy the back-end and middleware services using Azure App Service. We will leverage the Azure Web App Maven Plugin for deployment. If the required App Service instances do not already exist (as discussed in blog 2.b), the Maven plugin will automatically create them. Otherwise, it will simply deploy the application code to the existing instances.

Additionally, we will deploy the front-end ReactJS application separately to its own Azure App Service. This ensures a complete end-to-end deployment of all application components, enabling seamless communication between the front-end, back-end, and middleware services.

What we will cover:​

1. Preparing the Application for Deployment
2. Configuring Maven for Azure Web App Deployment
3. Deploying the Back-end and Middleware Services Using Maven
4. Deploying the Front-end Service
5. Verifying the Deployment

Step 1: Preparing the Application for Deployment​

Before deploying, ensure that your application is built and packaged correctly. This involves creating a WAR file for the back-end and middleware services.

1. Build the Application:

   • Use the Maven wrapper command to build the application:

     ./mvnw clean install

2. Package the Application:

   • After building, package the application to create the necessary WAR file for deployment:

     ./mvnw clean package

   • This will create a WAR file in the target directory of your project.

Step 2: Configuring the Azure Web App Maven Plugin​

1. Update the pom.xml File:
   • Open the pom.xml file for both the back-end and middleware services and add the following plugin configuration:

<plugin>
    <groupId>com.microsoft.azure</groupId>
    <artifactId>azure-webapp-maven-plugin</artifactId>
    <version>2.13.0</version>
    <!-- Refer: https://github.com/microsoft/azure-maven-plugins/wiki/Azure-Web-App:-Configuration-Details -->
    <configuration>
        <!-- Optional: If not mentioned, it will pick the default subscription -->
        <subscriptionId>03a4877f-5f78-4a6c-af93-d492e85d4e82</subscriptionId>
        <resourceGroup>ai-study-con-gen-1</resourceGroup>
        <appName>ai-study-backend-service</appName>
        <region>eastus</region>
        <pricingTier>B1</pricingTier>
        <runtime>
            <os>Linux</os>
            <javaVersion>Java 17</javaVersion>
            <webContainer>Tomcat 10.0</webContainer>
        </runtime>
        <deployment>
            <resources>
                <resource>
                    <directory>${project.basedir}/target</directory>
                    <includes>
                        <include>*.war</include>
                    </includes>
                </resource>
            </resources>
        </deployment>
    </configuration>
</plugin>

• Note: If the subscriptionId is not specified, the Maven plugin will use the default subscription configured in your Azure CLI or settings. Additionally, if the App Service defined in the appName field does not exist (as discussed in blog 2b), the Maven plugin will create it automatically during deployment.

Step 3: Deploying the Back-end and Middleware Services Using Maven​

1. Deploy the Back-end Service Using Maven:

   • Use the following Maven wrapper command to deploy the back-end service to Azure App Service:

     ./mvnw clean package azure-webapp:deploy

   • This command will either create the App Service instance (if it does not exist) or deploy the WAR file located in the target directory to the existing Azure Web App.

2. Deploy the Middleware Service Using Maven:

   • Similarly, use the Maven wrapper command for the middleware service:

     ./mvnw clean package azure-webapp:deploy

• Verify that the middleware service is deployed successfully to the specified Azure App Service instance.

Step 4: Deploying the Front-end Service​

In our earlier blog post 1.2b. Preparing the Azure OpenAI Service resource, we discussed creating the app services that will host our app. If you didn’t create the front-end service then, you can do it now.

If you already created a front-end app service, proceed to deploy the ZIP file in step 4.2.

4.1 Create App Service​

Azure portal instructions​

1. In the Azure portal, go to App Services.
2. Select Create and choose the Web App.
3. Select your subscription and resource group, and give the app a name (for example, frontend-service).
4. Choose the Runtime stack: Node.js and Operating System: Linux.
5. Configure Deployment and Monitoring options as needed.
6. Review and create the App Service.

Azure CLI instructions​

# Create Web App for Frontend (ReactJS)
az webapp create --resource-group <your-resource-group-name> --plan <plan-name> --name <your-frontend-name> --runtime "NODE:22-lts"

4.2 Deploy the ZIP File​

Build, package, and deploy the application using the following commands.

Note: Before deploying code, make sure that you’ve cloned the application source from the GitHub repository to your local machine and then navigate to the “frontend” folder.

# Build the application
npm run build  

# Conver into ZIP File
zip -r build.zip ./build

# Deploy the code into Azure App Service
az webapp deploy --resource-group <your-resource-group> --name <your-web-app-name> --src-path build.zip --type zip  

4.3 Set the correct startup command for React​

Azure App Service needs to know how to correctly serve requests to our front-end React app. For static sites, like this React app, you configure the startup command to serve static files and handle routing for single-page applications (SPAs).

You can configure the app service to use the correct React startup command for the app by running the following:

az webapp config set --resource-group <your-resource-group> --name <your-web-app-name> --startup-file "pm2 serve /home/site/wwwroot/build --no-daemon --spa"

4.4 Clear browser cache​

Your browser cache may prevent client-side updates from appearing correctly. After deploying the front-end app, clear your browser cache or use an incognito window before accessing the app to make sure that you're seeing the latest version.

4.5 Restart the web app​

Configuration changes to the app service may not be applied immediately. To be sure that the app is running correctly, after a successful deployment, restart the web app to make sure that everything is refreshed:

az webapp restart --name <your-web-app-name> --resource-group <your-resource-group>

4.6 Verify your deployment​

Open the newly deployed web app in a browser, and be sure that it’s running:

az webapp browse --resource-group <your-resource-group> --name <your-webapp-name>

Step 5: Verifying the Deployment​

1. Check Application Functionality:
   • Open the front-end application in a browser and test its functionality to ensure it communicates with the back-end and middleware services as intended.

Cleanup​

Once you have finished deploying and testing your application, it's essential to clean up the resources to avoid incurring any unnecessary costs. You can delete the Azure Resource Group that contains your resources using either the Azure CLI or the Azure portal. This will remove all the resources within that group, such as the Azure App Service, Azure API Management, and any other services created in the group.

Using Azure CLI​

To delete the resource group using the Azure CLI, follow these steps:

• Open your terminal or Azure Cloud Shell.
• Run the following command to delete the resource group:

az group delete --name <resource-group-name> --yes --no-wait

• Replace <resource-group-name> with the name of your Azure resource group.
• The --yes parameter confirms the deletion without a prompt.
• The --no-wait parameter initiates the deletion and immediately returns control to the terminal, so you don't have to wait for the deletion process to complete.

Using the Azure Portal​

If you prefer to delete the resources using the Azure portal, follow these instructions: 

• Navigate to the Azure Portal: Go to Azure Portal and sign in with your credentials.
• Select "Resource Groups": In the left-hand menu, select Resource groups. This will list all the resource groups available in your subscription.
• Find the Resource Group: Locate the resource group you want to delete (e.g., the resource group that contains your AKS and ACR resources).
• Click on the Resource Group: Click on the resource group name to open it.
• Delete the Resource Group: At the top of the resource group page, click on Delete resource group.
• Confirm Deletion: You will be prompted to type the name of the resource group to confirm deletion. Type the name exactly and click on the Delete button.

Deleting a resource group in the Azure portal will remove all resources associated with it. Be careful when performing this action, as it cannot be undone.

Conclusion​

In this section, we deployed our Java-based back-end and middleware services using the Maven Azure Web App Plugin and verified that they were hosted correctly in Azure App Service. We also deployed the front-end ReactJS application using the deployment center in Azure App Service. These steps ensure that all components are securely hosted and configured for seamless communication.

Additional Resources:​

• Azure Web App Maven Plugin Documentation
• Azure App Service Documentation
• Deploying Java Applications to Azure

In this blog, we will set up APIM, Key Vault, and Managed Identity for the Middleware and Back-end services. We will refer to Blog 1.4a for the APIM configuration, as it remains consistent. We will also include the necessary Azure CLI commands and create a Kubernetes pod to integrate with Key Vault using Managed Identity.

What we cover:​

1. Review of APIM Configuration
2. Setting Up Key Vault and Managed Identity
3. Creating Kubernetes Pod with Managed Identity Integration
4. Verifying Configurations 

Prerequisites​

Before proceeding, ensure that you have completed the environment setup described in Blog 1.2b, where we created the AKS cluster and Azure Container Registry (ACR).

Step 1: Review of APIM Configuration​

The APIM configuration for the Middleware and Back-end services remains the same as covered in Blog 1.4a. Please refer to Blog 1.4a for detailed steps on setting up APIM and configuring CORS policies, header checks, and other necessary configurations.

Step 2: Setting Up Key Vault and Managed Identity​

In this step, we will set up Azure Key Vault and configure Managed Identity to allow secure access to the secrets stored in Key Vault.

Create a Key Vault:​

• Use the Azure CLI to create a Key Vault if it does not already exist:

az aks create \
  --resource-group <RESOURCE_GROUP_NAME> \
  --name <AKS_NAME> \
  --node-count 1 \
  --enable-addons monitoring \
  --enable-oidc-issuer \
  --enable-workload-identity \
  --generate-ssh-keys

Enable Managed Identity in AKS:​

• Follow the Azure documentation to enable Managed Identity for the AKS cluster using workload identity: Workload Identity for AKS.
• Run the following commands to integrate workload identity:

# Retrieve the OIDC issuer URL
export AKS_OIDC_ISSUER="$(az aks show --name <AKS_NAME> \
  --resource-group <RESOURCE_GROUP_NAME> \
  --query "oidcIssuerProfile.issuerUrl" \
  --output tsv)"

# Create a Managed Identity for the AKS cluster
az identity create \
  --name <USER_ASSIGNED_IDENTITY_NAME> \
  --resource-group <RESOURCE_GROUP_NAME> \
  --location <LOCATION> \
  --subscription "$(az account show --query id --output tsv)"

export USER_ASSIGNED_CLIENT_ID="$(az identity show \
  --resource-group <RESOURCE_GROUP_NAME> \
  --name <USER_ASSIGNED_IDENTITY_NAME> \
  --query 'clientId' \
  --output tsv)"

# Create a Kubernetes service account
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ServiceAccount
metadata:
  annotations:
    azure.workload.identity/client-id: "${USER_ASSIGNED_CLIENT_ID}"
  name: workload-identity-sa
  namespace: default
EOF

# Create the federated identity credential
az identity federated-credential create \
  --name ${FEDERATED_IDENTITY_CREDENTIAL_NAME} \
  --identity-name "${USER_ASSIGNED_IDENTITY_NAME}" \
  --resource-group "${RESOURCE_GROUP}" \
  --issuer "${AKS_OIDC_ISSUER}" \
  --subject system:serviceaccount:"${SERVICE_ACCOUNT_NAMESPACE}":"${SERVICE_ACCOUNT_NAME}" \
  --audience api://AzureADTokenExchange

# Deploy your application
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: sample-workload-identity
  namespace: ${SERVICE_ACCOUNT_NAMESPACE}
  labels:
    azure.workload.identity/use: "true"  # Required. Only pods with this label can use workload identity.
spec:
  serviceAccountName: ${SERVICE_ACCOUNT_NAME}
  containers:
    - image: <image>
      name: <containerName>
EOF

Grant Access to Key Vault:​

• Use the following command to grant the Managed Identity access to the Key Vault:

export KEYVAULT_RESOURCE_ID=$(az keyvault show --resource-group <RESOURCE_GROUP_NAME> \
  --name <KEYVAULT_NAME> \
  --query id \
  --output tsv)

export IDENTITY_PRINCIPAL_ID=$(az identity show \
  --name <USER_ASSIGNED_IDENTITY_NAME> \
  --resource-group <RESOURCE_GROUP_NAME> \
  --query principalId \
  --output tsv)

az role assignment create \
  --assignee-object-id "${IDENTITY_PRINCIPAL_ID}" \
  --role "Key Vault Secrets User" \
  --scope "${KEYVAULT_RESOURCE_ID}" \
  --assignee-principal-type ServicePrincipal

export AZURE_KEYVAULT_URI=https://<KEYVAULT_NAME>.vault.azure.net/

Step 3: Creating Kubernetes Pod with Managed Identity Integration​

In this step, we will create a Kubernetes pod and integrate it with Key Vault using the Managed Identity.

Deploy a Kubernetes Pod reference:​

• To define the pod with the necessary labels and annotations for Managed Identity integration.

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: workload-identity-key-vault
  namespace: default
  labels:
    azure.workload.identity/use: "true"
spec:
  serviceAccountName: <SERVICE_ACCOUNT_NAME>
  containers:
    - image: ghcr.io/azure/azure-workload-identity/msal-go
      name: oidc
      env:
        - name: AZURE_KEYVAULT_URI
          value: ${AZURE_KEYVAULT_URI}
  nodeSelector:
    kubernetes.io/os: linux
EOF

Step 4: Verifying Configurations​

Verify Pod Creation:​

• Check if the pod is running successfully:

kubectl get pods

Conclusion​

In this blog, we configured APIM, Key Vault, and Managed Identity for the Middleware and Back-end services. We also created a Kubernetes pod with Managed Identity integration, ensuring secure access to Key Vault secrets.

Additional Resources​

• Azure API Management Documentation
• Azure Key Vault Documentation
• Managed Identities for Azure Resources
• Workload Identity for AKS

In this blog, we will deploy our application components (Middleware, Back-end, and Front-end services) to Azure Kubernetes Service (AKS) using Azure CLI and Kubernetes configurations. This approach allows us to leverage container orchestration for better management and scalability.

What we cover:​

1. Understanding Azure Kubernetes Service (AKS)
2. Preparing the Application for AKS Deployment
3. Creating the AKS Cluster
4. Configuring the Application Deployment
5. Deploying the Application to AKS
6. Verifying the Deployment

Prerequisites​

Before proceeding, ensure that the Managed Identity and Key Vault configurations are completed as described in Blog 1.5a. Additionally, refer to Blog 1.2b for details on the initial environment setup, where the AKS cluster and Azure Container Registry (ACR) were created.

Step 1: Understanding Azure Kubernetes Service (AKS)​

Azure Kubernetes Service (AKS) is a managed container orchestration service that simplifies the deployment, management, and operations of Kubernetes. It enables seamless scaling, monitoring, and management of containerized applications. With AKS, you can deploy, manage, and scale containerized applications with ease.

Step 2: Preparing the Application for AKS Deployment​

Containerize the Application:​

• Ensure that you have a separate Dockerfile for each service (Back-end, Middleware, and Front-end) for the containerization process. Example Dockerfile for Back-end:

# Use an official JDK base image
FROM mcr.microsoft.com/openjdk/jdk:17-mariner

# Set the working directory
WORKDIR /app

# Copy the WAR file into the container
COPY target/backend.war /app/backend.war

# Expose the port the app runs on
EXPOSE 8080

# Run the WAR file
ENTRYPOINT ["java", "-jar", "/app/backend.war"]

Build and Push Docker Images:​

• Build the Docker images and push them to Azure Container Registry (ACR).

Commands:​

# ACR Login
az acr login --name <ACR_NAME>

# Set your image build version
VERSION=1.0.1

# Build docker image
docker build -t aistudy/backend-service:${VERSION} .

# Tag the build image into acr repo
docker tag aistudy/backend-service:${VERSION} <ACR_Name>.azurecr.io/aistudy/backend-service:${VERSION}

# Push the backend service image to ACR
docker push <ACR_Name>.azurecr.io/backend-service:latest

# Build and push the Back-end service
docker tag backend-service:latest <ACR_Name>.azurecr.io/backend-service:latest
docker push <ACR_Name>.azurecr.io/backend-service:latest

# Build and push the Middleware service
docker tag middleware-service:latest <ACR_Name>.azurecr.io/middleware-service:latest
docker push <ACR_Name>.azurecr.io/middleware-service:latest

# Build and push the Front-end service
docker tag frontend-service:latest <ACR_Name>.azurecr.io/frontend-service:latest
docker push <ACR_Name>.azurecr.io/frontend-service:latest

Step 3: Creating the AKS Cluster​

If you have not already created the AKS cluster and ACR (Azure Container Registry), refer to Blog 1.2b for details on the environment setup.

Create AKS Cluster Using Azure CLI​

• Use the following Azure CLI command to create a new AKS cluster with workload identity enabled:

az aks create \
  --resource-group <RESOURCE_GROUP_NAME> \
  --name <CLUSTER_NAME> \
  --node-count 1 \
  --enable-addons monitoring \
  --enable-oidc-issuer \
  --enable-workload-identity \
  --generate-ssh-keys \
  --attach-acr <ACR_NAME>

• The --enable-oidc-issuer and --enable-workload-identity flags are required to enable workload identity on the cluster, allowing for secure and seamless integration with Azure AD and Key Vault.

Connect to the AKS Cluster​

• Use the following command to connect your local kubectl to the AKS cluster:

 az aks get-credentials --resource-group <RESOURCE_GROUP_NAME> --name <AKS_NAME>

• This command will download the credentials and configure your kubectl context.

Step 4: Configuring the Application Deployment​

In this step, we will create Kubernetes deployment and service YAML files for each service and configure them to run in the AKS cluster.

Create Deployment and Service YAML Files​

• Create separate deployment.yml files for each service. 
• Backend-deployment.yml for Back-end Service:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: backend-deployment
spec:
  replicas: 2
  selector:
    matchLabels:
      app: backend
  template:
    metadata:
      labels:
        app: backend
    spec:
      containers:
        - name: backend
          image: <ACR_NAME>.azurecr.io/aistudy/backend:latest
          ports:
            - containerPort: 8080
          env:
            - name: AZURE_KEYVAULT_URI
              value: https://<KEYVAULT_NAME>.vault.azure.net/

---
apiVersion: v1
kind: Service
metadata:
  name: backend-service
spec:
  selector:
    app: backend
  ports:
    - port: 80
      targetPort: 8080
  type: ClusterIP

Step 5: Deploying the Application to AKS​

Apply the Deployment and Service YML Files​

• Use the following commands to deploy the Back-end and Middleware services to AKS:

kubectl apply -f backend-deployment.yml
kubectl apply -f middleware-deployment.yml

Deploy the Front-end Service​

• Use similar commands to deploy the front-end service:

kubectl apply -f frontend-deployment.yml

Step 6: Verifying the Deployment​

Check the Status of Deployments​

• Use the following commands to check the status of each deployment:

kubectl get deployments  
kubectl get pods 

Access the Application​

• If the front-end service is exposed as a LoadBalancer type, use the following command to get the external IP address:

kubectl get services frontend-service

• Open the application in a browser using the external IP address.

Cleanup​

Once you have finished deploying and testing your application, it's essential to clean up the resources to avoid incurring any unnecessary costs. You can delete the Azure Resource Group that contains your resources using either the Azure CLI or the Azure portal. This will remove all the resources within that group, such as the Azure Kubernetes Service (AKS), Azure Container Registry (ACR), Azure API Management, and any other services created in the group.

Using Azure CLI​

To delete the resource group using the Azure CLI, follow these steps:

• Open your terminal or Azure Cloud Shell.
• Run the following command to delete the resource group:

az group delete --name <resource-group-name> --yes --no-wait

• Replace <resource-group-name> with the name of your Azure resource group.
• The --yes parameter confirms the deletion without a prompt.
• The --no-wait parameter initiates the deletion and immediately returns control to the terminal, so you don't have to wait for the deletion process to complete.

Using the Azure Portal​

If you prefer to delete the resources using the Azure portal, follow these instructions:

• Navigate to the Azure Portal: Go to Azure Portal and sign in with your credentials.
• Select "Resource Groups": In the left-hand menu, select Resource groups. This will list all the resource groups available in your subscription.
• Find the Resource Group: Locate the resource group you want to delete (e.g., the resource group that contains your AKS and ACR resources).
• Click on the Resource Group: Click on the resource group name to open it.
• Delete the Resource Group: At the top of the resource group page, click on Delete resource group.
• Confirm Deletion: You will be prompted to type the name of the resource group to confirm deletion. Type the name exactly and click on the Delete button.

Deleting a resource group in the Azure portal will remove all resources associated with it. Be careful when performing this action, as it cannot be undone.

Conclusion​

In this blog, we successfully deployed our Middleware, Back-end, and Front-end services to Azure Kubernetes Service (AKS) using Kubernetes configurations and the Azure CLI. By using the --enable-oidc-issuer and --enable-workload-identity flags, we enabled secure workload identity on the cluster, allowing for integration with Azure AD and seamless access to Key Vault secrets.

Additional Resources​

• Azure Kubernetes Service Documentation
• Getting Started with Azure Kubernetes Service
• Deploying Applications to Azure Kubernetes Service
• Managed Identities for Azure Resources
• Workload Identity for AKS

In this application development, Continuous Integration and Continuous Deployment (CI/CD) play a vital role in streamlining workflows and ensuring rapid delivery of software. In this blog, we will explore how to set up CI/CD pipelines for three applications—a Java Spring Boot backend, a Java Spring Boot middleware, and a ReactJS frontend—using Azure DevOps. We will deploy these applications to Azure App Service in parallel.

Prerequisites​

Before we start, ensure you have:

• An Azure DevOps account.
• A GitHub repository with your application code, specifically structured with three folders: backend, middleware, and frontend.
• An Azure subscription with access to Azure App Service.
• The GitHub URL for the project we will be using is: Java-AI-Based-Content-Generator.

Overview of the Pipeline​

The CI/CD pipeline will consist of the following key stages:

1. Build Stage: Each application is built in parallel, producing artifacts for deployment.
2. Deploy Stage: The built artifacts are deployed to Azure App Service and AKS.

Integrating GitHub Repository into Azure DevOps​

To connect your GitHub repository to Azure DevOps, follow these steps:

1. Navigate to Your Azure DevOps Project: Go to the Azure DevOps portal and select your project.

   

2. Select Project Settings: At the bottom left corner, click on the Project settings gear icon. 

3. Service Connections: Under the Pipelines section, select Service connections.

   

4. Create a New Service Connection: Click on New service connection and select GitHub from the list.

5. Authorize Azure DevOps: Follow the prompts to authorize Azure DevOps to access your GitHub account.

6. Select Repository: Choose the repository you want to integrate and complete the setup.

   

Creating Variable Groups​

Variable groups in Azure DevOps help centralize configuration management by allowing you to store and reuse variables across multiple pipelines. Here's how to create and configure a variable group:

1. Navigate to Pipelines: Open the Pipelines section in your Azure DevOps project.

2. Library: Click on Library under Pipelines.

   

3. Add Variable Group: Select + Variable group.

   

4. Define Variables: Add the necessary variables (e.g., AcrName, BackendAppServiceName, MiddlewareAppServiceName, and FrontendAppServiceName). Optionally, mark sensitive values as secrets.

   

5. Link to Azure Key Vault: If needed, link your variable group to an Azure Key Vault to manage secrets.

   

6. Save the Variable Group: Click Save to confirm your variable group configuration.

7. Add Pipeline Permission

   

Pipeline YAML Configuration​

Below is the YAML configuration for the Azure DevOps pipeline, followed by a detailed explanation of each block.

azure-pipeline.yml

trigger:
  branches:
    include:
      - main
      - feature/ci

pool:
  vmImage: 'ubuntu-latest'

variables:
  - group: ai-study-vg
  - group: ai-study-secrets

stages:
  - stage: Build
    displayName: 'Build Stage'
    jobs:
      - job: Build_Backend
        displayName: 'Build Backend Service'
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - checkout: self
          - task: Maven@4
            inputs:
              mavenPomFile: 'backend/pom.xml'
              mavenOptions: '-Xmx3072m'
              javaHomeOption: 'JDKVersion'
              jdkVersionOption: '1.17' # JDK Version set to 17
              jdkArchitectureOption: 'x64'
              publishJUnitResults: true
              testResultsFiles: '**/TEST-*.xml'
              goals: 'clean package'
          - publish: '$(System.DefaultWorkingDirectory)/backend/target/backend.war'
            artifact: backend

      - job: Build_Middleware
        displayName: 'Build Middleware Service'
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - checkout: self
          - task: Maven@4
            inputs:
              mavenPomFile: 'middleware/pom.xml'
              mavenOptions: '-Xmx3072m'
              javaHomeOption: 'JDKVersion'
              jdkVersionOption: '1.17' # JDK Version set to 17
              jdkArchitectureOption: 'x64'
              publishJUnitResults: true
              testResultsFiles: '**/TEST-*.xml'
              goals: 'clean package'
          - publish: '$(System.DefaultWorkingDirectory)/middleware/target/middleware.war'
            artifact: middleware

      - job: Build_Frontend
        displayName: 'Build Frontend Service'
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - checkout: self
          - script: |
              cd frontend
              npm install
              npm run build
          - publish: '$(System.DefaultWorkingDirectory)/frontend/build'
            artifact: frontend

  - stage: Deploy
    displayName: 'Deploy Stage'
    dependsOn: Build
    jobs:
      - job: Deploy_Backend_AppService
        displayName: 'Deploy Backend to Azure App Service'
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - download: current
            artifact: backend
          - task: AzureWebApp@1
            inputs:
              azureSubscription: '$(AzureSubscription)' # This will be set from Variable Group
              appName: '$(BackendAppServiceName)' # This will be set from Key Vault - Variable Group
              package: '$(Pipeline.Workspace)/backend/backend.war'
              appType: 'webAppLinux'
              appSettings: |
                -AZURE_KEYVAULT_URI "$(AZURE_KEYVAULT_URI)"

      - job: Deploy_Middleware_AppService
        displayName: 'Deploy Middleware to Azure App Service'
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - download: current
            artifact: middleware
          - task: AzureWebApp@1
            inputs:
              azureSubscription: '$(AzureSubscription)' # This will be set from Variable Group
              appName: '$(MiddlewareAppServiceName)' # This will be set from Key Vault - Variable Group
              package: '$(Pipeline.Workspace)/middleware/middleware.war'
              appType: 'webAppLinux'
              appSettings: |
                -AZURE_KEYVAULT_URI "$(AZURE_KEYVAULT_URI)"

      - job: Deploy_Frontend_AppService
        displayName: 'Deploy Frontend to Azure App Service'
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - download: current
            artifact: frontend
          - task: AzureWebApp@1
            inputs:
              azureSubscription: '$(AzureSubscription)' # This will be set from Variable Group
              appName: '$(FrontendAppServiceName)' # This will be set from Key Vault - Variable Group
              package: '$(Pipeline.Workspace)/frontend'
              startUpCommand: 'pm2 serve /home/site/wwwroot/build --no-daemon --spa'
              appType: 'webAppLinux'
              # These secrets will be retrieved from Key Vault - Variable Group
              appSettings: |
                -REACT_APP_SERVICE_BASE_URL "$(MiddlewareServiceBaseUrl)"
                -REACT_APP_CLIENT_ID "$(MsalAppId)"
                -REACT_APP_CONTENT_GENERATOR_ENDPOINT "$(MiddlewareServiceGenerateContentEndpoint)"
                -REACT_APP_SERVICE_ACCESS_KEY "$(MiddlewareServiceAccessKey)"

Testing the CI/CD Pipeline​

After setting up the pipeline, navigate to the Pipelines section in Azure DevOps and manually trigger a run to test your configuration.

1. Navigate to Pipelines: Open the Pipelines section.

   

2. Select the Pipeline: Choose the pipeline you configured.

   

3. Run Pipeline: Click on Run Pipeline and monitor the execution.

   

4. Monitor Job Logs: View the logs for each stage to ensure there are no errors.

   

Conclusion​

In this blog, we explored how to set up a CI/CD pipeline for Java and ReactJS applications using Azure DevOps. We integrated GitHub, configured variable groups, and deployed to Azure App Service. Testing the pipeline ensures that the setup works end-to-end.

Next​

In the next blog, we’ll cover advanced topics such as monitoring and scaling these deployments.

In the previous blog, we discussed the fundamentals of setting up CI/CD pipelines for Azure App Service. In this blog, we will focus on how to configure a CI/CD pipeline specifically for Azure Kubernetes Service (AKS) using Azure DevOps. We’ll cover prerequisites, detailed YAML configurations, and how to manage deployments based on a variable group setting.

Prerequisites​

Before diving into the pipeline configuration, ensure you have the following prerequisites:

• An Azure DevOps account.
• Access to your Azure subscription with permission to create resources.
• A GitHub repository with your application code, specifically structured with three folders: backend, middleware, and frontend. We will use the same repository as in the previous blog: Java-AI-Based-Content-Generator.
• Azure Container Registry (ACR) configured and accessible.
• A Kubernetes cluster (AKS) set up in your Azure environment.

Overview of the Pipeline​

The CI/CD pipeline for deploying to AKS will include:

1. Build Stage: Similar to the previous setup, each application is built in parallel to produce artifacts.
2. Push to ACR: Docker images are created from the build artifacts and pushed to Azure Container Registry.
3. Deploy Stage: The Docker images are pulled from ACR and deployed to the AKS cluster

Integrating GitHub Repository into Azure DevOps​

Follow similar steps as outlined in blog 1.6a.

Creating Variable Groups​

Variable groups in Azure DevOps help centralize configuration management by allowing you to store and reuse variables across multiple pipelines. Here's how to create and configure a variable group:

• Navigate to Pipelines: Open the Pipelines section in your Azure DevOps project.

• Library: Click on Library under Pipelines.

  

1. Add Variable Group: Select + Variable group.

   

2. Define Variables: Add the necessary variables (e.g., AcrName, BackendAppServiceName, MiddlewareAppServiceName, and FrontendAppServiceName). Optionally, mark sensitive values as secrets.

   

3. Link to Azure Key Vault: If needed, link your variable group to an Azure Key Vault to manage secrets.

   

4. Save the Variable Group: Click Save to confirm your variable group configuration.

5. Add Pipeline Permission: Ensure that the pipeline has permission to access the variable group.

   

Variable Groups​

We will utilize variable groups to manage our configurations efficiently. In particular, we will set the DeploymentTarget variable in the variable group ai-study-vg to determine whether we are deploying to AKS or App Service.

Adding Variables​

• DeploymentTarget: This variable will dictate the target deployment, allowing us to switch between AKS and App Service as needed.

  

Configuring Service Connection for Azure Container Registry (ACR)​

Before deploying your applications to Azure Kubernetes Service (AKS), it's essential to set up a service connection for Azure Container Registry (ACR). This allows Azure DevOps to authenticate and pull images from your ACR. Follow these steps to configure the service connection:

Step 1: Access Your Azure Container Registry​

1. Navigate to Azure Portal and find your Azure Container Registry resource.
2. In the left-hand menu, select Access keys.

Step 2: Copy the Required Values​

In the Access keys section, you will find two important values:

• Login Server: This will typically be in the format youracrname.azurecr.io.

• Admin User Password: This is used for authentication.

  

Step 3: Create the Service Connection​

6. In your Azure DevOps project, navigate to Project Settings > Service connections.

7. Click on New service connection and select Docker Registry.

8. Choose Others as the Docker Registry type.

9. Enter the following details:

   • Registry: $(DockerRegistry) (the variable you created in ai-study-vg)
   • Username: Your ACR username (usually the name of your ACR)
   • Password: Use the Admin User Password you copied earlier.

10. Name your service connection (e.g., AI Study Container) and save it.

    

Step 4: Store the Values in Variable Group​

11. Go to your Azure DevOps project.

12. Select Pipelines > Library.

13. Find the variable group ai-study-vg.

14. Add a new variable named DockerRegistry and service connection name (e.g., AI Study Container).

    

Now your Azure DevOps pipeline can authenticate to ACR and pull the necessary images for your AKS deployments.

Pipeline YAML Configuration​

Here's the YAML configuration for setting up the Azure DevOps pipeline for AKS deployment:

azure-pipeline.yml

trigger:
  branches:
    include:
      - main

pool:
  vmImage: 'ubuntu-latest'

variables:
  - group: ai-study-vg
  - group: ai-study-secrets

stages:
  - stage: Build
    displayName: 'Build Stage'
    jobs:
      - job: Build_Backend
        displayName: 'Build Backend Service'
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - checkout: self
          - task: Maven@4
            inputs:
              mavenPomFile: 'backend/pom.xml'
              mavenOptions: '-Xmx3072m'
              javaHomeOption: 'JDKVersion'
              jdkVersionOption: '1.17'  # JDK Version set to 17
              jdkArchitectureOption: 'x64'
              publishJUnitResults: true
              testResultsFiles: '**/TEST-*.xml'
              goals: 'clean package'
          - publish: 'backend/target/backend.war'
            artifact: backend

          # Docker build and push using the published WAR file
          - task: Docker@2
            condition: eq(variables['DeploymentTarget'], 'AKS')
            inputs:
              containerRegistry: '$(DockerRegistry)'
              repository: '$(AcrName).azurecr.io/aistudy/backend'
              command: 'buildAndPush'
              Dockerfile: 'backend/Dockerfile'
              buildContext: 'backend'
              tags: 'latest'

      - job: Build_Middleware
        displayName: 'Build Middleware Service'
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - checkout: self
          - task: Maven@4
            inputs:
              mavenPomFile: 'middleware/pom.xml'
              mavenOptions: '-Xmx3072m'
              javaHomeOption: 'JDKVersion'
              jdkVersionOption: '1.17'  # JDK Version set to 17
              jdkArchitectureOption: 'x64'
              publishJUnitResults: true
              testResultsFiles: '**/TEST-*.xml'
              goals: 'clean package'
          - publish: 'middleware/target/middleware.war'
            artifact: middleware

          # Docker build and push using the published WAR file
          - task: Docker@2
            condition: eq(variables['DeploymentTarget'], 'AKS')
            inputs:
              containerRegistry: '$(DockerRegistry)'
              repository: '$(AcrName).azurecr.io/aistudy/middleware'
              command: 'buildAndPush'
              Dockerfile: 'middleware/Dockerfile'
              buildContext: 'middleware'
              tags: 'latest'

      - job: Build_Frontend
        displayName: 'Build Frontend Service'
        condition: eq(variables['DeploymentTarget'], 'AppService')
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - checkout: self

          # Install dependencies and build the React app
          - task: Npm@1
            inputs:
              command: 'install'
              workingDir: 'frontend'

          - task: Npm@1
            inputs:
              command: 'custom'
              workingDir: 'frontend'
              customCommand: 'run build'

          - publish: 'frontend/build'
            artifact: frontend

      - job: Build_Frontend_AKS
        displayName: 'Build Frontend Service'
        condition: eq(variables['DeploymentTarget'], 'AKS')
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - checkout: self
          - task: Docker@2
            inputs:
              containerRegistry: '$(DockerRegistry)'
              repository: '$(AcrName).azurecr.io/aistudy/frontend'
              command: 'buildAndPush'
              Dockerfile: 'frontend/Dockerfile'
              buildContext: 'frontend'
              tags: 'latest'

  - stage: Deploy_AppServices
    displayName: 'Deploy to Azure App Services'
    condition: eq(variables['DeploymentTarget'], 'AppService')
    dependsOn: Build
    jobs:
      - job: Deploy_Backend_AppService
        displayName: 'Deploy Backend to Azure App Service'
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - download: current
            artifact: backend
          - task: AzureWebApp@1
            inputs:
              azureSubscription: '$(AzureSubscription)' # This will be set from Variable Group
              appName: '$(BackendAppServiceName)' # This will be set from Key Vault - Variable Group
              package: '$(Pipeline.Workspace)/backend/backend.war'
              appType: 'webAppLinux'
              appSettings: |
                -AZURE_KEYVAULT_URI "$(AZURE_KEYVAULT_URI)"

      - job: Deploy_Middleware_AppService
        displayName: 'Deploy Middleware to Azure App Service'
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - download: current
            artifact: middleware
          - task: AzureWebApp@1
            inputs:
              azureSubscription: '$(AzureSubscription)' # This will be set from Variable Group
              appName: '$(MiddlewareAppServiceName)' # This will be set from Key Vault - Variable Group
              package: '$(Pipeline.Workspace)/middleware/middleware.war'
              appType: 'webAppLinux'
              appSettings: |
                -AZURE_KEYVAULT_URI "$(AZURE_KEYVAULT_URI)"

      - job: Deploy_Frontend_AppService
        displayName: 'Deploy Frontend to Azure App Service'
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - download: current
            artifact: frontend
          - task: AzureWebApp@1
            inputs:
              azureSubscription: '$(AzureSubscription)' # This will be set from Variable Group
              appType: 'webAppLinux'
              appName: '$(FrontendAppServiceName)' # This will be set from Key Vault - Variable Group
              package: '$(Pipeline.Workspace)/frontend'
              startUpCommand: 'pm2 serve /home/site/wwwroot/build --no-daemon --spa'
              appSettings: |
                -REACT_APP_SERVICE_BASE_URL "$(MiddlewareServiceBaseUrl)"
                -REACT_APP_CLIENT_ID "$(MsalAppId)"
                -REACT_APP_CONTENT_GENERATOR_ENDPOINT "$(MiddlewareServiceGenerateContentEndpoint)"
                -REACT_APP_SERVICE_ACCESS_KEY "$(MiddlewareServiceAccessKey)"

  # Deploying to Azure Kubernetes Service
  - stage: Deploy_AKS
    displayName: 'Deploy to Azure Kubernetes Service'
    condition: eq(variables['DeploymentTarget'], 'AKS')
    dependsOn: Deploy_AppServices
    jobs:
      - job: Deploy_Backend_AKS
        displayName: 'Deploy Backend to Azure Kubernetes Service'
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - task: KubernetesManifest@1
            inputs:
              action: 'deploy'
              connectionType: 'azureResourceManager'
              azureSubscriptionConnection: '$(AzureSubscription)'
              azureResourceGroup: '$(ResourceGroupName)'
              kubernetesCluster: '$(AKS)'
              manifests: 'backend/backend-deployment.yml'

      - job: Deploy_Middleware_AKS
        displayName: 'Deploy Middleware to Azure Kubernetes Service'
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - task: KubernetesManifest@1
            inputs:
              action: 'deploy'
              connectionType: 'azureResourceManager'
              azureSubscriptionConnection: '$(AzureSubscription)'
              azureResourceGroup: '$(ResourceGroupName)'
              kubernetesCluster: '$(AKS)'
              manifests: 'middleware/middleware-deployment.yml'

      - job: Deploy_Frontend_AKS
        displayName: 'Deploy Frontend to Azure Kubernetes Service'
        pool:
          vmImage: 'ubuntu-latest'
        steps:
          - task: KubernetesManifest@1
            inputs:
              action: 'deploy'
              connectionType: 'azureResourceManager'
              azureSubscriptionConnection: '$(AzureSubscription)'
              azureResourceGroup: '$(ResourceGroupName)'
              kubernetesCluster: '$(AKS)'
              manifests: 'frontend/frontend-deployment.yml'

Explanation of the YAML Configuration​

• The pipeline triggers on changes to the main branch.
• The Build job compiles the applications and builds artifacts for deployment.
• The Deploy job checks the value of DeploymentTarget and decides whether to deploy to AKS or App Service.

Testing the CI/CD Pipeline​

Similar to Blog 1.6a, manually trigger the pipeline to ensure that the deployment is successful.

Conclusion​

In this blog, we extended our CI/CD pipelines to deploy Java Spring Boot and ReactJS applications to Azure Kubernetes Service. This approach allows for greater scalability and flexibility.

In our next blog, we will cover advanced topics such as monitoring and scaling these deployments to ensure optimal performance and reliability.

What we cover:​

• Monitoring your application health
• Scaling your application
• Setting up alerts and notifications
• Implementing continuous optimization

Introduction​

In our previous blog posts, we walked through building and deploying an example application using both Azure App Service and Azure Kubernetes Service (AKS) environments.

However, deployment is not the end of the journey. Post-production phases are essential to maintaining your application’s efficiency and ensuring that it scales effectively as demand increases.

This post covers the necessary steps for monitoring, scaling, and optimizing applications on both App Service and AKS, helping you maintain reliable and performant deployments.

Azure App Service​

Step 1: Monitor Application Health on App Service​

1. Set up Azure Monitor and Application Insights
   Enable Application Insights for your App Service to track performance metrics such as response times, request rates, and failure rates.

   • To enable:
     Go to your App Service in the Azure portal. Under Monitoring, enable Application Insights. This provides features like:

     

   • Live metrics: Real-time performance monitoring.

     

   • Custom dashboards: Create visual dashboards for metrics such as API response times and database latency.

     

2. Implement Logging and Tracing
   Enable diagnostic logs to capture application and server logs. Stream logs live or archive them in Azure Storage for future analysis.

3. Key Metrics to Monitor

   • Availability: Uptime and responsiveness of your App Service.
   • Performance: Response times, throughput, and latency.
   • Error Rates: Percentage of failed requests or exceptions.
   • Resource Utilization: CPU and memory usage.

Step 2: Scale Your Application on App Service​

1. Vertical Scaling (Scale Up)
   Increase the compute resources allocated to your App Service plan for handling intensive workloads.

2. Horizontal Scaling (Scale Out)
   Add more instances of your App Service to handle increased traffic. You can set the number of instances or configure Autoscale based on CPU or memory thresholds.

   • To Scale Out:
     Go to App Service > Scale Out (App Service plan), and configure scaling rules.

     

Step 3: Set Up Alerts and Notifications for App Service​

1. Configure Azure Monitor Alerts
   Set up alerts to notify you when key metrics (such as CPU usage, memory, or failure rates) exceed defined thresholds.

   • Steps:
     Azure Monitor > Alerts > New Alert Rule. Define your target resource, set conditions, and configure actions like sending emails or SMS alerts.

2. Set Up Action Groups
   Use action groups to standardize alert responses, ensuring the right teams are notified promptly.

Azure Kubernetes Service (AKS)​

Step 1: Monitor Application Health on AKS​

1. Set up Azure Monitor with Container Insights
   Enable Container Insights to monitor cluster performance, including CPU, memory, and node availability. 

2. Logging and Tracing on AKS
   Use kubectl logs to retrieve logs from your pods. Integrate with Azure Monitor Logs for centralized log management.

   

3. Key Metrics to Monitor

   • Availability: Uptime of the services deployed in your AKS cluster. 

   • Performance: CPU, memory usage, and network traffic.

   • Error Rates: Failed requests or service outages.

   • Resource Utilization: Node and pod-level resource usage.

     

Step 2: Scale Your Application on AKS​

4. Horizontal Pod Autoscaling (HPA)
   Automatically adjust the number of pod replicas based on observed CPU or memory usage. Configure HPA by defining resource thresholds in your Kubernetes deployment YAML:

   apiVersion: autoscaling/v1
   kind: HorizontalPodAutoscaler
   metadata:
     name: <your-hpa-name>
   spec:
     scaleTargetRef:
       apiVersion: apps/v1
       kind: Deployment
       name: <your-app-name>
     minReplicas: 1
     maxReplicas: 10
     targetCPUUtilizationPercentage: 80
   

5. Cluster Autoscaler
   Automatically scales the AKS cluster by adding or removing nodes based on workload demands.

   • Enable using Azure CLI:

     az aks update \
       --resource-group <RESOURCE_GROUP_NAME> \
       --name <AKS_NAME> \
       --enable-cluster-autoscaler \
       --min-count 1 \
       --max-count 5
     

     

Step 3: Set Up Alerts and Notifications for AKS​

6. Azure Monitor Alerts
   Configure alerts to detect resource thresholds being exceeded, such as CPU usage, memory, or network latency.

   

7. Action Groups
   Use action groups to define responses when an alert is triggered, standardizing notification and automated actions.

Common Continuous Optimization​

Step 4. Implement continuous optimization​

8. Performance Tuning

   • Database: Optimize queries and review the performance of your databases (e.g., Azure Cosmos DB for MongoDB).
   • Caching: Use caching mechanisms (e.g., Azure Cache for Redis) to offload work from the backend.
   • Code Optimization: Identify and resolve performance bottlenecks using profiling tools like Application Insights Profiler.

9. Cost Optimization

   • Monitor Resource Costs: Regularly review resource usage to avoid overprovisioning.
   • Reserved Instances: Use reserved instances for predictable workloads to save on costs.

Summary​

In this post, we provided a comprehensive guide on monitoring, scaling, and optimizing applications on Azure App Service and AKS. We discussed using monitoring tools, configuring scaling strategies, and setting up alerts. Additionally, we covered best practices for continuous optimization to ensure your applications remain reliable, scalable, and cost-effective.

In the next blog, we will demonstrate a live application setup, showcasing how all these components work together in real-world use cases.

What we cover:​

• A walkthrough of the app’s user-facing front end
• A walkthrough of the app’s middleware service
• A walkthrough of the app’s back-end data management service
• A demonstration of how the components work together

Introduction​

Over the last eight days, this blog series has walked you through creating an example application—the Best For You Organics Company’s marketing content generation app. You set up your local tools and the Azure services you needed for development, downloaded and built the app code, and performed an initial deployment to either Azure App Service or Azure Kubernetes Service (AKS). You also investigated deploying to a scalable production environment and explored the considerations you need to manage in the post-deployment phase.

In today’s blog post, we examine the functionality of our example app, now that you’ve deployed it. The app showcases how users can interact with a content generation service, while back-end systems integrate product similarity searches with AI-based content generation models. The app consists of three core services: a ReactJS-based user-facing front end, a Java Spring Boot middleware service, and a Java Spring Boot back-end service.

In the following demo, we present a high-level overview of the app’s interface and functionality, explaining how the app integrates with Azure services, like Azure API Management, Azure Key Vault, Azure Cosmos DB, Azure Blob Storage, and Azure OpenAI Service.

1. The front-end layer: How the user interacts with the app.​

The app’s front-end layer is a ReactJS application designed to provide users with an interface for requesting content through a seamless chat experience. Users enter their queries, select the type of content they want to generate, submit their request, and then receive results from the AI-powered middleware.

Key features:​

• Sign-in experience. Users are prompted to sign in before accessing the chat interface. This secures the app by making sure that only authenticated and authorized users can access it.

  

  

• Chat interface. After signing in, users can type their queries into a chat interface and select the content type they want. The submitted request is sent to the middleware service.

  

• Content display. The generated content is returned from the middleware service and displayed to the user.

  

2. The middleware service: Connecting users, product data, and generated content​

The middleware service is responsible for generating content using OpenAI’s GPT-4o model and retrieving related product data using an embedding vector model. It acts as a bridge between the front end and back end.

Key features:​

• API Management integration. All middleware APIs are exposed via Azure API Management, which applies security policies like API key checks, throttling, Cross-Origin Resource Sharing (CORS) rules, and rate limits.

• Azure OpenAI integration. The middleware connects to Azure OpenAI, using the completion model (GPT-4o) for content generation and the embedding model (text-embedding-3-small) for vector representation of user queries.

  

• Security. The middleware uses Azure Key Vault to securely retrieve API keys and credentials via a managed identity assigned to the App Service or AKS cluster running the service.

3. The back-end service: Managing product-related data​

The back end is responsible for managing product-related data, such as customer information, product details, and vector-based product similarity searches.

Key features:​

• API Management integration. Like the middleware, the back end exposes its APIs (for example, Customer API, Product API, Product Vector API) through API Management with API key checks, CORS, throttling, and rate limits.

• Product similarity search. When the middleware sends a query vector, the back end retrieves a list of similar products based on embeddings and product data stored in Azure Cosmos DB for MongoDB and Blob Storage.

  

• Key Vault integration. All back-end services securely connect to Azure Cosmos DB and Blob Storage, using secrets stored in Azure Key Vault. The key vault is accessed securely by using a managed identity assigned to the App Service or AKS cluster running the service.

4. End-to-end flow demonstration​

Step-by-step user experience​

1. User signs in. The user is prompted to sign in, helping to ensure secure access via identity management.
2. User submits query. After signing in, the user enters their query in the chat interface, selects the type of content, and then selects Submit. This request is sent to the middleware.
3. Middleware processes query. The middleware generates an embedding vector for the query and hits the back-end API Management endpoint to retrieve similar products based on this vector.
4. Back end returns results. The back end fetches product data (for example, similar products) from an Azure Cosmos DB database and then returns the results to the middleware.
5. Middleware generates content. The middleware then calls OpenAI's completion model (GPT-4o) to generate content based on the query and returns it to the front end.
6. User gets response. The generated content is displayed to the user in the chat interface, completing the process.

Summary​

In this blog post, we showcased the power of integrating multiple services and Azure technologies to create a straightforward user experience for AI-based content generation. From secure user sign-in to generating AI-driven content and retrieving relevant product data, the app efficiently uses Azure API Management, Azure Key Vault, and Azure OpenAI Service to meet the needs of modern applications.

In the next post, we’ll dive deeper into how the application works, covering technical details, code, and data flow diagrams.

What we cover:​

• A high-level architecture overview 
• A description of API calls made by the front-end app 
• An overview of the APIs offered by the middleware service 
• A walkthrough of the back-end service’s APIs and integration with other Azure services 

Introduction​

In the previous blog post, we gave you a high-level overview of our example AI-based content generation app, covering its user experience and how it integrated various services. Now we’d like to dive a little deeper into the technical side, explaining the app's architecture, key components, and code snippets and offering an overview of the app’s data flow.  

Today’s post breaks down how the front-end, middleware, and back-end services work together. We also discuss how the app uses Azure services, like Azure API Management, Azure Key Vault, Azure Cosmos DB, Azure Blob Storage, and Azure OpenAI Service, and how managed identity is used to securely connect these components. 

1. High-level architecture​

The application is composed of three core services:

1. Front-end app (ReactJS based). A chat-based UI where users submit queries and receive AI-generated content 

2. Middleware service (Java Spring Boot based). Responsible for content generation and handling product similarity searches 

3. Back-end service (Java Spring Boot based). Handles customer and product data and computes product similarity using vectors 

Architecture diagram​

The architecture diagram shows the interaction between: 

Front end -> Middleware Azure API Management endpoints -> Middleware service -> Back-end API Management endpoints -> Back end (Spring Boot).

How middleware and back-end services connect to Azure Key Vault, Azure Cosmos DB, Azure Blob Storage, and Azure OpenAI.

2. Front-end app​

The front end acts as the interface between the user and the application. It allows the user to sign in, enter queries, and receive generated content. 

Key components:​

• Sign-in and user identity management. The front end uses identity management to authenticate users before they access the main chat functionality. 

  • Code snippet: User authentication logic uses the Microsoft Authentication Library (MSAL). (@azure/msal-browser) 

• Chat interface. After authentication, the user can interact with the chat interface. On form submission, the query is sent to the Middleware API through an API Management endpoint.

3. Middleware service​

The middleware service is the central hub for content generation and managing product similarity searches. It connects to Azure OpenAI for content generation and interfaces with the back end to retrieve product vectors. 

Core APIs:​

• Generate content API. This API receives the user’s query, generates an embedding vector, and calls Azure OpenAI to generate relevant content. 

• Generate embedding vector API. Uses OpenAI's embedding model to transform the user query into an embedding vector that the back-end service can use for similarity matching. 

• OpenAI integration. The middleware connects to OpenAI's GPT-4o completion model for content generation and text embedding model for vector creation. Credentials and API keys are retrieved securely from Azure Key Vault using a managed identity.

4. Back-end service​

The back-end service handles customer and product data and uses the product vector API to find similar products based on embedding vectors sent from the middleware. 

Core APIs:​

• Customer API. Manages customer data and provides endpoints to retrieve and update customer information. 

• Product vector API. Retrieves a list of similar products based on the embedding vector generated by the middleware. 

5. Data flow diagram​

The data flow diagram shows how the user query flows from the front-end app to the middleware service and finally to the back-end service: 

1. User -> Front end. The user submits a query. 

2. Front end -> Middleware API Management endpoint. The query is sent to the middleware via API Management, which enforces API key checks and security. 

3. Middleware -> Back-end API Management endpoint. The middleware sends a vectorized query to the back end via API Management. 

4. Back-end -> Database. The back end retrieves similar products from Azure Cosmos DB. 

5. Middleware -> OpenAI. The middleware generates content using OpenAI’s models. 

6. Middleware -> Front end. The generated content is returned to the front end for display. 

Summary​

In today’s blog post, we discussed the inner workings of our example AI-based content generation app, detailing the interactions between the front-end, middleware, and back-end app tiers. We learned how our app uses various Azure services, like Azure API Management, Azure Key Vault, Azure Cosmos DB, Azure Blob Storage, and Azure OpenAI Service to build a secure and scalable solution.  

In the next post—the last in this series—we'll explore the outcomes of building this application, highlighting its scalability and performance, the overall impact of the architecture and technologies used, and the key learnings to take away from the project. 

What we cover:​

• How the app meets our scalability needs 
• How it addresses our performance needs 
• How it meets our security and compliance requirements 
• How our app’s AI integration improves business outcomes 
• How it optimizes costs 

Introduction​

In this blog series, we’ve walked through an example scenario—creating an AI-based content generation app to support the Best For You Organics Company’s marketing efforts. This app integrates various Azure services with AI models to offer a scalable, secure, and high-performance solution. 

In previous blog posts, we’ve examined our application architecture and covered the process of creating, deploying, and operating the application. In today’s post, the last in this series, we explore the scalability, performance, security, cost optimization, and business impacts of our example app, and we summarize the benefits and takeaways from the development process. 

1. Scalability​

One of the key design goals of this application was scalability, ensuring that the system could handle increasing user traffic and larger datasets without degrading performance. Here are the key factors that allowed us to achieve that in our example app, along with the business impacts of these capabilities. 

Key scalability factors​

• Azure Kubernetes Service (AKS). If we deploy our three app layers to AKS, we can take advantage of built-in scaling options, like horizontal pod autoscaling (HPA), allowing services to scale up based on CPU and memory usage. 
  • Impact. This ensures that as the demand increases, additional instances are spun up automatically, maintaining the app’s responsiveness. 
• Azure App Service. If our app is hosted on App Service, we can benefit from that service’s autoscaling capabilities to handle spikes in user traffic. 
  • Impact. During peak usage periods, the app can handle significant traffic, while maintaining fast response times. 
• Caching for efficiency. By using caching mechanisms at both the middleware and back-end levels, repeated queries can be served faster, reducing the load on back-end systems. 
  • Impact. Improves performance for common user queries and reduces API calls to services like Azure OpenAI Service. 

Business impacts​

• High availability. The app can handle thousands of users simultaneously, with no downtime. 
• Cost-effective scaling. The automatic scaling reduces operational costs by only allocating resources when demand increases.