What We'll Cover​

• What is Cloud-native New Year? (3 initiatives)
• How can I skill up (30 days)
• Who is behind this? (Team Contributors)
• Exercise: Take the Cloud Skills Challenge!
• Resources: #30DaysOfCloudNative Collection.

Welcome to Week 01 of 🥳 #CloudNativeNewYear ! Today, we kick off a full month of content and activities to skill you up on all things Cloud-native on Azure with content, events, and community interactions! Read on to learn about what we have planned!

Explore our initiatives​

We have a number of initiatives planned for the month to help you learn and skill up on relevant technologies. Click on the links to visit the relevant pages for each.

• #30DaysOfCloudNative - 4 themed weeks of daily articles in a structured roadmap
• Cloud Skills Challenge - skill up by competing with peers to complete modules
• Ask The Expert - watch the recorded Q&A sessions with Product Engineering teams

We'll go into more details about #30DaysOfCloudNative in this post - don't forget to subscribe to the blog to get daily posts delivered directly to your preferred feed reader!

Register for events!​

What are 3 things you can do today, to jumpstart your learning journey?

• Register for live Q&A sessions (free, online)
  • Feb 9 - Ask The Expert: Azure Kubernetes Service (PDT)
  • Feb 10 - Ask the Expert: Azure Kubernetes Service (SGT)
• Register for the Cloud Skills Challenge - 30 days to complete it!

#30DaysOfCloudNative​

#30DaysOfCloudNative is a month-long series of daily blog posts grouped into 4 themed weeks - taking you from core concepts to end-to-end solution examples in 30 days. Each article will be short (5-8 mins reading time) and provide exercises and resources to help you reinforce learnings and take next steps.

This series focuses on the Cloud-native On Azure learning journey in four stages, each building on the previous week to help you skill up in a beginner-friendly way:

• Week 1: Get started with Cloud-native Concepts
• Week 2: Build & deploy Kubernetes Apps on cloud.
• Week 3: Migrate your applications to Azure Kubernetes Service.
• Week 4: Go from Code to Containers to Cloud with Cloud-native solutions

We have a tentative weekly-themed roadmap for the topics we hope to cover and will keep this updated as we go with links to actual articles as they get published.

Let's Get Started!​

Now you know everything! We hope you are as excited as we are to dive into a full month of active learning and doing! Don't forget to subscribe for updates in your favorite feed reader! And look out for our first Cloud-native Fundamentals post on January 23rd!

You will often hear the term "cloud-native" when discussing modern application development, but even a quick online search will return a huge number of articles, tweets, and web pages with a variety of definitions. So, what does cloud-native actually mean? Also, what makes an application a cloud-native application versus a "regular" application?

Today, we will address these questions and more as we kickstart our learning journey (and our new year!) with an introductory dive into the wonderful world of cloud-native.

What We'll Cover​

• What is cloud-native?
• What is a cloud-native application?
• The benefits of cloud-native
• The five pillars of cloud-native
• Exercise: Take the Cloud Skills Challenge!

1. What is cloud-native?​

The term "cloud-native" can seem pretty self-evident (yes, hello, native to the cloud?), and in a way, it is. While there are lots of definitions of cloud-native floating around, at it's core, cloud-native simply refers to a modern approach to building software that takes advantage of cloud services and environments. This includes using cloud-native technologies, such as containers, microservices, and serverless, and following best practices for deploying, scaling, and managing applications in a cloud environment.

2. So, what exactly is a cloud-native application?​

Cloud-native applications are specifically designed to take advantage of the scalability, resiliency, and distributed nature of modern cloud infrastructure. But how does this differ from a "traditional" application?

Traditional applications are generally been built, tested, and deployed as a single, monolithic unit. The monolithic nature of this type of architecture creates close dependencies between components. This complexity and interweaving only increases as an application grows and can make it difficult to evolve (not to mention troubleshoot) and challenging to operate over time.

To contrast, in cloud-native architectures the application components are decomposed into loosely coupled services, rather than built and deployed as one block of code. This decomposition into multiple self-contained services enables teams to manage complexity and improve the speed, agility, and scale of software delivery. Many small parts enables teams to make targeted updates, deliver new features, and fix any issues without leading to broader service disruption.

3. The benefits of cloud-native​

Cloud-native architectures can bring many benefits to an organization, including:

1. Scalability: easily scale up or down based on demand, allowing organizations to adjust their resource usage and costs as needed.
2. Flexibility: deploy and run on any cloud platform, and easily move between clouds and on-premises environments.
3. High-availability: techniques such as redundancy, self-healing, and automatic failover help ensure that cloud-native applications are designed to be highly-available and fault tolerant.
4. Reduced costs: take advantage of the pay-as-you-go model of cloud computing, reducing the need for expensive infrastructure investments.
5. Improved security: tap in to cloud security features, such as encryption and identity management, to improve the security of the application.
6. Increased agility: easily add new features or services to your applications to meet changing business needs and market demand.

4. The pillars of cloud-native​

There are five areas that are generally cited as the core building blocks of cloud-native architecture:

1. Microservices: Breaking down monolithic applications into smaller, independent, and loosely-coupled services that can be developed, deployed, and scaled independently.
2. Containers: Packaging software in lightweight, portable, and self-sufficient containers that can run consistently across different environments.
3. Automation: Using automation tools and DevOps processes to manage and operate the cloud-native infrastructure and applications, including deployment, scaling, monitoring, and self-healing.
4. Service discovery: Using service discovery mechanisms, such as APIs & service meshes, to enable services to discover and communicate with each other.
5. Observability: Collecting and analyzing data from the infrastructure and applications to understand and optimize the performance, behavior, and health of the system.

These can (and should!) be used in combination to deliver cloud-native solutions that are highly scalable, flexible, and available.

Resources​

• Register for the Cloud Skills Challenge - 30 days to complete it!
• Resources: #30DaysOfCloudNative Collection
• eBook: Cloud Native Infrastructure with Azure

Don't forget to subscribe to the blog to get daily posts delivered directly to your favorite feed reader!

Today, we'll focus on building an understanding of containers.

What We'll Cover​

• Introduction
• How do Containers Work?
• Why are Containers Becoming so Popular?
• Conclusion
• Resources
• Learning Path

​

Introduction​

In the beginning, we deployed our applications onto physical servers. We only had a certain number of those servers, so often they hosted multiple applications. This led to some problems when those applications shared dependencies. Upgrading one application could break another application on the same server.

Enter virtualization. Virtualization allowed us to run our applications in an isolated operating system instance. This removed much of the risk of updating shared dependencies. However, it increased our overhead since we had to run a full operating system for each application environment.

To address the challenges created by virtualization, containerization was created to improve isolation without duplicating kernel level resources. Containers provide efficient and consistent deployment and runtime experiences for our applications and have become very popular as a way of packaging and distributing applications.

How do Containers Work?​

Containers build on two capabilities in the Linux operating system, namespaces and cgroups. These constructs allow the operating system to provide isolation to a process or group of processes, keeping their access to filesystem resources separate and providing controls on resource utilization. This, combined with tooling to help package, deploy, and run container images has led to their popularity in today’s operating environment. This provides us our isolation without the overhead of additional operating system resources.

When a container host is deployed on an operating system, it works at scheduling the access to the OS (operating systems) components. This is done by providing a logical isolated group that can contain processes for a given application, called a namespace. The container host then manages /schedules access from the namespace to the host OS. The container host then uses cgroups to allocate compute resources. Together, the container host with the help of cgroups and namespaces can schedule multiple applications to access host OS resources.

Overall, this gives the illusion of virtualizing the host OS, where each application gets its own OS. In actuality, all the applications are running on the same operating system and sharing the same kernel as the container host.

Why is Containerization so Popular?​

Containers are popular in the software development industry because they provide several benefits over traditional virtualization methods. Some of these benefits include:

• Portability: Containers make it easy to move an application from one environment to another without having to worry about compatibility issues or missing dependencies.
• Isolation: Containers provide a level of isolation between the application and the host system, which means that the application running in the container cannot access the host system's resources.
• Scalability: Containers make it easy to scale an application up or down as needed, which is useful for applications that experience a lot of traffic or need to handle a lot of data.
• Resource Efficiency: Containers are more resource-efficient than traditional virtualization methods because they don't require a full operating system to be running on each virtual machine.
• Cost-Effective: Containers are more cost-effective than traditional virtualization methods because they don't require expensive hardware or licensing fees.

Conclusion​

Containers are a powerful technology that allows developers to package and deploy applications in a portable and isolated environment. This technology is becoming increasingly popular in the world of software development and is being used by many companies and organizations to improve their application deployment and management processes. With the benefits of portability, isolation, scalability, resource efficiency, and cost-effectiveness, containers are definitely worth considering for your next application development project.

Containerizing applications is a key step in modernizing them, and there are many other patterns that can be adopted to achieve cloud-native architectures, including using serverless platforms, Kubernetes, and implementing DevOps practices.

Resources​

• What are Containers

• Containerizing .NET Applications

Learning Path​

• Introduction to Docker Containers

This week we'll focus on what Kubernetes is.

What We'll Cover​

• Introduction
• What is Kubernetes? (Video)
• How does Kubernetes Work? (Video)
• Conclusion

Introduction​

Kubernetes is an open source container orchestration engine that can help with automated deployment, scaling, and management of our applications.

Kubernetes takes physical (or virtual) resources and provides a consistent API over them, bringing a consistency to the management and runtime experience for our applications. Kubernetes provides us with a number of capabilities such as:

• Container scheduling
• Service discovery and load balancing
• Storage orchestration
• Automated rollouts and rollbacks
• Automatic bin packing
• Self-healing
• Secret and configuration management

We'll learn more about most of these topics as we progress through Cloud Native New Year.

What is Kubernetes?​

Let's hear from Brendan Burns, one of the founders of Kubernetes as to what Kubernetes actually is.

How does Kubernetes Work?​

And Brendan shares a bit more with us about how Kubernetes works.

Conclusion​

Kubernetes allows us to deploy and manage our applications effectively and consistently.

By providing a consistent API across many of the concerns our applications have, like load balancing, networking, storage, and compute, Kubernetes improves both our ability to build and ship new software.

There are standards for the applications to depend on for resources needed. Deployments, metrics, and logs are provided in a standardized fashion allowing more effecient operations across our application environments.

And since Kubernetes is an open source platform, it can be found in just about every type of operating environment - cloud, virtual machines, physical hardware, shared data centers, even small devices like Rasberry Pi's!

Want to learn more? Join us for a webinar on Kubernetes Concepts (or catch the playback) on Thursday, January 26th at 1 PM PST and watch for the rest of this series right here!

This week we'll focus on advanced topics and best practices for Cloud-Native practitioners, kicking off with this post on Serverless Container Options with Azure. We'll look at technologies, tools and best practices that range from managed services like Azure Kubernetes Service, to options allowing finer granularity of control and oversight.

What We'll Cover​

• What is Microservice Architecture?
• How do you design a Microservice?
• What challenges do Microservices introduce?
• Conclusion
• Resources

Microservices are a modern way of designing and building software that increases deployment velocity by decomposing an application into small autonomous services that can be deployed independently.

By deploying loosely coupled microservices your applications can be developed, deployed, and scaled independently. Because each service is independent, it can be updated or replaced without having to worry about the impact on the rest of the application. This means that if a bug is found in one service, it can be fixed without having to redeploy the entire application. All of which gives an organization the ability to deliver value to their customers faster.

In this article, we will explore the basics of microservices architecture, its benefits and challenges, and how it can help improve the development, deployment, and maintenance of software applications.

What is Microservice Architecture?​

Before explaining what Microservice architecture is, it’s important to understand what problems microservices aim to address.

Traditional software development is centered around building monolithic applications. Monolithic applications are built as a single, large codebase. Meaning your code is tightly coupled causing the monolithic app to suffer from the following:

Too much Complexity: Monolithic applications can become complex and difficult to understand and maintain as they grow. This can make it hard to identify and fix bugs and add new features.

Difficult to Scale: Monolithic applications can be difficult to scale as they often have a single point of failure, which can cause the whole application to crash if a service fails.

Slow Deployment: Deploying a monolithic application can be risky and time-consuming, as a small change in one part of the codebase can affect the entire application.

Microservice architecture (often called microservices) is an architecture style that addresses the challenges created by Monolithic applications. Microservices architecture is a way of designing and building software applications as a collection of small, independent services that communicate with each other through APIs. This allows for faster development and deployment cycles, as well as easier scaling and maintenance than is possible with a monolithic application.

How do you design a Microservice?​

Building applications with Microservices architecture requires a different approach. Microservices architecture focuses on business capabilities rather than technical layers, such as data access or messaging. Doing so requires that you shift your focus away from the technical stack and model your applications based upon the various domains that exist within the business.

Domain-driven design (DDD) is a way to design software by focusing on the business needs. You can use Domain-driven design as a framework that guides the development of well-designed microservices by building services that encapsulate knowledge in each domain and abstract that knowledge from clients.

In Domain-driven design you start by modeling the business domain and creating a domain model. A domain model is an abstract model of the business model that distills and organizes a domain of knowledge and provides a common language for developers and domain experts. It’s the resulting domain model that microservices a best suited to be built around because it helps establish a well-defined boundary between external systems and other internal applications.

In short, before you begin designing microservices, start by mapping the functions of the business and their connections to create a domain model for the microservice(s) to be built around.

What challenges do Microservices introduce?​

Microservices solve a lot of problems and have several advantages, but the grass isn’t always greener on the other side.

One of the key challenges of microservices is managing communication between services. Because services are independent, they need to communicate with each other through APIs. This can be complex and difficult to manage, especially as the number of services grows. To address this challenge, it is important to have a clear API design, with well-defined inputs and outputs for each service. It is also important to have a system for managing and monitoring communication between services, to ensure that everything is running smoothly.

Another challenge of microservices is managing the deployment and scaling of services. Because each service is independent, it needs to be deployed and scaled separately from the rest of the application. This can be complex and difficult to manage, especially as the number of services grows. To address this challenge, it is important to have a clear and consistent deployment process, with well-defined steps for deploying and scaling each service. Furthermore, it is advisable to host them on a system with self-healing capabilities to reduce operational burden.

It is also important to have a system for monitoring and managing the deployment and scaling of services, to ensure optimal performance.

Each of these challenges has created fertile ground for tooling and process that exists in the cloud-native ecosystem. Kubernetes, CI CD, and other DevOps practices are part of the package of adopting the microservices architecture.

Conclusion​

In summary, microservices architecture focuses on software applications as a collection of small, independent services that communicate with each other over well-defined APIs.

The main advantages of microservices include:

• increased flexibility and scalability per microservice,
• efficient resource utilization (with help from a container orchestrator like Kubernetes),
• and faster development cycles.

Continue following along with this series to see how you can use Kubernetes to help adopt microservices patterns in your own environments!

Resources​

• Microservice Applications
• Microservices architecture design - Azure Architecture Center | Microsoft Learn
• Design a microservices architecture - Azure Architecture Center | Microsoft Learn
• Domain analysis for microservices - Azure Architecture Center | Microsoft Learn

Today, we will do a brief recap of some of these technologies and provide some basic guidelines for when it is optimal to use each.

What We'll Cover​

• To Containerize or not to Containerize?
• The power of Kubernetes
• Where does Serverless fit?
• Resources
• What's coming next!

To Containerize or not to Containerize?​

As mentioned in our Containers 101 post earlier this week, containers can provide several benefits over traditional virtualization methods, which has made them popular within the software development community. Containers provide a consistent and predictable runtime environment, which can help reduce the risk of compatibility issues and simplify the deployment process. Additionally, containers can improve resource efficiency by allowing multiple applications to run on the same host while isolating their dependencies.

Some types of apps that are a particularly good fit for containerization include:

1. Microservices: Containers are particularly well-suited for microservices-based applications, as they can be used to isolate and deploy individual components of the system. This allows for more flexibility and scalability in the deployment process.
2. Stateless applications: Applications that do not maintain state across multiple sessions, such as web applications, are well-suited for containers. Containers can be easily scaled up or down as needed and replaced with new instances, without losing data.
3. Portable applications: Applications that need to be deployed in different environments, such as on-premises, in the cloud, or on edge devices, can benefit from containerization. The consistent and portable runtime environment of containers can make it easier to move the application between different environments.
4. Legacy applications: Applications that are built using older technologies or that have compatibility issues can be containerized to run in an isolated environment, without impacting other applications or the host system.
5. Dev and testing environments: Containerization can be used to create isolated development and testing environments, which can be easily created and destroyed as needed.

While there are many types of applications that can benefit from a containerized approach, it's worth noting that containerization is not always the best option, and it's important to weigh the benefits and trade-offs before deciding to containerize an application. Additionally, some types of applications may not be a good fit for containers including:

• Apps that require full access to host resources: Containers are isolated from the host system, so if an application needs direct access to hardware resources such as GPUs or specialized devices, it might not work well in a containerized environment.
• Apps that require low-level system access: If an application requires deep access to the underlying operating system, it may not be suitable for running in a container.
• Applications that have specific OS dependencies: Apps that have specific dependencies on a certain version of an operating system or libraries may not be able to run in a container.
• Stateful applications: Apps that maintain state across multiple sessions, such as databases, may not be well suited for containers. Containers are ephemeral by design, so the data stored inside a container may not persist between restarts.

The good news is that some of these limitations can be overcome with the use of specialized containerization technologies such as Kubernetes, and by carefully designing the architecture of the application.

The power of Kubernetes​

Speaking of Kubernetes...

Kubernetes is a powerful tool for managing and deploying containerized applications in production environments, particularly for applications that need to scale, handle large numbers of requests, or run in multi-cloud or hybrid environments.

Kubernetes is well-suited for a wide variety of applications, but it is particularly well-suited for the following types of applications:

1. Microservices-based applications: Kubernetes provides a powerful set of tools for managing and deploying microservices-based applications, making it easy to scale, update, and manage the individual components of the application.
2. Stateful applications: Kubernetes provides support for stateful applications through the use of Persistent Volumes and StatefulSets, allowing for applications that need to maintain state across multiple instances.
3. Large-scale, highly-available systems: Kubernetes provides built-in support for scaling, self-healing, and rolling updates, making it an ideal choice for large-scale, highly-available systems that need to handle large numbers of users and requests.
4. Multi-cloud and hybrid environments: Kubernetes can be used to deploy and manage applications across multiple cloud providers and on-premises environments, making it a good choice for organizations that want to take advantage of the benefits of multiple cloud providers or that need to deploy applications in a hybrid environment.

Where does Serverless fit in?

Serverless is a cloud computing model where the cloud provider (like Azure) is responsible for executing a piece of code by dynamically allocating the resources. With serverless, you only pay for the exact amount of compute time that you use, rather than paying for a fixed amount of resources. This can lead to significant cost savings, particularly for applications with variable or unpredictable workloads.

Serverless is commonly used for building applications like web or mobile apps, IoT, data processing, and real-time streaming - apps where the workloads are variable and high scalability is required. It's important to note that serverless is not a replacement for all types of workloads - it's best suited for stateless, short-lived and small-scale workloads.

For a detailed look into the world of Serverless and lots of great learning content, revisit #30DaysofServerless.

Resources​

• Register for the Cloud Skills Challenge - 30 days to complete it!
• Learning Resources: #30DaysOfCloudNative Collection
• eBook: Cloud Native Infrastructure with Azure
• eBook: Cloud-native Architecture Mapbook

What's up next in #CloudNativeNewYear?​

Week 1 has been all about the fundamentals of cloud-native. Next week, the team will be diving in to application deployment with Azure Kubernetes Service. Don't forget to subscribe to the blog to get daily posts delivered directly to your favorite feed reader!

The theme for this week is Kubernetes fundamentals. Last week we talked about Cloud Native architectures and the Cloud Native landscape. Today we'll explore the topic of Pods and Deployments in Kubernetes.

What We'll Cover​

• Setting Up A Kubernetes Environment in Azure
• Running Containers in Kubernetes Pods
• Making the Pods Resilient with Deployments
• Exercise
• Resources

Setting Up A Kubernetes Environment in Azure​

For this week, we'll be working with a simple app - the Azure Voting App. My teammate Paul Yu ported the app to Rust and we tweaked it a bit to let us highlight some of the basic features of Kubernetes.

You should be able to replicate this in just about any Kubernetes environment, but we'll use Azure Kubernetes Service (AKS) as our working environment for this week.

To make it easier to get started, there's a Bicep template to deploy an AKS cluster, an Azure Container Registry (ACR) (to host our container image), and connect the two so that we can easily deploy our application.

Step 0 - Prerequisites​

There are a few things you'll need if you want to work through this and the following examples this week.

Required:

• Git (and probably a GitHub account if you want to persist your work outside of your computer)
• Azure CLI
• An Azure subscription (if you want to follow along with the Azure steps)
• Kubectl (the command line tool for managing Kubernetes)

Helpful:

• Visual Studio Code (or equivalent editor)

Step 1 - Clone the application repository​

First, I forked the source repository to my account.

$GitHubOrg = 'smurawski' # Replace this with your GitHub account name or org name
git clone "https://github.com/$GitHubOrg/azure-voting-app-rust"
cd azure-voting-app-rust

Leave your shell opened with your current location inside the application repository.

Step 2 - Set up AKS​

Running the template deployment from the demo script (I'm using the PowerShell example in cnny23-week2-day1.ps1, but there's a Bash variant at cnny23-week2-day1.sh) stands up the environment. The second, third, and fourth commands take some of the output from the Bicep deployment to set up for later commands, so don't close out your shell after you run these commands.

az deployment sub create --template-file ./deploy/main.bicep --location eastus --parameters 'resourceGroup=cnny-week2'
$AcrName = az deployment sub show --name main --query 'properties.outputs.acr_name.value' -o tsv
$AksName = az deployment sub show --name main --query 'properties.outputs.aks_name.value' -o tsv
$ResourceGroup = az deployment sub show --name main --query 'properties.outputs.resource_group_name.value' -o tsv

az aks get-credentials --resource-group $ResourceGroup --name $AksName

Step 3 - Build our application container​

Since we have an Azure Container Registry set up, I'll use ACR Build Tasks to build and store my container image.

az acr build --registry $AcrName --% --image cnny2023/azure-voting-app-rust:{{.Run.ID}} .
$BuildTag = az acr repository show-tags `
                              --name $AcrName `
                              --repository cnny2023/azure-voting-app-rust `
                              --orderby time_desc `
                              --query '[0]' -o tsv

Running Containers in Kubernetes Pods​

Now that we have our AKS cluster and application image ready to go, let's look into how Kubernetes runs containers.

If you've been in tech for any length of time, you've seen that every framework, runtime, orchestrator, etc.. can have their own naming scheme for their concepts. So let's get into some of what Kubernetes calls things.

The Pod​

A container running in Kubernetes is called a Pod. A Pod is basically a running container on a Node or VM. It can be more. For example you can run multiple containers and specify some funky configuration, but we'll keep it simple for now - add the complexity when you need it.

Our Pod definition can be created via the kubectl command imperatively from arguments or declaratively from a configuration file. We'll do a little of both. We'll use the kubectl command to help us write our configuration files. Kubernetes configuration files are YAML, so having an editor that supports and can help you syntax check YAML is really helpful.

Creating a Pod Definition​

Let's create a few Pod definitions. Our application requires two containers to get working - the application and a database.

Let's create the database Pod first. And before you comment, the configuration isn't secure nor best practice. We'll fix that later this week. For now, let's focus on getting up and running.

This is a trick I learned from one of my teammates - Paul. By using the --output yaml and --dry-run=client options, we can have the command help us write our YAML. And with a bit of output redirection, we can stash it safely in a file for later use.

kubectl run azure-voting-db `
            --image "postgres:15.0-alpine" `
            --env "POSTGRES_PASSWORD=mypassword" `
            --output yaml `
            --dry-run=client > manifests/pod-db.yaml

This creates a file that looks like:

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: azure-voting-db
  name: azure-voting-db
spec:
  containers:
  - env:
    - name: POSTGRES_PASSWORD
      value: mypassword
    image: postgres:15.0-alpine
    name: azure-voting-db
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

The file, when supplied to the Kubernetes API, will identify what kind of resource to create, the API version to use, and the details of the container (as well as an environment variable to be supplied).

We'll get that container image started with the kubectl command. Because the details of what to create are in the file, we don't need to specify much else to the kubectl command but the path to the file.

kubectl apply -f ./manifests/pod-db.yaml

I'm going to need the IP address of the Pod, so that my application can connect to it, so we can use kubectl to get some information about our pod. By default, kubectl get pod only displays certain information but it retrieves a lot more. We can use the JSONPath syntax to index into the response and get the information you want.

$DB_IP = kubectl get pod azure-voting-db -o jsonpath='{.status.podIP}'

Now, let's create our Pod definition for our application. We'll use the same technique as before.

kubectl run azure-voting-app `
            --image "$AcrName.azurecr.io/cnny2023/azure-voting-app-rust:$BuildTag" `
            --env "DATABASE_SERVER=$DB_IP" `
            --env "DATABASE_PASSWORD=mypassword`
            --output yaml `
            --dry-run=client > manifests/pod-app.yaml

That command gets us a similar YAML file to the database container - you can see the full file here

Let's get our application container running.

kubectl apply -f ./manifests/pod-app.yaml

Now that the Application is Running​

We can check the status of our Pods with:

kubectl get pods

And we should see something like:

azure-voting-app-rust ❯  kubectl get pods
NAME               READY   STATUS    RESTARTS   AGE
azure-voting-app   1/1     Running   0          36s
azure-voting-db    1/1     Running   0          84s

Once our pod is running, we can check to make sure everything is working by letting kubectl proxy network connections to our Pod running the application. If we get the voting web page, we'll know the application found the database and we can start voting!

kubectl port-forward pod/azure-voting-app 8080:8080

When you are done voting, you can stop the port forwarding by using Control-C to break the command.

Clean Up​

Let's clean up after ourselves and see if we can't get Kubernetes to help us keep our application running. We can use the same configuration files to ensure that Kubernetes only removes what we want removed.

kubectl delete -f ./manifests/pod-app.yaml
kubectl delete -f ./manifests/pod-db.yaml

Summary - Pods​

A Pod is the most basic unit of work inside Kubernetes. Once the Pod is deleted, it's gone. That leads us to our next topic (and final topic for today.)

Making the Pods Resilient with Deployments​

We've seen how easy it is to deploy a Pod and get our containers running on Nodes in our Kubernetes cluster. But there's a problem with that. Let's illustrate it.

Breaking Stuff​

Setting Back Up​

First, let's redeploy our application environment. We'll start with our application container.

kubectl apply -f ./manifests/pod-db.yaml
kubectl get pod azure-voting-db -o jsonpath='{.status.podIP}'

The second command will report out the new IP Address for our database container. Let's open ./manifests/pod-app.yaml and update the container IP to our new one.

- name: DATABASE_SERVER
  value: YOUR_NEW_IP_HERE

Then we can deploy the application with the information it needs to find its database. We'll also list out our pods to see what is running.

kubectl apply -f ./manifests/pod-app.yaml
kubectl get pods

Feel free to look back and use the port forwarding trick to make sure your app is running if you'd like.

Knocking It Down​

The first thing we'll try to break is our application pod. Let's delete it.

kubectl delete pod azure-voting-app

Then, we'll check our pod's status:

kubectl get pods

Which should show something like:

azure-voting-app-rust ❯  kubectl get pods
NAME              READY   STATUS    RESTARTS   AGE
azure-voting-db   1/1     Running   0          50s

We should be able to recreate our application pod deployment with no problem, since it has the current database IP address and nothing else depends on it.

kubectl apply -f ./manifests/pod-app.yaml

Again, feel free to do some fun port forwarding and check your site is running.

Uncomfortable Truths​

Here's where it gets a bit stickier, what if we delete the database container?

If we delete our database container and recreate it, it'll likely have a new IP address, which would force us to update our application configuration. We'll look at some solutions for these problems in the next three posts this week.

Because our database problem is a bit tricky, we'll primarily focus on making our application layer more resilient and prepare our database layer for those other techniques over the next few days.

Let's clean back up and look into making things more resilient.

kubectl delete -f ./manifests/pod-app.yaml
kubectl delete -f ./manifests/pod-db.yaml

The Deployment​

One of the reasons you may want to use Kubernetes is it's ability to orchestrate workloads. Part of that orchestration includes being able to ensure that certain workloads are running (regardless of what Node they might be on).

We saw that we could delete our application pod and then restart it from the manifest with little problem. It just meant that we had to run a command to restart it. We can use the Deployment in Kubernetes to tell the orchestrator to ensure we have our application pod running.

The Deployment also can encompass a lot of extra configuration - controlling how many containers of a particular type should be running, how upgrades of container images should proceed, and more.

Creating the Deployment​

First, we'll create a Deployment for our database. We'll use a technique similar to what we did for the Pod, with just a bit of difference.

kubectl create deployment azure-voting-db `
                            --image "postgres:15.0-alpine" `
                            --port 5432 `
                            --output yaml `
                            --dry-run=client > manifests/deployment-db.yaml

Unlike our Pod definition creation, we can't pass in environment variable configuration from the command line. We'll have to edit the YAML file to add that.

So, let's open ./manifests/deployment-db.yaml in our editor and add the following in the spec/containers configuration.

        env:
        - name: POSTGRES_PASSWORD
          value: "mypassword"

Your file should look like this deployment-db.yaml.

Once we have our configuration file updated, we can deploy our database container image.

kubectl apply -f ./manifests/deployment-db.yaml

For our application, we'll use the same technique.

kubectl create deployment azure-voting-app `
                        --image "$AcrName.azurecr.io/cnny2023/azure-voting-app-rust:$BuildTag" `
                        --port 8080 `
                        --output yaml `
                        --dry-run=client > manifests/deployment-app.yaml

Next, we'll need to add an environment variable to the generated configuration. We'll also need the new IP address for the database deployment.

Previously, we named the pod and were able to ask for the IP address with kubectl and a bit of JSONPath. Now, the deployment created the pod for us, so there's a bit of random in the naming. Check out:

kubectl get pods

Should return something like:

azure-voting-app-rust ❯  kubectl get pods
NAME                               READY   STATUS    RESTARTS   AGE
azure-voting-db-686d758fbf-8jnq8   1/1     Running   0          7s

We can either ask for the IP with the new pod name, or we can use a selector to find our desired pod.

kubectl get pod --selector app=azure-voting-db -o jsonpath='{.items[0].status.podIP}'

Now, we can update our application deployment configuration file with:

        env:
        - name: DATABASE_SERVER
          value: YOUR_NEW_IP_HERE
        - name: DATABASE_PASSWORD
          value: mypassword

Your file should look like this deployment-app.yaml (but with IPs and image names matching your environment).

After we save those changes, we can deploy our application.

kubectl apply -f ./manifests/deployment-app.yaml

Let's test the resilience of our app now. First, we'll delete the pod running our application, then we'll check to make sure Kubernetes restarted our application pod.

kubectl get pods

azure-voting-app-rust ❯  kubectl get pods
NAME                                READY   STATUS    RESTARTS   AGE
azure-voting-app-56c9ccc89d-skv7x   1/1     Running   0          71s
azure-voting-db-686d758fbf-8jnq8    1/1     Running   0          12m

kubectl delete pod azure-voting-app-56c9ccc89d-skv7x
kubectl get pods

azure-voting-app-rust ❯  kubectl delete pod azure-voting-app-56c9ccc89d-skv7x
>> kubectl get pods
pod "azure-voting-app-56c9ccc89d-skv7x" deleted
NAME                                READY   STATUS    RESTARTS   AGE
azure-voting-app-56c9ccc89d-2b5mx   1/1     Running   0          2s
azure-voting-db-686d758fbf-8jnq8    1/1     Running   0          14m

As you can see, by the time the kubectl get pods command was run, Kubernetes had already spun up a new pod for the application container image. Thanks Kubernetes!

Clean up​

Since we can't just delete the pods, we have to delete the deployments.

kubectl delete -f ./manifests/deployment-app.yaml
kubectl delete -f ./manifests/deployment-db.yaml

Summary - Deployments​

Deployments allow us to create more durable configuration for the workloads we deploy into Kubernetes. As we dig deeper, we'll discover more capabilities the deployments offer. Check out the Resources below for more.

Exercise​

If you want to try these steps, head over to the source repository, fork it, clone it locally, and give it a spin!

You can check your manifests against the manifests in the week2/day1 branch of the source repository.

Resources​

Documentation​

• Azure Kubernetes Service
• Bicep
• Azure Voting App in Rust
• Pods.
• Nodes
• kubectl
• JSONPath syntax
• Deployment
• Labels and Selectors

Training​

• Learning Path - Introduction to Kubernetes on Azure

The theme for this week is #Kubernetes fundamentals. Yesterday we talked about how to deploy a containerized web app workload to Azure Kubernetes Service (AKS). Today we'll explore the topic of services and ingress and walk through the steps of making our containers accessible both internally as well as over the internet so that you can share it with the world 😊

What We'll Cover​

• Exposing Pods via Service
• Exposing Services via Ingress
• Takeaways
• Resources

Exposing Pods via Service​

There are a few ways to expose your pod in Kubernetes. One way is to take an imperative approach and use the kubectl expose command. This is probably the quickest way to achieve your goal but it isn't the best way. A better way to expose your pod by taking a declarative approach by creating a services manifest file and deploying it using the kubectl apply command.

Don't worry if you are unsure of how to make this manifest, we'll use kubectl to help generate it.

First, let's ensure we have the database deployed on our AKS cluster.

📝 NOTE: If you don't have an AKS cluster deployed, please head over to Azure-Samples/azure-voting-app-rust, clone the repo, and follow the instructions in the README.md to execute the Azure deployment and setup your kubectl context. Check out the first post this week for more on the environment setup.

kubectl apply -f ./manifests/deployment-db.yaml

Next, let's deploy the application. If you are following along from yesterday's content, there isn't anything you need to change; however, if you are deploy the app from scratch, you'll need to modify the deployment-app.yaml manifest and update it with your image tag and database pod's IP address.

kubectl apply -f ./manifests/deployment-app.yaml

Now, let's expose the database using a service so that we can leverage Kubernetes' built-in service discovery to be able to reference it by name; not pod IP. Run the following command.

kubectl expose deployment azure-voting-db \
  --port=5432 \
  --target-port=5432

With the database exposed using service, we can update the app deployment manifest to use the service name instead of pod IP. This way, if the pod ever gets assigned a new IP, we don't have to worry about updating the IP each time and redeploying our web application. Kubernetes has internal service discovery mechanism in place that allows us to reference a service by its name.

Let's make an update to the manifest. Replace the environment variable for DATABASE_SERVER with the following:

- name: DATABASE_SERVER
  value: azure-voting-db

Re-deploy the app with the updated configuration.

kubectl apply -f ./manifests/deployment-app.yaml

One service down, one to go. Run the following command to expose the web application.

kubectl expose deployment azure-voting-app \
  --type=LoadBalancer \
  --port=80 \
  --target-port=8080

Notice the --type argument has a value of LoadBalancer. This service type is implemented by the cloud-controller-manager which is part of the Kubernetes control plane. When using a managed Kubernetes cluster such as Azure Kubernetes Service, a public standard load balancer will be able to provisioned when the service type is set to LoadBalancer. The load balancer will also have a public IP assigned which will make your deployment publicly available.

Kubernetes supports four service types:

• ClusterIP: this is the default and limits service access to internal traffic within the cluster
• NodePort: this assigns a port mapping on the node's IP address and allows traffic from the virtual network (outside the cluster)
• LoadBalancer: as mentioned above, this creates a cloud-based load balancer
• ExternalName: this is used in special case scenarios where you want to map a service to an external DNS name

📝 NOTE: When exposing a web application to the internet, allowing external users to connect to your Service directly is not the best approach. Instead, you should use an Ingress, which we'll cover in the next section.

Now, let's confirm you can reach the web app from the internet. You can use the following command to print the URL to your terminal.

echo "http://$(kubectl get service azure-voting-app -o jsonpath='{.status.loadBalancer.ingress[0].ip}')"

Great! The kubectl expose command gets the job done, but as mentioned above, it is not the best method of exposing deployments. It is better to expose deployments declaratively using a service manifest, so let's delete the services and redeploy using manifests.

kubectl delete service azure-voting-db azure-voting-app

To use kubectl to generate our manifest file, we can use the same kubectl expose command that we ran earlier but this time, we'll include --output=yaml and --dry-run=client. This will instruct the command to output the manifest that would be sent to the kube-api server in YAML format to the terminal.

Generate the manifest for the database service.

kubectl expose deployment azure-voting-db \
  --type=ClusterIP \
  --port=5432 \
  --target-port=5432 \
  --output=yaml \
  --dry-run=client > ./manifests/service-db.yaml

Generate the manifest for the application service.

kubectl expose deployment azure-voting-app \
  --type=LoadBalancer \
  --port=80 \
  --target-port=8080 \
  --output=yaml \
  --dry-run=client > ./manifests/service-app.yaml

The command above redirected the YAML output to your manifests directory. Here is what the web application service looks like.

apiVersion: v1
kind: Service
metadata:
  creationTimestamp: null
  labels:
    app: azure-voting-app
  name: azure-voting-app
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 8080
  selector:
    app: azure-voting-app
  type: LoadBalancer
status:
  loadBalancer: {}

💡 TIP: To view the schema of any api-resource in Kubernetes, you can use the kubectl explain command. In this case the kubectl explain service command will tell us exactly what each of these fields do.

Re-deploy the services using the new service manifests.

kubectl apply -f ./manifests/service-db.yaml -f ./manifests/service-app.yaml

# You should see TYPE is set to LoadBalancer and the EXTERNAL-IP is set
kubectl get service azure-voting-db azure-voting-app

Confirm again that our application is accessible again. Run the following command to print the URL to the terminal.

echo "http://$(kubectl get service azure-voting-app -o jsonpath='{.status.loadBalancer.ingress[0].ip}')"

That was easy, right? We just exposed both of our pods using Kubernetes services. The database only needs to be accessible from within the cluster so ClusterIP is perfect for that. For the web application, we specified the type to be LoadBalancer so that we can access the application over the public internet.

But wait... remember that if you want to expose web applications over the public internet, a Service with a public IP is not the best way; the better approach is to use an Ingress resource.

Exposing Services via Ingress​

If you read through the Kubernetes documentation on Ingress you will see a diagram that depicts the Ingress sitting in front of the Service resource with a routing rule between it. In order to use Ingress, you need to deploy an Ingress Controller and it can be configured with many routing rules to forward traffic to one or many backend services. So effectively, an Ingress is a load balancer for your Services.

With that said, we no longer need a service type of LoadBalancer since the service does not need to be accessible from the internet. It only needs to be accessible from the Ingress Controller (internal to the cluster) so we can change the service type to ClusterIP.

Update your service.yaml file to look like this:

apiVersion: v1
kind: Service
metadata:
  creationTimestamp: null
  labels:
    app: azure-voting-app
  name: azure-voting-app
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 8080
  selector:
    app: azure-voting-app

📝 NOTE: The default service type is ClusterIP so we can omit the type altogether.

Re-apply the app service manifest.

kubectl apply -f ./manifests/service-app.yaml

# You should see TYPE set to ClusterIP and EXTERNAL-IP set to <none>
kubectl get service azure-voting-app

Next, we need to install an Ingress Controller. There are quite a few options, and the Kubernetes-maintained NGINX Ingress Controller is commonly deployed.

You could install this manually by following these instructions, but if you do that you'll be responsible for maintaining and supporting the resource.

I like to take advantage of free maintenance and support when I can get it, so I'll opt to use the Web Application Routing add-on for AKS.

💡 TIP: Whenever you install an AKS add-on, it will be maintained and fully supported by Azure Support.

Enable the web application routing add-on in our AKS cluster with the following command.

az aks addon enable \
  --name <YOUR_AKS_NAME> \
  --resource-group <YOUR_AKS_RESOURCE_GROUP>
  --addon web_application_routing

⚠️ WARNING: This command can take a few minutes to complete

Now, let's use the same approach we took in creating our service to create our Ingress resource. Run the following command to generate the Ingress manifest.

kubectl create ingress azure-voting-app \
  --class=webapprouting.kubernetes.azure.com \
  --rule="/*=azure-voting-app:80" \
  --output yaml \
  --dry-run=client > ./manifests/ingress.yaml

The --class=webapprouting.kubernetes.azure.com option activates the AKS web application routing add-on. This AKS add-on can also integrate with other Azure services such as Azure DNS and Azure Key Vault for TLS certificate management and this special class makes it all work.

The --rule="/*=azure-voting-app:80" option looks confusing but we can use kubectl again to help us understand how to format the value for the option.

kubectl create ingress --help

In the output you will see the following:

--rule=[]:
    Rule in format host/path=service:port[,tls=secretname]. Paths containing the leading character '*' are
    considered pathType=Prefix. tls argument is optional.

It expects a host and path separated by a forward-slash, then expects the backend service name and port separated by a colon. We're not using a hostname for this demo so we can omit it. For the path, an asterisk is used to specify a wildcard path prefix.

So, the value of /*=azure-voting-app:80 creates a routing rule for all paths following the domain (or in our case since we don't have a hostname specified, the IP) to route traffic to our azure-voting-app backend service on port 80.

📝 NOTE: Configuring the hostname and TLS is outside the scope of this demo but please visit this URL https://bit.ly/aks-webapp-routing for an in-depth hands-on lab centered around Web Application Routing on AKS.

Your ingress.yaml file should look like this:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  creationTimestamp: null
  name: azure-voting-app
spec:
  ingressClassName: webapprouting.kubernetes.azure.com
  rules:
  - http:
      paths:
      - backend:
          service:
            name: azure-voting-app
            port:
              number: 80
        path: /
        pathType: Prefix
status:
  loadBalancer: {}

Apply the app ingress manifest.

kubectl apply -f ./manifests/ingress.yaml

Validate the web application is available from the internet again. You can run the following command to print the URL to the terminal.

echo "http://$(kubectl get ingress azure-voting-app -o jsonpath='{.status.loadBalancer.ingress[0].ip}')"

Takeaways​

Exposing your applications both internally and externally can be easily achieved using Service and Ingress resources respectively. If your service is HTTP or HTTPS based and needs to be accessible from outsie the cluster, use Ingress with an internal Service (i.e., ClusterIP or NodePort); otherwise, use the Service resource. If your TCP-based Service needs to be publicly accessible, you set the type to LoadBalancer to expose a public IP for it. To learn more about these resources, please visit the links listed below.

Lastly, if you are unsure how to begin writing your service manifest, you can use kubectl and have it do most of the work for you 🥳

Resources​

• Services
• Ingress Controllers
• Hands-on Lab: Web Application Routing on AKS
• How-to Guide: Ingress Controller in AKS

The theme for this week is Kubernetes fundamentals. Yesterday we talked about Services and Ingress. Today we'll explore the topic of passing configuration and secrets to our applications in Kubernetes with ConfigMaps and Secrets.

What We'll Cover​

• Decouple configurations with ConfigMaps and Secerts
• Passing Environment Data with ConfigMaps and Secrets
• Conclusion

Decouple configurations with ConfigMaps and Secerts​

A ConfigMap is a Kubernetes object that decouples configuration data from pod definitions. Kubernetes secerts are similar, but were designed to decouple senstive information.

Separating the configuration and secerts from your application promotes better organization and security of your Kubernetes environment. It also enables you to share the same configuration and different secerts across multiple pods and deployments which can simplify scaling and management. Using ConfigMaps and Secerts in Kubernetes is a best practice that can help to improve the scalability, security, and maintainability of your cluster.

By the end of this tutorial, you'll have added a Kubernetes ConfigMap and Secret to the Azure Voting deployment.

Passing Environment Data with ConfigMaps and Secrets​

📝 NOTE: If you don't have an AKS cluster deployed, please head over to Azure-Samples/azure-voting-app-rust, clone the repo, and follow the instructions in the README.md to execute the Azure deployment and setup your kubectl context. Check out the first post this week for more on the environment setup.

Create the ConfigMap​

ConfigMaps can be used in one of two ways; as environment variables or volumes.

For this tutorial you'll use a ConfigMap to create three environment variables inside the pod; DATABASE_SERVER, FISRT_VALUE, and SECOND_VALUE. The DATABASE_SERVER provides part of connection string to a Postgres. FIRST_VALUE and SECOND_VALUE are configuration options that change what voting options the application presents to the users.

Follow the below steps to create a new ConfigMap:

1. Create a YAML file named 'config-map.yaml'. In this file, specify the environment variables for the application.

   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: azure-voting-config
   data:
     DATABASE_SERVER: azure-voting-db
     FIRST_VALUE: "Go"
     SECOND_VALUE: "Rust"
   

2. Create the config map in your Kubernetes cluster by running the following command:

   kubectl create -f config-map.yaml
   

Create the Secret​

The deployment-db.yaml and deployment-app.yaml are Kubernetes manifests that deploy the Azure Voting App. Currently, those deployment manifests contain the environment variables POSTGRES_PASSWORD and DATABASE_PASSWORD with the value stored as plain text. Your task is to replace that environment variable with a Kubernetes Secret.

Create a Secret running the following commands:

1. Encode mypassword.

   echo -n "mypassword" | base64
   

2. Create a YAML file named secret.yaml. In this file, add POSTGRES_PASSWORD as the key and the encoded value returned above under as the value in the data section.

   apiVersion: v1
   kind: Secret
   metadata:
     name: azure-voting-secret
   type: Opaque
   data:
     POSTGRES_PASSWORD: bXlwYXNzd29yZA==
   

3. Create the Secret in your Kubernetes cluster by running the following command:

   kubectl create -f secret.yaml
   

[!WARNING] base64 encoding is a simple and widely supported way to obscure plaintext data, it is not secure, as it can easily be decoded. If you want to store sensitive data like password, you should use a more secure method like encrypting with a Key Management Service (KMS) before storing it in the Secret.

Modify the app deployment manifest​

With the ConfigMap and Secert both created the next step is to replace the environment variables provided in the application deployment manuscript with the values stored in the ConfigMap and the Secert.

Complete the following steps to add the ConfigMap and Secert to the deployment mainifest:

1. Open the Kubernetes manifest file deployment-app.yaml.

2. In the containers section, add an envFrom section and upate the env section.

   envFrom:
   - configMapRef:
       name: azure-voting-config
   env:
   - name: DATABASE_PASSWORD
     valueFrom:
       secretKeyRef:
         name: azure-voting-secret
         key: POSTGRES_PASSWORD
   

   Using envFrom exposes all the values witin the ConfigMap as environment variables. Making it so you don't have to list them individually.

3. Save the changes to the deployment manifest file.

4. Apply the changes to the deployment by running the following command:

   kubectl apply -f deployment-app.yaml
   

Modify the database deployment manifest​

Next, update the database deployment manifest and replace the plain text environment variable with the Kubernetes Secert.

1. Open the deployment-db.yaml.

2. To add the secret to the deployment, replace the env section with the following code:

   env:
   - name: POSTGRES_PASSWORD
     valueFrom:
       secretKeyRef:
         name: azure-voting-secret
         key: POSTGRES_PASSWORD
   

3. Apply the updated manifest.

   kubectl apply -f deployment-db.yaml
   

Verify the ConfigMap and output environment variables​

Verify that the ConfigMap was added to your deploy by running the following command:

kubectl describe deployment azure-voting-app

Browse the output until you find the envFrom section with the config map reference.

You can also verify that the environment variables from the config map are being passed to the container by running the command kubectl exec -it <pod-name> -- printenv. This command will show you all the environment variables passed to the pod including the one from configmap.

By following these steps, you will have successfully added a config map to the Azure Voting App Kubernetes deployment, and the environment variables defined in the config map will be passed to the container running in the pod.

Verify the Secret and describe the deployment​

Once the secret has been created you can verify it exists by running the following command:

kubectl get secrets

You can view additional information, such as labels, annotations, type, and the Data by running kubectl describe:

kubectl describe secret azure-voting-secret

By default, the describe command doesn't output the encoded value, but if you output the results as JSON or YAML you'll be able to see the secret's encoded value.

 kubectl get secret azure-voting-secret -o json

Conclusion​

In conclusion, using ConfigMaps and Secrets in Kubernetes can help to improve the scalability, security, and maintainability of your cluster. By decoupling configuration data and sensitive information from pod definitions, you can promote better organization and security in your Kubernetes environment. Additionally, separating these elements allows for sharing the same configuration and different secrets across multiple pods and deployments, simplifying scaling and management.

The theme for this week is Kubernetes fundamentals. Yesterday we talked about how to set app configurations and secrets at runtime using Kubernetes ConfigMaps and Secrets. Today we'll explore the topic of persistent storage on Kubernetes and show you can leverage Persistent Volumes and Persistent Volume Claims to ensure your PostgreSQL data can survive container restarts.

What We'll Cover​

• Containers are ephemeral
• Persistent storage on Kubernetes
• Persistent storage on AKS
• Takeaways
• Resources

Containers are ephemeral​

In our sample application, the frontend UI writes vote values to a backend PostgreSQL database. By default the database container stores its data on the container's local file system, so there will be data loss when the pod is re-deployed or crashes as containers are meant to start with a clean slate each time.

Let's re-deploy our sample app and experience the problem first hand.

📝 NOTE: If you don't have an AKS cluster deployed, please head over to Azure-Samples/azure-voting-app-rust, clone the repo, and follow the instructions in the README.md to execute the Azure deployment and setup your kubectl context. Check out the first post this week for more on the environment setup.

kubectl apply -f ./manifests

Wait for the azure-voting-app service to be assigned a public IP then browse to the website and submit some votes. Use the command below to print the URL to the terminal.

echo "http://$(kubectl get ingress azure-voting-app -o jsonpath='{.status.loadBalancer.ingress[0].ip}')"

Now, let's delete the pods and watch Kubernetes do what it does best... that is, re-schedule pods.

# wait for the pod to come up then ctrl+c to stop watching
kubectl delete --all pod --wait=false && kubectl get po -w

Once the pods have been recovered, reload the website and confirm the vote tally has been reset to zero.

We need to fix this so that the data outlives the container.

Persistent storage on Kubernetes​

In order for application data to survive crashes and restarts, you must implement Persistent Volumes and Persistent Volume Claims.

A persistent volume represents storage that is available to the cluster. Storage volumes can be provisioned manually by an administrator or dynamically using Container Storage Interface (CSI) and storage classes, which includes information on how to provision CSI volumes.

When a user needs to add persistent storage to their application, a persistent volume claim is made to allocate chunks of storage from the volume. This "claim" includes things like volume mode (e.g., file system or block storage), the amount of storage to allocate, the access mode, and optionally a storage class. Once a persistent volume claim has been deployed, users can add the volume to the pod and mount it in a container.

In the next section, we'll demonstrate how to enable persistent storage on AKS.

Persistent storage on AKS​

With AKS, CSI drivers and storage classes are pre-deployed into your cluster. This allows you to natively use Azure Disks, Azure Files, and Azure Blob Storage as persistent volumes. You can either bring your own Azure storage account and use it with AKS or have AKS provision an Azure storage account for you.

To view the Storage CSI drivers that have been enabled in your AKS cluster, run the following command.

az aks show \
  --name <YOUR_AKS_NAME> \
  --resource-group <YOUR_AKS_RESOURCE_GROUP> \
  --query storageProfile

You should see output that looks like this.

{
  "blobCsiDriver": null,
  "diskCsiDriver": {
    "enabled": true,
    "version": "v1"
  },
  "fileCsiDriver": {
    "enabled": true
  },
  "snapshotController": {
    "enabled": true
  }
}

To view the storage classes that have been installed in your cluster, run the following command.

kubectl get storageclass

Workload requirements will dictate which CSI driver and storage class you will need to use.

If you need block storage, then you should use the blobCsiDriver. The driver may not be enabled by default but you can enable it by following instructions which can be found in the Resources section below.

If you need file storage you should leverage either diskCsiDriver or fileCsiDriver. The decision between these two boils down to whether or not you need to have the underlying storage accessible by one pod or multiple pods. It is important to note that diskCsiDriver currently supports access from a single pod only. Therefore, if you need data to be accessible by multiple pods at the same time, then you should opt for fileCsiDriver.

For our PostgreSQL deployment, we'll use the diskCsiDriver and have AKS create an Azure Disk resource for us. There is no need to create a PV resource, all we need to do to is create a PVC using the managed-csi-premium storage class.

Run the following command to create the PVC.

kubectl apply -f - <<EOF            
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-azuredisk
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
  storageClassName: managed-csi-premium
EOF

When you check the PVC resource, you'll notice the STATUS is set to Pending. It will be set to Bound once the volume is mounted in the PostgreSQL container.

kubectl get persistentvolumeclaim

Let's delete the azure-voting-db deployment.

kubectl delete deploy azure-voting-db

Next, we need to apply an updated deployment manifest which includes our PVC.

kubectl apply -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    app: azure-voting-db
  name: azure-voting-db
spec:
  replicas: 1
  selector:
    matchLabels:
      app: azure-voting-db
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: azure-voting-db
    spec:
      containers:
      - image: postgres:15.0-alpine
        name: postgres
        ports:
        - containerPort: 5432
        env:
        - name: POSTGRES_PASSWORD
          valueFrom:
            secretKeyRef:
              name: azure-voting-secret
              key: POSTGRES_PASSWORD
        resources: {}
        volumeMounts:
        - name: mypvc
          mountPath: "/var/lib/postgresql/data"
          subPath: "data"
      volumes:
      - name: mypvc
        persistentVolumeClaim:
          claimName: pvc-azuredisk
EOF

In the manifest above, you'll see that we are mounting a new volume called mypvc (the name can be whatever you want) in the pod which points to a PVC named pvc-azuredisk. With the volume in place, we can mount it in the container by referencing the name of the volume mypvc and setting the mount path to /var/lib/postgresql/data (which is the default path).

💡 IMPORTANT: When mounting a volume into a non-empty subdirectory, you must add subPath to the volume mount and point it to a subdirectory in the volume rather than mounting at root. In our case, when Azure Disk is formatted, it leaves a lost+found directory as documented here.

Watch the pods and wait for the STATUS to show Running and the pod's READY status shows 1/1.

# wait for the pod to come up then ctrl+c to stop watching
kubectl get po -w

Verify that the STATUS of the PVC is now set to Bound

kubectl get persistentvolumeclaim

With the new database container running, let's restart the application pod, wait for the pod's READY status to show 1/1, then head back over to our web browser and submit a few votes.

kubectl delete pod -lapp=azure-voting-app --wait=false && kubectl get po -lapp=azure-voting-app -w

Now the moment of truth... let's rip out the pods again, wait for the pods to be re-scheduled, and confirm our vote counts remain in tact.

kubectl delete --all pod --wait=false && kubectl get po -w

If you navigate back to the website, you'll find the vote are still there 🎉

Takeaways​

By design, containers are meant to be ephemeral and stateless workloads are ideal on Kubernetes. However, there will come a time when your data needs to outlive the container. To persist data in your Kubernetes workloads, you need to leverage PV, PVC, and optionally storage classes. In our demo scenario, we leveraged CSI drivers built into AKS and created a PVC using pre-installed storage classes. From there, we updated the database deployment to mount the PVC in the container and AKS did the rest of the work in provisioning the underlying Azure Disk. If the built-in storage classes does not fit your needs; for example, you need to change the ReclaimPolicy or change the SKU for the Azure resource, then you can create your own custom storage class and configure it just the way you need it 😊

We'll revisit this topic again next week but in the meantime, check out some of the resources listed below to learn more.

See you in the next post!

Resources​

• Kubernetes: Volumes
• Kubernetes: Persistent Volumes
• Container Storage Interface (CSI) for Kubernetes
• Container Storage Interface (CSI) drivers on Azure Kubernetes Service (AKS)
• Enable CSI driver on a new or existing AKS cluster
• AKS: Volumes
• AKS: Storage Classes
• AKS: Built-in Storage Classes

The theme for this week is Kubernetes fundamentals. Yesterday we talked about adding persistent storage to our deployment. Today we'll explore the topic of scaling pods and nodes in our Kubernetes cluster.

What We'll Cover​

• Scaling Our Application
• Scaling Pods
• Scaling Nodes
• Exercise
• Resources

Scaling Our Application​

One of our primary reasons to use a service like Kubernetes to orchestrate our workloads is the ability to scale. We've approached scaling in a multitude of ways over the years, taking advantage of the ever-evolving levels of hardware and software. Kubernetes allows us to scale our units of work, Pods, and the Nodes they run on. This allows us to take advantage of both hardware and software scaling abilities. Kubernetes can help improve the utilization of existing hardware (by scheduling Pods on Nodes that have resource capacity). And, with the capabilities of virtualization and/or cloud hosting (or a bit more work, if you have a pool of physical machines), Kubernetes can expand (or contract) the number of Nodes capable of hosting Pods. Scaling is primarily driven by resource utilization, but can be triggered by a variety of other sources thanks to projects like Kubernetes Event-driven Autoscaling (KEDA).

Scaling Pods​

Our first level of scaling is with our Pods. Earlier, when we worked on our deployment, we talked about how the Kubernetes would use the deployment configuration to ensure that we had the desired workloads running. One thing we didn't explore was running more than one instance of a pod. We can define a number of replicas of a pod in our Deployment.

Manually Scale Pods​

So, if we wanted to define more pods right at the start (or at any point really), we could update our deployment configuration file with the number of replicas and apply that configuration file.

spec:
  replicas: 5

Or we could use the kubectl scale command to update the deployment with a number of pods to create.

kubectl scale --replicas=5 deployment/azure-voting-app

Both of these approaches modify the running configuration of our Kubernetes cluster and request that it ensure that we have that set number of replicas running. Because this was a manual change, the Kubernetes cluster won't automatically increase or decrease the number of pods. It'll just ensure that there are always the specified number of pods running.

Autoscale Pods with the Horizontal Pod Autoscaler​

Another approach to scaling our pods is to allow the Horizontal Pod Autoscaler to help us scale in response to resources being used by the pod. This requires a bit more configuration up front. When we define our pod in our deployment, we need to include resource requests and limits. The requests help Kubernetes determine what nodes may have capacity for a new instance of a pod. The limit tells us where the node should cap utilization for a particular instance of a pod. For example, we'll update our deployment to request 0.25 CPU and set a limit of 0.5 CPU.

    spec:
      containers:
      - image: acrudavoz.azurecr.io/cnny2023/azure-voting-app-rust:ca4
        name: azure-voting-app-rust
        ports:
        - containerPort: 8080
        env:
        - name: DATABASE_URL
          value: postgres://postgres:mypassword@10.244.0.29
        resources:
          requests:
            cpu: 250m
          limits:
            cpu: 500m

Now that we've given Kubernetes an allowed range and an idea of what free resources a node should have to place new pods, we can set up autoscaling. Because autoscaling is a persistent configuration, I like to define it in a configuration file that I'll be able to keep with the rest of my cluster configuration. We'll use the kubectl command to help us write the configuration file. We'll request that Kubernetes watch our pods and when the average CPU utilization if 50% of the requested usage (in our case if it's using more than 0.375 CPU across the current number of pods), it can grow the number of pods serving requests up to 10. If the utilization drops, Kubernetes will have the permission to deprovision pods down to the minimum (three in our example).

kubectl autoscale deployment azure-voting-app --cpu-percent=50 --min=3 --max=10 -o YAML --dry-run=client

Which would give us:

apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
  creationTimestamp: null
  name: azure-voting-app
spec:
  maxReplicas: 10
  minReplicas: 3
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: azure-voting-app
  targetCPUUtilizationPercentage: 50
status:
  currentReplicas: 0
  desiredReplicas: 0

So, how often does the autoscaler check the metrics being monitored? The autoscaler checks the Metrics API every 15 seconds, however the pods stats are only updated every 60 seconds. This means that an autoscale event may be evaluated about once a minute. Once an autoscale down event happens however, Kubernetes has a cooldown period to give the new pods a chance to distribute the workload and let the new metrics accumulate. There is no delay on scale up events.

Application Architecture Considerations​

We've focused in this example on our front end, which is an easier scaling story. When we start talking about scaling our database layers or anything that deals with persistent storage or has primary/replica configuration requirements things get a bit more complicated. Some of these applications may have built-in leader election or could use sidecars to help use existing features in Kubernetes to perform that function. For shared storage scenarios, persistent volumes (or persistent volumes with Azure) can be of help, if the application knows how to play well with shared file access.

Ultimately, you know your application architecture and, while Kubernetes may not have an exact match to how you are doing things today, the underlying capability is probably there under a different name. This abstraction allows you to more effectively use Kubernetes to operate a variety of workloads with the levels of controls you need.

Scaling Nodes​

We've looked at how to scale our pods, but that assumes we have enough resources in our existing pool of nodes to accomodate those scaling requests. Kubernetes can also help scale our available nodes to ensure that our applications have the necessary resources to meet their performance requirements.

Manually Scale Nodes​

Manually scaling nodes isn't a direct function of Kubernetes, so your operating environment instructions may vary. On Azure, it's pretty straight forward. Using the Azure CLI (or other tools), we can tell our AKS cluster to scale up or scale down the number of nodes in our node pool.

First, we'll check out how many nodes we currently have in our working environment.

kubectl get nodes

This will show us

azure-voting-app-rust ❯  kubectl get nodes
NAME                            STATUS   ROLES   AGE     VERSION
aks-pool0-37917684-vmss000000   Ready    agent   5d21h   v1.24.6

Then, we'll scale it up to three nodes.

az aks scale --resource-group $ResourceGroup --name $AksName --node-count 3

Then, we'll check out how many nodes we now have in our working environment.

kubectl get nodes

Which returns:

azure-voting-app-rust ❯  kubectl get nodes
NAME                            STATUS   ROLES   AGE     VERSION
aks-pool0-37917684-vmss000000   Ready    agent   5d21h   v1.24.6
aks-pool0-37917684-vmss000001   Ready    agent   5m27s   v1.24.6
aks-pool0-37917684-vmss000002   Ready    agent   5m10s   v1.24.6

Autoscale Nodes with the Cluster Autoscaler​

Things get more interesting when we start working with the Cluster Autoscaler. The Cluster Autoscaler watches for the inability of Kubernetes to schedule the required number of pods due to resource constraints (and a few other criteria like affinity/anti-affinity). If there are insufficient resources available on the existing nodes, the autoscaler can provision new nodes into the nodepool. Likewise, the autoscaler watches to see if the existing pods could be consolidated to a smaller set of nodes and can remove excess nodes.

Enabling the autoscaler is likewise an update that can be dependent on where and how your Kubernetes cluster is hosted. Azure makes it easy with a simple Azure CLI command.

az aks update `
  --resource-group $ResourceGroup `
  --name $AksName `
  --update-cluster-autoscaler `
  --min-count 1 `
  --max-count 5

There are a variety of settings that can be configured to tune how the autoscaler works.

Scaling on Different Events​

CPU and memory utilization are the primary drivers for the Horizontal Pod Autoscaler, but those might not be the best measures as to when you might want to scale workloads. There are other options for scaling triggers and one of the more common plugins to help with that is the Kubernetes Event-driven Autoscaling (KEDA) project. The KEDA project makes it easy to plug in different event sources to help drive scaling. Find more information about using KEDA on AKS here.

Exercise​

Let's try out the scaling configurations that we just walked through using our sample application. If you still have your environment from Day 1, you can use that.

📝 NOTE: If you don't have an AKS cluster deployed, please head over to Azure-Samples/azure-voting-app-rust, clone the repo, and follow the instructions in the README.md to execute the Azure deployment and setup your kubectl context. Check out the first post this week for more on the environment setup.

Configure Horizontal Pod Autoscaler​

• Edit ./manifests/deployment-app.yaml to include resource requests and limits.

        resources:
          requests:
            cpu: 250m
          limits:
            cpu: 500m

• Apply the updated deployment configuration.

kubectl apply -f ./manifests/deployment-app.yaml

• Create the horizontal pod autoscaler configuration and apply it

kubectl autoscale deployment azure-voting-app --cpu-percent=50 --min=3 --max=10 -o YAML --dry-run=client > ./manifests/scaler-app.yaml
kubectl apply -f ./manifests/scaler-app.yaml

• Check to see your pods scale out to the minimum.

kubectl get pods

Configure Cluster Autoscaler​

Configuring the basic behavior of the Cluster Autoscaler is a bit simpler. We just need to run the Azure CLI command to enable the autoscaler and define our lower and upper limits.

• Check the current nodes available (should be 1).

kubectl get nodes

• Update the cluster to enable the autoscaler

az aks update `
  --resource-group $ResourceGroup `
  --name $AksName `
  --update-cluster-autoscaler `
  --min-count 2 `
  --max-count 5

• Check to see the current number of nodes (should be 2 now).

kubectl get nodes

Resources​

Documentation​

• Manually Scaling Pods and Nodes
• Deployments
• Horizontal Pod Autoscaler
• Leader Election in Kubernetes
• Persistent Volumes
• Persistent Volumes with Azure
• Cluster Autoscaler
• Cluster Autoscaler Profile Settings
• Kubernetes Event-driven Autoscaling (KEDA) project
• KEDA on AKS

Training​

• Application scalability on AKS with HorizontalPodAutoscalers
• Cluster Autoscaling with AKS
• Scale container applications in Azure Kubernetes Services using KEDA

The theme for this week is Bringing Your Application to Kubernetes. Last we talked about Kubernetes Fundamentals. Today we'll explore getting an existing application running in Kubernetes with a full pipeline in GitHub Actions.

What We'll Cover​

• Our Application
• Adding Some Infrastructure as Code
• Building and Publishing a Container Image
• Deploying to Kubernetes
• Summary
• Resources

Our Application​

This week we'll be taking an exisiting application - something similar to a typical line of business application - and setting it up to run in Kubernetes. Over the course of the week, we'll address different concerns. Today we'll focus on updating our CI/CD process to handle standing up (or validating that we have) an Azure Kubernetes Service (AKS) environment, building and publishing container images for our web site and API server, and getting those services running in Kubernetes.

The application we'll be starting with is eShopOnWeb. This application has a web site and API which are backed by a SQL Server instance. It's built in .NET 7, so it's cross-platform.

Adding Some Infrastructure as Code​

Just like last week, we need to stand up an AKS environment. This week, however, rather than running commands in our own shell, we'll set up GitHub Actions to do that for us.

There is a LOT of plumbing in this section, but once it's set up, it'll make our lives a lot easier. This section ensures that we have an environment to deploy our application into configured the way we want. We can easily extend this to accomodate multiple environments or add additional microservices with minimal new effort.

Federated Identity​

Setting up a federated identity will allow us a more securable and auditable way of accessing Azure from GitHub Actions. For more about setting up a federated identity, Microsoft Learn has the details on connecting GitHub Actions to Azure.

Here, we'll just walk through the setup of the identity and configure GitHub to use that idenity to deploy our AKS environment and interact with our Azure Container Registry.

The examples will use PowerShell, but a Bash version of the setup commands is available in the week3/day1 branch.

Prerequisites​

To follow along, you'll need:

• a GitHub account
• an Azure Subscription
• the Azure CLI
• and the Git CLI.

You'll need to fork the source repository under your GitHub user or organization where you can manage secrets and GitHub Actions.

It would be helpful to have the GitHub CLI, but it's not required.

Set Up Some Defaults​

You will need to update one or more of the variables (your user or organization, what branch you want to work off of, and possibly the Azure AD application name if there is a conflict).

# Replace the gitHubOrganizationName value
# with the user or organization you forked
# the repository under.

$githubOrganizationName = 'Azure-Samples'
$githubRepositoryName  = 'eShopOnAKS'
$branchName = 'week3/day1'
$applicationName = 'cnny-week3-day1'

Create an Azure AD Application​

Next, we need to create an Azure AD application.

# Create an Azure AD application
$aksDeploymentApplication = New-AzADApplication -DisplayName $applicationName

Set Up Federation for that Azure AD Application​

And configure that application to allow federated credential requests from our GitHub repository for a particular branch.

# Create a federated identity credential for the application
New-AzADAppFederatedCredential `
   -Name $applicationName `
   -ApplicationObjectId $aksDeploymentApplication.Id `
   -Issuer 'https://token.actions.githubusercontent.com' `
   -Audience 'api://AzureADTokenExchange' `
   -Subject "repo:$($githubOrganizationName)/$($githubRepositoryName):ref:refs/heads/$branchName"

Create a Service Principal for the Azure AD Application​

Once the application has been created, we need a service principal tied to that application. The service principal can be granted rights in Azure.

# Create a service principal for the application
New-AzADServicePrincipal -AppId $($aksDeploymentApplication.AppId)

Give that Service Principal Rights to Azure Resources​

Because our Bicep deployment exists at the subscription level and we are creating role assignments, we need to give it Owner rights. If we changed the scope of the deployment to just a resource group, we could apply more scoped permissions.

$azureContext = Get-AzContext
New-AzRoleAssignment `
   -ApplicationId $($aksDeploymentApplication.AppId) `
   -RoleDefinitionName Owner `
   -Scope $azureContext.Subscription.Id

Add Secrets to GitHub Repository​

If you have the GitHub CLI, you can use that right from your shell to set the secrets needed.

gh secret set AZURE_CLIENT_ID --body $aksDeploymentApplication.AppId
gh secret set AZURE_TENANT_ID --body $azureContext.Tenant.Id
gh secret set AZURE_SUBSCRIPTION_ID --body $azureContext.Subscription.Id

Otherwise, you can create them through the web interface like I did in the Learn Live event below.

Creating a Bicep Deployment​

Resuable Workflows​

We'll create our Bicep deployment in a reusable workflows. What are they? The previous link has the documentation or the video below has my colleague Brandon Martinez and I talking about them.

This workflow is basically the same deployment we did in last week's series, just in GitHub Actions.

Start by creating a file called deploy_aks.yml in the .github/workflows directory with the below contents.

name: deploy

on:
  workflow_call:
    inputs:
      resourceGroupName:
        required: true
        type: string
    secrets:
      AZURE_CLIENT_ID:
        required: true
      AZURE_TENANT_ID:
        required: true
      AZURE_SUBSCRIPTION_ID:
        required: true
    outputs:
      containerRegistryName:
        description: Container Registry Name
        value: ${{ jobs.deploy.outputs.containerRegistryName }}
      containerRegistryUrl:
        description: Container Registry Login Url
        value: ${{ jobs.deploy.outputs.containerRegistryUrl }}
      resourceGroupName:
        description: Resource Group Name
        value: ${{ jobs.deploy.outputs.resourceGroupName }}
      aksName:
        description: Azure Kubernetes Service Cluster Name
        value: ${{ jobs.deploy.outputs.aksName }}

permissions:
  id-token: write
  contents: read

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - uses: azure/login@v1
      name: Sign in to Azure
      with:
        client-id: ${{ secrets.AZURE_CLIENT_ID }}
        tenant-id: ${{ secrets.AZURE_TENANT_ID }}
        subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
    - uses: azure/arm-deploy@v1
      name: Run preflight validation
      with:
        deploymentName: ${{ github.run_number }}
        scope: subscription
        region: eastus
        template: ./deploy/main.bicep
        parameters: >
          resourceGroup=${{ inputs.resourceGroupName }}
        deploymentMode: Validate

  deploy:
    needs: validate
    runs-on: ubuntu-latest
    outputs:
      containerRegistryName: ${{ steps.deploy.outputs.acr_name }}
      containerRegistryUrl: ${{ steps.deploy.outputs.acr_login_server_url }}
      resourceGroupName: ${{ steps.deploy.outputs.resource_group_name }}
      aksName: ${{ steps.deploy.outputs.aks_name }}
    steps:
    - uses: actions/checkout@v2
    - uses: azure/login@v1
      name: Sign in to Azure
      with:
        client-id: ${{ secrets.AZURE_CLIENT_ID }}
        tenant-id: ${{ secrets.AZURE_TENANT_ID }}
        subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
    - uses: azure/arm-deploy@v1
      id: deploy
      name: Deploy Bicep file
      with:
        failOnStdErr: false
        deploymentName: ${{ github.run_number }}
        scope: subscription
        region: eastus
        template: ./deploy/main.bicep
        parameters: >
          resourceGroup=${{ inputs.resourceGroupName }}

Adding the Bicep Deployment​

Once we have the Bicep deployment workflow, we can add it to the primary build definition in .github/workflows/dotnetcore.yml

Permissions​

First, we need to add a permissions block to let the workflow request our Azure AD token. This can go towards the top of the YAML file (I started it on line 5).

permissions:
  id-token: write
  contents: read

Deploy AKS Job​

Next, we'll add a reference to our reusable workflow. This will go after the build job.

  deploy_aks:
    needs: [build]
    uses: ./.github/workflows/deploy_aks.yml
    with:
      resourceGroupName: 'cnny-week3'
    secrets:
      AZURE_CLIENT_ID: ${{ secrets.AZURE_CLIENT_ID }}
      AZURE_TENANT_ID: ${{ secrets.AZURE_TENANT_ID }}
      AZURE_SUBSCRIPTION_ID: ${{ secrets.AZURE_SUBSCRIPTION_ID }}

Building and Publishing a Container Image​

Now that we have our target environment in place and an Azure Container Registry, we can build and publish our container images.

Add a Reusable Workflow​

First, we'll create a new file for our reusable workflow at .github/workflows/publish_container_image.yml.

We'll start the file with a name, the parameters it needs to run, and the permissions requirements for the federated identity request.

name: Publish Container Images

on: 
  workflow_call:
    inputs:
      containerRegistryName:
        required: true
        type: string
      containerRegistryUrl:
        required: true
        type: string
      githubSha:
        required: true
        type: string
    secrets:
      AZURE_CLIENT_ID:
        required: true
      AZURE_TENANT_ID:
        required: true
      AZURE_SUBSCRIPTION_ID:
        required: true

permissions:
  id-token: write
  contents: read

Build the Container Images​

Our next step is to build the two container images we'll need for the application, the website and the API. We'll build the container images on our build worker and tag it with the git SHA, so there'll be a direct tie between the point in time in our codebase and the container images that represent it.

jobs:
  publish_container_image:
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v2
    - name: docker build
      run: |
        docker build . -f src/Web/Dockerfile -t ${{ inputs.containerRegistryUrl }}/web:${{ inputs.githubSha }}
        docker build . -f src/PublicApi/Dockerfile -t ${{ inputs.containerRegistryUrl }}/api:${{ inputs.githubSha}}

Scan the Container Images​

Before we publish those container images, we'll scan them for vulnerabilities and best practice violations. We can add these two steps (one scan for each image).

    - name: scan web container image
      uses: Azure/container-scan@v0
      with:
        image-name: ${{ inputs.containerRegistryUrl }}/web:${{ inputs.githubSha}}
    - name: scan api container image
      uses: Azure/container-scan@v0
      with:
        image-name: ${{ inputs.containerRegistryUrl }}/web:${{ inputs.githubSha}}

The container images provided have a few items that'll be found. We can create an allowed list at .github/containerscan/allowedlist.yaml to define vulnerabilities or best practice violations that we'll explictly allow to not fail our build.

general:
  vulnerabilities:
    - CVE-2022-29458
    - CVE-2022-3715
    - CVE-2022-1304
    - CVE-2021-33560
    - CVE-2020-16156
    - CVE-2019-8457
    - CVE-2018-8292
  bestPracticeViolations:
    - CIS-DI-0001
    - CIS-DI-0005  
    - CIS-DI-0006 
    - CIS-DI-0008  

Publish the Container Images​

Finally, we'll log in to Azure, then log in to our Azure Container Registry, and push our images.

    - uses: azure/login@v1
      name: Sign in to Azure
      with:
        client-id: ${{ secrets.AZURE_CLIENT_ID }}
        tenant-id: ${{ secrets.AZURE_TENANT_ID }}
        subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
    - name: acr login 
      run: az acr login --name ${{ inputs.containerRegistryName  }}
    - name: docker push
      run: |
        docker push ${{ inputs.containerRegistryUrl }}/web:${{ inputs.githubSha}}
        docker push ${{ inputs.containerRegistryUrl }}/api:${{ inputs.githubSha}}

Update the Build With the Image Build and Publish​

Now that we have our reusable workflow to create and publish our container images, we can include that in our primary build defnition at .github/workflows/dotnetcore.yml.

  publish_container_image:
    needs: [deploy_aks]
    uses: ./.github/workflows/publish_container_image.yml
    with:
      containerRegistryName: ${{ needs.deploy_aks.outputs.containerRegistryName }}
      containerRegistryUrl: ${{ needs.deploy_aks.outputs.containerRegistryUrl }}
      githubSha: ${{ github.sha }}
    secrets:
      AZURE_CLIENT_ID: ${{ secrets.AZURE_CLIENT_ID }}
      AZURE_TENANT_ID: ${{ secrets.AZURE_TENANT_ID }}
      AZURE_SUBSCRIPTION_ID: ${{ secrets.AZURE_SUBSCRIPTION_ID }}

Deploying to Kubernetes​

Finally, we've gotten enough set up that a commit to the target branch will:

• build and test our application code
• set up (or validate) our AKS and ACR environment
• and create, scan, and publish our container images to ACR

Our last step will be to deploy our application to Kubernetes. We'll use the basic building blocks we worked with last week, deployments and services.

Starting the Reusable Workflow to Deploy to AKS​

We'll start our workflow with our parameters that we need, as well as the permissions to access the token to log in to Azure.

We'll check out our code, then log in to Azure, and use the az CLI to get credentials for our AKS cluster.

name: deploy_to_aks

on:
  workflow_call:
    inputs:
      aksName:
        required: true
        type: string
      resourceGroupName:
        required: true
        type: string
      containerRegistryUrl:
        required: true
        type: string
      githubSha:
        required: true
        type: string
    secrets:
      AZURE_CLIENT_ID:
        required: true
      AZURE_TENANT_ID:
        required: true
      AZURE_SUBSCRIPTION_ID:
        required: true

permissions:
  id-token: write
  contents: read

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:  
      - uses: actions/checkout@v2
      - uses: azure/login@v1
        name: Sign in to Azure
        with:
          client-id: ${{ secrets.AZURE_CLIENT_ID }}
          tenant-id: ${{ secrets.AZURE_TENANT_ID }}
          subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
      - name: Get AKS Credentials
        run: |
          az aks get-credentials --resource-group ${{ inputs.resourceGroupName }} --name ${{ inputs.aksName }}

Edit the Deployment For Our Current Image Tag​

Let's add the Kubernetes manifests to our repo. This post is long enough, so you can find the content for the manifests folder in the manifests folder in the source repo under the week3/day1 branch.

git remote add upstream https://github.com/Azure-Samples/eShopOnAks
git fetch upstream week3/day1
git checkout upstream/week3/day1 manifests

The deployments and the service defintions should be familiar from last week's content (but not the same). This week, however, there's a new file in the manifests - ./manifests/kustomization.yaml

This file helps us more dynamically edit our kubernetes manifests and support is baked right in to the kubectl command.

Kustomize Definition​

Kustomize allows us to specify specific resource manifests and areas of that manifest to replace. We've put some placeholders in our file as well, so we can replace those for each run of our CI/CD system.

In ./manifests/kustomization.yaml you will see:

resources:
- deployment-api.yaml
- deployment-web.yaml

# Change the image name and version
images:
- name: notavalidregistry.azurecr.io/api:v0.1.0
  newName: <YOUR_ACR_SERVER>/api
  newTag: <YOUR_IMAGE_TAG>
- name: notavalidregistry.azurecr.io/web:v0.1.0
  newName: <YOUR_ACR_SERVER>/web
  newTag: <YOUR_IMAGE_TAG>

Replacing Values in our Build​

Now, we encounter a little problem - our deployment files need to know the tag and ACR server. We can do a bit of sed magic to edit the file on the fly.

In .github/workflows/deploy_to_aks.yml, we'll add:

      - name: replace_placeholders_with_current_run
        run: |
          sed -i "s/<YOUR_ACR_SERVER>/${{ inputs.containerRegistryUrl }}/g" ./manifests/kustomization.yaml
          sed -i "s/<YOUR_IMAGE_TAG>/${{ inputs.githubSha }}/g" ./manifests/kustomization.yaml

Deploying the Manifests​

We have our manifests in place and our kustomization.yaml file (with commands to update it at runtime) ready to go, we can deploy our manifests.

First, we'll deploy our database (deployment and service). Next, we'll use the -k parameter on kubectl to tell it to look for a kustomize configuration, transform the requested manifests and apply those. Finally, we apply the service defintions for the web and API deployments.

        run: |
          kubectl apply -f ./manifests/deployment-db.yaml \
                        -f ./manifests/service-db.yaml
          kubectl apply -k ./manifests
          kubectl apply -f ./manifests/service-api.yaml \
                        -f ./manifests/service-web.yaml

Summary​

We've covered a lot of ground in today's post. We set up federated credentials with GitHub. Then we added reusable workflows to deploy an AKS environment and build/scan/publish our container images, and then to deploy them into our AKS environment.

This sets us up to start making changes to our application and Kubernetes configuration and have those changes automatically validated and deployed by our CI/CD system. Tomorrow, we'll look at updating our application environment with runtime configuration, persistent storage, and more.

Resources​

• Azure Kubernetes Service (AKS)
• Reusable workflows in GitHub Actions
• Connecting GitHub Actions to Azure
• Kustomize
• GitHub CLI
• eShopOnAKS

The theme for this week is Bringing Your Application to Kubernetes. Yesterday we talked about getting an existing application running in Kubernetes with a full pipeline in GitHub Actions. Today we'll evaluate our sample application's configuration, storage, and networking requirements and implement using Kubernetes and Azure resources.

What We'll Cover​

• Gather requirements
• Implement environment variables using ConfigMaps
• Implement persistent volumes using Azure Files
• Implement secrets using Azure Key Vault
• Re-package deployments
• Conclusion
• Resources

Gather requirements​

The eShopOnWeb application is written in .NET 7 and has two major pieces of functionality. The web UI is where customers can browse and shop. The web UI also includes an admin portal for managing the product catalog. This admin portal, is packaged as a WebAssembly application and relies on a separate REST API service. Both the web UI and the REST API connect to the same SQL Server container.

Looking through the source code which can be found here we can identify requirements for configs, persistent storage, and secrets.

Database server​

• Need to store the password for the sa account as a secure secret
• Need persistent storage volume for data directory
• Need to inject environment variables for SQL Server license type and EULA acceptance

Web UI and REST API service​

• Need to store database connection string as a secure secret
• Need to inject ASP.NET environment variables to override app settings
• Need persistent storage volume for ASP.NET key storage

Implement environment variables using ConfigMaps​

ConfigMaps are relatively straight-forward to create. If you were following along with the examples last week, this should be review 😉

Create a ConfigMap to store database environment variables.

kubectl apply -f - <<EOF
apiVersion: v1
kind: ConfigMap
metadata:
  name: mssql-settings
data:
  MSSQL_PID: Developer
  ACCEPT_EULA: "Y"
EOF

Create another ConfigMap to store ASP.NET environment variables.

kubectl apply -f - <<EOF
apiVersion: v1
kind: ConfigMap
metadata:
  name: aspnet-settings
data:
  ASPNETCORE_ENVIRONMENT: Development
EOF

Implement persistent volumes using Azure Files​

Similar to last week, we'll take advantage of storage classes built into AKS. For our SQL Server data, we'll use the azurefile-csi-premium storage class and leverage an Azure Files resource as our PersistentVolume.

Create a PersistentVolumeClaim (PVC) for persisting SQL Server data.

kubectl apply -f - <<EOF
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mssql-data
spec:
  accessModes:
  - ReadWriteMany
  storageClassName: azurefile-csi-premium
  resources:
    requests:
      storage: 5Gi
EOF

Create another PVC for persisting ASP.NET data.

kubectl apply -f - <<EOF
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: aspnet-data
spec:
  accessModes:
  - ReadWriteMany
  storageClassName: azurefile-csi-premium
  resources:
    requests:
      storage: 5Gi
EOF

Implement secrets using Azure Key Vault​

It's a well known fact that Kubernetes secretes are not really secrets. They're just base64-encoded values and not secure, especially if malicious users have access to your Kubernetes cluster.

In a production scenario, you will want to leverage an external vault like Azure Key Vault or HashiCorp Vault to encrypt and store secrets.

With AKS, we can enable the Secrets Store CSI driver add-on which will allow us to leverage Azure Key Vault.

# Set some variables
RG_NAME=<YOUR_RESOURCE_GROUP_NAME>
AKS_NAME=<YOUR_AKS_CLUSTER_NAME>
ACR_NAME=<YOUR_ACR_NAME>

az aks enable-addons \
  --addons azure-keyvault-secrets-provider \
  --name $AKS_NAME \
  --resource-group $RG_NAME

With the add-on enabled, you should see aks-secrets-store-csi-driver and aks-secrets-store-provider-azure resources installed on each node in your Kubernetes cluster.

Run the command below to verify.

kubectl get pods \
  --namespace kube-system \
  --selector 'app in (secrets-store-csi-driver, secrets-store-provider-azure)'

The Secrets Store CSI driver allows us to use secret stores via Container Storage Interface (CSI) volumes. This provider offers capabilities such as mounting and syncing between the secure vault and Kubernetes Secrets. On AKS, the Azure Key Vault Provider for Secrets Store CSI Driver enables integration with Azure Key Vault.

You may not have an Azure Key Vault created yet, so let's create one and add some secrets to it.

AKV_NAME=$(az keyvault create \
  --name akv-eshop$RANDOM \
  --resource-group $RG_NAME \
  --query name -o tsv)

# Database server password
az keyvault secret set \
  --vault-name $AKV_NAME \
  --name mssql-password \
  --value "@someThingComplicated1234"

# Catalog database connection string
az keyvault secret set \
  --vault-name $AKV_NAME \
  --name mssql-connection-catalog \
  --value "Server=db;Database=Microsoft.eShopOnWeb.CatalogDb;User Id=sa;Password=@someThingComplicated1234;TrustServerCertificate=True;"

# Identity database connection string
az keyvault secret set \
  --vault-name $AKV_NAME \
  --name mssql-connection-identity \
  --value "Server=db;Database=Microsoft.eShopOnWeb.Identity;User Id=sa;Password=@someThingComplicated1234;TrustServerCertificate=True;"

Pods authentication using Azure Workload Identity​

In order for our Pods to retrieve secrets from Azure Key Vault, we'll need to set up a way for the Pod to authenticate against Azure AD. This can be achieved by implementing the new Azure Workload Identity feature of AKS.

The workload identity feature within AKS allows us to leverage native Kubernetes resources and link a Kubernetes ServiceAccount to an Azure Managed Identity to authenticate against Azure AD.

For the authentication flow, our Kubernetes cluster will act as an Open ID Connect (OIDC) issuer and will be able issue identity tokens to ServiceAccounts which will be assigned to our Pods.

The Azure Managed Identity will be granted permission to access secrets in our Azure Key Vault and with the ServiceAccount being assigned to our Pods, they will be able to retrieve our secrets.

For more information on how the authentication mechanism all works, check out this doc.

To implement all this, start by enabling the new preview feature for AKS.

az feature register \
  --namespace "Microsoft.ContainerService" \
  --name "EnableWorkloadIdentityPreview"

Check the status and ensure the state shows Regestered before moving forward.

az feature show \
  --namespace "Microsoft.ContainerService" \
  --name "EnableWorkloadIdentityPreview"

Update your AKS cluster to enable the workload identity feature and enable the OIDC issuer endpoint.

az aks update \
  --name $AKS_NAME \
  --resource-group $RG_NAME \
  --enable-workload-identity \
  --enable-oidc-issuer 

Create an Azure Managed Identity and retrieve its client ID.

MANAGED_IDENTITY_CLIENT_ID=$(az identity create \
  --name aks-workload-identity \
  --resource-group $RG_NAME \
  --subscription $(az account show --query id -o tsv) \
  --query 'clientId' -o tsv)

Create the Kubernetes ServiceAccount.

# Set namespace (this must align with the namespace that your app is deployed into)
SERVICE_ACCOUNT_NAMESPACE=default

# Set the service account name
SERVICE_ACCOUNT_NAME=eshop-serviceaccount

# Create the service account
kubectl apply -f - <<EOF
apiVersion: v1
kind: ServiceAccount
metadata:
  annotations:
    azure.workload.identity/client-id: ${MANAGED_IDENTITY_CLIENT_ID}
  labels:
    azure.workload.identity/use: "true"
  name: ${SERVICE_ACCOUNT_NAME}
  namespace: ${SERVICE_ACCOUNT_NAMESPACE}
EOF

That was a lot... Let's review what we just did.

We have an Azure Managed Identity (object in Azure AD), an OIDC issuer URL (endpoint in our Kubernetes cluster), and a Kubernetes ServiceAccount.

The next step is to "tie" these components together and establish a Federated Identity Credential so that Azure AD can trust authentication requests from your Kubernetes cluster.

To establish the federated credential, we'll need the OIDC issuer URL, and a subject which points to your Kubernetes ServiceAccount.

# Get the OIDC issuer URL
OIDC_ISSUER_URL=$(az aks show \
  --name $AKS_NAME \
  --resource-group $RG_NAME \
  --query "oidcIssuerProfile.issuerUrl" -o tsv)

# Set the subject name using this format: `system:serviceaccount:<YOUR_SERVICE_ACCOUNT_NAMESPACE>:<YOUR_SERVICE_ACCOUNT_NAME>`
SUBJECT=system:serviceaccount:$SERVICE_ACCOUNT_NAMESPACE:$SERVICE_ACCOUNT_NAME

az identity federated-credential create \
  --name aks-federated-credential \
  --identity-name aks-workload-identity \
  --resource-group $RG_NAME \
  --issuer $OIDC_ISSUER_URL \
  --subject $SUBJECT

With the authentication components set, we can now create a SecretProviderClass which includes details about the Azure Key Vault, the secrets to pull out from the vault, and identity used to access the vault.

# Get the tenant id for the key vault
TENANT_ID=$(az keyvault show \
  --name $AKV_NAME \
  --resource-group $RG_NAME \
  --query properties.tenantId -o tsv)

# Create the secret provider for azure key vault
kubectl apply -f - <<EOF
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: eshop-azure-keyvault
spec:
  provider: azure
  parameters:
    usePodIdentity: "false"
    useVMManagedIdentity: "false"   
    clientID: "${MANAGED_IDENTITY_CLIENT_ID}"
    keyvaultName: "${AKV_NAME}"
    cloudName: ""
    objects:  |
      array:
        - |
          objectName: mssql-password
          objectType: secret
          objectVersion: ""
        - |
          objectName: mssql-connection-catalog
          objectType: secret
          objectVersion: ""
        - |
          objectName: mssql-connection-identity
          objectType: secret
          objectVersion: ""
    tenantId: "${TENANT_ID}"
  secretObjects:
  - secretName: eshop-secrets
    type: Opaque
    data:
      - objectName: mssql-password
        key: mssql-password
      - objectName: mssql-connection-catalog
        key: mssql-connection-catalog
      - objectName: mssql-connection-identity
        key: mssql-connection-identity
EOF

Finally, lets grant the Azure Managed Identity permissions to retrieve secrets from the Azure Key Vault.

az keyvault set-policy \
  --name $AKV_NAME \
  --secret-permissions get \
  --spn $MANAGED_IDENTITY_CLIENT_ID

Re-package deployments​

Update your database deployment to load environment variables from our ConfigMap, attach the PVC and SecretProviderClass as volumes, mount the volumes into the Pod, and use the ServiceAccount to retrieve secrets.

Additionally, you may notice the database Pod is set to use fsGroup:10001 as part of the securityContext. This is required as the MSSQL container runs using a non-root account called mssql and this account has the proper permissions to read/write data at the /var/opt/mssql mount path.

kubectl apply -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: db
  labels:
    app: db
spec:
  replicas: 1
  selector:
    matchLabels:
      app: db
  template:
    metadata:
      labels:
        app: db
    spec:
      securityContext:
        fsGroup: 10001
      serviceAccountName: ${SERVICE_ACCOUNT_NAME}
      containers:
        - name: db
          image: mcr.microsoft.com/mssql/server:2019-latest
          ports:
            - containerPort: 1433
          envFrom:
            - configMapRef:
                name: mssql-settings
          env:
            - name: MSSQL_SA_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: eshop-secrets
                  key: mssql-password
          resources: {}
          volumeMounts:
            - name: mssqldb
              mountPath: /var/opt/mssql
            - name: eshop-secrets
              mountPath: "/mnt/secrets-store"
              readOnly: true
      volumes:
        - name: mssqldb
          persistentVolumeClaim:
            claimName: mssql-data
        - name: eshop-secrets
          csi:
            driver: secrets-store.csi.k8s.io
            readOnly: true
            volumeAttributes:
              secretProviderClass: eshop-azure-keyvault
EOF

We'll update the API and Web deployments in a similar way.

# Set the image tag
IMAGE_TAG=<YOUR_IMAGE_TAG>

# API deployment
kubectl apply -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
  labels:
    app: api
spec:
  replicas: 1
  selector:
    matchLabels:
      app: api
  template:
    metadata:
      labels:
        app: api
    spec:
      serviceAccount: ${SERVICE_ACCOUNT_NAME}
      containers:
        - name: api
          image: ${ACR_NAME}.azurecr.io/api:${IMAGE_TAG}
          ports:
            - containerPort: 80
          envFrom:
            - configMapRef:
                name: aspnet-settings
          env:
            - name: ConnectionStrings__CatalogConnection
              valueFrom:
                secretKeyRef:
                  name: eshop-secrets
                  key: mssql-connection-catalog
            - name: ConnectionStrings__IdentityConnection
              valueFrom:
                secretKeyRef:
                  name: eshop-secrets
                  key: mssql-connection-identity
          resources: {}
          volumeMounts:
            - name: aspnet
              mountPath: ~/.aspnet/https:/root/.aspnet/https:ro
            - name: eshop-secrets
              mountPath: "/mnt/secrets-store"
              readOnly: true
      volumes:
        - name: aspnet
          persistentVolumeClaim:
            claimName: aspnet-data
        - name: eshop-secrets
          csi:
            driver: secrets-store.csi.k8s.io
            readOnly: true
            volumeAttributes:
                secretProviderClass: eshop-azure-keyvault
EOF

## Web deployment
kubectl apply -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
  labels:
    app: web
spec:
  replicas: 1
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      serviceAccount: ${SERVICE_ACCOUNT_NAME}
      containers:
        - name: web
          image: ${ACR_NAME}.azurecr.io/web:${IMAGE_TAG}
          ports:
            - containerPort: 80
          envFrom:
            - configMapRef:
                name: aspnet-settings
          env:
            - name: ConnectionStrings__CatalogConnection
              valueFrom:
                secretKeyRef:
                  name: eshop-secrets
                  key: mssql-connection-catalog
            - name: ConnectionStrings__IdentityConnection
              valueFrom:
                secretKeyRef:
                  name: eshop-secrets
                  key: mssql-connection-identity
          resources: {}
          volumeMounts:
            - name: aspnet
              mountPath: ~/.aspnet/https:/root/.aspnet/https:ro
            - name: eshop-secrets
              mountPath: "/mnt/secrets-store"
              readOnly: true
      volumes:
        - name: aspnet
          persistentVolumeClaim:
            claimName: aspnet-data
        - name: eshop-secrets
          csi:
            driver: secrets-store.csi.k8s.io
            readOnly: true
            volumeAttributes:
                secretProviderClass: eshop-azure-keyvault
EOF