Module 7 - ACA Scheduled Jobs with Dapr Cron Binding¶

Module Duration

60 minutes

Curious about Azure Container Apps jobs?

There is a new kid on the block. Azure Container Apps jobs became generally available in late August 2023. This workshop is not yet updated to account for this new type of container app. Stay tuned for updates!

Objective¶

In this module, we will accomplish three objectives:

Learn how the Cron binding can trigger actions.
Add a Cron binding to the Backend Background Processor.
Deploy updated Background Processor and API projects to Azure.

Module Sections¶

From the VS Code Terminal tab, open developer command prompt or PowerShell terminal in the project folder TasksTracker.ContainerApps (root):
```
cd ~\TasksTracker.ContainerApps
```
Restore the previously-stored variables by executing the local script. The output informs you how many variables have been set.
```
.\Variables.ps1
```

1. The Cron Binding¶

In the preceding module, we discussed how Dapr bindings can simplify the integration process with external systems by facilitating the handling of events and the invocation of external resources.

In this module we will focus on a special type of Dapr input binding named Cron Binding.

The Cron binding doesn't subscribe to events coming from an external system. Instead, this binding can be used to trigger application code in our service periodically based on a configurable interval. The binding provides a simple way to implement a background worker to wake up and do some work at a regular interval, without the need to implement an endless loop with a configurable delay. We intend to utilize this binding for a specific use case, wherein it will be triggered once daily at a particular time (12:05 am), and search for tasks that have a due date matching the previous day of its execution and are still pending. Once the service identifies tasks that meet these criteria, it will designate them as overdue tasks and save the revised status on Azure Cosmos DB.

Contrasting the binding to Azure Container Apps jobs, we do not need a separate container app and can integrate this binding into our existing backend service.

2. Updating the Backend Background Processor Project¶

2.1 Add Cron Binding Configuration¶

To set up the Cron binding, we add a component file that specifies the code that requires triggering and the intervals at which it should occur.

To accomplish this, create a new file called dapr-scheduled-cron.yaml within the components folder and insert the following code:

dapr-scheduled-cron.yaml

apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
  name: ScheduledTasksManager
  namespace: default
spec:
  type: bindings.cron
  version: v1
  metadata:
    - name: schedule
      value: "5 0 * * *" # Everyday at 12:05am
scopes:
  - tasksmanager-backend-processor

Curious to learn more about above yaml file configuration?

The actions performed above are as follows:

Added a new input binding of type bindings.cron.
Provided the name ScheduledTasksManager for this binding. This means that an HTTP POST endpoint on the URL /ScheduledTasksManager should be added as it will be invoked when the job is triggered based on the Cron interval.
Setting the interval for this Cron job to be triggered once a day at 12:05am. For full details and available options on how to set this value, visit the Cron binding specs..

2.2 Add the Endpoint Which Will be Invoked by Cron Binding¶

Let's add an endpoint which will be triggered when the Cron configuration is met. This endpoint will contain the routine needed to run at a regular interval.

Add a new file under the Controllers folder in the project TasksTracker.Processor.Backend.Svc as shown below:

ScheduledTasksManagerController.cs

using Dapr.Client;
using Microsoft.AspNetCore.Mvc;
using TasksTracker.Processor.Backend.Svc.Models;

namespace TasksTracker.Processor.Backend.Svc.Controllers
{
    [Route("ScheduledTasksManager")]
    [ApiController]
    public class ScheduledTasksManagerController : ControllerBase
    {
        private readonly ILogger<ScheduledTasksManagerController> _logger;
        private readonly DaprClient _daprClient;
        public ScheduledTasksManagerController(ILogger<ScheduledTasksManagerController> logger, DaprClient daprClient)
        {
            _logger = logger;
            _daprClient = daprClient;
        }

        [HttpPost]
        public async Task CheckOverDueTasksJob()
        {
            var runAt = DateTime.UtcNow;

            _logger.LogInformation($"ScheduledTasksManager::Timer Services triggered at: {runAt}");

            var overdueTasksList = new List<TaskModel>();

            var tasksList = await _daprClient.InvokeMethodAsync<List<TaskModel>>(HttpMethod.Get, "tasksmanager-backend-api", $"api/overduetasks");

            _logger.LogInformation($"ScheduledTasksManager::completed query state store for tasks, retrieved tasks count: {tasksList?.Count()}");

            tasksList?.ForEach(taskModel =>
            {
                if (runAt.Date> taskModel.TaskDueDate.Date)
                {
                    overdueTasksList.Add(taskModel);
                }
            });

            if (overdueTasksList.Count> 0)
            {
                _logger.LogInformation($"ScheduledTasksManager::marking {overdueTasksList.Count()} as overdue tasks");

                await _daprClient.InvokeMethodAsync(HttpMethod.Post, "tasksmanager-backend-api", $"api/overduetasks/markoverdue", overdueTasksList);
            }
        }
    }
}

Here, we have added a new action method called CheckOverDueTasksJob, which includes the relevant business logic that will be executed by the Cron job configuration at specified intervals. This action method must be of the POST type, allowing it to be invoked when the job is triggered in accordance with the Cron interval.

2.3 Update the Backend Web API Project¶

Now we need to add two new methods which are used by the scheduled job.

Update these files under the Services folder in the project TasksTracker.TasksManager.Backend.Api as highlighted below:

ITasksManager.csTasksStoreManager.cs

using TasksTracker.TasksManager.Backend.Api.Models;

namespace TasksTracker.TasksManager.Backend.Api.Services
{
    public interface ITasksManager
    {
        Task<List<TaskModel>> GetTasksByCreator(string createdBy);
        Task<TaskModel?> GetTaskById(Guid taskId);
        Task<Guid> CreateNewTask(string taskName, string createdBy, string assignedTo, DateTime dueDate);
        Task<bool> UpdateTask(Guid taskId, string taskName, string assignedTo, DateTime dueDate);
        Task<bool> MarkTaskCompleted(Guid taskId);
        Task<bool> DeleteTask(Guid taskId);
        Task MarkOverdueTasks(List<TaskModel> overdueTasksList);
        Task<List<TaskModel>> GetYesterdaysDueTasks();
    }
}

using Dapr.Client;
using System.Text.Json;
using System.Text.Encodings.Web;
using System.Text.Json.Serialization;
using TasksTracker.TasksManager.Backend.Api.Models;

namespace TasksTracker.TasksManager.Backend.Api.Services
{
    public class TasksStoreManager : ITasksManager
    {
        private static string STORE_NAME = "statestore";
        private readonly DaprClient _daprClient;
        private readonly IConfiguration _config;
        private readonly ILogger<TasksStoreManager> _logger;

        public TasksStoreManager(DaprClient daprClient, IConfiguration config, ILogger<TasksStoreManager> logger)
        {
            _daprClient = daprClient;
            _config = config;
            _logger = logger;
        }
        public async Task<Guid> CreateNewTask(string taskName, string createdBy, string assignedTo, DateTime dueDate)
        {
            var taskModel = new TaskModel()
            {
                TaskId = Guid.NewGuid(),
                TaskName = taskName,
                TaskCreatedBy = createdBy,
                TaskCreatedOn = DateTime.UtcNow,
                TaskDueDate = dueDate,
                TaskAssignedTo = assignedTo,
            };

            _logger.LogInformation("Save a new task with name: '{0}' to state store", taskModel.TaskName);
            await _daprClient.SaveStateAsync<TaskModel>(STORE_NAME, taskModel.TaskId.ToString(), taskModel);
            return taskModel.TaskId;
        }

        public async Task<bool> DeleteTask(Guid taskId)
        {
            _logger.LogInformation("Delete task with Id: '{0}'", taskId);
            await _daprClient.DeleteStateAsync(STORE_NAME, taskId.ToString());
            return true;
        }

        public async Task<TaskModel?> GetTaskById(Guid taskId)
        {
            _logger.LogInformation("Getting task with Id: '{0}'", taskId);
            var taskModel = await _daprClient.GetStateAsync<TaskModel>(STORE_NAME, taskId.ToString());
            return taskModel;
        }

        public async Task<List<TaskModel>> GetTasksByCreator(string createdBy)
        {
            var query = "{" +
                    "\"filter\": {" +
                        "\"EQ\": { \"taskCreatedBy\": \"" + createdBy + "\" }" +
                    "}}";

            var queryResponse = await _daprClient.QueryStateAsync<TaskModel>(STORE_NAME, query);

            var tasksList = queryResponse.Results.Select(q => q.Data).OrderByDescending(o=>o.TaskCreatedOn);
            return tasksList.ToList();
        }

        public async Task<bool> MarkTaskCompleted(Guid taskId)
        {
            _logger.LogInformation("Mark task with Id: '{0}' as completed", taskId);
            var taskModel = await _daprClient.GetStateAsync<TaskModel>(STORE_NAME, taskId.ToString());
            if (taskModel != null)
            {
                taskModel.IsCompleted = true;
                await _daprClient.SaveStateAsync<TaskModel>(STORE_NAME, taskModel.TaskId.ToString(), taskModel);
                return true;
            }
            return false;
        }

        public async Task<bool> UpdateTask(Guid taskId, string taskName, string assignedTo, DateTime dueDate)
        {
            _logger.LogInformation("Update task with Id: '{0}'", taskId);
            var taskModel = await _daprClient.GetStateAsync<TaskModel>(STORE_NAME, taskId.ToString());
            var currentAssignee = taskModel.TaskAssignedTo;
            if (taskModel != null)
            {
                taskModel.TaskName = taskName;
                taskModel.TaskAssignedTo = assignedTo;
                taskModel.TaskDueDate = dueDate;
                await _daprClient.SaveStateAsync<TaskModel>(STORE_NAME, taskModel.TaskId.ToString(), taskModel);
                return true;
            }
            return false;
        }

        public async Task<List<TaskModel>> GetYesterdaysDueTasks()
        {
            var options = new JsonSerializerOptions
            {
                PropertyNamingPolicy = JsonNamingPolicy.CamelCase,
                WriteIndented = true,
                Converters =
                {
                    new JsonStringEnumConverter(),
                    new DateTimeConverter("yyyy-MM-ddTHH:mm:ss")
                },
                PropertyNameCaseInsensitive = true,
                DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull,
                Encoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping
            };
            var yesterday = DateTime.Today.AddDays(-1);

            var jsonDate = JsonSerializer.Serialize(yesterday, options);

            _logger.LogInformation("Getting overdue tasks for yesterday date: '{0}'", jsonDate);

            var query = "{" +
                    "\"filter\": {" +
                        "\"EQ\": { \"taskDueDate\": " + jsonDate + " }" +
                    "}}";

            var queryResponse = await _daprClient.QueryStateAsync<TaskModel>(STORE_NAME, query);
            var tasksList = queryResponse.Results.Select(q => q.Data).Where(q=>q.IsCompleted==false && q.IsOverDue==false).OrderBy(o=>o.TaskCreatedOn);
            return tasksList.ToList();
        }

        public async Task MarkOverdueTasks(List<TaskModel> overDueTasksList)
        {
            foreach (var taskModel in overDueTasksList)
            {
                _logger.LogInformation("Mark task with Id: '{0}' as OverDue task", taskModel.TaskId);
                taskModel.IsOverDue = true;
                await _daprClient.SaveStateAsync<TaskModel>(STORE_NAME, taskModel.TaskId.ToString(), taskModel);
            }
        }
    }
}

Add a new file in a new Utilities folder in the project TasksTracker.TasksManager.Backend.Api as shown below:

DateTimeConverter.cs

using System.Text.Json;
using System.Text.Json.Serialization;

namespace TasksTracker.TasksManager.Backend.Api.Services
{
    public class DateTimeConverter : JsonConverter<DateTime>
    {
        private readonly string _dateFormatString;

        public DateTimeConverter(string dateFormatString)
        {
            _dateFormatString = dateFormatString;
        }

        public override DateTime Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
        {
            var dateString = reader.GetString();

            if (dateString != null) {
                return DateTime.ParseExact(dateString, _dateFormatString, System.Globalization.CultureInfo.InvariantCulture);
            } else {
                throw new("Date string from reader is null.");
            }
        }

        public override void Write(Utf8JsonWriter writer, DateTime value, JsonSerializerOptions options)
        {
            writer.WriteStringValue(value.ToString(_dateFormatString));
        }
    }
}

Curious to learn more about above code?

What we've implemented here is the following:

Method GetYesterdaysDueTasks will query the Cosmos DB state store using Dapr State API to lookup all the yesterday's task which are not completed yet. Remember that Cron job is configured to run each day at 12:05am so we are interested to check only the day before when the service runs. We initially made this implementation simple. There might be some edge cases not handled with the current implementation.
Method MarkOverdueTasks will take list of all tasks which passed the due date and set the flag IsOverDue to true.

Add the new methods to the fake implementation for class FakeTasksManager.cs so the project TasksTracker.TasksManager.Backend.Api builds successfully.

FakeTasksManager.cs

using TasksTracker.TasksManager.Backend.Api.Models;

namespace TasksTracker.TasksManager.Backend.Api.Services
{
    public class FakeTasksManager : ITasksManager
    {
        List<TaskModel> _tasksList = new List<TaskModel>();
        Random rnd = new Random();

        private void GenerateRandomTasks()
        {
            for (int i = 0; i < 10; i++)
            {
                var task = new TaskModel()
                {
                    TaskId = Guid.NewGuid(),
                    TaskName = $"Task number: {i}",
                    TaskCreatedBy = "tjoudeh@bitoftech.net",
                    TaskCreatedOn = DateTime.UtcNow.AddMinutes(i),
                    TaskDueDate = DateTime.UtcNow.AddDays(i),
                    TaskAssignedTo = $"assignee{rnd.Next(50)}@mail.com",
                };
                _tasksList.Add(task);
            }
        }

        public FakeTasksManager()
        {
            GenerateRandomTasks();
        }

        public Task<Guid> CreateNewTask(string taskName, string createdBy, string assignedTo, DateTime dueDate)
        {
            var task = new TaskModel()
            {
                TaskId = Guid.NewGuid(),
                TaskName = taskName,
                TaskCreatedBy = createdBy,
                TaskCreatedOn = DateTime.UtcNow,
                TaskDueDate = dueDate,
                TaskAssignedTo = assignedTo,
            };

            _tasksList.Add(task);

            return Task.FromResult(task.TaskId);
        }

        public Task<bool> DeleteTask(Guid taskId)
        {
            var task = _tasksList.FirstOrDefault(t => t.TaskId.Equals(taskId));

            if (task != null)
            {
                _tasksList.Remove(task);
                return Task.FromResult(true);
            }

            return Task.FromResult(false);
        }

        public Task<TaskModel?> GetTaskById(Guid taskId)
        {
            var taskModel = _tasksList.FirstOrDefault(t => t.TaskId.Equals(taskId));
            return Task.FromResult(taskModel);
        }

        public Task<List<TaskModel>> GetTasksByCreator(string createdBy)
        {
            var tasksList = _tasksList.Where(t => t.TaskCreatedBy.Equals(createdBy)).OrderByDescending(o => o.TaskCreatedOn).ToList();
            return Task.FromResult(tasksList);
        }

        public Task<bool> MarkTaskCompleted(Guid taskId)
        {
            var task = _tasksList.FirstOrDefault(t => t.TaskId.Equals(taskId));

            if (task != null)
            {
                task.IsCompleted = true;
                return Task.FromResult(true);
            }

            return Task.FromResult(false);
        }

        public Task<bool> UpdateTask(Guid taskId, string taskName, string assignedTo, DateTime dueDate)
        {
            var task = _tasksList.FirstOrDefault(t => t.TaskId.Equals(taskId));

            if (task != null)
            {
                task.TaskName = taskName;
                task.TaskAssignedTo = assignedTo;
                task.TaskDueDate = dueDate;
                return Task.FromResult(true);
            }

            return Task.FromResult(false);
        }

        public Task MarkOverdueTasks(List<TaskModel> overDueTasksList)
        {
            throw new NotImplementedException();
        }

        public Task<List<TaskModel>> GetYesterdaysDueTasks()
        {
            var tasksList = _tasksList.Where(t => t.TaskDueDate.Equals(DateTime.Today.AddDays(-1))).ToList();

            return Task.FromResult(tasksList);
        }  
    }
}

2.4 Add Action Methods to Backend Web API project¶

As you've seen previously, we are using a Dapr Service-to-Service invocation API to call methods api/overduetasks and api/overduetasks/markoverdue in the Backend Web API from the Backend Background Processor.

Add a new file under the Controllers folder in the project TasksTracker.TasksManager.Backend.Api as shown below:

OverdueTasksController.cs

using Microsoft.AspNetCore.Mvc;
using TasksTracker.TasksManager.Backend.Api.Models;
using TasksTracker.TasksManager.Backend.Api.Services;

namespace TasksTracker.TasksManager.Backend.Api.Controllers
{
    [Route("api/overduetasks")]
    [ApiController]
    public class OverdueTasksController : ControllerBase
    {
        private readonly ILogger<TasksController> _logger;
        private readonly ITasksManager _tasksManager;

        public OverdueTasksController(ILogger<TasksController> logger, ITasksManager tasksManager)
        {
            _logger = logger;
            _tasksManager = tasksManager;
        }

        [HttpGet]
        public async Task<IEnumerable<TaskModel>> Get()
        {
            return await _tasksManager.GetYesterdaysDueTasks();
        }

        [HttpPost("markoverdue")]
        public async Task<IActionResult> Post([FromBody] List<TaskModel> overdueTasksList)
        {
            await _tasksManager.MarkOverdueTasks(overdueTasksList);

            return Ok();
        }
    }
}

2.5 Add Cron Binding Configuration Matching ACA Specs¶

Add a new file folder aca-components. This file will be used when updating the Azure Container App Env and enable this binding.

containerapps-scheduled-cron.yaml

componentType: bindings.cron
version: v1
metadata:
  - name: schedule
    value: "5 0 * * *" # Everyday at 12:05am
scopes:
  - tasksmanager-backend-processor

Note

The name of the binding is not part of the file metadata. We are going to set the name of the binding to the value ScheduledTasksManager when we update the Azure Container Apps Env.

3. Deploy the Backend Background Processor and the Backend API Projects to Azure Container Apps¶

3.1 Build the Backend Background Processor and the Backend API App Images and Push them to ACR¶

To prepare for deployment to Azure Container Apps, we must build and deploy both application images to ACR, just as we did before. We can use the same PowerShell console use the following code (make sure you are on directory TasksTracker.ContainerApps):

az acr build `
--registry $AZURE_CONTAINER_REGISTRY_NAME `
--image "tasksmanager/$BACKEND_API_NAME" `
--file 'TasksTracker.TasksManager.Backend.Api/Dockerfile' . 

az acr build `
--registry $AZURE_CONTAINER_REGISTRY_NAME `
--image "tasksmanager/$BACKEND_SERVICE_NAME" `
--file 'TasksTracker.Processor.Backend.Svc/Dockerfile' .

3.2 Add the Cron Dapr Component to ACA Environment¶

# Cron binding component
az containerapp env dapr-component set `
--name $ENVIRONMENT --resource-group $RESOURCE_GROUP `
--dapr-component-name scheduledtasksmanager `
--yaml '.\aca-components\containerapps-scheduled-cron.yaml'

3.3 Deploy New Revisions of the Backend API and Backend Background Processor to ACA¶

As we did before, we need to update the Azure Container App hosting the Backend API & Backend Background Processor with a new revision so our code changes are available for the end users. To accomplish this run the PowerShell script below:

# Update Backend API App container app and create a new revision 
az containerapp update `
--name $BACKEND_API_NAME `
--resource-group $RESOURCE_GROUP `
--revision-suffix v$TODAY-4

# Update Backend Background Processor container app and create a new revision 
az containerapp update `
--name $BACKEND_SERVICE_NAME `
--resource-group $RESOURCE_GROUP `
--revision-suffix v$TODAY-4

Note

The service ScheduledTasksManager which will be triggered by the Cron job on certain intervals is hosted in the ACA service ACA-Processor Backend. In the future module we are going to scale this ACA ACA-Processor Backend to multiple replicas/instances.

It is highly recommended that background periodic jobs are hosted in a container app with one single replica, you don't want your background periodic job to run on multiple replicas trying to do the same thing. This, in fact, would be a limitation that could call for a separate Azure Container App jobs implementation as we typically want or app/API/service to scale.

Success

With those changes in place and deployed, from the Azure portal, you can open the log streams of the container app hosting the ACA-Processor-Backend and check the logs generated when the Cron job is triggered, you should see logs similar to the below image

Note

Keep in mind though that you won't be able to see the results instantaneously as the cron job searches for tasks that have a due date matching the previous day of its execution and are still pending.

Navigate to the root and persist the module to Git.
```
git add .
git commit -m "Add Module 7"
```

Review¶

In this module, we have accomplished three objectives:

Learned how the Cron binding can trigger actions.
Added a Cron binding to the Backend Background Processor.
Deployed updated Background Processor and API projects to Azure.