Module 7 - ACA Scheduled Jobs with Dapr Cron Binding¶
In the preceding module, we discussed how Dapr bindings can simplify the integration process with external systems by facilitating the handling of events and the invocation of external resources. In this module we will focus on a special type of Dapr input binding named Cron Binding.
The Cron binding doesn't subscribe for events coming from an external system. Instead, this binding can be used to trigger application code in our service periodically based on a configurable interval. The binding provides a simple way to implement a background worker to wake up and do some work at a regular interval, without the need to implement an endless loop with a configurable delay.
We intend to utilize this binding for a specific use case, wherein it will be triggered once daily at a particular time (12:05 am), and search for tasks that have a due date matching the previous day of its execution and are still pending. Once the service identifies tasks that meet these criteria, it will designate them as overdue tasks and save the revised status on Azure Cosmos DB.
Updating the Backend Background Processor Project¶
1. Add Cron Binding Configuration¶
To set up the Cron binding, the initial step involves adding a component file that specifies the location of the code that requires triggering and the intervals at which it should occur. To accomplish this, create a new file called dapr-scheduled-cron.yaml within the components folder and insert the following code:
Add new file under components as shown below:
Curious to learn more about above yaml file configuration?
The actions performed above are as follows:
- Added a new input binding of type
bindings.cron
. - Provided the name
ScheduledTasksManager
for this binding. This means that an HTTP POST endpoint on the URL/ScheduledTasksManager
should be added as it will be invoked when the job is triggered based on the Cron interval. - Setting the interval for this Cron job to be triggered once a day at 12:05am. For full details and available options on how to set this value, visit the Cron binding specs..
2. Add the Endpoint Which Will be Invoked by Cron Binding¶
Let's add an endpoint which will be triggered when the Cron configuration is met. This endpoint will contain the routine needed to run at a regular interval.
Add new file under controllers folder in the project TasksTracker.Processor.Backend.Svc as shown below:
using Dapr.Client;
using Microsoft.AspNetCore.Mvc;
using TasksTracker.Processor.Backend.Svc.Models;
namespace TasksTracker.Processor.Backend.Svc.Controllers
{
[Route("ScheduledTasksManager")]
[ApiController]
public class ScheduledTasksManagerController : ControllerBase
{
private readonly ILogger<ScheduledTasksManagerController> _logger;
private readonly DaprClient _daprClient;
public ScheduledTasksManagerController(ILogger<ScheduledTasksManagerController> logger,
DaprClient daprClient)
{
_logger = logger;
_daprClient = daprClient;
}
[HttpPost]
public async Task CheckOverDueTasksJob()
{
var runAt = DateTime.UtcNow;
_logger.LogInformation($"ScheduledTasksManager::Timer Services triggered at: {runAt}");
var overdueTasksList = new List<TaskModel>();
var tasksList = await _daprClient.InvokeMethodAsync<List<TaskModel>>(HttpMethod.Get, "tasksmanager-backend-api", $"api/overduetasks");
_logger.LogInformation($"ScheduledTasksManager::completed query state store for tasks, retrieved tasks count: {tasksList?.Count()}");
tasksList?.ForEach(taskModel =>
{
if (runAt.Date> taskModel.TaskDueDate.Date)
{
overdueTasksList.Add(taskModel);
}
});
if (overdueTasksList.Count> 0)
{
_logger.LogInformation($"ScheduledTasksManager::marking {overdueTasksList.Count()} as overdue tasks");
await _daprClient.InvokeMethodAsync(HttpMethod.Post, "tasksmanager-backend-api", $"api/overduetasks/markoverdue", overdueTasksList);
}
}
}
}
Here, we have added a new action method called CheckOverDueTasksJob
, which includes the relevant business logic that will be executed by the Cron job configuration at specified intervals.
This action method must be of the POST
type, allowing it to be invoked when the job is triggered in accordance with the Cron interval.
3. Update the Backend Web API Project¶
Now we need to add two new methods which are used by the scheduled job.
Update below files under services folder in the project TasksTracker.TasksManager.Backend.Api as highlighted below:
using System.Text.Json;
using System.Text.Encodings.Web;
using System.Text.Json.Serialization;
public async Task<List<TaskModel>> GetYesterdaysDueTasks()
{
var options = new JsonSerializerOptions
{
PropertyNamingPolicy = JsonNamingPolicy.CamelCase,
WriteIndented = true,
Converters =
{
new JsonStringEnumConverter(),
new DateTimeConverter("yyyy-MM-ddTHH:mm:ss")
},
PropertyNameCaseInsensitive = true,
DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull,
Encoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping
};
var yesterday = DateTime.Today.AddDays(-1);
var jsonDate = JsonSerializer.Serialize(yesterday, options);
_logger.LogInformation("Getting overdue tasks for yesterday date: '{0}'", jsonDate);
var query = "{" +
"\"filter\": {" +
"\"EQ\": { \"taskDueDate\": " + jsonDate + " }" +
"}}";
var queryResponse = await _daprClient.QueryStateAsync<TaskModel>(STORE_NAME, query);
var tasksList = queryResponse.Results.Select(q => q.Data).Where(q=>q.IsCompleted==false && q.IsOverDue==false).OrderBy(o=>o.TaskCreatedOn);
return tasksList.ToList();
}
public async Task MarkOverdueTasks(List<TaskModel> overDueTasksList)
{
foreach (var taskModel in overDueTasksList)
{
_logger.LogInformation("Mark task with Id: '{0}' as OverDue task", taskModel.TaskId);
taskModel.IsOverDue = true;
await _daprClient.SaveStateAsync<TaskModel>(STORE_NAME, taskModel.TaskId.ToString(), taskModel);
}
}
Add below file under Utilities folder in the project TasksTracker.TasksManager.Backend.Api as shown below:
using System.Text.Json;
using System.Text.Json.Serialization;
namespace TasksTracker.TasksManager.Backend.Api.Services
{
public class DateTimeConverter : JsonConverter<DateTime>
{
private readonly string _dateFormatString;
public DateTimeConverter(string dateFormatString)
{
_dateFormatString = dateFormatString;
}
public override DateTime Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
{
return DateTime.ParseExact(reader.GetString(), _dateFormatString, System.Globalization.CultureInfo.InvariantCulture);
}
public override void Write(Utf8JsonWriter writer, DateTime value, JsonSerializerOptions options)
{
writer.WriteStringValue(value.ToString(_dateFormatString));
}
}
}
Curious to learn more about above code?
What we've implemented here is the following:
- Method
GetYesterdaysDueTasks
will query the Cosmos DB state store using Dapr State API to lookup all the yesterday's task which are not completed yet. Remember that Cron job is configured to run each day at 12:05am so we are interested to check only the day before when the service runs. We initially made this implementation simple. There might be some edge cases not handled with the current implementation. - Method
MarkOverdueTasks
will take list of all tasks which passed the due date and set the flagIsOverDue
totrue
.
Do not forget to add fake implementation for class FakeTasksManager.cs
so the project TasksTracker.TasksManager.Backend.Api builds successfully.
4. Add Action Methods to Backend Web API project¶
As you've seen in the previous step, we are using Dapr Service to Service invocation API to call
methods api/overduetasks
and api/overduetasks/markoverdue
in the Backend Web API from the Backend Background Processor.
Add below file under controllers folder in the project TasksTracker.TasksManager.Backend.Api as shown below:
using Microsoft.AspNetCore.Mvc;
using TasksTracker.TasksManager.Backend.Api.Models;
using TasksTracker.TasksManager.Backend.Api.Services;
namespace TasksTracker.TasksManager.Backend.Api.Controllers
{
[Route("api/overduetasks")]
[ApiController]
public class OverdueTasksController : ControllerBase
{
private readonly ILogger<TasksController> _logger;
private readonly ITasksManager _tasksManager;
public OverdueTasksController(ILogger<TasksController> logger, ITasksManager tasksManager)
{
_logger = logger;
_tasksManager = tasksManager;
}
[HttpGet]
public async Task<IEnumerable<TaskModel>> Get()
{
return await _tasksManager.GetYesterdaysDueTasks();
}
[HttpPost("markoverdue")]
public async Task<IActionResult> Post([FromBody] List<TaskModel> overdueTasksList)
{
await _tasksManager.MarkOverdueTasks(overdueTasksList);
return Ok();
}
}
}
5. Add Cron Binding Configuration Matching ACA Specs¶
Add a new file folder aca-components. This file will be used when updating the Azure Container App Env and enable this binding.
Note
The name of the binding is not part of the file metadata. We are going to set the name of the binding to the value ScheduledTasksManager
when we update the Azure Container Apps Env.
Deploy the Backend Background Processor and the Backend API Projects to Azure Container Apps¶
1. Build the Backend Background Processor and the Backend API App Images and Push them to ACR¶
To prepare for deployment to Azure Container Apps, we must build and deploy both application images to ACR, just as we did before. We can use the same PowerShell console use the following code (make sure you are on directory TasksTracker.ContainerApps):
az acr build --registry $ACR_NAME --image "tasksmanager/$BACKEND_API_NAME" --file 'TasksTracker.TasksManager.Backend.Api/Dockerfile' .
az acr build --registry $ACR_NAME --image "tasksmanager/$BACKEND_SVC_NAME" --file 'TasksTracker.Processor.Backend.Svc/Dockerfile' .
2. Add Cron Dapr Component to ACA Environment¶
##Cron binding component
az containerapp env dapr-component set `
--name $ENVIRONMENT --resource-group $RESOURCE_GROUP `
--dapr-component-name scheduledtasksmanager `
--yaml '.\aca-components\containerapps-scheduled-cron.yaml'
3. Deploy New Revisions of the Backend API and Backend Background Processor to ACA¶
As we did before, we need to update the Azure Container App hosting the Backend API & Backend Background Processor with a new revision so our code changes are available for the end users. To accomplish this run the PowerShell script below:
## Update Backend API App container app and create a new revision
az containerapp update `
--name $BACKEND_API_NAME `
--resource-group $RESOURCE_GROUP `
--revision-suffix v20230227-1
## Update Backend Background Processor container app and create a new revision
az containerapp update `
--name $BACKEND_SVC_NAME `
--resource-group $RESOURCE_GROUP `
--revision-suffix v20230227-1
Note
The service ScheduledTasksManager
which will be triggered by the Cron job on certain intervals is hosted in the ACA service ACA-Processor Backend
. In the future module we are going to scale this
ACA ACA-Processor Backend
to multiple replicas/instances.
It is highly recommended that background periodic jobs are hosted in a container app with one single replica, you don't want your background periodic job to run on multiple replicas trying to do the same thing.
Success
With those changes in place and deployed, from the Azure Portal, you can open the log streams of the container app hosting the ACA-Processor-Backend
and check the logs generated when the Cron job is triggered,
you should see logs similar to the below image
Note
Keep in mind though that you won't be able to see the results instantaneously as the cron job searches for tasks that have a due date matching the previous day of its execution and are still pending.