Intelligent Alarm Solution


Summary
Intelligent Alarm project utilizes Vision AI DevKit to create captures of persons arriving to monitored area. These captures are further processed using Face API in order to identify entrants, while processing results can be surfaced thru chat bot. Intelligent Alarm is also capable of alert invocation either as chat bot proactive message or thru SMS.
These capabilities make this project perfect base for example for automated card/chip less attendance systems with alerting capabilities or even as the name says for alarm solution :).
In comparison to DevKit Integration with Azure Face API Services project this solutions implements different way of image capture on Vision AI DevKit (using node.js). It also utilizes cloud for further image processing and data storing and it provides alerting and presentation layer built using Conversational AI. On the contrary DevKit Integration with Azure Face API Services project meant to deliver standalone edge device capable of Face API invocation.
Implementation
There are two important parts to the solution. First is intelligent edge device - Vision AI DevKit. Thanks to default custom vision model, it is capable (almost out of the box) to detect persons. If such event occurs, it captures the image and uploads it to Azure Storage for further processing (face identification).
Second part to the solution is cloud based backend and presentation layer. Backend, built using Azure Functions and Cosmos DB, is responsible for further image analysis, data storing and data access. Presentation layer is built as conversational AI interface using Bot Framework.
Software and Services used Hardware
  • Azure IoT Edge
  • IoT Hub
  • Azure Storage
  • Azure Functions
  • Face API
  • Cosmos DB
  • Twilio
  • Bot Framework
  • LUIS
  • C#
  • Node.js
  • Vision AI DevKit camera
Repository
Find all relevant information, including code, for full implementation of this product here.
Users are always encouraged to innovate and continue to improve the functionality of current projects.
Future Improvements and Project Suggestions
Solution was built as pet project and so there certainly are several areas for improvements:
  • Replace Cosmos with Azure SQL Database
  • Improvement for logic determining when the image will be captured and sent for further processing. Ideally, solution should limit number of pictures per entrant and per entry that are being sent up to the cloud, while it should be uploading pictures with visible faces. There haven’t been done much work in this area yet.
  • Solution is missing console for Face API person group training
  • Solution is not supporting multi tenancy at this point, neither multi user usage
  • LUIS model might be trained more extensively and support more intents
  • Once Face API is exportable also to docker container compatible with ARM devices, the plan is to run face identification directly on the Vision AI DevKit
    • About the Creator
    Marek Lani is Software Engineer working closely with Microsoft customers on their projects with ultimate goal to make them even more successful using latest Azure services. Marek is tech evangelist by heart, with passion for IoT, Serverless, Cognitive Services, Conversational AI…
    The check out more of Marek's projects please visit his GitHub

    Updated:

    Leave a comment