Overview of My Submission

My submission to this app is a client app that uses Azure Cognitive services to provide a natural language interface to collect various health parameters.
I created this app to help people who have difficulties to enter data via a keyboard because they have an impairment or because they think it is easier to communicate with the computer via a speech interface.

Submission Category: AI Aces

My submission is in the category AI Aces because the core of the app is the Natural Language processing of the speech input in the Azures Cognitive Services. This LUIS app transforms speech into text and analyses this text and transforms it into intents and entities.
These intents and entities are evaluated in the client app and translated into actions/behavior of the app.

Link to Code on GitHub

The Github repository contains mainly the Xamarin.Forms app and the definition of the LUIS app.

FrankPohl / healthmate

Using an Azure Bot and LUIS to input blood pressure, pulse and blood sugar

Health Mate

This is a contribution to the Azure hackathon on DEV. See this post for more details https://dev.to/devteam/hack-the-microsoft-azure-trial-on-dev-2ne5.

Why Health Mate

Health Mate was created with people in mind, that are not familiar with apps or used to use apps on the phone or desktop To help these people collect important health information like blood pressure or pulse on a regular basis Health Mate uses a speech as the interface instead of a keyboard. What is measured with which value and when is captured in a natural language dialog between the computer and the user.

Using the app

The app can collect data for Temperature, Blood Pressure, Pulse and Glucose. After the app is started it is ready for input and prompts the user to say what was measured. An example dialog may look like that:

The system welcomes the user and asks for his input
    User says:

…

View on GitHub

The code on Github contains everything that is necessary to compile and run the code with Visual Studio 2022. The necessary keys to connect to the LUIS app are part of the repository, too. They shouldn't be but this was the easiest way to make my submission testable.

The MIT license can be found in the LICENSE file and is part of the repository.

Additional Resources / Info

I have published a video with a demo on YouTube.
DEV Hackathon- Healthmate Demo

The readme.md file in Github contains more details about the implementation.

There were no other developers involved in this work.

Blog

Health Mate - Collect body parameters with speech

FrankPohl