MLOps with AML and Azure Devops

Few years ago, when someone used to say machine learning or AI the focus used to be on data prep and cleaning, EDA, modeling etc. Inferencing/ prediction used to be final steps for most people. But the situation has changed today, people are now talking about more elusive and less tangible final state — often termed “deployment”, “delivery”.

When is comes to deploying your machine learning solution in production there are various ways to do that. Some of them are as follows:

Python code behind e.g. Flask: This is the simplest way to go into production. write python or flask wrapper for your machine learning model inferencing and you're ready. But this way is very hard to scale.
Execution service from cloud provider: All major cloud service providers easy and scalable out of the box solutions.
Runtime
- TensorFlow serving
- Clipper
- NVIDIA TensorRT inference server
- MXNet Model Server
- …
Bespoke solutions (C++, …): Write your own runtime
kubernetes-ize everything above: Any solution above can be scaled using Kubernetes.

There are some things we need to consider before we deploy our model in production. Whatever deployment solution you choose must fulfill following criteria:

Must fit the technology stack
Not just about languages, but about semantics, scalability, guarantees
Run anywhere, any size
Composable building blocks
Must try to limit the amount of moving parts and the amount of moving data
Must make best use of resources

Now, if you have built some machine learning solution and if you will like to set up a minimal, robust, repeatable set of pipelines for productionising Machine Learning models in the cloud using Microsoft Azure, this article will help you with steps as well as resources you should refer.

First let's go through some terms:

What is MLOps?

According to Nisha Talagala, MLOps (a compound of Machine Learning and “information technology OPerationS”) is new discipline/focus/practice for collaboration and communication between data scientists and information technology (IT) professionals while automating and productizing machine learning algorithms. Via practice and tools, MLOps aims to establish a culture and environment where ML technologies can generate business benefits by rapidly, frequently and reliably building, testing, and releasing ML technology into production.

Azure Machine Learning:

Azure Machine Learning service provides a cloud-based environment you can use to develop, train, test, deploy, manage, and track machine learning models.

Azure DevOps:

We will be using the Azure DevOps project for build and release pipelines along with Azure ML services for ML/AI model management and operationalization.

Azure Pipelines:

Azure Pipelines are cloud-hosted pipelines that are fully integrated with Azure DevOps. You can either use a yaml file or a UI-based tool in Azure DevOps to set up your pipelines. It allows us to frequently update models, test new models, and continuously roll out new ML models alongside your other applications and services.

The end-to-end Machine learning pipeline includes data-prep, training, packaging and validating model, deploying model and continuous testing. It looks like below:

Our end-to-end CI/CD pipeline will look like below:

Let's get started:

Prerequisites

Microsoft Azure Account: You will need a valid and active Azure account for the Azure labs. If you do not have one, you can sign up for a free trial.
You will need an Azure DevOps account. If you do not have one, you can sign up for free here.

Setup

We will be using Azure Demo Generator to initiate setup. Azure DevOps Demo Generator helps you create team projects on your Azure DevOps Organization with sample content that include source code, work items, iterations, service endpoints, build and release definitions based on the template you choose during the configuration.

Use the Azure DevOps Demo Generator to provision the project on your Azure DevOps organization. This URL will automatically select Azure Machine Learning template in the demo generator. This template contains code and pipeline definition for a machine learning project demonstrating how to automate the end to end ML/AI project.

Step 1: Configure CI pipeline

In this step, we will configure CI pipeline for your ML/AI project. This pipeline will include DevOps tasks for

data sanity test,
model training on different compute targets,
model version management,
model evaluation/model selection etc.

a. Let's go to Azure Devops Dashboard:

b. Select your project and Navigate to Pipeline > Builds. Select the pipeline DevOps-for-AI-CI and click on edit.

Your current pipeline configs will look like below:

c. In this, Python environment and Install Requirements tasks will be required to setup and prepare python environment for subsequent builds.

Select next task, which will be Create or get Workspace (currently it's showing some setting need attention). In task configurations, Select the Azure subscription from the drop-down list and click Authorize to configure Azure service connection. Make sure your subscription allows to create AML workspace as this task will be used to create Workspace for Azure Machine learning service.

d. Follow similar procedure for all other tasks and select same subscription. Your end configs will look like below. save this configs.

e. Also, click on triggers tab and make sure CI is enabled.

f. Now, we are done with our CI pipeline which does following tasks:

Prepare the python environment for our upcoming builds
Get or Create the workspace for AML service
Submit Training job on the remote DSVM / Local Python Env
Compare performance of different models and select the best
Register model to the workspace
Create Docker Image for Scoring Web service
Copy and Publish the Artifacts to Release Pipeline

Step 2: Configure CD pipeline

So, as we are done with CI pipeline let's configure Release pipeline which will deploy the image created from the build pipeline to Azure Container Instance and Azure Kubernetes Services.

a. Now, navigate to Pipeline > Releases and select Deploy Web service and click Edit pipeline.

b. As we can see here, pipeline has two steps QA and Prod. We have to modify configs for QA so click on QA.

Similar to sub-step c in step-1,
Python environment and Install Requirements tasks will be required to setup and prepare python environment for subsequent builds.

c. Select next task, which will be Deploy webservice of ACI (currently it's showing some setting need attention). In task configurations, Select the Azure subscription from the drop-down list and click Authorize to configure Azure service connection. This task creates ACI (Azure Container Instance) and deploys web service image created in Build Pipeline to ACI.

d. Follow similar steps for Prod-Deploy on AKS which you can find form drop-down in top bar.

This task configures AKS (Azure Kubernetes Service) so we can deploy our web-service on AKS in production.

Step 3: Update config file in the source code to trigger CI and CD

a. Navigate to Repos and choose files. Go to directory aml_config/ and open config.json file.

b. Update your Azure subscription ID in place of <>. If required Change resource group name, AML workspace name and the location where you want to deploy your Azure ML service workspace. Click Commit to commit the changes.

Note: Navigate to environment_setup/install_requirements.sh, change the Azure CLI version from existing 2.0.X to 2.5.0.

c. Since we have enabled in CI trigger a build will be queued automatically.

d. If you want to check build, you can navigate to Pipelines > Builds and you will see a build is queued. Open the build to see the progress.

e. Once the build is success a release will be queued automatically. Navigate to Pipelines –> Releases to see the release in progress.


Deployment process in action (courtesy: Azure DevOps Labs)

f. If you want to view the resources provisioned and deployed by CI-CD pipelines, navigate to your Azure portal as shown below!

This was a very short and high-level overview to play with Azure DevOps and Azure pipelines to create manage machine learning deployments. We used the Azure DevOps project for build and release pipelines along with Azure ML services for ML/AI model management and operationalization.

What's next?

If you want to go through detailed step-by-step explanation, you can check out blog by our friend Ben Alex Keen available here.

For more information about various services we used here, you can go through list of references and learn more.

Blog