Azure DevOps and GitHub Integration for a Python Project

Photo by Quinten de Graaf on Unsplash

Have you been wondering how to set up a CI and CD environment in Azure DevOps for your Python project while continuing to use GitHub? Here you will find the solution!

For you as an orientation we proceed as follows:

1. Demo project setup
2. Creating requirements.txt
3. Azure Portal — App Service Setup
4. Azure DevOps Setup
5. Using your VM

Let’s start!

Demo Project Setup

With the help of an example, I would like to explain the setup to you. You may save the following code in a file called calculate.py.

For demonstration purposes, we use two modules: Numpy and Pandas, which you need to install on your computer via the pip-command.

The program is simple, it generates random numbers in the range that the user enters via the console.

Creating requirements.txt

To be able to use the application in a VM (Virtual Machine), we need to install the Numpy module and Pandas as well. In order not to have to do this every time the project is restarted inside the VM, a solution is needed. With the help of a requirements.txt, it would be possible to fix this problem.

A requirements.txt lists all the relevant modules that your application needs to run successfully. These are usually installed on your computer via the pip command since you will need them when you test your application. We need the text file to successfully set up our pipelines.

To generate a requirements.txt you have to execute the following commands:

$ pip install pipreqs
$ pipreqs /home/project/location
Successfully saved requirements file in /home/project/location/requirements.txt

Your text file should contain the following modules:

numpy==1.21.4
pandas==1.3.4

Push your requirements.txt to your main branch.

Azure Portal — App Service Setup

First of all, you need to create an account with Microsoft. Then, log in to the Azure Portal, follow the instructions, and create a new resource.

Select the Web App.

In the “Basics” section, you need to create the following instances:

1. Subscription: here you select the ones you have previously set.
2. Resource Group: Create a new one and name it “RandomNumberCalculator”.
3. Name: “RandomNumberCalculator”
4. Runtime stack: Python 3.7
5. Region: Choose the one that is near you
6. Linux Plan: Create a new one and name it „RandomNumberCalculator”
7. Sku and size: It depends (as always). For this project select the cheapest version.

Click on the “Review + create” button. The price will be determined, which may take a few minutes, once this is done you can create the Web App.

In the overview, the resource groups and the app service should now have been added under “Recent resources”. We will take a closer look at the App Service later since it will host our application.

Azure DevOps Setup

Log in to Azure DevOps with your Microsoft account. Create a new organization and a new project.

Once your project is successfully created, several menu items will open for you. If you navigate to “Pipeline”, you can create a new one.

Build Pipeline

In the next step, we choose GitHub as the source, since that’s where our source code is stored.

Afterward, you log in to GitHub via the popup and select the appropriate project, preferably the one in which you have your calculate.py stored.

Once you connect to your GitHub account, select Python to Linux Web App on Azure and choose the suggested subscription.

Since we have already created our App Service via the Azure Portal, we can select our Web App from the list.

Subsequently, an azure-pipelines.yml is created that processes jobs every time a change is made to the main branch. When the build pipeline is successfully created, we get an artifact. Into this, we want to install the necessary modules such as numpy and pandas. We do this by calling the previously created requirements.txt via a pip command.

Override your existing .yml with the one below.

If you have overwritten the .yaml, start the Build Pipeline! Excited?

Once you have created your Build Pipeline, you can view the logs under “Jobs”. In the “Install dependencies” job you get an overview of the installed modules. These were processed based on your requirements.txt and loaded into the artifact.

Parallelism Request

If you get the following error, don’t worry!

You follow this link (https://aka.ms/azpipelines-parallelism-request) and will be redirected to the following form. Unfortunately, you have to fill it out before you can start your first pipeline.

Release Pipeline

To push the artifact to a production system, we need to create a Release Pipeline. You select or search for “Azure App Service deployment” from the suggestions.

In the next step select the created artifact.

To ensure CD and that the current state of your project is always saved on your production system, you should set the following triggers.

After that, you go to the stage and finish the deployment for the Azure App Service.

Do not forget to set the agent specification to ubuntu-latest.

Once you have everything done, you can create your first Release Pipeline. Wuhuu!

Running calculate.py in your VM

Before we run our app in the VM, we need to do one last configuration. To do this, go back to the Azure portal and select your App Service. Then navigate to “Settings” and select “Configuration”.

You add two new “Application settings”.

SCM_COMMAND_IDLE_TIMEOUT

SCM_DO_BUILD_DURING_DEPLOYMENT

Only after you have made the settings in your app service, you can successfully run your application. Don’t forget to save your application settings and restart your release pipeline.

You should then be able to run your application in your VM. Navigate here to “Development Tools” and open the console.

Inside your VM, you start the application and as you can see, it works like a charm.

May the pipeline be with you!

Congratulations if you made it this far and haven’t closed the tab yet from boredom! Should you now make changes to your code, you can rely on your successful CI and CD environment.