Posts Deploy every Pull Request into a dedicated Namespace in Kubernetes
Deploy every Pull Request into a dedicated Namespace in Kubernetes
Cancel

Deploy every Pull Request into a dedicated Namespace in Kubernetes

Kubernetes enables developers to deploy quickly into test and production environments. Following the principles of DevOps of deploying fast and often, we should not stop there. Rather DevOps should enable developers to deploy each feature to a dedicated environment for it to be tested and therefore be deployed into production as fast as possible.

This post will show you how to deploy every pull request into its own Kubernetes namespace with its own unique URL. Doing so will bring projects one step closer to continuous deployments.

This post is part of “Microservice Series - From Zero to Hero”.

Deploy the Helm package during a Pull Request Build

You can find the code of the demo on Github.

If you have not enabled CI builds during pull requests, see Run the CI Pipeline during a Pull Request for more information on how to run your pipeline.

Deploying the Helm package during the pull request is almost the same as the deployment to the test environment:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
- stage: PR_Deploy
  condition: startsWith(variables['resources.pipeline.CustomerApiBuild.sourcebranch'], 'refs/pull/') 
  variables:      
    DeploymentEnvironment: 'pr-$(prId)'
    K8sNamespace: '$(ApiName)-$(DeploymentEnvironment)'
    ConnectionString: "Server=tcp:$(SQLServerName),1433;Initial Catalog=$(DatabaseName)-$(DeploymentEnvironment);Persist Security Info=False;User ID=$(DbUser);Password=$(DbPassword);MultipleActiveResultSets=False;Encrypt=True;TrustServerCertificate=False;Connection Timeout=30;"
    URL: $(DeploymentEnvironment).customer.programmingwithwolfgang.com # replace with your service url     
  jobs:
  - deployment: Web_PR
    environment: "customerapi-PR-Deploy"
    displayName: 'Deploy CustomerApi to the PR environment'
    strategy:
      runOnce:
        deploy:
          steps:
          - download: CustomerApiBuild
            artifact: $(ArtifactName)
          - template: templates/GetPrId.yml
          - template: templates/DeployHelmPackage.yml
            parameters:
              apiName: $(ApiName)
              azureSubscription: '$(AzureSubscription)'
              clusterResourceGroup: '$(ClusterResourceGroup)'
              chartPackage: '$(ChartPackage)'
              helmVersion: $(HelmVersion)
              k8sNamespace: $(K8sNamespace)
              kubernetesCluster: $(KubernetesCluster)
              releaseValuesFile: '$(ReleaseValuesFile)'

The most notable change is the condition that checks the source branch of the build. Since the CI and CD pipeline are separated and the CD pipeline is triggered by the CI pipeline, you can not access the source branch directly. Therefore you have to reference the source branch of the CI build. A pull request branch always starts with refs/pull. You can check this with the following condition:

1
condition: startsWith(variables['resources.pipeline.CustomerApiBuild.sourcebranch'], 'refs/pull/') 

The next change to the regular deployment is obtaining the pull request id. The pull request id is great to distinguish between pull requests and therefore will be used as part of the URL and namespace. Since the CD pipeline is triggered by the CI pipeline and not the pull request directly, it is not possible to access the pull request id through the built-in variable System.PullRequest.PullRequestId. Instead, I use the following PowerShell command which is placed inside the GetPrId.yaml template:

1
2
3
4
- pwsh: |
    $prId = [regex]::match('$(resources.pipeline.OrderApiBuild.sourcebranch)','(refs/pull/)(\d*)(/merge)').Groups[2].Value
    Write-Host "##vso[task.setvariable variable=prId;]$prId"    
  displayName: 'Get PR Id'

For the Kubernetes namespace, use a meaningful name and add the pull request id, for example, customerapi-pr-123. The URL almost works the same way, for example, pr-123.customerapi.programmingwithwolfgang.com. If you followed this series and have the DNS already configured for subdomains, then you do not have to change anything in the DNS settings. If you haven’t made the DNS configuration, see Configure custom URLs to access Microservices running in Kubernetes for more information.

Cleaning up the Pull Request Deployments

As you have seen, deploying pull requests is quite easy. It is also important to not forget to clean them up after someone reviewed them. Otherwise, your Kubernetes cluster will be full soon.

Deleting the previously created namespace inside the Azure DevOps YAML pipeline is quite simple:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
- stage: PR_Delete
  dependsOn: PR_Deploy  
  condition: succeeded('PR_Deploy')
  variables:
    K8sNamespace: '$(ApiName)-pr-$(prId)'
  jobs:  
  - deployment: Delete_PR_Namespace
    environment: "customerapi-PR-Delete" 
    displayName: Build and push Docker image and create Helm package
    strategy:
      runOnce:
        deploy:
         steps:
         - download: none
         - template: templates/GetPrId.yml
         - task: Kubernetes@1       
           inputs:
             connectionType: 'Azure Resource Manager'
             azureSubscriptionEndpoint: 'AzureServiceConnection'
             azureResourceGroup: $(ClusterResourceGroup)
             kubernetesCluster: $(KubernetesCluster)
             useClusterAdmin: true
             command: 'delete'
             arguments: 'namespaces $(k8sNamespace)'
           displayName: 'Delete PR namespace'

This code reads the pull request id to find the correct namespace name and then uses kubectl to delete the Kubernetes namespace. The more interesting question is how you can control when the namespace gets deleted. The code above runs immediately after the deployment is finished. This means that you create a new namespace and in the next step delete it again.

Having control over when the stage runs can be achieved with approvals in Azure DevOps.

Add an Approval before deleting the Pull Request Namespace

To add an approval go to Pipelines and then Environment in your Azure DevOps project. There you can see your environment for deleting the namespace, in my case customerapi-PR-Delete, when you ran the deployment already at least once. If you can’t see it, click on New environment and add it with the same name as in your pipeline.

Click on the environment and then click on the three dots on the top right and select Approvals and checks.

Open the Environment Settings

Open the Environment Settings

This redirects you to a new page where you can click on Approvals.

Add an Approval before the deployment

Add an Approval before the deployment

Select who is allowed to approve and optionally configure the timeout and if the approver is allowed to approve their own deployment.

Configure the Approval

Configure the Approval

When you create a new Pull Request, you will see that after the deployment succeeded, an approval is necessary to execute the deletion step. Often the approver is someone from QA or marketing. This person often checks if the feature looks as requested and if the basic functionality is as expected. More in-depth tests should be automated.

The Deployment needs to be approved

The Deployment needs to be approved

For more detailed explanations about approvals in Azure DevOps, see Approvals for YAML Pipelines in Azure DevOps.

Testing the Pull Request Deployment

Create a new pull request to trigger the CI pipeline.

Create a new Pull Request

Create a new Pull Request

As you can see on the screenshot above, the pull request id is 61.

After the Help deployment is finished, the task also prints the URL of the new service. As you can see the URL pr-61.customer.programmingwithwolfgang.com matches the pull request id.

The Pull Request got deployed to an unique URL

The Pull Request got deployed to an unique URL

Click on the URL and your browser opens the previously deployed microservice. As you can see, the URL works and also has a valid SSL certificate that was created automatically. For more information about creating SSL certificates inside your Kubernetes cluster, see Automatically issue SSL Certificates and use SSL Termination in Kubernetes

Everything works as expected

Everything works as expected

When you open your Kubernetes dashboard or check your namespaces in the Azure portal, you will see the Kubernetes namespace corresponding to the pull request id.

The namespace matches the PR id

The namespace matches the PR id

Further Improvements

As you can see, this demo uses a very simple pull request deployment and only uses the Helm package for the microservice. This microservice has no connection to a queue service or a database. You have to consider how much the PR deployment in your project needs.

In case you want a full application with queue and database, then add them in the PR deployment. The steps are basically the same as in the other deployments and you can use the PR id again to distinguish the PR database from the test or production database.

If you add additional services, make sure to not forget to delete them. Especially database can be expensive in Azure if you never delete them.

Conclusion

Modern DevOps environments should strive to deploy as fast and as often as possible. To achieve this, deploy every feature into a dedicated environment and if it passes the tests, deploy it to your production environment. This allows developers to deliver features faster and also helps to narrow down bugs due to the small scope of each deployment.

You can find the code of the demo on Github.

This post is part of “Microservice Series - From Zero to Hero”.

This post is licensed under CC BY 4.0 by the author.