Deploy, promote, and roll back a model with GitHub Actions

With the model registered and production promotion protected by a GitHub environment, you're ready to serve predictions without exposing every user to an untested version.

Create an endpoint and a deployment

A managed online endpoint provides a stable HTTPS address that a client application, such as Proseware's scheduling system, calls to get a prediction. The endpoint itself doesn't run the model. One or more deployments, each pairing a specific registered model version with a compute configuration and, for MLflow models, an autogenerated environment and scoring script, do the actual serving. Keeping the endpoint URL stable while you replace deployments underneath it means the scheduling system never needs to change, even as the no-show model gets retrained and redeployed.

Promote a new version safely

An endpoint can host more than one deployment at a time, and you control what percentage of incoming traffic each deployment receives. This setup lets you roll out a new model version the same way you'd roll out any production software change: deploy the new version alongside the current one at 0% traffic, confirm it behaves as expected, then gradually shift traffic toward it, an approach commonly called a blue-green deployment. If the new version underperforms, you shift traffic back to the previous deployment instead of taking the endpoint down.

Test before you shift traffic

Before a new deployment receives any live traffic, you send requests directly to that deployment and compare its responses against what you expect. Only after the new deployment passes these checks do you increase its traffic allocation. Because a deployment can take a few minutes to reach a ready state, an automated test needs to wait for the deployment to finish provisioning before it sends requests.

Automate the pipeline with GitHub Actions

Manually repeating registration, deployment, and testing for every retrained model doesn't scale, so Proseware's team automates the pipeline with GitHub Actions and the Azure Machine Learning CLI (v2). The current recommended way to authenticate a workflow to Azure is OpenID Connect (OIDC) federated credentials, which let GitHub Actions request a short-lived access token instead of storing a client secret as a repository secret. You configure a federated credential on a Microsoft Entra application, scoped to your repository and branch, then reference it in the workflow with the azure/login action:

permissions:
  id-token: write
  contents: read

env:
  RESOURCE_GROUP: ${{ vars.AZURE_RESOURCE_GROUP }}
  WORKSPACE_NAME: ${{ vars.AZURE_WORKSPACE_NAME }}
  ENDPOINT_NAME: ${{ vars.AZURE_ENDPOINT_NAME }}

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Sign in to Azure
        uses: azure/login@v2
        with:
          client-id: ${{ secrets.AZURE_CLIENT_ID }}
          tenant-id: ${{ secrets.AZURE_TENANT_ID }}
          subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
      - name: Register and deploy the model
        run: |
          az extension add -n ml -y
          az ml model create \
            --file model.yml \
            --resource-group $RESOURCE_GROUP \
            --workspace-name $WORKSPACE_NAME
          az ml online-deployment create \
            --name green \
            --endpoint-name $ENDPOINT_NAME \
            --file green-deployment.yml \
            --resource-group $RESOURCE_GROUP \
            --workspace-name $WORKSPACE_NAME
      - name: Test the new deployment
        run: |
          az ml online-endpoint invoke \
            --name $ENDPOINT_NAME \
            --deployment-name green \
            --request-file sample-request.json \
            --resource-group $RESOURCE_GROUP \
            --workspace-name $WORKSPACE_NAME
      - name: Send limited traffic to the new deployment
        run: |
          az ml online-endpoint update \
            --name $ENDPOINT_NAME \
            --traffic "blue=90 green=10" \
            --resource-group $RESOURCE_GROUP \
            --workspace-name $WORKSPACE_NAME

The workflow's permissions block grants the job the id-token it needs to request a token from Azure. The azure/login step exchanges that token for an authenticated Azure CLI session. After a direct test succeeds, the workflow sends 10% of traffic to the new green deployment. Promote it further only after its production behavior meets your acceptance criteria.

Roll back to the previous model

Keep the previous blue deployment available until the new model completes its observation period. If errors or unacceptable predictions appear after promotion, route all traffic back to blue:

az ml online-endpoint update \
  --name $ENDPOINT_NAME \
  --traffic "blue=100 green=0" \
  --resource-group $RESOURCE_GROUP \
  --workspace-name $WORKSPACE_NAME

Changing traffic preserves the endpoint URL and provides a faster recovery than deleting and recreating the endpoint. After traffic returns to the previous model, investigate the green deployment without exposing users to it. Delete the previous deployment only after the new model meets its acceptance criteria for the required observation period.