CI/CD integration

Anaconda Platform integrates with CI/CD systems, so your teams can test and deploy flows automatically as part of their existing delivery pipelines. For more informational background on deploying ML/AI systems to production, read How To Organize Continuous Delivery of ML/AI Systems.

GitOps for Anaconda Platform

The following diagram illustrates a typical CI/CD pattern:

Overview of a GitOps workflow with Anaconda Platform, from local development through pull request, automated testing, review, and deployment

Experiment

The user experiments and prototypes code on a cloud workstation or locally on a laptop. The user can test their code at scale quickly and autonomously on the platform’s cluster.

Pull request

When the code works adequately, the user commits it and opens a pull request. They authenticate with the CI/CD system using their personal credentials.

Run tests

The CI/CD system, such as GitHub Actions or CircleCI, launches a test suite automatically when a pull request is opened. The CI/CD system submits workloads to the platform, authenticating as a machine user.

Review and approve

After tests pass, a human reviewer reviews the pull request. The reviewer can tag the pull request as approved, and a corresponding Metaflow tag can be applied to test runs as well, signaling a successful PR.

Deploy

After the PR is approved, the CI/CD system deploys the flow either as a new production version or as a new @project variant, running concurrently with the production version so its performance can be evaluated live.

Supported CI/CD platforms

Anaconda Platform supports all major CI/CD platforms through OIDC-based authentication:

GitHub Actions: Native OIDC support
GitLab CI/CD: Native OIDC support via id_tokens
Azure DevOps: Azure AD federation
CircleCI: OIDC token support

Each platform uses a similar pattern:

Configure a machine user on the platform. See Programmatic access via machine users for per-provider claim fields.
Set up OIDC authentication in your CI/CD config (YAML examples below).
Use obproject-deploy to deploy your project.

Verifying your deploy

obproject-deploy tags each workflow it deploys with lineage metadata: the commit hash being deployed, and an identifier for the CI run that triggered the deploy when running in CI. To confirm a deployment succeeded, open the workflow in the UI: select the project, then Workflows, and then select the workflow. The workflow details show the tags applied to its runs:

The workflow detail view showing the commit-hash tag the workflow applies to runs, above a table of successful runs

You can also see the tags in the deploy output, which prints a line such as Tagging deployments with: commit-hash:15ecbcb5ecf0f21295bf7085a6b9a568f92f85ab. If you do not see the tags, the deploy did not complete. Check the CI job’s logs for the obproject-deploy step and verify the machine user authenticated successfully against auth.<YOUR_DEPLOYMENT_URL>.

Machine user naming convention

The YAML examples below derive the machine user name from your project name:

PROJECT_NAME=$(yq .project obproject.toml)
CICD_USER="${PROJECT_NAME//_/-}-cicd"

Two implications:

The machine user must be named with hyphens rather than underscores. For example, my-project-cicd, not my_project-cicd. The underscore-to-hyphen normalization in the YAML exists so a project named my_project resolves to a machine user named my-project-cicd.
To use a different name, such as a team-shared machine user across multiple projects, set cicd_user in obproject.toml:

project = "my_project"
cicd_user = "team-shared-cicd"

The example YAML files read this value with yq ".cicd_user // \"$DEFAULT\"", so an explicit value takes precedence over the derived default.

The `outerbounds service-principal-configure` flag reference

The auth command takes a different flag per provider: In all cases, the command also takes --name <SERVICE_PRINCIPAL_NAME>, --deployment-domain <DEPLOYMENT_DOMAIN>, and --perimeter <PERIMETER>. To read these values from obproject.toml instead of passing them explicitly, use --from-obproject-toml.

Using Anaconda Platform with GitHub Actions

The GitHub Actions on OBP demo repository walks through the key workflows in practice.

Example GitHub Actions workflow

name: Deploy Project
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

permissions:
  id-token: write
  contents: read

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v4
      with:
        fetch-depth: 0

    - name: Set up Python
      uses: actions/setup-python@v5
      with:
        python-version: "3.12"

    - name: Install dependencies
      run: pip install outerbounds ob-project-utils pyyaml

    - name: Configure the machine user
      run: |
        PROJECT_NAME=$(yq .project obproject.toml)
        PLATFORM=$(yq .platform obproject.toml)
        CICD_USER="${PROJECT_NAME//_/-}-cicd"
        outerbounds service-principal-configure \
          --name $CICD_USER \
          --deployment-domain $PLATFORM \
          --perimeter default \
          --github-actions

    - name: Deploy Project
      run: obproject-deploy

Using Anaconda Platform with GitLab CI/CD

GitLab CI/CD supports OIDC authentication via the id_tokens keyword. Use the example below as a starting template.

The aud value in id_tokens must match your platform URL. GitLab evaluates id_tokens at pipeline creation time, before any scripts run, so the value cannot be read dynamically from obproject.toml. Update this value when setting up a new project.

Example `.gitlab-ci.yml`

stages:
  - deploy

deploy:
  stage: deploy
  image: python:3.12
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
    - if: $CI_COMMIT_BRANCH == "main"

  id_tokens:
    OUTERBOUNDS_ID_TOKEN:
      aud: <YOUR_PLATFORM_URL>

  script:
    - wget -qO /usr/local/bin/yq https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64
    - chmod +x /usr/local/bin/yq
    - pip install outerbounds ob-project-utils pyyaml

    - PROJECT_NAME=$(yq .project obproject.toml)
    - PLATFORM=$(yq .platform obproject.toml)
    - CICD_USER="${PROJECT_NAME//_/-}-cicd"
    - |
      outerbounds service-principal-configure \
        --name $CICD_USER \
        --deployment-domain $PLATFORM \
        --perimeter default \
        --jwt-token $OUTERBOUNDS_ID_TOKEN

    - obproject-deploy

  artifacts:
    paths:
      - deployment_summary.md
    when: always

Using Anaconda Platform with Azure DevOps

Azure DevOps supports OIDC through Azure AD federation.

Example `azure-pipelines.yml`

trigger:
  - main

pr:
  - main

pool:
  vmImage: ubuntu-latest

steps:
- checkout: self

- task: AzureCLI@2
  displayName: 'Configure the machine user'
  inputs:
    azureSubscription: '<YOUR_SERVICE_CONNECTION>'
    addSpnToEnvironment: true
    scriptType: bash
    scriptLocation: inlineScript
    inlineScript: |
      wget -qO /usr/local/bin/yq https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64
      chmod +x /usr/local/bin/yq

      PROJECT_NAME=$(yq .project obproject.toml)
      PLATFORM=$(yq .platform obproject.toml)
      CICD_USER="${PROJECT_NAME//_/-}-cicd"

      pip install outerbounds ob-project-utils pyyaml
      outerbounds service-principal-configure \
        --name $CICD_USER \
        --deployment-domain $PLATFORM \
        --perimeter default \
        --jwt-token $idToken

- script: obproject-deploy
  displayName: 'Deploy Project'
  env:
    SYSTEM_ACCESSTOKEN: $(System.AccessToken)

Using Anaconda Platform with CircleCI

CircleCI supports OIDC tokens for secure authentication.

Example `.circleci/config.yml`

version: 2.1

jobs:
  deploy:
    docker:
      - image: cimg/python:3.12
    steps:
      - checkout
      - run:
          name: Install dependencies
          command: |
            wget -qO /usr/local/bin/yq https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64
            chmod +x /usr/local/bin/yq
            pip install outerbounds ob-project-utils pyyaml
      - run:
          name: Configure the machine user
          command: |
            PROJECT_NAME=$(yq .project obproject.toml)
            PLATFORM=$(yq .platform obproject.toml)
            CICD_USER="${PROJECT_NAME//_/-}-cicd"
            outerbounds service-principal-configure \
              --name $CICD_USER \
              --deployment-domain $PLATFORM \
              --perimeter default \
              --jwt-token $CIRCLE_OIDC_TOKEN_V2
      - run:
          name: Deploy Project
          command: obproject-deploy

workflows:
  deploy:
    jobs:
      - deploy:
          context: [<YOUR_OIDC_CONTEXT>]
          filters:
            branches:
              only: [main]

If you need help setting up GitOps in your environment, contact Anaconda support.

Anaconda Platform

Getting Started

Platform Guides

CI/CD integration

GitOps for Anaconda Platform

Supported CI/CD platforms

Verifying your deploy

Machine user naming convention

The `outerbounds service-principal-configure` flag reference

Using Anaconda Platform with GitHub Actions

Example GitHub Actions workflow

Using Anaconda Platform with GitLab CI/CD

Example `.gitlab-ci.yml`

Using Anaconda Platform with Azure DevOps

Example `azure-pipelines.yml`

Using Anaconda Platform with CircleCI

Example `.circleci/config.yml`

​GitOps for Anaconda Platform

​Supported CI/CD platforms

​Verifying your deploy

​Machine user naming convention

​The outerbounds service-principal-configure flag reference

​Using Anaconda Platform with GitHub Actions

​Example GitHub Actions workflow

​Using Anaconda Platform with GitLab CI/CD

​Example .gitlab-ci.yml

​Using Anaconda Platform with Azure DevOps

​Example azure-pipelines.yml

​Using Anaconda Platform with CircleCI

​Example .circleci/config.yml

GitOps for Anaconda Platform

Supported CI/CD platforms

Verifying your deploy

Machine user naming convention

The `outerbounds service-principal-configure` flag reference

Using Anaconda Platform with GitHub Actions

Example GitHub Actions workflow

Using Anaconda Platform with GitLab CI/CD

Example `.gitlab-ci.yml`

Using Anaconda Platform with Azure DevOps

Example `azure-pipelines.yml`

Using Anaconda Platform with CircleCI

Example `.circleci/config.yml`