Deploy to production

The examples so far have been run manually. A defining feature of production deployments is that they run automatically and reliably without human supervision. Anaconda Platform includes a workflow orchestrator that allows you to deploy any flow to production with one command.

This page assumes you have installed the outerbounds package and connected to your platform instance. If you haven’t, start with Connect to the platform and run your first flow.

Deploying a scheduled flow to production

Weather forecasts are a good example of a workflow that needs to run in production at a regular cadence. This example calls the Open-Meteo API to retrieve a forecast for a specified location.

Create a directory for the example and navigate to it:
mkdir weatherflow && cd weatherflow

Create a file named weatherflow.py in that directory with the following contents:

from metaflow import FlowSpec, step, Parameter, card, current, project, trigger, schedule, retry
from metaflow.cards import Markdown, VegaChart

GEOCODING = "https://geocoding-api.open-meteo.com/v1/search"
FORECAST = "https://api.open-meteo.com/v1/forecast"
CHART = {
    "$schema": "https://vega.github.io/schema/vega-lite/v5.json",
    "width": 600,
    "height": 400,
    "mark": {"type": "line", "tooltip": True},
    "encoding": {
        "x": {"field": "time", "type": "temporal"},
        "y": {"field": "temperature", "type": "quantitative"},
    },
}

# ⬇️ Enable this decorator if you want hourly forecasts
# @schedule(hourly=True)

@trigger(event="forecast_request")
@project(name="weather")
class WeatherFlow(FlowSpec):
    location = Parameter("location", default="San Francisco")
    unit = Parameter("unit", default="fahrenheit")

    def parse_location(self):
        import requests

        resp = requests.get(GEOCODING, {"name": self.location, "count": 1}).json()
        if "results" not in resp:
            raise Exception(f"Location {self.location} not found")
        else:
            match = resp["results"][0]
            self.latitude = match["latitude"]
            self.longitude = match["longitude"]
            self.loc_name = match["name"]
            self.country = match["country"]

    def get_forecast(self):
        import requests

        resp = requests.get(
            FORECAST,
            {
                "latitude": self.latitude,
                "longitude": self.longitude,
                "hourly": "temperature_2m",
                "forecast_days": 3,
                "temperature_unit": self.unit,
            },
        ).json()
        forecast = resp["hourly"]
        self.forecast = [
            {"time": t, "temperature": x}
            for t, x in zip(forecast["time"], forecast["temperature_2m"])
        ]

    @retry
    @card(type="blank")
    @step
    def start(self):
        self.parse_location()
        self.get_forecast()
        current.card.append(
            Markdown(f"# Temperature forecast for {self.loc_name}, {self.country} 🌤️")
        )
        CHART["data"] = {"values": self.forecast}
        current.card.append(VegaChart(CHART))
        self.next(self.end)

    @step
    def end(self):
        pass


if __name__ == "__main__":
    WeatherFlow()

The flow uses four decorators for production readiness:

@schedule runs the flow automatically on a cadence. Uncomment it to run hourly. For details, see scheduling flows.
@trigger starts the flow when an external event occurs. For details, see event triggering.
@project isolates deployments by developer, so multiple people can deploy the same flow without interfering with each other. For details, see coordinating larger projects.
@retry automatically retries a step if it fails. This example uses it to handle cases where the forecast API is temporarily unresponsive. For details, see retrying tasks.

Testing the flow locally

Before deploying, test the flow locally:

python weatherflow.py run --location ulaanbaatar

The result is stored as an artifact called forecast and displayed as a card:

A card showing the temperature forecast for Ulaanbaatar, Mongolia as a line chart

A test run like this is isolated from production by default through Metaflow namespaces. You can continue developing and running the flow locally after it is deployed, without affecting production.

Deploying to production

To deploy the flow to production, run:

python weatherflow.py argo-workflows create

The flow is now deployed and runs automatically without human intervention. You can shut down your laptop and the flow continues to run on the platform. You can see deployed workflows in the Workflows view. By default, the @project decorator prefixes the deployment with your username, so the deployment appears as weather.user.<YOUR_EMAIL>.weatherflow. This allows multiple developers to create their own isolated deployments. To promote a deployment to be the singular production version, add the --production flag:

python weatherflow.py --production argo-workflows create

To create branched deployments for use cases like A/B testing, use the --branch option.

Triggering a production run

To trigger a production run from the command line:

python weatherflow.py argo-workflows trigger --location 'Las Vegas'

You can pass any parameters to the trigger command. A key difference between run and trigger is that trigger starts a production run that is independent of your local machine. Even if you shut down your computer, the run continues on the platform.

Triggered runs appear in the Workflows view a few seconds before they appear in the Runs view, as the runs might take a while to get scheduled. You can also trigger the flow in the Workflows view. Open the workflow’s detail page, click Actions, and select Trigger a run.

The workflow detail page with the Actions dropdown open, showing Trigger a run as the first option

The Trigger Run panel opens with the flow’s parameters pre-filled with their default values. Adjust them as needed, then click Trigger.

The Trigger Run panel showing the location and unit parameters with default values

The triggered run produces the same forecast card as the local run:

A triggered production run showing the temperature forecast for Las Vegas, United States as a line chart

Deploying with stable production environments

An important reason for taking care of dependency management, as we covered in defining the environment, is to ensure stable and reproducible production environments. You can use the @conda/@pypi approach or custom images to define production environments. For instance, you could deploy our earlier example, TorchTestFlow, to production as follows (the environment variable MYIMAGE is defined for readability):

export MYIMAGE=763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-training:2.3.0-gpu-py311-cu121-ubuntu20.04-ec2
python torchtest.py --with kubernetes:image=$MYIMAGE argo-workflows create

Note that the --with option comes before the argo-workflows command.

GitOps and continuous delivery

For production deployments at scale, users typically do not call argo-workflows create directly. Instead, deployment happens through a CI/CD pipeline, such as GitHub Actions. Anaconda Platform gives you tools to separate staging and production environments securely, test flows before deployment automatically, deploy A/B experiments, and set up end-to-end continuous delivery workflows. Read more here. Also, occasionally things fail in production. Thanks to artifacts and consistent environments, you can reproduce production issues locally and deploy fixes back to production quickly.

What’s next

You are now ready to develop, scale, and deploy flows on Anaconda Platform:

Develop code in your existing environment or on a cloud workstation.
Scale to the cloud using your preferred libraries and hardware.
Deploy flows to run automatically in production.

To explore additional Anaconda Platform features, see the documentation. For questions, contact your account team.

Anaconda Platform

Getting Started

Platform Guides

Deploy to production

Deploying a scheduled flow to production

Testing the flow locally

Deploying to production

Triggering a production run

Deploying with stable production environments

GitOps and continuous delivery

What’s next

​Deploying a scheduled flow to production

​Testing the flow locally

​Deploying to production

​Triggering a production run

​Deploying with stable production environments

​GitOps and continuous delivery

​What’s next

Deploying a scheduled flow to production

Testing the flow locally

Deploying to production

Triggering a production run

Deploying with stable production environments

GitOps and continuous delivery

What’s next