Skip to main content
Pandas has utility functions that make it one line to create a table, store it in a database, and later run queries against the data. This page shows how to run a SQL query against a self-hosted database from a Metaflow flow, transform the results in a dataframe, and write them back to the database.
1

Add a table to the MySQL database

To run the full example locally, install MySQL and set up a database called test. This example uses a Python function defined in the script containing the flow to create the table, but you can set up the table any way you prefer to interact with the database.
2

Run the flow

The flow shows how to:
  • Access data in a pandas dataframe by running a SQL query on a local database.
    • This example uses a MySQL database, but you could also store data in PostgreSQL.
  • Make a transformation to the dataframe.
  • Save the result to a separate table in the database.
sql_query_local.py
3

Access artifacts outside of the flow

Run the following in any script or notebook to access the contents of the dataframe that was stored as a flow artifact with self.result: