The Conda Configuration Engine for Power Users

Released last fall, conda 4.2 brought with it configuration superpowers. The capabilities are extensive, and they’re designed with conda power users, devops engineers, and sysadmins in mind.

Configuration information comes from four basic sources:

1. hard-coded defaults,
2. configuration files,
3. environment variables, and
4. command-line arguments.

Each time a conda process initializes, an operating context is built that in a cascading fashion merges configuration sources. Command-line arguments hold the highest precedence, and hard-coded defaults the lowest.

The configuration file search path has been dramatically expanded. In order from lowest to highest priority, and directly from the conda code,

SEARCH_PATH = ( 
 '/etc/conda/.condarc', 
 '/etc/conda/condarc', 
 '/etc/conda/condarc.d/', 
 '/var/lib/conda/.condarc', 
 '/var/lib/conda/condarc', 
 '/var/lib/conda/condarc.d/', 
 '$CONDA_ROOT/.condarc', 
 '$CONDA_ROOT/condarc', 
 '$CONDA_ROOT/condarc.d/', 
 '~/.conda/.condarc', 
 '~/.conda/condarc', 
 '~/.conda/condarc.d/', 
 '~/.condarc', 
 '$CONDA_PREFIX/.condarc', 
 '$CONDA_PREFIX/condarc', 
 '$CONDA_PREFIX/condarc.d/', 
 '$CONDARC', 
)

where environment variables and user home directory are expanded on first use. $CONDA_ROOT is automatically set to the root environment prefix (and shouldn’t be set by users), and $CONDA_PREFIX is automatically set for activated conda environments. Thus, conda environments can have their own individualized and customized configurations. For the “.d” directories in the search path, conda will read in sorted order any (and only) files ending with .yml or .yaml extensions. The $CONDARC environment variable can be any path to a file having a .yml or .yaml extension, or containing “condarc” in the file name; it can also be a directory.

Environment variables hold second-highest precedence, and all configuration parameters are able to be specified as environment variables. To convert from the condarc file-based configuration parameter name to the environment variable parameter name, make the name all uppercase and prepend CONDA_. For example, conda’s always_yes configuration parameter can be specified using a CONDA_ALWAYS_YES environment variable.

Configuration parameters in some cases have aliases. For example, setting always_yes: true or yes: true in a configuration file is equivalent to the command-line flag --yes. They’re all also equivalent to both CONDA_ALWAYS_YES=true and CONDA_YES=true environment variables. A validation error is thrown if multiple parameters aliased to each other are specified within a single configuration source.

There are three basic configuration parameter types: primitive, map, and sequence. Each follow a slightly different set of merge rules.

The primitive configuration parameter is the easiest to merge. Within the linearized chain of information sources, the last source that sets the parameter wins. There is one caveat: if the parameter is trailed by a #!final flag, the merge cascade stops for that parameter. (Indeed, the markup concept is borrowed from the !important rule in CSS.) While still giving end-users extreme flexibility in most cases, we also give sysadmins the ability to lock down as much configuration as needed by making files read-only.

Map configuration parameters have elements that are key-value pairs. Merges are at the per-key level. Given two files with the contents

# file: /etc/conda/condarc.d/proxies.yml
proxy_servers: https: 
 https://prod-proxy

# file: ~/.conda/condarc.d/proxies.yml
proxy_servers: 
 https: https://dev-proxy:1080 
 https: https://dev-proxy:1081

the merged proxy_servers configuration will be

proxy_servers: 
 https: https://dev-proxy:1080 
 https: https://dev-proxy:1081

However, by modifying the contents of the first file to be

# file: /etc/conda/condarc.d/proxies.yml
proxy_servers:
 https: https://prod-proxy #!final

the merged settings will be

proxy_servers: 
 https: https://dev-proxy:1080 
 https: https://prod-proxy

Note the use of the !final flag acts at the per-key level. A !final flag can also be set for the parameter as a whole. With the first file again changed to

# file: /etc/conda/condarc.d/proxies.yml
proxy_servers: #!final 
 https: https://prod-proxy

the merged settings will be

proxy_servers: 
 https: https://prod-proxy

with no http key defined.

The sequence parameter merges are the most involved. Consider contents of the three files

# file: /etc/conda/condarc
channels: 
 - one 
 - two

# file: ~/.condarc
channels: 
 - three 
 - four

# file: $CONDA_PREFIX/.condarc
channels: 
 - five 
 - six
the final merged configuration will be
channels: 
 - five 
 - six 
 - three 
 - four 
 - one 
 - two

Sequence order within each individual configuration source is preserved, while still respecting sources’ overall precedence. Just like map parameters, a !final flag can be used for a sequence parameter as a whole. However, the !final flag does not apply to individual elements of sequence parameters, and instead !top and !bottom flags are available. Modifying the sequence example to the following

# file: /etc/conda/condarc
channels: 
 - one #!top 
 - two

# file: ~/.condarc
channels: #!final 
 - three 
 - four #!bottom

# file: $CONDA_PREFIX/.condarc
channels: 
 - five 
 - six

will yield a final merged configuration

channels: 
 - one 
 - three 
 - two 
 - four

Managing all of these new sources of configuration could become difficult without some new tools. The most basic is conda config --validate, which simply exits 0 if conda’s configured state passes all validation tests. The command conda config --describe (recently added in 4.3.16) gives a detailed description of available configuration parameters.

We’ve also added the commands conda config --show-sources and conda config --show. The first displays all of the configuration information conda recognizes–in its non-merged form broken out per source. The second gives the final, merged values for all configuration parameters.

Conda’s configuration engine gives power users tools for ultimate control. If you’ve read to this point, that’s probably you. And as a conda power user, please consider participating in the conda canary program. Be on the cutting edge, and also help influence new conda features and behaviors before they’re solidified in general availability releases.