User Tools

Site Tools


perun_and_custom_environments

Author: Dmytro Tymoshenko | Last update: November 6, 2025

🤯 Difficulty: easy | ⏱️ Time: 5-10 minutes | 🛠️ Ability to break something: absent (but it can give some ideas on how to do so…)

- Mom, can we access the lab's pre-installed tools on the cluster?
- We have conda environments at home.
- Conda at home: 47 environments named test, test2, test_old, env_backup_v3 with symlinks pointing nowhere

Occasionally, you need to create your own environment — either you are testing a new software or comparing outputs from different versions of the same one. This guide will assist you in doing so!

Custom environment creation

1. Configuration of your conda (one-time step)

Start by configuring your conda by editing your ~/.condarc file. Any editor (i.e., built-in nano or your preferred editor) will work fine. The configuration file can be empty/non-existent, it is expected. To the mentioned .condarc, located at your home directory (~/ — technically, shortcut for /home/username/) Add the following:

envs_dirs:
  - ~/.local/envs/
  - /scratch5/software/miniconda3/envs/
channels:
  - conda-forge
  - bioconda
  - defaults
channel_priority: strict

Save the file and run cat ~/.condarc to verify it looks right (you should see the same lines you have parsed above).

What exactly and why I am I configuring here?

2. Creating your environment

Now you're ready to create environments.

- Here you define environment name, not executable software alias, so feel free to name it as you wish but the way you remember it, like my_multiqc or multiqc_backward_compatibility, etc.
- When creating an environment, use the –prefix ~/conda_envs/[env_name] flag. Otherwise, it is the same workflow as for regular conda environments.
- I recommend using mamba instead of conda for speed, but you can use conda. As an example for software, I am using multiqc.

mamba create --prefix ~/conda_envs/multiqc multiqc

Critical note: You might see some libmamba lock file errors for shared cache directories — it can look scary, but those are fine and actually a good sign. You should be more concerned if you don't see them :)

3. Activation, verification and usage

If creation was successful, activate it with source activate [environment name] and verify the installation with which [executable software name] (you should see it pointing towards your home/prefix directory installation folders), then check the version ([executable software name] –version).

For example:

username@perun: source activate multiqc
(multiqc) username@perun: which multiqc
/home/username/.local/envs/multiqc/bin/multiqc
(multiqc) username@perun: multiqc --version
multiqc, version 1.32

It did work, but the installed software is old

Extra FAQ (will be updated over time based on questions)

General tips:

  1. If both your and shared environments have the environment with the same name, it will activate the one with higher priority (higher) in envs_dirs of ~/.condarc. You can switch local and shared entries as you wish, or add extra entries!
  2. You can create an empty environment: just don't provide packages to install: mamba create –prefix ~/conda_envs/[env_name]. I would suggest providing a specific Python version during installation to ensure compatibility, as in mamba create -n [env_name] python=3.12.
  3. Your environments support `pip`, direct unpacking and creating custom aliases. `make` installations or ones that require deep systemwide changes can be tricky for obvious (permission) issues. If you don't know what you are doing, contact Balagopal Pillai (Balagopal.Pillai@Dal.Ca) to fix/assist you in doing so, and if you know what you are doing — especially 🙂
  4. I generally recommend to add `which` and `–version` commands into the script to be logged, so you are always keeping an eye on what is happening in your pipelines.

Conda configuration explanation

Why are we configuring condarc in the first place? You're telling conda where to look for environments (env_dirs) — –prefix doesn't automatically add the directory there; and which package channels to prioritize (channels) — this prevents installing ancient software versions.

env_dirs: — this tells conda where to look for environments. It's a list of directories searched in order (priority: top to bottom). When you activate an environment by name, conda searches these paths sequentially.
~/.local/envs/ — a local user directory. It can be different, but it is common practice to assign it the mentioned way. In the current setup, conda prioritizes this, so any environment here takes precedence over duplicates elsewhere.
/scratch5/software/miniconda3/envs/ — shared cluster location which is our current “default”. If conda doesn't find an environment in the local user directory, it looks here. Can be modified in the future/new locations added.
channels: — this specifies where conda downloads packages from (the remote repositories). Order matters — conda checks them sequentially when resolving dependencies. If you don't want to add channel on persistent basis, just specify it (or them) during installation with -c flag (separately for each entry). For example, mamba install -c conda-forge -c bioconda multiqc
conda-forge/bioconda — 97%+ of all the software you are usually using you are getting from these channels.
channel_priority: strict — Optional but highly recommended. This enforces strict priority: conda won't mix versions across channels to satisfy dependencies. It prevents conda from grabbing an “ol“ but compatible” p”ckage from `defaults` when a newer one exists in `conda-forge`. Without this, you might get surprising outdated software.

Getting old software

It did run, but you're getting ancient builds. Python 2.7 in 2025? Welcome to conda's default channel!

Double-check your ~/.condarc and that the correct environment is activated (which). If you would rather not add channels in condarc and this is the issue, you can specify channels during install: mamba install -c conda-forge -c bioconda “multiqc==1.32” for a specific channels/software version during install.

Contacts

Questions? Suggestions? Assistance? > Dmytro Tymoshenko

perun_and_custom_environments.txt · Last modified: by dmytrot