Working with Python

Python is available on most clusters by loading it as a module. To see available versions, run module spider python.

Environments

A good practice and strong recommendation when working with Python is to always use virtual environments while working on any project. This allows you to disentangle the different versions needed for specific packages without messing with other projects. Here are two possible ways one can manage virtual environments:

venv

When run, venv will create a virtual environment no top of the existing python installation that was loaded using module load .... In order to use it, the environment needs to be activated: source venv/bin/activate. This will put environment-specific python and pip in your $PATH variable. To check, run which python to confirm it now points to the python executable in your environment directory. To deactivate the environment, run deactivate. venv comes with the standard python library.

Miniconda or Micromamba

In addition to virtual environment management, these packages also allow you to download and use pre-compiled software that can be found in the conda-forge repository. For example, to install a conda environment with Python 3.12 and Pytorch, you can run conda create -n "pytorch_environment" python=3.12 pytorch, this will also download the Python 3.12 binary. It is possible to use pip together with conda when installing packages, but keep in mind that this can lead to issues. To install conda (or the equivalent albeit faster mamba) check out miniconda or micromamba.