Executors

Note

Executors are currently only available for the Slurm backend. Setting an executor for other backends will not have any effect. Other backends will be supported in the near future.

Executors are used to enable runtime behavior for targets. This means that you can now use executors to run your target inside Conda, Pixi, Apptainer, and Singularity environments:

from gwf import Workflow
from gwf.executors import Conda

gwf = Workflow()

gwf.target("Test", inputs=[], outputs=[], executor=Conda("myenv")) <<< """
echo this will run inside the `myenv` Conda environment.
"""

A default executor can also be specified for the workflow:

from gwf import Workflow
from gwf.executors import Conda

gwf = Workflow(executor=Conda("myenv"))

gwf.target("Test", inputs=[], outputs=[]) <<< """
echo this will run inside the `myenv` Conda environment.
"""

Available executors

class gwf.executors.Bash

Executes a target directly with Bash.

This is the default behavior and is basically the same as not using the executor mechanism. Specs will be run using a Bash shell which must be available on the system.

class gwf.executors.Conda(env: str, debug_mode: bool = False)

Executes a target in a Conda environment.

This executor will run specs inside the Conda environment specified by env. The executor does not create or update Conda environments for you, so the environment must already exist.

The env can either be the name of a Conda environment (that can be looked up with conda env list) or a path to a Conda environment directory.

If debug_mode is set to true, the executor will print detailed debug info when activating the environment and running the script.

Conda must be installed and available on the system path, or the CONDA_EXE environment variable must be point to a Conda installation.

class gwf.executors.Pixi(project: str | None = None, env: str = 'default', debug_mode: bool = False)

Executes a target in a Pixi environment.

This executor will run specs inside a Pixi environment in a given Pixi project, as specified by project. If project is not specified, it is assumed that there’s a Pixi project in the workflow root.

The executor does not create or update Pixi environments for you, so the environment must already exist.

The env is the name of the environment to run in and defaults to default.

If debug_mode is set to true, the executor will print detailed debug info when activating the environment and running the script.

Pixi must be installed and available on the system path, or the PIXI_EXE environment variable must be point to a Pixi installation.

class gwf.executors.Singularity(image: str, flags: Iterable[str] = NOTHING, debug_mode: bool = False)

Executes a target in a Singularity container.

This executor will execute a target inside the Singularity container specified by image, which is the path to the image file (often .sif).

The executor will execute the target with the singularity exec command. Additional flags to singularity exec may be specified with the flags argument.

If debug_mode is set to true, the executor will print detailed debug info when activating the environment and running the script.

Singularity must be available on the system path.

class gwf.executors.Apptainer(image: str, flags: Iterable[str] = NOTHING, debug_mode: bool = False)

Executes a target in a Apptainer container.

This executor will execute a target inside the Apptainer container specified by image, which is the path to the image file (often .sif).

The executor will execute the target with the singularity exec command. Additional flags to singularity exec may be specified with the flags argument.

If debug_mode is set to true, the executor will print detailed debug info when activating the environment and running the script.

Apptainer must be available on the system path.