Welcome to gwf¶
gwf is a flexible, pragmatic workflow tool for building and running large, scientific workflows. It runs on Python 3.7+ and is developed at GenomeDK, Aarhus University.
- Examples
To get a feeling for what a gwf workflow looks like, have a look at a few examples.
- Getting started
To quickly get started writing workflows in gwf you can read the Tutorial.
Features¶
Easy to adopt, there’s no special syntax to learn if you already know Python
Automatically resolves dependencies between targets based on filenames
Only submits targets when their output files are not up to date
Supports multiple backends like Slurm, SGE and a local backend for testing
Fire-and-forget, does not require you to use screen or tmux to keep your workflow running
Commands for cleaning temporary data from your workflow
Friendly to your system administrator!
Installation¶
We’ll assume that you have the Anaconda distribution installed and that you are familiar with how to install and manage packages and environments through the conda package manager.
To install gwf via Conda:
conda config --add channels gwforg
conda install gwf
We recommend that you install gwf in a project-specific environment:
conda config --add channels gwforg
conda create -n myproject gwf dep1 dep2 ...
conda activate myproject
You can find the code for gwf here. You are encouraged to report any issues through the issue tracker, which is also a good place to ask questions.
User’s Guide¶
- User’s Guide
- Tutorial
- A Minimal Workflow
- Running Your First Workflow
- Setting the Default Verbosity
- Debugging a Workflow
- Defining Targets with Dependencies
- Named Inputs and Outputs
- Specifying Target Resources
- Observing Target Execution
- What Happens When a Target Fails?
- Reusable Targets with Templates
- Viewing Logs
- Cleaning Up
- Protecting Files From Being Cleaned Up
- A Note About Reproducibility
- Mapping over Inputs
- Configuration
- Tips and Tricks
- Tutorial
Extending¶
Release history¶
- Change Log
- Version 2.0.5
- Version 2.0.4
- Version 2.0.3
- Version 2.0.2
- Version 2.0.1
- Version 2.0.0
- Version 1.8.5
- Version 1.8.4
- Version 1.8.3
- Version 1.8.2
- Version 1.8.1
- Version 1.8.0
- Version 1.7.2
- Version 1.7.1
- Version 1.7.0
- Version 1.6.0
- Version 1.5.1
- Version 1.5.0
- Version 1.4.0
- Version 1.3.2
- Version 1.3.1
- Version 1.3.0
- Version 1.2.1
- Version 1.2
- Version 1.1
- Version 1.0
- Version 1.0b10
- Version 1.0b9
- Version 1.0b8
- Version 1.0b7
- Version 1.0b6
- Version 1.0b5