Speakers: Geert Jan BEX ([email protected];[email protected]), Olivier Mattelaer (Université catholique de Louvain), Raphaël Leplae (Université libre de Bruxelles), Damien François (Université catholique de Louvain), Orian Louant (université de Liège)
                               

To connect

Logged user should have a link "zoom link" on the left menu bar to connect. All registered user have received an email this Monday Morning with a link to the event (please check your spam).

Overview

Whenever one has to deal with multiple jobs on a HPC system, the idea of automating parts or all of the job management process involves describing and implementing so-called 'workflows'. Options for managing workflows are numerous and range from using basic scheduler features such as job arrays and job dependencies, up to using a complex system managed by a central, multi-user, database. Moreover, workflows tools are also available from the software development and deployment ecosystem and the whole "devops" movement. This online workshop aims at guiding participants towards the right tool for their use and help them reduce the time they spend managing their jobs by automating what can be automated and follow best practices.

After attending the workshop, you will

know about generally available GNU/Linux tools to manage multiple tasks and jobs
learn about some easy-to-install scientific workflow tools for single-user usage
learn how to automate packaging and deploying software with containers or git-based web sites
see how some research groups used around workflow management system tools

Prerequisites

General knowledge of Linux and the command line
Knowing how to install software as a regular user
Knowing about Slurm and job submission and management

Agenda

Day1: February 15th 2022

9h-9h15: Introduction/Welcome (Raphaël Leplae)
9h15-9h30: Introduction to Workflow (Damien François)
9h30-11h + 30 min Q/A+break: Workflows with basic GNU tools (and Maestro) (Damien François)
11h30-12h30 + 15min Q/A: Checkpoint/Restart (Olivier Mattelaer)
Lunch break
14h-15h + 30 min Q/A+break: Atools (Geert Jan Bex)
15h30-16h15 + 15 min Q/A+break: Makeflow (Damien François)

Day2: February 16th 2022

9h-10h + 30 min Q/A+break: Github (CI/CD)  (Olivier Mattelaer)
10h30-11h30 + 30 min Q/A+break: Singularity (Olivier Mattelaer)
Lunch break:
14h-15h + 30 min Q/A+break: Snakemake (Orian Louant)
15h30-17h: Users testimonials

    15h30-15h50: NextFlow for bioinformatics by Luc Cornet
    15h50-16h10: Fireworks for material science by Guillaume Brunin
    16h10-16h30: Coral, a home made workflow system tool to manage numerical climate simulations by François Klein
    16h30-16h50: Examples of CI/CD in research codebases using git-based websites and HPC Denis-Gabriel Caprace

https://events.prace-ri.eu/event/1321/