View on GitHub


Repository for participants of the "worker & atools" training

An important type of scientific workload is quite easy to parallelize, e.g.,

It is in fact so easy that it is called embarrassingly parallel. Since this workload is so common, we developed some frameworks to support them, and take the bookkeeping burden of your shoulders as much as possible: worker and atools.

Learning outcomes

When you complete this training you will be able to


Total duration: 3 hours.

Subject Duration
introduction and motivation 15 min.
worker framework 75 min.
coffee break 15 min.
atools 45 min.
use cases & comparison 20 min.
wrap up 10 min.

Training materials

Slides are available in the GitHub repository, as well as example code and job scripts.

Repository and documentation of the tools covered:

Video materials

Video recordings of this training are available on YouTube.

  1. Introduction (1 minute)
  2. worker: parameter exploration (11 minutes)
  3. Get example scripts (1 minute)
  4. worker: Bash example (6 minutes)
  5. worker: MapReduce (2.5 minutes)
  6. worker features (17 minutes)
  7. parameter-weaver (5 minutes)
  8. worker: tuning (19 minutes)
  9. atools: parameter exploration (6.5 minutes)
  10. atools: features (7 minutes)
  11. atools: demo (6 minutes)
  12. atools: tuning (2 minutes)
  13. Comparison between worker and atools (2 minutes)
  14. File I/O and performance (4.5 minutes)
  15. Conclusions (1 minute)
  16. Implementation (3.5 minutes)

Target audience

This training is for you if you need to use HPC resources effectively for embarrassingly parallel workloads.


You will need to be comfortable using Linux and the HPC environment. If necessary, attend the appropriate training sessions on those subjects.