Although vanilla Python is fairly slow and hence not a good candidate, there are several options to significantly increase the efficiency of Python programs.
Learning outcomes
When you complete this training you will
- understand and identify performance bottlenecks of Python;
- know some libraries that can help improve performance for scientific computing such as numpy, numexpr and numba;
- be able to use Cython to improve your code’s performance;
- be able to wrap C, C++ and Fortran code to use it from Python;
- understand the opportunities and pitfalls of multi-threaded programming with Python;
- be able to write distributed application using MPI;
- have an understanding of how frameworks for distributed computing such as dask and pyspark work.
Schedule
Total duration: 8 hours.
| Subject | Duration |
|---|---|
| introduction and motivation | 5 min. |
| performance and profiling | 45 min. |
| libraries | 10 min. |
| Numba | 60 min. |
| Cython | 90 min. |
| interfacing with C/C++/Fortran | 30 min. |
| multi-threaded programming | 60 min. |
| MPI | 45 min. |
| dask | 45 min. |
| pyspark | 45 min. |
| wrap up | 15 min. |
Training materials
Slides are available in the GitHub repository, as well as example code and hands-on material.
Software environment
Instructions on how to create the required software environment are available.
Target audience
This training is for you if you need to use Python for computationally intensive scientific computing.
Prerequisites
You will need experience programming in Python, using numpy, and have a passing familiarity with C/C++. This is not a training that starts from scratch.
If you plan to do Python programming in a Linux or HPC environment you should be familiar with these as well.
For following along hands-on, you need
- laptop or desktop with internet access.
- a system set up so you can connect to an HPC system, an account on an HPC system (e.g., VSC, CECI, …), compute credits if that is required to run jobs on the HPC system if you want to use an HPC system;
- a Python environment that can run Jupyter Lab if you want to use your own system;
- access to Google Colaboratory if you prefer not to install software.
Level
- Introductory: 10 %
- Intermediate: 40 %
- Advanced: 50 %
Trainer(s)
- Geert Jan Bex (geertjan.bex@uhasselt.be)