I/O performance

Geert Jan Bex

2024-10-21

Overview

• Motivation
• Types of data & solutions
  • Tabular data
  • Text data
  • Multidimensional arrays
  • Images
• Applications & environments
• Wrap up

Motivation

Why worry?

• Input/Output may take time
• Large dataset, e.g., machine learning, data science
• File systems have characteristics

HPC file systems tuned for

• reading large files/large chunks
• writing large files/large chunks

Many small read/writes

Metadata updates

• access time
• modification time
• file size

Warning

Major performance degradation for… entire system/all users!

Tip

Use file systems well!

Types of data & solutions

Many data types

• Tabular data: .csv, …
• Text data: .txt, .json, …
• Image data: .jpeg, .tiff, …
• Video data: .mp4, …

Many small files == many metadata operations

Solutions

• Use binary file formats
• Bundle many small files into large one
• Read/Write multiple items at once
• Use caching
• Use node-local scratch space

Tabular data

Reading CSV is… very slow

Alternatives:

• Parquet files
• Arrow files

Can be read by pandas/polars/R

Parquet versus Arrow

Parquet

• I/O & space efficient
• More CPU intensive for I/O (de)compression
• Not optimal for random access

Arrow

• I/O efficient
• In-memory format
• Random access okay, sequential operations very efficient

Many text files

• Use TAR, ZIP
• Use Arrow/Hugging Face dataset

Tip

Read/write multiple “files” per operation

Perfornamce comparison: texts

Random 200,000 reads of 100,000 “files” (\(\le\) 1,024 chars)

data format	time (s)	metadata IOPs
naive	64.7	100,000
ZIP file	97.4	1
dataset	22.7	3

Read 100,000 “files” (\(\le\) 1,024 chars)

data format	time (s)	metadata IOPs
naive	74.7	100,000
ZIP file	3.1	1
dataset	17.2	3

Multidimensional arrays

• Use HDF5
• Use Pytorch/TensorFlow tensors in Arrow file
• Use binary files

Warning

Don’t use ASCII files!

HDF5

Advantages

• Programming language agnostic
• Cross-platform
• Self-documenting
• Multiple datasets
• Parallel I/O

Mostly for numerical data

HDF5 dataset layout

Tip

Data layout matters a lot!

layout	time (s)
row-major	144.4
column-major	8760.6
stacked	8346.8

Random 10,000 reads of 2,000 matrices (648 \(\times\) 1152 \(\times\) 4, uint8)

Image data

Many formats: JPEG, GIF, PNG, TIFF

Can have

• 1 channel: grayscale \(\mapsto\) 2D array
• 3 channels: RGB or BGR \(\mapsto\) 3D array
• 4 channels: RGB or BGR + alpha \(\mapsto\) 3D array

Store as

• HDF5 file: dataset 3D array of uint8
• Arrow file: Pytorch or TensorFlow tensors

Arrow files

How? Use Hugging Face 🤗 datasets library

• Python only
• Data + metadata
• Easily split datasets for training & test
• Directory with Arrow file and JSON metadata
• Easy to use with Pytorch/TensorFlow

Perfornamce comparison: images

Contiguous read

data format	time (s)	nr. metadata IOPs
naive	12.6	2,000
HDF5	15.7	1
datasets	14.3	3

Warning

Reading individual files \(\mapsto\) many more metadata IOPs

Local disks

Random 10,000 reads of 2,000 matrices (\(648 \times 1152 \times 4\), uint8)

• Scenario 1: read from lustre
• Scenario 2: copy from lustre to local SSD, read from SSD

data format	Scenario 1		Scenario 2
	time (s)	metadata IOPs	time (s)	metadata IOPs
naive	41.2	10,000	11.0 + 36.9	2,000
HDF5	144.4	1	10.6 + 41.2	1
datasets	64.5	3	17.0 + 52.4	3

Applications & environments

The problem with Python/R

Running script loads lots of modules/packages == lots of metadata operations

Solution: use containers

Apptainer is supported on VSC-systems

• lmod module loads + Python script module load \(\mapsto\) 14 seconds, gazillion IOPs
• start container (3 GB image) + Python scirpt module load \(\mapsto\) 7 seconds, 2 IOPs

Best practices & caveats

• Build containers on target system (--fakeroot)
• Compile critical software components
• Be weary of .local et al.
• Use hpccm

Wrap up

Conclusions

• Understaning I/O constraints is crucial
• Adapting workflows is (relatively) easy

Tip

Remember: machine learning is not I/O bound: it’s all about metadata IOPs!

How to proceed?

If necessary, take trainings

• Linux intro
• HPC intro
• Containers on HPC
• Python for HPC
• Parallel programming with MPI

Contact support