The Lustre file system is a high-performance shared parallel file system dedicated for HPC and Data centers. It allows to efficiently share the disk resources to hundreds of computer clinets at the same time.
On supercomputer, there are several storage systems based on Lustre:
Unfortunately, an improper use of the Lustre file systems can lead to unnecessary overload or, in extreme cases, subsystem failures. Users of the supercomputer should comply with the best practices regarding the Lustre file system and its usage, which will enable the optimal utilization of resources as well.
When a program wants to access data stored on Lustre, it first queries the Metadata Server (MDS) for the file's location. With this information, it can directly communicate with the Object Storage Server (OSS) or servers, which allow access to the data, appending data to files, and so on.
ls -l should not be abused. When displaying the output of the ls -l command, the client sends a request to both the Metadata Server (MDS) and the Object Storage Server(s) for each listed file. It is also advised not to use ls with syntax coloring. To list the contents of a directory, it is recommended to run ls without additional arguments, or alternatively, use programs like find or lfs with the find option./home directory.read-only mode. If a program does not intend to write to a file, it should be opened with the O_RDONLY flag. Additionally, if the access time of the file is not important, it is good practice to use the O_NOATIME flag.