The Slurm Workload Manager, formerly known as Simple Linux Utility for Resource Management (SLURM), or simply Slurm, is a free and open-source

job scheduler A job scheduler is a computer application for controlling unattended background program execution of job (computing), jobs. This is commonly called batch scheduling, as execution of non-interactive jobs is often called batch processing, though tr ...

for

Linux Linux ( or ) is a family of open-source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991, by Linus Torvalds. Linux is typically packaged as a Linux distribution, w ...

and

Unix-like A Unix-like (sometimes referred to as UN*X or *nix) operating system is one that behaves in a manner similar to a Unix system, although not necessarily conforming to or being certified to any version of the Single UNIX Specification. A Unix-li ...

kernels Kernel may refer to: Computing * Kernel (operating system), the central component of most operating systems * Kernel (image processing), a matrix used for image convolution * Compute kernel, in GPGPU programming * Kernel method, in machine learnin ...

, used by many of the world's supercomputers and computer clusters. It provides three key functions: * allocating exclusive and/or non-exclusive access to resources (computer nodes) to users for some duration of time so they can perform work, * providing a framework for starting, executing, and monitoring work, typically a parallel job such as Message Passing Interface (MPI) on a set of allocated nodes, and * arbitrating contention for resources by managing a queue of pending jobs. Slurm is the workload manager on about 60% of the

TOP500 The TOP500 project ranks and details the 500 most powerful non- distributed computer systems in the world. The project was started in 1993 and publishes an updated list of the supercomputers twice a year. The first of these updates always coinci ...

supercomputers. Slurm uses a best fit algorithm based on

Hilbert curve scheduling In parallel processing, the Hilbert curve scheduling method turns a multidimensional task allocation problem into a one-dimensional space filling problem using Hilbert curves, assigning related tasks to locations with higher levels of proximity.''S ...

fat tree The fat tree network is a universal network for provably efficient communication. It was invented by Charles E. Leiserson of the Massachusetts Institute of Technology in 1985. k-ary n-trees, the type of fat-trees commonly used in most high-perf ...

network topology in order to optimize locality of task assignments on parallel computers.

History

Slurm began development as a collaborative effort primarily by Lawrence Livermore National Laboratory,

SchedMD SchedMD LLC is an American software company that is the main developer of the Slurm Workload Manager (or Slurm), an open-source workload management system. SchedMD also provides support, training and consulting services around Slurm. SchedMD ...

, Linux NetworX, Hewlett-Packard, and Groupe Bull as a Free Software resource manager. It was inspired by the closed source Quadrics RMS and shares a similar syntax. The name is a reference to the

soda Soda or SODA may refer to: Chemistry * Some chemical compounds containing sodium ** Sodium carbonate, washing soda or soda ash ** Sodium bicarbonate, baking soda ** Sodium hydroxide, caustic soda ** Sodium oxide, an alkali metal oxide * Sod ...

in Futurama. Over 100 people around the world have contributed to the project. It has since evolved into a sophisticated batch scheduler capable of satisfying the requirements of many large computer centers. ,

list of most powerful computers in the world indicates that Slurm is the workload manager on more than half of the top ten systems.

Structure

Slurm's design is very modular with about 100 optional plugins. In its simplest configuration, it can be installed and configured in a couple of minutes. More sophisticated configurations provide database integration for accounting, management of resource limits and workload prioritization.

Features

Slurm features include: * No single point of failure, backup daemons, fault-tolerant job options * Highly scalable (schedules up to 100,000 independent jobs on the 100,000 sockets of

IBM Sequoia IBM Sequoia was a petascale Blue Gene/Q supercomputer constructed by IBM for the National Nuclear Security Administration as part of the Advanced Simulation and Computing Program (ASC). It was delivered to the Lawrence Livermore National Labora ...

) * High performance (up to 1000 job submissions per second and 600 job executions per second) * Free and open-source software (

GNU General Public License The GNU General Public License (GNU GPL or simply GPL) is a series of widely used free software licenses that guarantee end users the four freedoms to run, study, share, and modify the software. The license was the first copyleft for general ...

) * Highly configurable with about 100 plugins * Fair-share scheduling with hierarchical bank accounts * Preemptive and gang scheduling (time-slicing of parallel jobs) * Integrated with database for accounting and configuration * Resource allocations optimized for network topology and on-node topology (sockets, cores and hyperthreads) * Advanced reservation * Idle nodes can be powered down * Different operating systems can be booted for each job * Scheduling for generic resources (e.g.

Graphics processing unit A graphics processing unit (GPU) is a specialized electronic circuit designed to manipulate and alter memory to accelerate the creation of images in a frame buffer intended for output to a display device. GPUs are used in embedded systems, m ...

) * Real-time accounting down to the task level (identify specific tasks with high CPU or memory usage) * Resource limits by user or bank account * Accounting for power consumption by job * Support of IBM Parallel Environment (PE/POE) * Support for job arrays * Job profiling (periodic sampling of each task's CPU use, memory use, power consumption, network and file system use) * Sophisticated multifactor job prioritization algorithms * Support for MapReduce+ * Support for burst buffer that accelerates scientific data movement The following features are announced for version 14.11 of Slurm, was released in November 2014: * Improved job array data structure and scalability * Support for heterogeneous generic resources * Add user options to set the CPU governor * Automatic job requeue policy based on exit value * Report API use by user, type, count and time consumed * Communication gateway nodes improve scalability

Supported platforms

Slurm is primarily developed to work alongside

distributions, although there is also support for a few other

POSIX The Portable Operating System Interface (POSIX) is a family of standards specified by the IEEE Computer Society for maintaining compatibility between operating systems. POSIX defines both the system- and user-level application programming inter ...

-based

operating system An operating system (OS) is system software that manages computer hardware, software resources, and provides common services for computer programs. Time-sharing operating systems schedule tasks for efficient use of the system and may also i ...

s, including BSDs ( FreeBSD, NetBSD and OpenBSD).Slurm Platforms
/ref> Slurm also supports several unique computer architectures, including: * IBM

BlueGene Blue Gene is an IBM project aimed at designing supercomputers that can reach operating speeds in the petaFLOPS (PFLOPS) range, with low power consumption. The project created three generations of supercomputers, Blue Gene/L, Blue Gene/P, ...

/Q models, including the 20 petaflop

* Cray XT, XE and Cascade *

Tianhe-2 Tianhe-2 or TH-2 (, i.e. 'Milky Way 2') is a 33.86- petaflops supercomputer located in the National Supercomputer Center in Guangzhou, China. It was developed by a team of 1,300 scientists and engineers. It was the world's fastest supercomputer ...

a 33.9 petaflop system with 32,000 Intel Ivy Bridge chips and 48,000 Intel Xeon Phi chips with a total of 3.1 million cores * IBM Parallel Environment *

Anton Anton may refer to: People *Anton (given name), including a list of people with the given name *Anton (surname) Places *Anton Municipality, Bulgaria **Anton, Sofia Province, a village *Antón District, Panama **Antón, a town and capital of th ...

License

Slurm is available under the GNU General Public License v2.

Commercial support

In 2010, the developers of Slurm founded SchedMD, which maintains the canonical source, provides development, level 3 commercial support and training services. Commercial support is also available from Bull, Cray, and Science + Computing.

References

External links

Slurm Documentation

SchedMD

Slurm Workload Manager Architecture Configuration and Use
{{DEFAULTSORT:Slurm Job scheduling Parallel computing Grid computing Cluster computing Free software