MLIR (software)

MLIR (Multi-Level Intermediate Representation) is an open-source compiler infrastructure project developed as a sub-project of the LLVM project. It provides a modular and extensible intermediate representation (IR) framework intended to facilitate the construction of domain-specific compilers and improve compilation for heterogeneous computing platforms. MLIR supports multiple abstraction levels in a single IR and introduces dialects, a mechanism for defining custom operations, types, and attributes tailored to specific domains. The name "Multi-Level Intermediate Representation" reflects the system’s ability to model computations at various abstraction levels and progressively lower them toward machine code.

MLIR was originally developed in 2018 by Chris Lattner at Google, and publicly released as part of LLVM in 2019. It was designed to address challenges in building compilers for modern workloads such as machine learning, hardware acceleration, and high-level synthesis by providing reusable components and standardizing the representation of intermediate computations across different programming languages and hardware targets.

MLIR is used in a range of systems including TensorFlow, Mojo, TPU-MLIR, and others. It is released under the Apache License 2.0 with LLVM exceptions and is maintained as part of the LLVM project.

History

Work on MLIR began in 2018, led by Chris Lattner at Google in collaboration with Mehdi Amini, River Riddle, and others, as a response to the growing complexity of modern compiler toolchains. The project aimed to improve the modularity, composability, and maintainability of compiler infrastructures, particularly in domains such as machine learning, high-level synthesis, and hardware acceleration. It was formally introduced at the 2019 LLVM Developer Meeting and was open-sourced later that year as part of the LLVM monorepository.

MLIR’s architecture was shaped by prior experiences building compilers such as XLA and LLVM, where limitations in existing intermediate representations hindered optimization and reuse across abstraction levels. To address this, MLIR introduced a novel concept of multi-level IRs that could coexist in the same system and be gradually lowered through well-defined transformations. A foundational design feature was the use of dialects, allowing different domains and hardware targets to define custom operations and type systems while maintaining interoperability.

Since its release, MLIR has been adopted by multiple compiler ecosystems and research efforts. In TensorFlow, MLIR serves as the foundation for rewriting and lowering transformations in components such as XLA and TensorFlow Runtime. The language Mojo, developed by Modular Inc., relies on MLIR to achieve ahead-of-time compilation for artificial intelligence workloads. Additional projects that have built on MLIR include TPU-MLIR for compiling models to Tensor Processing Unit hardware, ONNX-MLIR for interoperable machine learning models, MLIR-AIE for targeting Xilinx AI Engines, IREE for compiling and executing machine learning models across CPUs, GPUs, and accelerators, DSP-MLIR, a compiler infrastructure tailored for digital signal processing (DSP) applications, and torch-mlir, which brings MLIR-based compilation capabilities to the PyTorch ecosystem.

MLIR continues to evolve as part of the LLVM Project and follows the project's release schedule and development policies. It is developed collaboratively by contributors from industry, academia, and the broader open-source community.

Dialects

In MLIR, a dialect defines a self-contained namespace of operations, types, attributes, and other constructs. Dialects are the primary mechanism for extensibility, allowing developers to introduce domain-specific abstractions while maintaining compatibility within the broader MLIR framework. Each operation within a dialect is identified by a unique name and may include optional operands, results, attributes, and regions. Operands and results follow the static single-assignment form (SSA), and each result is associated with a type. Attributes represent compile-time metadata, such as constant values. Regions consist of ordered blocks, each of which may take input arguments and contain a sequence of nested operations. While MLIR is designed around SSA, it avoids traditional PHI nodes by using block arguments in conjunction with the operands of control-flow operations to model value merging.

The general syntax for an operation is the following:

%res:2 = "mydialect.morph"(%input#3) ()
: (!mydialect<"custom_type">) -> (!mydialect<"other_type">, !mydialect<"other_type">)
loc(callsite("foo" at "mysource.cc":10:8))
This operation, named morph, belongs to the mydialect dialect. It takes one input operand (%input#3) of type custom_type and produces two output values of type other_type. The operation includes two attributes—some.attribute and other_attribute—and contains a region with a single block (^bb0) that accepts one argument. The loc keyword specifies source-level location information, which can be used for debugging or diagnostic reporting.
The syntax of operations, types and attributes can also be customized according to the user preferences by implementing proper parsing and printing functions within the operation definition.

Core dialects

The MLIR dialects ecosystem is open and extensible, allowing end-users to define new dialects that capture the semantics of specific computational domains. At the same time, the MLIR codebase provides a variety of built-in dialects that address common patterns found in intermediate representations. These core dialects are designed to be self-contained and interoperable, making them suitable for reuse across different compiler stacks.

For example, the arith dialect includes basic mathematical operations over integers and floating-point types, while the memref dialect provides operations for memory allocation and access. Control-flow abstractions are handled by dialects such as affine, which supports affine loop nests suitable for polyhedral optimization, and scf, which provides structured control flow using constructs like for, if, and while. The func dialect supports function definitions and calls, while the gpu dialect introduces primitives for GPU programming models. Additionally, the tosa dialect defines a portable and quantization-friendly operator set for machine learning inference. Finally, the llvm dialect provides a one-to-one mapping to LLVM IR, enabling seamless lowering to LLVM’s backend and reuse of its optimization and code generation infrastructure.

The following code defines a function that takes two floating point matrices and performs the sum between the values at the same positions:

func.func @matrix_add(%arg0: memref<10x20xf32>, %arg1: memref<10x20xf32>) -> memref<10x20xf32>
Although different dialects may be used to express similar computations, the level of abstraction and the intended compilation flow may vary. In the example above, the affine dialect enables polyhedral analysis and optimizations, while the memref and arith dialects express memory and arithmetic operations, respectively.

Operation definition specification

The operations of a dialect can be defined using the C++ language, but also in a more convenient and robust way by using the Operation definition specification (ODS). By using TableGen, the C++ code for declarations and definitions can be then automatically generated.

The autogenerated code can include parsing and printing methods – which are based on a simple string mapping the structure of desired textual representation – together with all the boilerplate code for accessing fields and perform common actions such verification of the semantics of each operation, canonicalization or folding.

The same declaration mechanism can be used also for types and attributes, which are the other two categories of elements constituting a dialect.

The following example illustrates how to specify the ''assembly format'' of an operation expecting a variadic number of operands and producing zero results. The textual representation consists in the optional list of attributes, followed by the optional list of operands, a colon, and types of the operands.

let assemblyFormat = "attr-dict ($operands^ `:` type($operands))?";

Transformations

Transformations can always be performed directly on the IR, without having to rely on built-in coordination mechanisms. However, in order to ease both implementation and maintenance, MLIR provides an infrastructure for IR rewriting that is composed by different rewrite drivers. Each driver receives a set of objects named ''patterns'', each of which has its own internal logic to match operations with certain properties. When an operation is matched, the rewrite process is performed and the IR is modified according to the logic within the pattern.

Dialect conversion driver

This driver operates according to the ''legality'' of existing operations, meaning that the driver receives a set of rules determining which operations have to be considered ''illegal'' and expects the patterns to match and convert them into ''legal'' ones. The logic behind those rules can be arbitrarily complex: it may be based just on the dialect to which the operations belong, but can also inspect more specific properties such as attributes or nested operations.

As the names suggests, this driver is typically used for converting the operations of a dialect into operations belonging to a different one. In this scenario, the whole source dialect would be marked as illegal, the destination one as legal, and patterns for the source dialect operations would be provided. The dialect conversion framework also provides support for type conversion, which has to be performed on operands and results to convert them to the type system of the destination dialect.

MLIR allows for multiple conversion paths to be taken. Considering the example about the sum of matrices, a possible lowering strategy may be to generate for-loops belonging to the ''scf'' dialect, obtaining code to be executed on CPUs:

#map = affine_map<(d0, d1) -> (d0, d1)>

module
Another possible strategy, however, could have been to use the ''gpu'' dialect to generate code for GPUs:

#map = affine_map<(d0, d1) -> (d0, d1)>

module

Greedy pattern rewrite driver

The driver greedily applies the provided patterns according to their benefit, until a fixed point is reached or the maximum number of iterations is reached. The benefit of a pattern is self-attributed. In case of equalities, the relative order within the patterns list is used.

Traits and interfaces

MLIR allows to apply existing optimizations (e.g., common subexpression elimination, loop-invariant code motion) on custom dialects by means of traits and interfaces. These two mechanisms enable transformation passes to operate on operations without knowing their actual implementation, relying only on some properties that traits or interfaces provide.

Traits are meant to be attached to operations without requiring any additional implementation. Their purpose is to indicate that the operation satisfies certain properties (e.g. having exactly two operands). Interfaces, instead, represent a more powerful tool through which the operation can be queried about some specific aspect, whose value may change between instances of the same kind of operation. An example of interface is the representation of memory effects: each operation that operates on memory may have such interface attached, but the actual effects may depend on the actual operands (e.g., a function call with arguments possibly being constants or references to memory).

Applications

The freedom in modeling intermediate representations enables MLIR to be used in a wide range of scenarios. This includes traditional programming languages, but also high-level synthesis, quantum computing and homomorphic encryption. Machine learning applications also take advantage of built-in polyhedral compilation techniques, together with dialects targeting accelerators and other heterogeneous systems.

See also

* TensorFlow
* Tensor Processing Unit

References

External links

*
* {{GitHub, llvm/mlir-www, MLIR WWW, source code

Code documentation

Free and open source compilers
Software using the University of Illinois/NCSA Open Source License
Software using the Apache license