TheInfoList

OR:

Linear algebra is the branch of
mathematics Mathematics is an area of knowledge that includes the topics of numbers, formulas and related structures, shapes and the spaces in which they are contained, and quantities and their changes. These topics are represented in modern mathematics ...
concerning
linear equation In mathematics, a linear equation is an equation that may be put in the form a_1x_1+\ldots+a_nx_n+b=0, where x_1,\ldots,x_n are the variables (or unknowns), and b,a_1,\ldots,a_n are the coefficients, which are often real numbers. The coefficie ...
s such as: :$a_1x_1+\cdots +a_nx_n=b,$
linear map In mathematics, and more specifically in linear algebra, a linear map (also called a linear mapping, linear transformation, vector space homomorphism, or in some contexts linear function) is a mapping V \to W between two vector spaces that pre ...
s such as: :$\left(x_1, \ldots, x_n\right) \mapsto a_1x_1+\cdots +a_nx_n,$ and their representations in
vector space In mathematics and physics, a vector space (also called a linear space) is a set whose elements, often called '' vectors'', may be added together and multiplied ("scaled") by numbers called '' scalars''. Scalars are often real numbers, but ...
s and through
matrices Matrix most commonly refers to: * ''The Matrix'' (franchise), an American media franchise ** ''The Matrix'', a 1999 science-fiction action film ** "The Matrix", a fictional setting, a virtual reality environment, within ''The Matrix'' (franchis ...
. Linear algebra is central to almost all areas of mathematics. For instance, linear algebra is fundamental in modern presentations of
geometry Geometry (; ) is, with arithmetic, one of the oldest branches of mathematics. It is concerned with properties of space such as the distance, shape, size, and relative position of figures. A mathematician who works in the field of geometry is c ...
, including for defining basic objects such as lines, planes and
rotations Rotation, or spin, is the circular movement of an object around a '' central axis''. A two-dimensional rotating object has only one possible central axis and can rotate in either a clockwise or counterclockwise direction. A three-dimensional ...
. Also,
functional analysis Functional analysis is a branch of mathematical analysis, the core of which is formed by the study of vector spaces endowed with some kind of limit-related structure (e.g. inner product, norm, topology, etc.) and the linear functions defined o ...
, a branch of mathematical analysis, may be viewed as the application of linear algebra to spaces of functions. Linear algebra is also used in most sciences and fields of
engineering Engineering is the use of scientific principles to design and build machines, structures, and other items, including bridges, tunnels, roads, vehicles, and buildings. The discipline of engineering encompasses a broad range of more speciali ...
, because it allows modeling many natural phenomena, and computing efficiently with such models. For
nonlinear system In mathematics and science, a nonlinear system is a system in which the change of the output is not proportional to the change of the input. Nonlinear problems are of interest to engineers, biologists, physicists, mathematicians, and many other ...
s, which cannot be modeled with linear algebra, it is often used for dealing with first-order approximations, using the fact that the differential of a
multivariate function In mathematical analysis and its applications, a function of several real variables or real multivariate function is a function with more than one argument, with all arguments being real variables. This concept extends the idea of a function ...
at a point is the linear map that best approximates the function near that point.

History

The procedure (using counting rods) for solving simultaneous linear equations now called
Gaussian elimination In mathematics, Gaussian elimination, also known as row reduction, is an algorithm for solving systems of linear equations. It consists of a sequence of operations performed on the corresponding matrix of coefficients. This method can also be use ...
appears in the ancient Chinese mathematical text Chapter Eight: ''Rectangular Arrays'' of ''
The Nine Chapters on the Mathematical Art ''The Nine Chapters on the Mathematical Art'' () is a Chinese mathematics book, composed by several generations of scholars from the 10th–2nd century BCE, its latest stage being from the 2nd century CE. This book is one of the earliest sur ...
''. Its use is illustrated in eighteen problems, with two to five equations. Systems of linear equations arose in Europe with the introduction in 1637 by
René Descartes René Descartes ( or ; ; Latinized: Renatus Cartesius; 31 March 1596 – 11 February 1650) was a French philosopher, scientist, and mathematician, widely considered a seminal figure in the emergence of modern philosophy and science. Mathe ...
of
coordinates In geometry, a coordinate system is a system that uses one or more numbers, or coordinates, to uniquely determine the position of the points or other geometric elements on a manifold such as Euclidean space. The order of the coordinates is sig ...
in
geometry Geometry (; ) is, with arithmetic, one of the oldest branches of mathematics. It is concerned with properties of space such as the distance, shape, size, and relative position of figures. A mathematician who works in the field of geometry is c ...
. In fact, in this new geometry, now called Cartesian geometry, lines and planes are represented by linear equations, and computing their intersections amounts to solving systems of linear equations. The first systematic methods for solving linear systems used
determinant In mathematics, the determinant is a scalar value that is a function of the entries of a square matrix. It characterizes some properties of the matrix and the linear map represented by the matrix. In particular, the determinant is nonzero if an ...
s and were first considered by
Leibniz Gottfried Wilhelm (von) Leibniz . ( – 14 November 1716) was a German polymath active as a mathematician, philosopher, scientist and diplomat. He is one of the most prominent figures in both the history of philosophy and the history of mathe ...
in 1693. In 1750, Gabriel Cramer used them for giving explicit solutions of linear systems, now called Cramer's rule. Later,
Gauss Johann Carl Friedrich Gauss (; german: Gauß ; la, Carolus Fridericus Gauss; 30 April 177723 February 1855) was a German mathematician and physicist who made significant contributions to many fields in mathematics and science. Sometimes refer ...
further described the method of elimination, which was initially listed as an advancement in
geodesy Geodesy ( ) is the Earth science of accurately measuring and understanding Earth's figure ( geometric shape and size), orientation in space, and gravity. The field also incorporates studies of how these properties change over time and equiv ...
. In 1844
Hermann Grassmann Hermann Günther Grassmann (german: link=no, Graßmann, ; 15 April 1809 – 26 September 1877) was a German polymath known in his day as a linguist and now also as a mathematician. He was also a physicist, general scholar, and publisher. His mat ...
published his "Theory of Extension" which included foundational new topics of what is today called linear algebra. In 1848, James Joseph Sylvester introduced the term ''matrix'', which is Latin for ''womb''. Linear algebra grew with ideas noted in the
complex plane In mathematics, the complex plane is the plane formed by the complex numbers, with a Cartesian coordinate system such that the -axis, called the real axis, is formed by the real numbers, and the -axis, called the imaginary axis, is formed by the ...
. For instance, two numbers and in $\mathbb$ have a difference , and the line segments and are of the same length and direction. The segments are equipollent. The four-dimensional system $\mathbb$ of
quaternion In mathematics, the quaternion number system extends the complex numbers. Quaternions were first described by the Irish mathematician William Rowan Hamilton in 1843 and applied to mechanics in three-dimensional space. Hamilton defined a quat ...
s was started in 1843. The term ''vector'' was introduced as representing a point in space. The quaternion difference also produces a segment equipollent to . Other hypercomplex number systems also used the idea of a linear space with a
basis Basis may refer to: Finance and accounting * Adjusted basis, the net cost of an asset after adjusting for various tax-related items * Basis point, 0.01%, often used in the context of interest rates * Basis trading, a trading strategy consisting ...
.
Arthur Cayley Arthur Cayley (; 16 August 1821 – 26 January 1895) was a prolific British mathematician who worked mostly on algebra. He helped found the modern British school of pure mathematics. As a child, Cayley enjoyed solving complex maths problems ...
introduced
matrix multiplication In mathematics, particularly in linear algebra, matrix multiplication is a binary operation that produces a matrix from two matrices. For matrix multiplication, the number of columns in the first matrix must be equal to the number of rows in the ...
and the inverse matrix in 1856, making possible the
general linear group In mathematics, the general linear group of degree ''n'' is the set of invertible matrices, together with the operation of ordinary matrix multiplication. This forms a group, because the product of two invertible matrices is again invertible ...
. The mechanism of
group representation In the mathematical field of representation theory, group representations describe abstract groups in terms of bijective linear transformations of a vector space to itself (i.e. vector space automorphisms); in particular, they can be used ...
became available for describing complex and hypercomplex numbers. Crucially, Cayley used a single letter to denote a matrix, thus treating a matrix as an aggregate object. He also realized the connection between matrices and determinants, and wrote "There would be many things to say about this theory of matrices which should, it seems to me, precede the theory of determinants". Benjamin Peirce published his ''Linear Associative Algebra'' (1872), and his son
Charles Sanders Peirce Charles Sanders Peirce ( ; September 10, 1839 – April 19, 1914) was an American philosopher, logician, mathematician and scientist who is sometimes known as "the father of pragmatism". Educated as a chemist and employed as a scientist for ...
extended the work later. The
telegraph Telegraphy is the long-distance transmission of messages where the sender uses symbolic codes, known to the recipient, rather than a physical exchange of an object bearing the message. Thus flag semaphore is a method of telegraphy, whereas ...
required an explanatory system, and the 1873 publication of ''
A Treatise on Electricity and Magnetism ''A Treatise on Electricity and Magnetism'' is a two-volume treatise on electromagnetism written by James Clerk Maxwell in 1873. Maxwell was revising the ''Treatise'' for a second edition when he died in 1879. The revision was completed by W ...
'' instituted a field theory of forces and required
differential geometry Differential geometry is a mathematical discipline that studies the geometry of smooth shapes and smooth spaces, otherwise known as smooth manifolds. It uses the techniques of differential calculus, integral calculus, linear algebra and mult ...
for expression. Linear algebra is flat differential geometry and serves in tangent spaces to
manifold In mathematics, a manifold is a topological space that locally resembles Euclidean space near each point. More precisely, an n-dimensional manifold, or ''n-manifold'' for short, is a topological space with the property that each point has a ne ...
s. Electromagnetic symmetries of spacetime are expressed by the
Lorentz transformation In physics, the Lorentz transformations are a six-parameter family of linear transformations from a coordinate frame in spacetime to another frame that moves at a constant velocity relative to the former. The respective inverse transformatio ...
s, and much of the history of linear algebra is the history of Lorentz transformations. The first modern and more precise definition of a vector space was introduced by
Peano Giuseppe Peano (; ; 27 August 1858 – 20 April 1932) was an Italian mathematician and glottologist. The author of over 200 books and papers, he was a founder of mathematical logic and set theory, to which he contributed much notation. The sta ...
in 1888; by 1900, a theory of linear transformations of finite-dimensional vector spaces had emerged. Linear algebra took its modern form in the first half of the twentieth century, when many ideas and methods of previous centuries were generalized as
abstract algebra In mathematics, more specifically algebra, abstract algebra or modern algebra is the study of algebraic structures. Algebraic structures include groups, rings, fields, modules, vector spaces, lattices, and algebras over a field. The term ...
. The development of computers led to increased research in efficient
algorithm In mathematics and computer science, an algorithm () is a finite sequence of rigorous instructions, typically used to solve a class of specific problems or to perform a computation. Algorithms are used as specifications for performing ...
s for Gaussian elimination and matrix decompositions, and linear algebra became an essential tool for modelling and simulations.

Vector spaces

Until the 19th century, linear algebra was introduced through systems of linear equations and
matrices Matrix most commonly refers to: * ''The Matrix'' (franchise), an American media franchise ** ''The Matrix'', a 1999 science-fiction action film ** "The Matrix", a fictional setting, a virtual reality environment, within ''The Matrix'' (franchis ...
. In modern mathematics, the presentation through ''vector spaces'' is generally preferred, since it is more synthetic, more general (not limited to the finite-dimensional case), and conceptually simpler, although more abstract. A vector space over a field (often the field of the
real number In mathematics, a real number is a number that can be used to measure a ''continuous'' one- dimensional quantity such as a distance, duration or temperature. Here, ''continuous'' means that values can have arbitrarily small variations. Every ...
s) is a
set Set, The Set, SET or SETS may refer to: Science, technology, and mathematics Mathematics *Set (mathematics), a collection of elements *Category of sets, the category whose objects and morphisms are sets and total functions, respectively Electro ...
equipped with two
binary operation In mathematics, a binary operation or dyadic operation is a rule for combining two elements (called operands) to produce another element. More formally, a binary operation is an operation of arity two. More specifically, an internal binary ...
s satisfying the following
axiom An axiom, postulate, or assumption is a statement that is taken to be true, to serve as a premise or starting point for further reasoning and arguments. The word comes from the Ancient Greek word (), meaning 'that which is thought worthy or ...
s. Elements of are called ''vectors'', and elements of ''F'' are called ''scalars''. The first operation, ''
vector addition In mathematics, physics, and engineering, a Euclidean vector or simply a vector (sometimes called a geometric vector or spatial vector) is a geometric object that has magnitude (or length) and direction. Vectors can be added to other vectors a ...
'', takes any two vectors and and outputs a third vector . The second operation, ''
scalar multiplication In mathematics, scalar multiplication is one of the basic operations defining a vector space in linear algebra (or more generally, a module in abstract algebra). In common geometrical contexts, scalar multiplication of a real Euclidean vect ...
'', takes any scalar and any vector and outputs a new . The axioms that addition and scalar multiplication must satisfy are the following. (In the list below, and are arbitrary elements of , and and are arbitrary scalars in the field .) : The first four axioms mean that is an
abelian group In mathematics, an abelian group, also called a commutative group, is a group in which the result of applying the group operation to two group elements does not depend on the order in which they are written. That is, the group operation is commu ...
under addition. An element of a specific vector space may have various nature; for example, it could be a
sequence In mathematics, a sequence is an enumerated collection of objects in which repetitions are allowed and order matters. Like a set, it contains members (also called ''elements'', or ''terms''). The number of elements (possibly infinite) is cal ...
, a function, a
polynomial In mathematics, a polynomial is an expression consisting of indeterminates (also called variables) and coefficients, that involves only the operations of addition, subtraction, multiplication, and positive-integer powers of variables. An ex ...
or a
matrix Matrix most commonly refers to: * ''The Matrix'' (franchise), an American media franchise ** ''The Matrix'', a 1999 science-fiction action film ** "The Matrix", a fictional setting, a virtual reality environment, within ''The Matrix'' (franchis ...
. Linear algebra is concerned with those properties of such objects that are common to all vector spaces.

Linear maps

Linear maps are mappings between vector spaces that preserve the vector-space structure. Given two vector spaces and over a field , a linear map (also called, in some contexts, linear transformation or linear mapping) is a
map A map is a symbolic depiction emphasizing relationships between elements of some space, such as objects, regions, or themes. Many maps are static, fixed to paper or some other durable medium, while others are dynamic or interactive. Altho ...
: $T:V\to W$ that is compatible with addition and scalar multiplication, that is : $T\left(\mathbf u + \mathbf v\right)=T\left(\mathbf u\right)+T\left(\mathbf v\right), \quad T\left(a \mathbf v\right)=aT\left(\mathbf v\right)$ for any vectors in and scalar in . This implies that for any vectors in and scalars in , one has : $T\left(a \mathbf u + b \mathbf v\right)= T\left(a \mathbf u\right) + T\left(b \mathbf v\right) = aT\left(\mathbf u\right) + bT\left(\mathbf v\right)$ When are the same vector space, a linear map is also known as a ''linear operator'' on . A
bijective In mathematics, a bijection, also known as a bijective function, one-to-one correspondence, or invertible function, is a function between the elements of two sets, where each element of one set is paired with exactly one element of the other ...
linear map between two vector spaces (that is, every vector from the second space is associated with exactly one in the first) is an
isomorphism In mathematics, an isomorphism is a structure-preserving mapping between two structures of the same type that can be reversed by an inverse mapping. Two mathematical structures are isomorphic if an isomorphism exists between them. The word i ...
. Because an isomorphism preserves linear structure, two isomorphic vector spaces are "essentially the same" from the linear algebra point of view, in the sense that they cannot be distinguished by using vector space properties. An essential question in linear algebra is testing whether a linear map is an isomorphism or not, and, if it is not an isomorphism, finding its range (or image) and the set of elements that are mapped to the zero vector, called the
kernel Kernel may refer to: Computing * Kernel (operating system), the central component of most operating systems * Kernel (image processing), a matrix used for image convolution * Compute kernel, in GPGPU programming * Kernel method, in machine learni ...
of the map. All these questions can be solved by using
Gaussian elimination In mathematics, Gaussian elimination, also known as row reduction, is an algorithm for solving systems of linear equations. It consists of a sequence of operations performed on the corresponding matrix of coefficients. This method can also be use ...
or some variant of this
algorithm In mathematics and computer science, an algorithm () is a finite sequence of rigorous instructions, typically used to solve a class of specific problems or to perform a computation. Algorithms are used as specifications for performing ...
.

Subspaces, span, and basis

The study of those subsets of vector spaces that are in themselves vector spaces under the induced operations is fundamental, similarly as for many mathematical structures. These subsets are called
linear subspace In mathematics, and more specifically in linear algebra, a linear subspace, also known as a vector subspaceThe term ''linear subspace'' is sometimes used for referring to flats and affine subspaces. In the case of vector spaces over the reals, ...
s. More precisely, a linear subspace of a vector space over a field is a
subset In mathematics, set ''A'' is a subset of a set ''B'' if all elements of ''A'' are also elements of ''B''; ''B'' is then a superset of ''A''. It is possible for ''A'' and ''B'' to be equal; if they are unequal, then ''A'' is a proper subset o ...
of such that and are in , for every , in , and every in . (These conditions suffice for implying that is a vector space.) For example, given a linear map , the
image An image is a visual representation of something. It can be two-dimensional, three-dimensional, or somehow otherwise feed into the visual system to convey information. An image can be an artifact, such as a photograph or other two-dimensi ...
of , and the inverse image of (called
kernel Kernel may refer to: Computing * Kernel (operating system), the central component of most operating systems * Kernel (image processing), a matrix used for image convolution * Compute kernel, in GPGPU programming * Kernel method, in machine learni ...
or null space), are linear subspaces of and , respectively. Another important way of forming a subspace is to consider linear combinations of a set of vectors: the set of all sums : $a_1 \mathbf v_1 + a_2 \mathbf v_2 + \cdots + a_k \mathbf v_k,$ where are in , and are in form a linear subspace called the span of . The span of is also the intersection of all linear subspaces containing . In other words, it is the smallest (for the inclusion relation) linear subspace containing . A set of vectors is linearly independent if none is in the span of the others. Equivalently, a set of vectors is linearly independent if the only way to express the zero vector as a linear combination of elements of is to take zero for every coefficient . A set of vectors that spans a vector space is called a spanning set or generating set. If a spanning set is ''linearly dependent'' (that is not linearly independent), then some element of is in the span of the other elements of , and the span would remain the same if one remove from . One may continue to remove elements of until getting a ''linearly independent spanning set''. Such a linearly independent set that spans a vector space is called a
basis Basis may refer to: Finance and accounting * Adjusted basis, the net cost of an asset after adjusting for various tax-related items * Basis point, 0.01%, often used in the context of interest rates * Basis trading, a trading strategy consisting ...
of . The importance of bases lies in the fact that they are simultaneously minimal generating sets and maximal independent sets. More precisely, if is a linearly independent set, and is a spanning set such that , then there is a basis such that . Any two bases of a vector space have the same
cardinality In mathematics, the cardinality of a set is a measure of the number of elements of the set. For example, the set A = \ contains 3 elements, and therefore A has a cardinality of 3. Beginning in the late 19th century, this concept was generalized ...
, which is called the
dimension In physics and mathematics, the dimension of a mathematical space (or object) is informally defined as the minimum number of coordinates needed to specify any point within it. Thus, a line has a dimension of one (1D) because only one coord ...
of ; this is the dimension theorem for vector spaces. Moreover, two vector spaces over the same field are
isomorphic In mathematics, an isomorphism is a structure-preserving mapping between two structures of the same type that can be reversed by an inverse mapping. Two mathematical structures are isomorphic if an isomorphism exists between them. The word i ...
if and only if they have the same dimension. If any basis of (and therefore every basis) has a finite number of elements, is a ''finite-dimensional vector space''. If is a subspace of , then . In the case where is finite-dimensional, the equality of the dimensions implies . If and are subspaces of , then :$\dim\left(U_1 + U_2\right) = \dim U_1 + \dim U_2 - \dim\left(U_1 \cap U_2\right),$ where denotes the span of .

Matrices

Matrices allow explicit manipulation of finite-dimensional vector spaces and
linear map In mathematics, and more specifically in linear algebra, a linear map (also called a linear mapping, linear transformation, vector space homomorphism, or in some contexts linear function) is a mapping V \to W between two vector spaces that pre ...
s. Their theory is thus an essential part of linear algebra. Let be a finite-dimensional vector space over a field , and be a basis of (thus is the dimension of ). By definition of a basis, the map :$\begin \left(a_1, \ldots, a_m\right)&\mapsto a_1 \mathbf v_1+\cdots a_m \mathbf v_m\\ F^m &\to V \end$ is a
bijection In mathematics, a bijection, also known as a bijective function, one-to-one correspondence, or invertible function, is a function between the elements of two sets, where each element of one set is paired with exactly one element of the other ...
from , the set of the
sequences In mathematics, a sequence is an enumerated collection of objects in which repetitions are allowed and order matters. Like a set, it contains members (also called ''elements'', or ''terms''). The number of elements (possibly infinite) is called ...
of elements of , onto . This is an
isomorphism In mathematics, an isomorphism is a structure-preserving mapping between two structures of the same type that can be reversed by an inverse mapping. Two mathematical structures are isomorphic if an isomorphism exists between them. The word i ...
of vector spaces, if is equipped of its standard structure of vector space, where vector addition and scalar multiplication are done component by component. This isomorphism allows representing a vector by its inverse image under this isomorphism, that is by the coordinate vector or by the column matrix :$\begina_1\\\vdots\\a_m\end.$ If is another finite dimensional vector space (possibly the same), with a basis , a linear map from to is well defined by its values on the basis elements, that is . Thus, is well represented by the list of the corresponding column matrices. That is, if :$f\left(w_j\right)=a_v_1 + \cdots+a_v_m,$ for , then is represented by the matrix :$\begin a_&\cdots&a_\\ \vdots&\ddots&\vdots\\ a_&\cdots&a_ \end,$ with rows and columns.
Matrix multiplication In mathematics, particularly in linear algebra, matrix multiplication is a binary operation that produces a matrix from two matrices. For matrix multiplication, the number of columns in the first matrix must be equal to the number of rows in the ...
is defined in such a way that the product of two matrices is the matrix of the
composition Composition or Compositions may refer to: Arts and literature *Composition (dance), practice and teaching of choreography *Composition (language), in literature and rhetoric, producing a work in spoken tradition and written discourse, to include v ...
of the corresponding linear maps, and the product of a matrix and a column matrix is the column matrix representing the result of applying the represented linear map to the represented vector. It follows that the theory of finite-dimensional vector spaces and the theory of matrices are two different languages for expressing exactly the same concepts. Two matrices that encode the same linear transformation in different bases are called similar. It can be proved that two matrices are similar if and only if one can transform one into the other by elementary row and column operations. For a matrix representing a linear map from to , the row operations correspond to change of bases in and the column operations correspond to change of bases in . Every matrix is similar to an
identity matrix In linear algebra, the identity matrix of size n is the n\times n square matrix with ones on the main diagonal and zeros elsewhere. Terminology and notation The identity matrix is often denoted by I_n, or simply by I if the size is immaterial ...
possibly bordered by zero rows and zero columns. In terms of vector spaces, this means that, for any linear map from to , there are bases such that a part of the basis of is mapped bijectively on a part of the basis of , and that the remaining basis elements of , if any, are mapped to zero.
Gaussian elimination In mathematics, Gaussian elimination, also known as row reduction, is an algorithm for solving systems of linear equations. It consists of a sequence of operations performed on the corresponding matrix of coefficients. This method can also be use ...
is the basic algorithm for finding these elementary operations, and proving these results.

Linear systems

A finite set of linear equations in a finite set of variables, for example, , or is called a system of linear equations or a linear system. Systems of linear equations form a fundamental part of linear algebra. Historically, linear algebra and matrix theory has been developed for solving such systems. In the modern presentation of linear algebra through vector spaces and matrices, many problems may be interpreted in terms of linear systems. For example, let be a linear system. To such a system, one may associate its matrix : and its right member vector :$\mathbf = \begin 8\\-11\\-3 \end.$ Let be the linear transformation associated to the matrix . A solution of the system () is a vector :$\mathbf=\begin x\\y\\z \end$ such that :$T\left(\mathbf\right) = \mathbf,$ that is an element of the
preimage In mathematics, the image of a function is the set of all output values it may produce. More generally, evaluating a given function f at each element of a given subset A of its domain produces a set, called the "image of A under (or through ...
of by . Let () be the associated homogeneous system, where the right-hand sides of the equations are put to zero: The solutions of () are exactly the elements of the
kernel Kernel may refer to: Computing * Kernel (operating system), the central component of most operating systems * Kernel (image processing), a matrix used for image convolution * Compute kernel, in GPGPU programming * Kernel method, in machine learni ...
of or, equivalently, . The Gaussian-elimination consists of performing elementary row operations on the
augmented matrix In linear algebra Linear algebra is the branch of mathematics Mathematics is an area of knowledge that includes the topics of numbers, formulas and related structures, shapes and the spaces in which they are contained, and quantities and ...
: for putting it in
reduced row echelon form In linear algebra, a matrix is in echelon form if it has the shape resulting from a Gaussian elimination. A matrix being in row echelon form means that Gaussian elimination has operated on the rows, and column echelon form means that Gaussian e ...
. These row operations do not change the set of solutions of the system of equations. In the example, the reduced echelon form is : showing that the system () has the unique solution :$\beginx&=2\\y&=3\\z&=-1.\end$ It follows from this matrix interpretation of linear systems that the same methods can be applied for solving linear systems and for many operations on matrices and linear transformations, which include the computation of the ranks, kernels, matrix inverses.

Endomorphisms and square matrices

A linear
endomorphism In mathematics, an endomorphism is a morphism from a mathematical object to itself. An endomorphism that is also an isomorphism is an automorphism. For example, an endomorphism of a vector space is a linear map , and an endomorphism of a gr ...
is a linear map that maps a vector space to itself. If has a basis of elements, such an endomorphism is represented by a square matrix of size . With respect to general linear maps, linear endomorphisms and square matrices have some specific properties that make their study an important part of linear algebra, which is used in many parts of mathematics, including
geometric transformation In mathematics, a geometric transformation is any bijection of a set to itself (or to another such set) with some salient geometrical underpinning. More specifically, it is a function whose domain and range are sets of points — most often b ...
s,
coordinate change In mathematics, an ordered basis of a vector space of finite dimension allows representing uniquely any element of the vector space by a coordinate vector, which is a sequence of scalars called coordinates. If two different bases are consid ...
s,
quadratic form In mathematics, a quadratic form is a polynomial with terms all of degree two (" form" is another name for a homogeneous polynomial). For example, :4x^2 + 2xy - 3y^2 is a quadratic form in the variables and . The coefficients usually belong to ...
s, and many other part of mathematics.

Determinant

The ''determinant'' of a square matrix is defined to be :$\sum_ \left(-1\right)^ a_ \cdots a_,$ where is the group of all permutations of elements, is a permutation, and the parity of the permutation. A matrix is
invertible In mathematics, the concept of an inverse element generalises the concepts of opposite () and reciprocal () of numbers. Given an operation denoted here , and an identity element denoted , if , one says that is a left inverse of , and that ...
if and only if the determinant is invertible (i.e., nonzero if the scalars belong to a field). Cramer's rule is a
closed-form expression In mathematics, a closed-form expression is a mathematical expression that uses a finite number of standard operations. It may contain constants, variables, certain well-known operations (e.g., + − × ÷), and functions (e.g., ''n''th ...
, in terms of determinants, of the solution of a system of linear equations in unknowns. Cramer's rule is useful for reasoning about the solution, but, except for or , it is rarely used for computing a solution, since
Gaussian elimination In mathematics, Gaussian elimination, also known as row reduction, is an algorithm for solving systems of linear equations. It consists of a sequence of operations performed on the corresponding matrix of coefficients. This method can also be use ...
is a faster algorithm. The ''determinant of an endomorphism'' is the determinant of the matrix representing the endomorphism in terms of some ordered basis. This definition makes sense, since this determinant is independent of the choice of the basis.

Eigenvalues and eigenvectors

If is a linear endomorphism of a vector space over a field , an ''eigenvector'' of is a nonzero vector of such that for some scalar in . This scalar is an ''eigenvalue'' of . If the dimension of is finite, and a basis has been chosen, and may be represented, respectively, by a square matrix and a column matrix ; the equation defining eigenvectors and eigenvalues becomes :$Mz=az.$ Using the
identity matrix In linear algebra, the identity matrix of size n is the n\times n square matrix with ones on the main diagonal and zeros elsewhere. Terminology and notation The identity matrix is often denoted by I_n, or simply by I if the size is immaterial ...
, whose entries are all zero, except those of the main diagonal, which are equal to one, this may be rewritten :$\left(M-aI\right)z=0.$ As is supposed to be nonzero, this means that is a
singular matrix In linear algebra, an -by- square matrix is called invertible (also nonsingular or nondegenerate), if there exists an -by- square matrix such that :\mathbf = \mathbf = \mathbf_n \ where denotes the -by- identity matrix and the multiplicati ...
, and thus that its determinant equals zero. The eigenvalues are thus the
roots A root is the part of a plant, generally underground, that anchors the plant body, and absorbs and stores water and nutrients. Root or roots may also refer to: Art, entertainment, and media * ''The Root'' (magazine), an online magazine focusing ...
of the
polynomial In mathematics, a polynomial is an expression consisting of indeterminates (also called variables) and coefficients, that involves only the operations of addition, subtraction, multiplication, and positive-integer powers of variables. An ex ...
:$\det\left(xI-M\right).$ If is of dimension , this is a
monic polynomial In algebra, a monic polynomial is a single-variable polynomial (that is, a univariate polynomial) in which the leading coefficient (the nonzero coefficient of highest degree) is equal to 1. Therefore, a monic polynomial has the form: :x^n+c_x^+ ...
of degree , called the
characteristic polynomial In linear algebra, the characteristic polynomial of a square matrix is a polynomial which is invariant under matrix similarity and has the eigenvalues as roots. It has the determinant and the trace of the matrix among its coefficients. The ch ...
of the matrix (or of the endomorphism), and there are, at most, eigenvalues. If a basis exists that consists only of eigenvectors, the matrix of on this basis has a very simple structure: it is a
diagonal matrix In linear algebra, a diagonal matrix is a matrix in which the entries outside the main diagonal are all zero; the term usually refers to square matrices. Elements of the main diagonal can either be zero or nonzero. An example of a 2×2 diagonal ...
such that the entries on the
main diagonal In linear algebra Linear algebra is the branch of mathematics concerning linear equations such as: :a_1x_1+\cdots +a_nx_n=b, linear maps such as: :(x_1, \ldots, x_n) \mapsto a_1x_1+\cdots +a_nx_n, and their representations in vector spaces a ...
are eigenvalues, and the other entries are zero. In this case, the endomorphism and the matrix are said to be
diagonalizable In linear algebra, a square matrix A is called diagonalizable or non-defective if it is similar to a diagonal matrix, i.e., if there exists an invertible matrix P and a diagonal matrix D such that or equivalently (Such D are not unique. ...
. More generally, an endomorphism and a matrix are also said diagonalizable, if they become diagonalizable after extending the field of scalars. In this extended sense, if the characteristic polynomial is square-free, then the matrix is diagonalizable. A
symmetric matrix In linear algebra, a symmetric matrix is a square matrix that is equal to its transpose. Formally, Because equal matrices have equal dimensions, only square matrices can be symmetric. The entries of a symmetric matrix are symmetric with r ...
is always diagonalizable. There are non-diagonalizable matrices, the simplest being :$\begin0&1\\0&0\end$ (it cannot be diagonalizable since its square is the
zero matrix In mathematics, particularly linear algebra, a zero matrix or null matrix is a matrix all of whose entries are zero. It also serves as the additive identity of the additive group of m \times n matrices, and is denoted by the symbol O or 0 followe ...
, and the square of a nonzero diagonal matrix is never zero). When an endomorphism is not diagonalizable, there are bases on which it has a simple form, although not as simple as the diagonal form. The Frobenius normal form does not need of extending the field of scalars and makes the characteristic polynomial immediately readable on the matrix. The
Jordan normal form In linear algebra, a Jordan normal form, also known as a Jordan canonical form (JCF), is an upper triangular matrix of a particular form called a Jordan matrix representing a linear operator on a finite-dimensional vector space with respect to ...
requires to extend the field of scalar for containing all eigenvalues, and differs from the diagonal form only by some entries that are just above the main diagonal and are equal to 1.

Duality

A
linear form In mathematics, a linear form (also known as a linear functional, a one-form, or a covector) is a linear map from a vector space to its field of scalars (often, the real numbers or the complex numbers). If is a vector space over a field , the ...
is a linear map from a vector space over a field to the field of scalars , viewed as a vector space over itself. Equipped by
pointwise In mathematics, the qualifier pointwise is used to indicate that a certain property is defined by considering each value f(x) of some function f. An important class of pointwise concepts are the ''pointwise operations'', that is, operations defined ...
addition and multiplication by a scalar, the linear forms form a vector space, called the dual space of , and usually denoted or . If is a basis of (this implies that is finite-dimensional), then one can define, for , a linear map such that and if . These linear maps form a basis of , called the
dual basis In linear algebra, given a vector space ''V'' with a basis ''B'' of vectors indexed by an index set ''I'' (the cardinality of ''I'' is the dimension of ''V''), the dual set of ''B'' is a set ''B''∗ of vectors in the dual space ''V''∗ with ...
of . (If is not finite-dimensional, the may be defined similarly; they are linearly independent, but do not form a basis.) For in , the map :$f\to f\left(\mathbf v\right)$ is a linear form on . This defines the canonical linear map from into , the dual of , called the bidual of . This canonical map is an
isomorphism In mathematics, an isomorphism is a structure-preserving mapping between two structures of the same type that can be reversed by an inverse mapping. Two mathematical structures are isomorphic if an isomorphism exists between them. The word i ...
if is finite-dimensional, and this allows identifying with its bidual. (In the infinite dimensional case, the canonical map is injective, but not surjective.) There is thus a complete symmetry between a finite-dimensional vector space and its dual. This motivates the frequent use, in this context, of the
bra–ket notation In quantum mechanics, bra–ket notation, or Dirac notation, is used ubiquitously to denote quantum states. The notation uses angle brackets, and , and a vertical bar , to construct "bras" and "kets". A ket is of the form , v \rangle. Mathem ...
:$\langle f, \mathbf x\rangle$ for denoting .

Dual map

Let :$f:V\to W$ be a linear map. For every linear form on , the composite function is a linear form on . This defines a linear map :$f^*:W^*\to V^*$ between the dual spaces, which is called the dual or the transpose of . If and are finite dimensional, and is the matrix of in terms of some ordered bases, then the matrix of over the dual bases is the
transpose In linear algebra, the transpose of a matrix is an operator which flips a matrix over its diagonal; that is, it switches the row and column indices of the matrix by producing another matrix, often denoted by (among other notations). The t ...
of , obtained by exchanging rows and columns. If elements of vector spaces and their duals are represented by column vectors, this duality may be expressed in
bra–ket notation In quantum mechanics, bra–ket notation, or Dirac notation, is used ubiquitously to denote quantum states. The notation uses angle brackets, and , and a vertical bar , to construct "bras" and "kets". A ket is of the form , v \rangle. Mathem ...
by :$\langle h^\mathsf T , M \mathbf v\rangle = \langle h^\mathsf T M, \mathbf v\rangle.$ For highlighting this symmetry, the two members of this equality are sometimes written :$\langle h^\mathsf T \mid M \mid \mathbf v\rangle.$

Inner-product spaces

Besides these basic concepts, linear algebra also studies vector spaces with additional structure, such as an
inner product In mathematics, an inner product space (or, rarely, a Hausdorff pre-Hilbert space) is a real vector space or a complex vector space with an operation called an inner product. The inner product of two vectors in the space is a scalar, often ...
. The inner product is an example of a
bilinear form In mathematics, a bilinear form is a bilinear map on a vector space (the elements of which are called '' vectors'') over a field ''K'' (the elements of which are called '' scalars''). In other words, a bilinear form is a function that is line ...
, and it gives the vector space a geometric structure by allowing for the definition of length and angles. Formally, an ''inner product'' is a map :$\langle \cdot, \cdot \rangle : V \times V \to F$ that satisfies the following three
axiom An axiom, postulate, or assumption is a statement that is taken to be true, to serve as a premise or starting point for further reasoning and arguments. The word comes from the Ancient Greek word (), meaning 'that which is thought worthy or ...
s for all vectors in and all scalars in : * Conjugate symmetry: *:$\langle \mathbf u, \mathbf v\rangle =\overline.$ :In $\mathbb$, it is symmetric. *
Linear Linearity is the property of a mathematical relationship ('' function'') that can be graphically represented as a straight line. Linearity is closely related to '' proportionality''. Examples in physics include rectilinear motion, the linear ...
ity in the first argument: *:$\begin \langle a \mathbf u, \mathbf v\rangle &= a \langle \mathbf u, \mathbf v\rangle. \\ \langle \mathbf u + \mathbf v, \mathbf w\rangle &= \langle \mathbf u, \mathbf w\rangle+ \langle \mathbf v, \mathbf w\rangle. \end$ * Positive-definiteness: *:$\langle \mathbf v, \mathbf v\rangle \geq 0$ :with equality only for . We can define the length of a vector v in ''V'' by :$\, \mathbf v\, ^2=\langle \mathbf v, \mathbf v\rangle,$ and we can prove the
Cauchy–Schwarz inequality The Cauchy–Schwarz inequality (also called Cauchy–Bunyakovsky–Schwarz inequality) is considered one of the most important and widely used inequalities in mathematics. The inequality for sums was published by . The corresponding inequality fo ...
: :$, \langle \mathbf u, \mathbf v\rangle, \leq \, \mathbf u\, \cdot \, \mathbf v\, .$ In particular, the quantity :$\frac \leq 1,$ and so we can call this quantity the cosine of the angle between the two vectors. Two vectors are orthogonal if . An orthonormal basis is a basis where all basis vectors have length 1 and are orthogonal to each other. Given any finite-dimensional vector space, an orthonormal basis could be found by the Gram–Schmidt procedure. Orthonormal bases are particularly easy to deal with, since if , then :$a_i = \langle \mathbf v, \mathbf v_i \rangle.$ The inner product facilitates the construction of many useful concepts. For instance, given a transform , we can define its Hermitian conjugate as the linear transform satisfying :$\langle T \mathbf u, \mathbf v \rangle = \langle \mathbf u, T^* \mathbf v\rangle.$ If satisfies , we call normal. It turns out that normal matrices are precisely the matrices that have an orthonormal system of eigenvectors that span .

Relationship with geometry

There is a strong relationship between linear algebra and
geometry Geometry (; ) is, with arithmetic, one of the oldest branches of mathematics. It is concerned with properties of space such as the distance, shape, size, and relative position of figures. A mathematician who works in the field of geometry is c ...
, which started with the introduction by
René Descartes René Descartes ( or ; ; Latinized: Renatus Cartesius; 31 March 1596 – 11 February 1650) was a French philosopher, scientist, and mathematician, widely considered a seminal figure in the emergence of modern philosophy and science. Mathe ...
, in 1637, of
Cartesian coordinates A Cartesian coordinate system (, ) in a plane is a coordinate system that specifies each point uniquely by a pair of numerical coordinates, which are the signed distances to the point from two fixed perpendicular oriented lines, measured in ...
. In this new (at that time) geometry, now called Cartesian geometry, points are represented by
Cartesian coordinates A Cartesian coordinate system (, ) in a plane is a coordinate system that specifies each point uniquely by a pair of numerical coordinates, which are the signed distances to the point from two fixed perpendicular oriented lines, measured in ...
, which are sequences of three real numbers (in the case of the usual
three-dimensional space Three-dimensional space (also: 3D space, 3-space or, rarely, tri-dimensional space) is a geometric setting in which three values (called ''parameters'') are required to determine the position of an element (i.e., point). This is the informa ...
). The basic objects of geometry, which are lines and planes are represented by linear equations. Thus, computing intersections of lines and planes amounts to solving systems of linear equations. This was one of the main motivations for developing linear algebra. Most
geometric transformation In mathematics, a geometric transformation is any bijection of a set to itself (or to another such set) with some salient geometrical underpinning. More specifically, it is a function whose domain and range are sets of points — most often b ...
, such as
translations Translation is the communication of the meaning of a source-language text by means of an equivalent target-language text. The English language draws a terminological distinction (which does not exist in every language) between ''transl ...
,
rotation Rotation, or spin, is the circular movement of an object around a '' central axis''. A two-dimensional rotating object has only one possible central axis and can rotate in either a clockwise or counterclockwise direction. A three-dimensional ...
s, reflections, rigid motions, isometries, and projections transform lines into lines. It follows that they can be defined, specified and studied in terms of linear maps. This is also the case of homographies and
Möbius transformation In geometry and complex analysis, a Möbius transformation of the complex plane is a rational function of the form f(z) = \frac of one complex variable ''z''; here the coefficients ''a'', ''b'', ''c'', ''d'' are complex numbers satisfying ''a ...
s, when considered as transformations of a
projective space In mathematics, the concept of a projective space originated from the visual effect of perspective, where parallel lines seem to meet ''at infinity''. A projective space may thus be viewed as the extension of a Euclidean space, or, more generally ...
. Until the end of the 19th century, geometric spaces were defined by
axiom An axiom, postulate, or assumption is a statement that is taken to be true, to serve as a premise or starting point for further reasoning and arguments. The word comes from the Ancient Greek word (), meaning 'that which is thought worthy or ...
s relating points, lines and planes (
synthetic geometry Synthetic geometry (sometimes referred to as axiomatic geometry or even pure geometry) is the study of geometry without the use of coordinates or formulae. It relies on the axiomatic method and the tools directly related to them, that is, compas ...
). Around this date, it appeared that one may also define geometric spaces by constructions involving vector spaces (see, for example,
Projective space In mathematics, the concept of a projective space originated from the visual effect of perspective, where parallel lines seem to meet ''at infinity''. A projective space may thus be viewed as the extension of a Euclidean space, or, more generally ...
and Affine space). It has been shown that the two approaches are essentially equivalent. In classical geometry, the involved vector spaces are vector spaces over the reals, but the constructions may be extended to vector spaces over any field, allowing considering geometry over arbitrary fields, including
finite field In mathematics, a finite field or Galois field (so-named in honor of Évariste Galois) is a field that contains a finite number of elements. As with any field, a finite field is a set on which the operations of multiplication, addition, sub ...
s. Presently, most textbooks, introduce geometric spaces from linear algebra, and geometry is often presented, at elementary level, as a subfield of linear algebra.

Usage and applications

Linear algebra is used in almost all areas of mathematics, thus making it relevant in almost all scientific domains that use mathematics. These applications may be divided into several wide categories.

Geometry of ambient space

The modeling of ambient space is based on
geometry Geometry (; ) is, with arithmetic, one of the oldest branches of mathematics. It is concerned with properties of space such as the distance, shape, size, and relative position of figures. A mathematician who works in the field of geometry is c ...
. Sciences concerned with this space use geometry widely. This is the case with
mechanics Mechanics (from Ancient Greek: μηχανική, ''mēkhanikḗ'', "of machines") is the area of mathematics and physics concerned with the relationships between force, matter, and motion among physical objects. Forces applied to objec ...
and
robotics Robotics is an interdisciplinary branch of computer science and engineering. Robotics involves design, construction, operation, and use of robots. The goal of robotics is to design machines that can help and assist humans. Robotics integrate ...
, for describing
rigid body dynamics In the physical science of dynamics, rigid-body dynamics studies the movement of systems of interconnected bodies under the action of external forces. The assumption that the bodies are ''rigid'' (i.e. they do not deform under the action ...
;
geodesy Geodesy ( ) is the Earth science of accurately measuring and understanding Earth's figure ( geometric shape and size), orientation in space, and gravity. The field also incorporates studies of how these properties change over time and equiv ...
for describing Earth shape; perspectivity,
computer vision Computer vision is an interdisciplinary scientific field that deals with how computers can gain high-level understanding from digital images or videos. From the perspective of engineering, it seeks to understand and automate tasks that the huma ...
, and
computer graphics Computer graphics deals with generating images with the aid of computers. Today, computer graphics is a core technology in digital photography, film, video games, cell phone and computer displays, and many specialized applications. A great deal ...
, for describing the relationship between a scene and its plane representation; and many other scientific domains. In all these applications,
synthetic geometry Synthetic geometry (sometimes referred to as axiomatic geometry or even pure geometry) is the study of geometry without the use of coordinates or formulae. It relies on the axiomatic method and the tools directly related to them, that is, compas ...
is often used for general descriptions and a qualitative approach, but for the study of explicit situations, one must compute with
coordinates In geometry, a coordinate system is a system that uses one or more numbers, or coordinates, to uniquely determine the position of the points or other geometric elements on a manifold such as Euclidean space. The order of the coordinates is sig ...
. This requires the heavy use of linear algebra.

Functional analysis

Functional analysis Functional analysis is a branch of mathematical analysis, the core of which is formed by the study of vector spaces endowed with some kind of limit-related structure (e.g. inner product, norm, topology, etc.) and the linear functions defined o ...
studies
function space In mathematics, a function space is a set of functions between two fixed sets. Often, the domain and/or codomain will have additional structure which is inherited by the function space. For example, the set of functions from any set into a ve ...
s. These are vector spaces with additional structure, such as
Hilbert space In mathematics, Hilbert spaces (named after David Hilbert) allow generalizing the methods of linear algebra and calculus from (finite-dimensional) Euclidean vector spaces to spaces that may be infinite-dimensional. Hilbert spaces arise natural ...
s. Linear algebra is thus a fundamental part of functional analysis and its applications, which include, in particular,
quantum mechanics Quantum mechanics is a fundamental theory in physics that provides a description of the physical properties of nature at the scale of atoms and subatomic particles. It is the foundation of all quantum physics including quantum chemistry ...
(
wave function A wave function in quantum physics is a mathematical description of the quantum state of an isolated quantum system. The wave function is a complex-valued probability amplitude, and the probabilities for the possible results of measurements ...
s).

Study of complex systems

Most physical phenomena are modeled by
partial differential equation In mathematics, a partial differential equation (PDE) is an equation which imposes relations between the various partial derivatives of a multivariable function. The function is often thought of as an "unknown" to be solved for, similarly to ...
s. To solve them, one usually decomposes the space in which the solutions are searched into small, mutually interacting
cells Cell most often refers to: * Cell (biology), the functional basic unit of life Cell may also refer to: Locations * Monastic cell, a small room, hut, or cave in which a religious recluse lives, alternatively the small precursor of a monastery ...
. For
linear system In systems theory, a linear system is a mathematical model of a system based on the use of a linear operator. Linear systems typically exhibit features and properties that are much simpler than the nonlinear case. As a mathematical abstraction or ...
s this interaction involves
linear function In mathematics, the term linear function refers to two distinct but related notions: * In calculus and related areas, a linear function is a function whose graph is a straight line, that is, a polynomial function of degree zero or one. For dis ...
s. For nonlinear systems, this interaction is often approximated by linear functions.This is called a linear model or first-order approximation. Linear models are frequently used for complex nonlinear real-world systems because it makes parametrization more manageable. In both cases, very large matrices are generally involved.
Weather forecasting Weather forecasting is the application of science and technology to predict the conditions of the atmosphere for a given location and time. People have attempted to predict the weather informally for millennia and formally since the 19th cent ...
(or more specifically, parametrization for atmospheric modeling) is a typical example of a real-world application, where the whole Earth
atmosphere An atmosphere () is a layer of gas or layers of gases that envelop a planet, and is held in place by the gravity of the planetary body. A planet retains an atmosphere when the gravity is great and the temperature of the atmosphere is low. ...
is divided into cells of, say, 100km of width and 100km of height.

Scientific computation

Nearly all scientific computations involve linear algebra. Consequently, linear algebra algorithms have been highly optimized. BLAS and
LAPACK LAPACK ("Linear Algebra Package") is a standard software library for numerical linear algebra. It provides routines for solving systems of linear equations and linear least squares, eigenvalue problems, and singular value decomposition. It a ...
are the best known implementations. For improving efficiency, some of them configure the algorithms automatically, at run time, for adapting them to the specificities of the computer (
cache Cache, caching, or caché may refer to: Places United States * Cache, Idaho, an unincorporated community * Cache, Illinois, an unincorporated community * Cache, Oklahoma, a city in Comanche County * Cache, Utah, Cache County, Utah * Cache Co ...
size, number of available cores, ...). Some processors, typically
graphics processing units A graphics processing unit (GPU) is a specialized electronic circuit designed to manipulate and alter memory to accelerate the creation of images in a frame buffer intended for output to a display device. GPUs are used in embedded systems, mobi ...
(GPU), are designed with a matrix structure, for optimizing the operations of linear algebra.

Extensions and generalizations

This section presents several related topics that do not appear generally in elementary textbooks on linear algebra, but are commonly considered, in advanced mathematics, as parts of linear algebra.

Module theory

The existence of multiplicative inverses in fields is not involved in the axioms defining a vector space. One may thus replace the field of scalars by a
ring Ring may refer to: * Ring (jewellery), a round band, usually made of metal, worn as ornamental jewelry * To make a sound with a bell, and the sound made by a bell :(hence) to initiate a telephone connection Arts, entertainment and media Film and ...
, and this gives a structure called module over , or -module. The concepts of linear independence, span, basis, and linear maps (also called
module homomorphism In algebra, a module homomorphism is a function between modules that preserves the module structures. Explicitly, if ''M'' and ''N'' are left modules over a ring ''R'', then a function f: M \to N is called an ''R''-''module homomorphism'' or an ' ...
s) are defined for modules exactly as for vector spaces, with the essential difference that, if is not a field, there are modules that do not have any basis. The modules that have a basis are the
free module In mathematics, a free module is a module that has a basis – that is, a generating set consisting of linearly independent elements. Every vector space is a free module, but, if the ring of the coefficients is not a division ring (not a fie ...
s, and those that are spanned by a finite set are the finitely generated modules. Module homomorphisms between finitely generated free modules may be represented by matrices. The theory of matrices over a ring is similar to that of matrices over a field, except that
determinant In mathematics, the determinant is a scalar value that is a function of the entries of a square matrix. It characterizes some properties of the matrix and the linear map represented by the matrix. In particular, the determinant is nonzero if an ...
s exist only if the ring is
commutative In mathematics, a binary operation is commutative if changing the order of the operands does not change the result. It is a fundamental property of many binary operations, and many mathematical proofs depend on it. Most familiar as the name of ...
, and that a square matrix over a commutative ring is
invertible In mathematics, the concept of an inverse element generalises the concepts of opposite () and reciprocal () of numbers. Given an operation denoted here , and an identity element denoted , if , one says that is a left inverse of , and that ...
only if its determinant has a
multiplicative inverse In mathematics, a multiplicative inverse or reciprocal for a number ''x'', denoted by 1/''x'' or ''x''−1, is a number which when multiplied by ''x'' yields the multiplicative identity, 1. The multiplicative inverse of a fraction ''a''/' ...
in the ring. Vector spaces are completely characterized by their dimension (up to an isomorphism). In general, there is not such a complete classification for modules, even if one restricts oneself to finitely generated modules. However, every module is a
cokernel The cokernel of a linear mapping of vector spaces is the quotient space of the codomain of by the image of . The dimension of the cokernel is called the ''corank'' of . Cokernels are dual to the kernels of category theory, hence the nam ...
of a homomorphism of free modules. Modules over the integers can be identified with
abelian group In mathematics, an abelian group, also called a commutative group, is a group in which the result of applying the group operation to two group elements does not depend on the order in which they are written. That is, the group operation is commu ...
s, since the multiplication by an integer may identified to a repeated addition. Most of the theory of abelian groups may be extended to modules over a
principal ideal domain In mathematics, a principal ideal domain, or PID, is an integral domain in which every ideal is principal, i.e., can be generated by a single element. More generally, a principal ideal ring is a nonzero commutative ring whose ideals are princip ...
. In particular, over a principal ideal domain, every submodule of a free module is free, and the fundamental theorem of finitely generated abelian groups may be extended straightforwardly to finitely generated modules over a principal ring. There are many rings for which there are algorithms for solving linear equations and systems of linear equations. However, these algorithms have generally a computational complexity that is much higher than the similar algorithms over a field. For more details, see Linear equation over a ring.

Multilinear algebra and tensors

In
multilinear algebra Multilinear algebra is a subfield of mathematics that extends the methods of linear algebra. Just as linear algebra is built on the concept of a vector and develops the theory of vector spaces, multilinear algebra builds on the concepts of ''p' ...
, one considers multivariable linear transformations, that is, mappings that are linear in each of a number of different variables. This line of inquiry naturally leads to the idea of the dual space, the vector space consisting of linear maps where ''F'' is the field of scalars. Multilinear maps can be described via
tensor product In mathematics, the tensor product V \otimes W of two vector spaces and (over the same field) is a vector space to which is associated a bilinear map V\times W \to V\otimes W that maps a pair (v,w),\ v\in V, w\in W to an element of V \otimes ...
s of elements of . If, in addition to vector addition and scalar multiplication, there is a bilinear vector product , the vector space is called an
algebra Algebra () is one of the broad areas of mathematics. Roughly speaking, algebra is the study of mathematical symbols and the rules for manipulating these symbols in formulas; it is a unifying thread of almost all of mathematics. Elementary ...
; for instance, associative algebras are algebras with an associate vector product (like the algebra of square matrices, or the algebra of polynomials).

Topological vector spaces

Vector spaces that are not finite dimensional often require additional structure to be tractable. A
normed vector space In mathematics, a normed vector space or normed space is a vector space over the real or complex numbers, on which a norm is defined. A norm is the formalization and the generalization to real vector spaces of the intuitive notion of "length ...
is a vector space along with a function called a
norm Naturally occurring radioactive materials (NORM) and technologically enhanced naturally occurring radioactive materials (TENORM) consist of materials, usually industrial wastes or by-products enriched with radioactive elements found in the envir ...
, which measures the "size" of elements. The norm induces a
metric Metric or metrical may refer to: * Metric system, an internationally adopted decimal system of measurement * An adjective indicating relation to measurement in general, or a noun describing a specific type of measurement Mathematics In mathem ...
, which measures the distance between elements, and induces a
topology In mathematics, topology (from the Greek words , and ) is concerned with the properties of a geometric object that are preserved under continuous deformations, such as stretching, twisting, crumpling, and bending; that is, without closin ...
, which allows for a definition of continuous maps. The metric also allows for a definition of limits and completeness – a metric space that is complete is known as a
Banach space In mathematics, more specifically in functional analysis, a Banach space (pronounced ) is a complete normed vector space. Thus, a Banach space is a vector space with a metric that allows the computation of vector length and distance between vec ...
. A complete metric space along with the additional structure of an
inner product In mathematics, an inner product space (or, rarely, a Hausdorff pre-Hilbert space) is a real vector space or a complex vector space with an operation called an inner product. The inner product of two vectors in the space is a scalar, often ...
(a conjugate symmetric sesquilinear form) is known as a
Hilbert space In mathematics, Hilbert spaces (named after David Hilbert) allow generalizing the methods of linear algebra and calculus from (finite-dimensional) Euclidean vector spaces to spaces that may be infinite-dimensional. Hilbert spaces arise natural ...
, which is in some sense a particularly well-behaved Banach space.
Functional analysis Functional analysis is a branch of mathematical analysis, the core of which is formed by the study of vector spaces endowed with some kind of limit-related structure (e.g. inner product, norm, topology, etc.) and the linear functions defined o ...
applies the methods of linear algebra alongside those of
mathematical analysis Analysis is the branch of mathematics dealing with continuous functions, limits, and related theories, such as differentiation, integration, measure, infinite sequences, series, and analytic functions. These theories are usually studied ...
to study various function spaces; the central objects of study in functional analysis are spaces, which are Banach spaces, and especially the space of square integrable functions, which is the only Hilbert space among them. Functional analysis is of particular importance to quantum mechanics, the theory of partial differential equations, digital signal processing, and electrical engineering. It also provides the foundation and theoretical framework that underlies the Fourier transform and related methods.

Homological algebra

* Fundamental matrix (computer vision) *
Geometric algebra In mathematics, a geometric algebra (also known as a real Clifford algebra) is an extension of elementary algebra to work with geometrical objects such as vectors. Geometric algebra is built out of two fundamental operations, addition and the g ...
*
Linear programming Linear programming (LP), also called linear optimization, is a method to achieve the best outcome (such as maximum profit or lowest cost) in a mathematical model whose requirements are represented by linear relationships. Linear programming is ...
*
Linear regression In statistics, linear regression is a linear approach for modelling the relationship between a scalar response and one or more explanatory variables (also known as dependent and independent variables). The case of one explanatory variable is ca ...
, a statistical estimation method * List of linear algebra topics *
Multilinear algebra Multilinear algebra is a subfield of mathematics that extends the methods of linear algebra. Just as linear algebra is built on the concept of a vector and develops the theory of vector spaces, multilinear algebra builds on the concepts of ''p' ...
*
Numerical linear algebra Numerical linear algebra, sometimes called applied linear algebra, is the study of how matrix operations can be used to create computer algorithms which efficiently and accurately provide approximate answers to questions in continuous mathematic ...
*
Transformation matrix In linear algebra, linear transformations can be represented by matrices. If T is a linear transformation mapping \mathbb^n to \mathbb^m and \mathbf x is a column vector with n entries, then T( \mathbf x ) = A \mathbf x for some m \times n matri ...

General and cited sources

* * * * * * * * *

History

* Fearnley-Sander, Desmond,
Hermann Grassmann and the Creation of Linear Algebra
, American Mathematical Monthly 86 (1979), pp. 809–817. *

Introductory textbooks

* * * * * * * * * Murty, Katta G. (2014)
Computational and Algorithmic Linear Algebra and n-Dimensional Geometry
', World Scientific Publishing, .
Chapter 1: Systems of Simultaneous Linear Equations
' * Noble, B. & Daniel, J.W. (2nd Ed. 1977)

', Pearson Higher Education, . * * * * * The Manga Guide to Linear Algebra (2012), by Shin Takahashi, Iroha Inoue and Trend-Pro Co., Ltd.,

* * * * * * * * * * * * * * * * * * * * * * * * *

* * * * *

Online Resources

MIT Linear Algebra Video Lectures
a series of 34 recorded lectures by Professor Gilbert Strang (Spring 2010)
International Linear Algebra Society
*

on
MathWorld ''MathWorld'' is an online mathematics reference work, created and largely written by Eric W. Weisstein. It is sponsored by and licensed to Wolfram Research, Inc. and was partially funded by the National Science Foundation's National Science ...

Matrix and Linear Algebra Terms
o

o

Essence of linear algebra
a video presentation from 3Blue1Brown of the basics of linear algebra, with emphasis on the relationship between the geometric, the matrix and the abstract points of view

Online books

* * * * * * * Sharipov, Ruslan,
Course of linear algebra and multidimensional geometry
' * Treil, Sergei,

' {{DEFAULTSORT:Linear Algebra Numerical analysis