|
Automatic Basis Function Construction
In machine learning, automatic basis function construction, also known as basis discovery, is a technique that finds a set of general-purpose (task-independent) basis functions to simplify complex data (state space) into a smaller, manageable form (lower-dimensional embedding) while accurately capturing the value function. Unlike methods relying on expert-designed functions, this approach works without prior knowledge of the specific problem area (domain), making it effective in situations where creating tailored basis functions is challenging or impractical. Motivation In reinforcement learning (RL), many real-world problems modeled as Markov Decision Processes (MDPs) involve large or continuous state spaces—sets of all possible situations an agent might encounter—which are too complex to handle directly and often need approximation for efficient computation. o reference provided in original/ref> Linear function approximatorsKeller, Philipp; Mannor, Shie; Precup, Doina. (2 ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Machine Learning
Machine learning (ML) is a field of inquiry devoted to understanding and building methods that 'learn', that is, methods that leverage data to improve performance on some set of tasks. It is seen as a part of artificial intelligence. Machine learning algorithms build a model based on sample data, known as training data, in order to make predictions or decisions without being explicitly programmed to do so. Machine learning algorithms are used in a wide variety of applications, such as in medicine, email filtering, speech recognition, agriculture, and computer vision, where it is difficult or unfeasible to develop conventional algorithms to perform the needed tasks.Hu, J.; Niu, H.; Carrasco, J.; Lennox, B.; Arvin, F.,Voronoi-Based Multi-Robot Autonomous Exploration in Unknown Environments via Deep Reinforcement Learning IEEE Transactions on Vehicular Technology, 2020. A subset of machine learning is closely related to computational statistics, which focuses on making pred ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Random Walk
In mathematics, a random walk is a random process that describes a path that consists of a succession of random steps on some mathematical space. An elementary example of a random walk is the random walk on the integer number line \mathbb Z which starts at 0, and at each step moves +1 or −1 with equal probability. Other examples include the path traced by a molecule as it travels in a liquid or a gas (see Brownian motion), the search path of a foraging animal, or the price of a fluctuating stock and the financial status of a gambler. Random walks have applications to engineering and many scientific fields including ecology, psychology, computer science, physics, chemistry, biology, economics, and sociology. The term ''random walk'' was first introduced by Karl Pearson in 1905. Lattice random walk A popular random walk model is that of a random walk on a regular lattice, where at each step the location jumps to another site according to some probability distribution. ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Optimal Control
Optimal control theory is a branch of mathematical optimization that deals with finding a control for a dynamical system over a period of time such that an objective function is optimized. It has numerous applications in science, engineering and operations research. For example, the dynamical system might be a spacecraft with controls corresponding to rocket thrusters, and the objective might be to reach the moon with minimum fuel expenditure. Or the dynamical system could be a nation's economy, with the objective to minimize unemployment; the controls in this case could be fiscal and monetary policy. A dynamical system may also be introduced to embed operations research problems within the framework of optimal control theory. Optimal control is an extension of the calculus of variations, and is a mathematical optimization method for deriving control policies. The method is largely due to the work of Lev Pontryagin and Richard Bellman in the 1950s, after contributions to ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Bellman Equation
A Bellman equation, named after Richard E. Bellman, is a necessary condition for optimality associated with the mathematical optimization method known as dynamic programming. It writes the "value" of a decision problem at a certain point in time in terms of the payoff from some initial choices and the "value" of the remaining decision problem that results from those initial choices. This breaks a dynamic optimization problem into a sequence of simpler subproblems, as Bellman's “principle of optimality" prescribes. The equation applies to algebraic structures with a total ordering; for algebraic structures with a partial ordering, the generic Bellman's equation can be used. The Bellman equation was first applied to engineering control theory and to other topics in applied mathematics, and subsequently became an important tool in economic theory; though the basic concepts of dynamic programming are prefigured in John von Neumann and Oskar Morgenstern's ''Theory of Games and Econ ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Dynamic Programming
Dynamic programming is both a mathematical optimization method and a computer programming method. The method was developed by Richard Bellman in the 1950s and has found applications in numerous fields, from aerospace engineering to economics. In both contexts it refers to simplifying a complicated problem by breaking it down into simpler sub-problems in a recursive manner. While some decision problems cannot be taken apart this way, decisions that span several points in time do often break apart recursively. Likewise, in computer science, if a problem can be solved optimally by breaking it into sub-problems and then recursively finding the optimal solutions to the sub-problems, then it is said to have '' optimal substructure''. If sub-problems can be nested recursively inside larger problems, so that dynamic programming methods are applicable, then there is a relation between the value of the larger problem and the values of the sub-problems.Cormen, T. H.; Leiserson, C. E.; Riv ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Markov Chain
A Markov chain or Markov process is a stochastic model describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event. Informally, this may be thought of as, "What happens next depends only on the state of affairs ''now''." A countably infinite sequence, in which the chain moves state at discrete time steps, gives a discrete-time Markov chain (DTMC). A continuous-time process is called a continuous-time Markov chain (CTMC). It is named after the Russian mathematician Andrey Markov. Markov chains have many applications as statistical models of real-world processes, such as studying cruise control systems in motor vehicles, queues or lines of customers arriving at an airport, currency exchange rates and animal population dynamics. Markov processes are the basis for general stochastic simulation methods known as Markov chain Monte Carlo, which are used for simulating sampling from complex probability ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Function Space
In mathematics, a function space is a set of functions between two fixed sets. Often, the domain and/or codomain will have additional structure which is inherited by the function space. For example, the set of functions from any set into a vector space has a natural vector space structure given by pointwise addition and scalar multiplication. In other scenarios, the function space might inherit a topological or metric structure, hence the name function ''space''. In linear algebra Let be a vector space over a field and let be any set. The functions → can be given the structure of a vector space over where the operations are defined pointwise, that is, for any , : → , any in , and any in , define \begin (f+g)(x) &= f(x)+g(x) \\ (c\cdot f)(x) &= c\cdot f(x) \end When the domain has additional structure, one might consider instead the subset (or subspace) of all such functions which respect that structure. For example, if is also a vector space over , ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Eigenvalues And Eigenvectors
In linear algebra, an eigenvector () or characteristic vector of a linear transformation is a nonzero vector that changes at most by a scalar factor when that linear transformation is applied to it. The corresponding eigenvalue, often denoted by \lambda, is the factor by which the eigenvector is scaled. Geometrically, an eigenvector, corresponding to a real nonzero eigenvalue, points in a direction in which it is stretched by the transformation and the eigenvalue is the factor by which it is stretched. If the eigenvalue is negative, the direction is reversed. Loosely speaking, in a multidimensional vector space, the eigenvector is not rotated. Formal definition If is a linear transformation from a vector space over a field into itself and is a nonzero vector in , then is an eigenvector of if is a scalar multiple of . This can be written as T(\mathbf) = \lambda \mathbf, where is a scalar in , known as the eigenvalue, characteristic value, or characteristic root ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Ilse Ipsen
Ilse Clara Franziska Ipsen is a German-American mathematician who works as a professor of mathematics at North Carolina State University. She was formerly associate director of the Statistical and Applied Mathematical Sciences Institute, a joint venture of North Carolina State and other nearby universities.. Education and career Ipsen earned a diploma from the Kaiserslautern University of Technology in 1977, and completed her doctorate from the Pennsylvania State University in 1983 under the supervision of Don Heller. Her dissertation was ''Systolic Arrays for VLSI'' and concerned Very Large Scale Integration hardware implementations of the systolic array parallel computing architecture. After working at Yale University for ten years beginning in 1983, she joined North Carolina State University in 1993. Ipsen is Founding Editor in Chief of the SIAM Book Series on Data Science. Book Ipsen is the author of the book ''Numerical Matrix Analysis: Linear Systems and Least Squares'' (SI ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Neumann Series
A Neumann series is a mathematical series of the form : \sum_^\infty T^k where T is an operator and T^k := T^\circ its k times repeated application. This generalizes the geometric series. The series is named after the mathematician Carl Neumann, who used it in 1877 in the context of potential theory. The Neumann series is used in functional analysis. It forms the basis of the Liouville-Neumann series, which is used to solve Fredholm integral equations. It is also important when studying the spectrum of bounded operators. Properties Suppose that T is a bounded linear operator on the normed vector space X. If the Neumann series converges in the operator norm, then \text-T is invertible and its inverse is the series: : (\mathrm - T)^ = \sum_^\infty T^k , where \mathrm is the identity operator in X. To see why, consider the partial sums :S_n := \sum_^n T^k. Then we have :\lim_(\mathrm-T)S_n = \lim_\left(\sum_^n T^k - \sum_^n T^\right) = \lim_\left(\mathrm - T^\right) = \mathr ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Diffusion Wavelets
Diffusion wavelets are a fast multiscale framework for the analysis of functions on discrete (or discretized continuous) structures like graphs, manifolds, and point clouds in Euclidean space. Diffusion wavelets are an extension of classical wavelet theory from harmonic analysis. Unlike classical wavelets whose basis functions are predetermined, diffusion wavelets are adapted to the geometry of a given diffusion operator T (e.g., a heat kernel or a random walk). Moreover, the diffusion wavelet basis functions are constructed by dilation using the dyadic powers (powers of two) of T. These dyadic powers of T diffusion over the space and propagate local relationships in the function throughout the space until they become global. And if the rank of higher powers of T decrease (i.e., its spectrum decays), then these higher powers become compressible. From these decaying dyadic powers of T comes a chain of decreasing subspaces. These subspaces are the scaling function approximation subspac ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
Function Approximation
In general, a function approximation problem asks us to select a function among a that closely matches ("approximates") a in a task-specific way. The need for function approximations arises in many branches of applied mathematics, and computer science in particular , such as predicting the growth of microbes in microbiology. Function approximations are used where theoretical models are unavailable or hard to compute. One can distinguish two major classes of function approximation problems: First, for known target functions approximation theory is the branch of numerical analysis that investigates how certain known functions (for example, special functions) can be approximated by a specific class of functions (for example, polynomials or rational functions) that often have desirable properties (inexpensive computation, continuity, integral and limit values, etc.). Second, the target function, call it ''g'', may be unknown; instead of an explicit formula, only a set of point ... [...More Info...] [...Related Items...] OR: [Wikipedia] [Google] [Baidu] |
