An in-memory database (IMDb, or main memory database system (MMDB) or memory resident database) is a
database management system
In computing, a database is an organized collection of data or a type of data store based on the use of a database management system (DBMS), the software that interacts with end users, applications, and the database itself to capture and an ...
that primarily relies on
main memory
Computer data storage or digital data storage is a technology consisting of computer components and recording media that are used to retain digital data. It is a core function and fundamental component of computers.
The central processin ...
for
computer data storage
Computer data storage or digital data storage is a technology consisting of computer components and Data storage, recording media that are used to retain digital data. It is a core function and fundamental component of computers.
The cent ...
. It is contrasted with database management systems that employ a
disk storage
Disc or disk may refer to:
* Disk (mathematics)
In geometry, a disk (Spelling of disc, also spelled disc) is the region in a plane (geometry), plane bounded by a circle. A disk is said to be ''closed'' if it contains the circle that constitut ...
mechanism. In-memory databases are faster than disk-optimized databases because disk access is slower than memory access and the internal optimization algorithms are simpler and execute fewer CPU instructions. Accessing data in memory eliminates
seek time
Higher performance in hard disk drives comes from devices which have better performance characteristics. These performance characteristics can be grouped into two categories: #Access time, access time and #Data transfer rate, data transfer time (o ...
when querying the data, which provides faster and more predictable performance than disk.
Applications where response time is critical, such as those running telecommunications network equipment and
mobile advertising
Mobile advertising is a form of advertising via mobile (wireless) phones or other mobile devices. It is a subset of mobile marketing, mobile advertising can take place as text ads via SMS, or banner advertisements that appear embedded in a ...
networks, often use main-memory databases. IMDBs have gained much traction, especially in the
data analytics
Analytics is the systematic computational analysis of data or statistics. It is used for the discovery, interpretation, and communication of meaningful patterns in data, which also falls under and directly relates to the umbrella term, data sci ...
space, starting in the
mid-2000s – mainly due to multi-core processors that can address large memory and due to less expensive
RAM
Ram, ram, or RAM most commonly refers to:
* A male sheep
* Random-access memory, computer memory
* Ram Trucks, US, since 2009
** List of vehicles named Dodge Ram, trucks and vans
** Ram Pickup, produced by Ram Trucks
Ram, ram, or RAM may also ref ...
.
A potential technical hurdle with in-memory data storage is the volatility of RAM. Specifically in the event of a power loss, intentional or otherwise, data stored in
volatile RAM is lost.
With the introduction of
non-volatile random-access memory
Non-volatile random-access memory (NVRAM) is random-access memory that retains data without applied power. This is in contrast to dynamic random-access memory (DRAM) and static random-access memory (SRAM), which both maintain data only for as l ...
technology, in-memory databases will be able to run at full speed and maintain data in the event of power failure.
ACID support
In its simplest form, main memory databases store data on
volatile memory
Volatile memory, in contrast to non-volatile memory, is computer memory that requires power to maintain the stored information; it retains its contents while powered on but when the power is interrupted, the stored data is quickly lost.
Volatile ...
devices. These devices lose all stored information when the device loses power or is reset. In this case, IMDBs can be said to lack support for the "durability" portion of the
ACID
An acid is a molecule or ion capable of either donating a proton (i.e. Hydron, hydrogen cation, H+), known as a Brønsted–Lowry acid–base theory, Brønsted–Lowry acid, or forming a covalent bond with an electron pair, known as a Lewis ...
(atomicity, consistency, isolation, durability) properties. Volatile memory-based IMDBs can, and often do, support the other three ACID properties of atomicity, consistency and isolation.
Many IMDBs have added durability via the following mechanisms:
*
Snapshot files, or,
checkpoint images, which record the state of the database at a given moment in time. The system typically generates these periodically, or at least when the IMDb does a controlled shut-down. While they give a measure of persistence to the data (in that the database does not lose everything in the case of a system crash) they only offer partial durability (as 'recent" changes will be lost). For full durability, they need supplementing with one of the following:
*
Transaction log
In the field of databases in computer science, a transaction log (also transaction journal, database log, binary log or audit trail) is a history of actions executed by a database management system used to guarantee ACID properties over crashes ...
ging, which records changes to the database in a journal file and facilitates automatic recovery of an in-memory database.
* Non-Volatile DIMM (
NVDIMM), a memory module that has a DRAM interface, often combined with NAND flash for the Non-Volatile data security. The first NVDIMM solutions were designed with
supercapacitors instead of batteries for the backup power source. With this storage, IMDb can resume securely from its state upon reboot.
*
Non-volatile random-access memory
Non-volatile random-access memory (NVRAM) is random-access memory that retains data without applied power. This is in contrast to dynamic random-access memory (DRAM) and static random-access memory (SRAM), which both maintain data only for as l ...
(NVRAM), usually in the form of static RAM backed up with battery power (battery RAM), or an electrically erasable programmable ROM (
EEPROM
EEPROM or E2PROM (electrically erasable programmable read-only memory) is a type of non-volatile memory. It is used in computers, usually integrated in microcontrollers such as smart cards and remote keyless systems, or as a separate chip d ...
). With this storage, the re-booting IMDb system can recover the data store from its last consistent state.
*
High availability
High availability (HA) is a characteristic of a system that aims to ensure an agreed level of operational performance, usually uptime, for a higher than normal period.
There is now more dependence on these systems as a result of modernization ...
implementations that rely on database
replication, with automatic
failover
Failover is switching to a redundant or standby computer server, system, hardware component or network upon the failure or abnormal termination of the previously active application, server, system, hardware component, or network in a computer ...
to an identical standby database in the event of primary database failure. To protect against loss of data in the case of a complete system crash, replication of an IMDb is normally used in addition to one or more of the mechanisms listed above.
Some IMDBs allow the database schema to specify different durability requirements for selected areas of the database thus, faster-changing data that can easily be regenerated or that has no meaning after a system shut-down would not need to be journaled for durability (though it would have to be replicated for high availability), whereas configuration information would be flagged as needing preservation.
Hybrids with on-disk databases
While storing data in-memory confers performance advantages, it is an expensive method of data storage. An approach to realising the benefits of in-memory storage while limiting its costs is to store the most frequently accessed data in-memory and the rest on disk. Since there is no hard distinction between which data should be stored in-memory and which should be stored on disk, some systems dynamically update where data is stored based on the data's usage.
This approach is subtly different from
caching, in which the most ''recently accessed'' data is cached, as opposed to the most ''frequently accessed'' data being stored in-memory.
The flexibility of hybrid approaches allow a balance to be struck between:
* performance (which is enhanced by sorting, storing and retrieving specified data entirely in memory, rather than going to disk)
* cost, because a less costly hard disk can be substituted for more memory
*
persistence
Persistence or Persist may refer to:
Math and computers
* Image persistence, in LCD monitors
* Persistence (computer science), the characteristic of data that outlives the execution of the program that created it
* Persistence of a number, a ma ...
* form factor, because RAM chips cannot approach the
density
Density (volumetric mass density or specific mass) is the ratio of a substance's mass to its volume. The symbol most often used for density is ''ρ'' (the lower case Greek letter rho), although the Latin letter ''D'' (or ''d'') can also be u ...
of a small hard drive
In the
cloud computing
Cloud computing is "a paradigm for enabling network access to a scalable and elastic pool of shareable physical or virtual resources with self-service provisioning and administration on-demand," according to International Organization for ...
industry the terms "data temperature", or "hot data" and "cold data" have emerged to describe how data is stored in this respect.
Hot data is used to describe mission-critical data that needs to be accessed frequently while cold data describes data that is needed less often and less urgently, such as data kept for archiving or auditing purposes. Hot data should be stored in ways offering fast retrieval and modification, often accomplished by in-memory storage but not always. Cold data on the other hand can be stored in a more cost-effective way and is accepted that data access will likely be slower compared to hot data. While these descriptions are useful, "hot" and "cold" lack concrete definitions.
[
Manufacturing efficiency provides another reason for selecting a combined in-memory/on-disk database system. Some device product lines, especially in ]consumer electronics
Consumer electronics, also known as home electronics, are electronic devices intended for everyday household use. Consumer electronics include those used for entertainment, Communication, communications, and recreation. Historically, these prod ...
, include some units with permanent storage, and others that rely on memory for storage (set-top box
A set-top box (STB), also known as a cable converter box, cable box, receiver, or simply box, and historically television decoder or a converter, is an information appliance device that generally contains a Tuner (radio)#Television, TV tuner inpu ...
es, for example). If such devices require a database system, a manufacturer can adopt a hybrid database system at lower and ''upper'' cost, and with less customization of code, rather than using separate in-memory and on-disk databases, respectively, for its disk-less and disk-based products.
The first database engine
A database engine (or storage engine) is the underlying software component that a database management system (DBMS) uses to create, read, update and delete (CRUD) data from a database. Most database management systems include their own application ...
to support both in-memory and on-disk tables in a single database, WebDNA, was released in 1995.
Storage memory
Another variation involves large amounts of nonvolatile memory in the server, for example, flash memory chips as addressable memory rather than structured as disk arrays. A database in this form of memory combines very fast access speed with persistence over reboots and power losses.
Notable In-memory Databases
* SAP HANA: This is a column-orientated in-memory database that stores data in its memory instead of keeping it on a disk. It claims to store data in columnar fashion in main memory and supports both online analytical processing
In computing, online analytical processing (OLAP) (), is an approach to quickly answer multi-dimensional analytical (MDA) queries. The term ''OLAP'' was created as a slight modification of the traditional database term online transaction proces ...
(OLAP) and online transactional processing (OLTP) in the same system.
* Oracle TimesTen: This is an In-Memory Database which is memory-optimized, relational database that claims to deliver microsecond response and extremely high throughput performance.
See also
* Comparison of relational database management systems
The following tables compare general and technical information for a number of relational database management systems. Please see the individual products' articles for further information. Unless otherwise specified in footnotes, comparisons are ba ...
* Distributed data store
A distributed data store is a computer network where information is stored on more than one node, often in a replicated fashion. It is usually specifically used to refer to either a distributed database where users store information on a ''numb ...
* Embedded database
An embedded database system is a database management system (DBMS) which is tightly integrated with an application software; it is embedded in the application (instead of coming as a standalone application). It is a broad technology category that ...
s
* In-memory processing
* List of in-memory databases
Notable in-memory database system software includes:
References
{{cite web , title=Stackshare Top In-Memory Databases , url=https://stackshare.io/in-memory-databases#:~:text=Redis%2C%20Hazelcast%2C%20Aerospike%2C%20SAP,reason%20why%20Hazelcast% ...
* NoSQL
NoSQL (originally meaning "Not only SQL" or "non-relational") refers to a type of database design that stores and retrieves data differently from the traditional table-based structure of relational databases. Unlike relational databases, which ...
* NewSQL
NewSQL is a class of relational database management system, relational database management systems that seek to provide the scalability of NoSQL systems for online transaction processing (OLTP) workloads while maintaining the ACID guarantees of a t ...
Notes
References
* {{cite book, author=Jack Belzer, title=''Encyclopedia of Computer Science and Technology'' - Volume 14: Very Large Data Base Systems to Zero-Memory and Markov Information Source, date=April 1980 , publisher=Marcel Dekker Inc, isbn=978-0-8247-2214-2
Computer memory
Database management systems
In-memory database management systems