Expanded Storage

z/Architecture, initially and briefly called ESA Modal Extensions (ESAME), is IBM's 64-bit complex instruction set computer (CISC) instruction set architecture, implemented by its mainframe computers. IBM introduced its first z/Architecture-based system, the z900, in late 2000. Subsequent z/Architecture systems include the IBM z800, z990, z890, System z9, System z10, zEnterprise 196, zEnterprise 114, zEC12, zBC12, z13, z14, z15, z16, and z17.

z/Architecture retains backward compatibility with previous 32-bit-data/31-bit-addressing architecture ESA/390 and its predecessors back to the 32-bit-data/24-bit-addressing System/360. The IBM z13 is the last z Systems server to support running an operating system in ESA/390 architecture mode. However, all 24-bit and 31-bit problem-state application programs originally written to run on the ESA/390 architecture will be unaffected by this change.

Features

z/Architecture includes almost all of the features of ESA/390, and adds some new features. Among the features of z/Architecture are
:A channel subsystem with the architecture introduced by S/370-XA
:Branch relative instructions introduced by ESA/390
:Trimodal (24/31/64-bit) addresses
:16 32-bit access registers (ARs) introduced by ESA/370
:16 64-bit general registers (GRs), 32-bit on older architectures
:16 64-bit control registers (CRs) introduced by System/370 as 32-bit
:16 64-bit floating-point registers (FPRs)
:32 128-bit vector registers (VRs); bits 0–63 of VR0–VR15 contain FPR0–FPR15
:1 32-bit floating-point control (FPC) register
:1 128-bit program-status word (PSW), which includes a 64-bit instruction address
:An 8-KiB prefix storage area (PSA)
:Cryptographic Facility
: IEEE Binary-floating-point instructions added by ESA/390
: IEEE Decimal-floating-point instructions
For information on when each feature was introduced, consult the Principles of Operation.

Vector facility

The z13 introduced the Vector Facility for z/Architecture. It adds 32 vector registers, each 128 bits wide; the existing 16 floating-point registers are overlaid on the new vector registers. The new architecture adds over 150 new instructions to operate on data in vector registers, including integer, floating-point, and string data types. The z13 implementation includes two independent SIMD units to operate on vector data.

Neural-network-processing-assist facility

The z16 introduced the Neural-network-processing-assist facility, which introduces several instructions performing operations on model-dependent data types. For the z16 this is the 16-bit NNP-Data-Type-1 Format.

The new instructions include tensor operations useful for AI and neural network applications.

Registers

Each processor has these registers

* Access registers
* Breaking-event-address register (BEAR)
* Control registers
* Floating-point Control (FPC) register
* Floating-point registers
* General registers
* Prefix register
* Program status word (PSW)
* Vector registers

Access registers

Each CPU has 16 32-bit access registers. When a program running in AR mode specifies register 1–15 as a base register or as a register operand containing an address, the CPU uses the associated access register during address translation.

Breaking-event-address register (BEAR)

The 64-bit BEAR contains the address of the last instruction that broke the sequential execution of instructions; an interrupt stores the BEAR in the doubleword at real address 272 (). After an Execute of a branch, the BEAR contains the address of the execute, not that of the branch.

Control registers

The 16 64-bit control registers provide controls over and the status of a CPU, except for information included in the PSW. They are an evolutionary enhancement to the control registers on the earlier ESA/390 on the IBM S/390 processors. For details on which fields are dependent on specific features, consult the Principles of Operation.
Because z/Architecture expands the control registers from 32 bits to 64, the bit numbering differs from that in ESA/390.

Floating-point Control (FPC) register

The FPC register contains Interrupt Masks (IM), Status Flags (SF), Data Exception Code (DXC), Decimal Rounding Mode (DRM) and Binary Rounding Mode (BRM). An interruption only stores the DXC if the FPC register if the AFP-register (additional floating-point register) control bit, bit 13 of control register 0, is one. Also, while individual bits of the DXC usually have significance, programs should normally treat it as an 8-bit integer rather than querying individual bits.

Floating-point registers

Each CPU had 16 64-bit floating-point registers; FP0–15 occupy bits 0–63 of VR0–15.

General registers

Each CPU has 16 64-bit general registers, which serve as accumulators, base registers and index registers. Instructions designated as ''Grandé'' operate on all 64 bits; some instructions added by the Extended-Immediate Facility operate on any halfword or word in the register; most other instructions do not change or use bits 0–31.

Prefix register

The prefix register is used in translating a real address to an absolute address. In z/Architecture mode, the PSA is 2 pages (8 KiB). Bits 0–32 and 51–63 are always zero. If bits 0–50 of a real address are zero then they are replaced by bits 0–50 of the prefix register; if bits 0–50 of the real address are equal to bits 0–50 of the prefix register then they are replaced with zeros.

Program status word (PSW)

The PSW holds the instruction address and other fields reflecting the status of the program currently running on a CPU. The status of the program is also affected by the contents of the Control registers.

Vector registers

Each CPU has 32 128-bit vector registers; bits 0–63 of VR0–15 are also FPR0–15. A vector register may contain 16 8-bit fields, 8 16-bit fields, 4 32-bit fields, 2 64-bit fields or 1 128-bit field.

Memory

IBM classifies memory in z/Architecture into Main Storage and Expanded Storage.

Main storage is addressed in 8-bit bytes (octets), with larger aligned groupings:
;Halfword
:Two bytes
:16 bits
;Word
:Four bytes
:32 bits
;Doubleword
:8 bytes
:64 bits
;Quadword
:16 bytes
:128 bits
;Page
:4096 bytes

Although z/Architecture allows real and virtual addresses from 0 to 2⁶⁴-1, engineering constraints limit current and planned models to far less.

Expanded storage is address in 4 KiB blocks, with block numbers ranging fom 0 to 2³².

Addressing

Types of main storage addresses

There are three types of main storage addresses in z/Architecture
;Virtual address
:The address as seen by application programs. It is an offset into an address space and is subject to address translation via page and segment tables.
;Real address
:The address after address translation, or the address seen by an OS component running with translation off. It is subject to prefixing.
;Absolute address
:The address after prefixing references to the first two pages via the prefix register.

Address encoding

z/Architecture uses the same truncated addressing as ESA, with some additional instruction formats. As with ESA, in AR mode each nonzero base register is associated with a base register specifying the address space. Depending on the instruction, an address may be provided in several different formats.
;R
:The address is contained in a general register
;Relative
:A signed 16-bit halfword offset from the current instruction.
;Relative long
:A signed 32-bit halfword offset from the current instruction.
;RS
:A base register and a 12-bit displacement
;RX
:A base register, an index register, and a 12-bit displacement
;Y
:A base register, an index register, and a 20-bit displacement; colloquially known as "Yonder".

Addressing modes

In addition to the two addressing modes supported by S/370-XA and ESA, a/Architecture has an extended addressing mode with 64-bit virtual addresses. The addressing mode is controlled by the EA (bit 31) and BA (bit 32) bits in the PSW. The valid combinations are
* 00 24-bit addressing
* 01 31-bit addressing
* 11 64-bit addressing

Translation modes

z/Architecture supports four virtual ''translation modes'', controlled by bit 5, the DAT-mode bit, and bits 16–17, the Address-Space Control (AS) bits, of the PSW.

;Primary-space mode
:All storage references use the translation tables for the primary address space
;Access-register mode
:All storage references use the translation tables designated by the access register associated with the base register.
;Secondary-space mode
:All storage references use the translation tables for the secondary address space
;Home-space mode
:All storage references use the translation tables for the home address space

Operating system support

IBM's operating systems z/OS, z/VSE, z/TPF, and z/VM are versions of MVS, VSE, Transaction Processing Facility (TPF), and VM that support z/Architecture. Older versions of z/OS, z/VSE, and z/VM continued to support 32-bit systems; z/OS version 1.6 and later, z/VSE Version 4 and later, and z/VM Version 5 and later require z/Architecture.

Linux also supports z/Architecture with Linux on IBM Z.

z/Architecture supports running multiple concurrent operating systems and applications even if they use different address sizes. This allows software developers to choose the address size that is most advantageous for their applications and data structures.

On July 7, 2009, IBM on occasion of announcing a new version of one of its operating systems implicitly stated that Architecture Level Set 4 (ALS 4) exists, and is implemented on the System z10 and subsequent machines. The ALS 4 is also specified in LOADxx as ARCHLVL 3, whereas the earlier z900, z800, z990, z890, System z9 specified ARCHLVL 2. Earlier announcements of System z10 simply specified that it implements z/Architecture with some additions: 50+ new machine instructions, 1 MB page frames, and hardware decimal floating-point unit (HDFU).

Most operating systems for the z/Architecture, including z/OS, generally restrict code execution to the first 2 GB (31 address bits, or 2³¹ addressable bytes) of each virtual address space for reasons of efficiency and compatibility rather than because of architectural limits. Linux on IBM Z allows code to execute within 64-bit address ranges.

z/OS

Each z/OS address space, called a 64-bit address space, is 16 exabytes in size.

Code (or mixed) spaces

The z/OS implementation of the Java programming language is an exception. The z/OS virtual memory implementation supports multiple 2 GB address spaces, permitting more than 2 GB of concurrently resident program code.

Data-only spaces

Data-only spaces are memory regions that can be read from and written to, but not used as executable code. (Similar to the NX bit on other modern processors.)
By default, the z/Architecture memory space is indexed by 64-bit pointers, allowing up to 16 exabytes of memory to be visible to an executing program.

=Dataspaces and hiperspaces

=
Applications that need more than a 16 exabyte data address space can employ extended addressability techniques, using additional address spaces or data-only spaces. The data-only spaces that are available for user programs are called:
* dataspaces (sometimes referred to as "data spaces") and
* hiperspaces (High performance space).

These spaces are similar in that both are areas of virtual storage that a program can create, and can be up to 2 gigabytes. Unlike an address space, a dataspace or hiperspace contains only user data; it does not contain system control blocks or common areas. Program code cannot run in a dataspace or a hiperspace.

A dataspace differs from a hiperspace in that dataspaces are byte-addressable, whereas hiperspaces are page-addressable.

IBM mainframe expanded storage

Traditionally IBM Mainframe memory has been byte-addressable. This kind of memory is termed "Central Storage". IBM Mainframe processors through much of the 1980s and 1990s supported another kind of memory: Expanded Storage. It was first introduced with the IBM 3090 high-end mainframe series in 1985.

Expanded Storage is 4KB-page addressable. When an application wants to access data in Expanded Storage it must first be moved into Central Storage. Similarly, data movement from Central Storage to Expanded Storage is done in multiples of 4 KB pages. Initially page movement was performed using relatively expensive instructions, by paging subsystem code.

The overhead of moving single and groups of pages between Central and Expanded Storage was reduced with the introduction
of the MVPG (Move Page) instruction and the ADMF (Asynchronous Data Mover Facility) capability.

The MVPG instruction and ADMF are explicitly invoked—generally by middleware in z/OS or z/VM (and ACP?)—to access data in expanded storage. Some uses are namely:
* MVPG is used by VSAM Local Shared Resources (LSR) buffer pool management to access buffers in a hiperspace in Expanded Storage.
* Both MVPG and ADMF are used by IBM Db2 to access hiperpools. Hiperpools are portions of a buffer pool located in a hiperspace.
* VM Minidisk Caching.

Until the mid-1990s Central and Expanded Storage were physically different areas of memory on the processor. Since the mid-1990s Central and Expanded Storage were merely assignment choices for the underlying processor memory.
These choices were made based on specific expected uses:
For example, Expanded Storage is required for the Hiperbatch function (which uses the MVPG instruction to access its hiperspaces).

In addition to the hiperspace and paging cases mentioned above there are other uses of expanded storage, including:
* Virtual I/O (VIO) to Expanded Storage which stored temporary data sets in simulated devices in Expanded Storage. (This function has been replaced by VIO in Central Storage.)
* VM Minidisk Caching.

z/OS removed the support for Expanded Storage. All memory in z/OS is now Central Storage. z/VM 6.4 fulfills Statement of Direction to drop support for all use of Expanded Storage.

MVPG and ADMF

MVPG

IBM described MVPG as "moves a single page and the central processor cannot execute any other instructions until the page move is completed."

The MVPG mainframe instruction (MoVe PaGe, opcode X'B254') has been compared to the MVCL (MoVe Character Long) instruction, both of which can move more than 256 bytes within main memory using a single instruction. These instructions do not comply with definitions for atomicity, although they can be used as a single instruction within documented timing and non-overlap restrictions.

The need to move more than 256 bytes within main memory had historically been addressed with software (MVC loops), MVCL, which was introduced with the 1970 announcement of the System/370, and MVPG, patented and announced by IBM in 1989, each have advantages.

ADMF

ADMF (Asynchronous Data Mover Facility), which was introduced in 1992, goes beyond the capabilities of the MVPG (Move Page) instruction, which is limited to a single page, and can move groups of pages between Central and Expanded Storage.

A macro instruction named IOSADMF, which has been described as an API that avoids "direct, low-level use of ADMF", can be used to read or write data to or from a hiperspace. Hiperspaces are created using DSPSERV CREATE.

To provide reentrancy, IOSADMF is used together with a "List form" and "Execute form."

Non-IBM implementations

Platform Solutions Inc. (PSI) previously marketed Itanium-based servers which were compatible with z/Architecture. IBM bought PSI in July 2008, and the PSI systems are no longer available. FLEX-ES, zPDT and the Hercules emulator also implement z/Architecture. Hitachi mainframes running newer releases of the VOS3 operating system implement ESA/390 plus Hitachi-unique CPU instructions, including a few 64-bit instructions. While Hitachi formally collaborated with IBM on the z900-G2/z800 CPUs introduced in 2002, Hitachi's machines are not z/Architecture-compatible.

Notes

References

;z
:

Further reading

Preshing on Programming – Atomic vs. Non-Atomic Operations

Principles of Computer Design – Atomicity

{{DEFAULTSORT:Z Architecture
IBM mainframe technology
Instruction set architectures
Computer-related introductions in 2000
mainframe expanded storage
64-bit computers