Proxmox Virtual Environment

Proxmox Virtual Environment (Proxmox VE or PVE) is an open-source software server for virtualization management. It is a hosted hypervisor that can run operating systems including Linux and Windows on x64 hardware. It is a Debian-based Linux distribution with a modified Ubuntu LTS kernel and allows deployment and management of virtual machines and containers. Proxmox VE includes a web console and command-line tools, and provides a REST API for third-party tools. Two types of virtualization are supported: container-based with LXC (starting from version 4.0 replacing OpenVZ used in version up to 3.4, included), and full virtualization with KVM.
It includes a web-based management interface.

Proxmox VE is licensed under the GNU Affero General Public License, version 3.

History

Development of Proxmox VE started when Dietmar Maurer and Martin Maurer, two Linux developers, found out OpenVZ had no backup tool and no management GUI. KVM was appearing at the same time in Linux, and was added shortly afterwards.

The first public release took place in April 2008. It supported container and full virtualization, managed with a web-based user interface similar to other commercial offerings.

Features

Proxmox VE is an open-source server virtualization platform to manage two virtualization technologies: Kernel-based Virtual Machine (KVM) for virtual machines and LXC for containers - with a single web-based interface.
The source is open, based on the GNU AGPL, v3. The company sells optional subscription-based customer support. With a subscription, users get access to an enterprise software repository.

VM emulated/paravirtualized hardware

Since Proxmox VE for Virtual Machines are using KVM (+QEMU) as hypervisor, it natively (from the GUI) supports many emulated and paravirtualized hardware components such as:

* Motherboard chipset: i440fx or q35 (many versions)
* BIOS: SeaBIOS or OVMF (UEFI)
* Processor types: host, kvm32, kvm64, qemu32, qemu64 or specific CPU generation (features/flags) types like: athlon, opteron, phenom, broadwell, coreduo
* Disk controllers: LSI53C895A, LSI53C810, MegaRAID SAS 8708EI, VirtIO SCSI (paravirtualized), VirtIO SCSI Single (paravirtualized), VMware PVSCSI
* Display: Standard VGA, VMware compatible, SPICE, Serial, VirtIO GPU (paravirtualized), VirGL GPU (paravirtualized)
* Network: Intel E1000, VirtIO (paravirtualized), Realtek 8139, VMware vmxnet3
* RNG: VirtIO RNG
* Balloon capabilities (VirtIO)
* Support for QEMU guest agent.

Proxmox VE also supports PCI and USB pass-through devices from the physical server. Additional options like CPU flags, different type of emulated hardware (that QEMU supports) etc. are also available from the CLI (shell).

Supported VM/Linux container operating systems

Since Proxmox VE is using KVM in the background, it supports all operating systems that KVM supports for virtual machines, like:

* Linux: any distribution (Red Hat Enterprise Linux, CentOS, Rocky Linux, Fedora, Oracle Enterprise Linux, Debian, Ubuntu, SUSE Linux Enterprise Server, openSUSE, Gentoo, Arch Linux, ... ).
* BSD: FreeBSD,OpenBSD, NetBSD, DragonflyBSD, ...
* Solaris (like): Solaris, OpenSolaris, OpenIndiana, Nexenta Core, ...
* Windows: almost any version starting from Windows 95 (Windows 95, Windows NT, Windows 98SE, Windows Server 2000, Windows 2000, Windows XP, Windows Server 2003/ R2, Windows Vista, Windows Server 2008/ R2, Windows 7/ 8, Windows 10, Windows Server 2008R2, Windows Server 2012/ R2, Windows 11).
* Haiku, ReactOS, Plan 9, AROS Research Operating System and macOS.
* In addition it supports: Android, GNU/Hurd (Debian K16), Minix 3.1.2a, together with other operating systems.

Relating to Linux containers, since Proxmox VE is using LXC Linux containers, it supports any Linux distribution as a Linux container.

Central management

Proxmox VE use integrated web based management based on cluster and Proxmox VE cluster filesystem (pmxcfs). It means that there is no single point of failure in cluster, since every node (physical host) has all the configuration files stored in database-driver multi master cluster wide filesystem, shared among the nodes via corosync. Even more, every node has all the roles/functions (management/firewall/storage/network/...) needed for work. As the cluster grows, level of redundancy also grows with them. It also means that user can configure anything on any node inside the cluster, since all the changes are synchronized inside the cluster and immediately available to the other nodes.

Network model

Proxmox VE use bridge model by default. Bridges are like physical network switches that work in OSI layer 2 (OSI 2), but implemented in software. All virtual machines or Linux containers can share a single bridge, or you can create multiple bridges to isolate (segment) the network. For the bridge itself you can choose between Linux native bridge and/or Open vSwitch bridge. In addition to that you can also natively use a bond (aggregation): Linux native or Open vSwitch kind, with the mode that is best suited for you (e.g. balance-rr, active-backup, balance-xor, broadcast, LACP, balance-tlb, balance-alb).

All of the above interfaces support VLANs ( 802.1Q). There is an additional SDN module (currently in testing in v.7.2.x) with which you can utilize: VLAN, QinQ, VxLAN and EVPN for network isolation/segmentation, among the other options specific to the SDN.

Storage model

Proxmox VE supports flexible local and remote (network) storages.

Supported local storage types are:
* LVM group, LVM-Thin, directory, ZFS and Btrfs.

Supported remote (network) storage types are:
* iSCSI, Fibre Channel, NFS, GlusterFS, SMB/CIFS, ZFS over iSCSI, Ceph (Ceph FS and/or RBD) and Proxmox Backup Server.

In addition, storages can be divided into two groups:

* File level storage - which allow access to fully featured POSIX compliant filesystem.
* Block level storage - which allow access to raw (block) data.

Even more, there is a need (and possibilities) to define the type of the storage (for each and every storage), depending on the usage of the specific storage (for storing the VM/Container disk images, ISO images, LXC templates, backups, etc).

Storage replication

Proxmox VE supports redundancy for guests that are using the local storage in the way that it can replicate guest volumes to another PVE node (host) using the method so called storage replication (available only for ZFS). Those replication tasks can be started automatically in selected replication time interval.

High-availability cluster

Proxmox VE (PVE) can be clustered across multiple server nodes.

Official documentation guarantees/tested the cluster size of up to 32 physical servers (nodes), no matter the CPU core number or any other hardware limitations, but even bigger cluster size is possible. Those limitation are not imposed by PVE itself, but come from Corosync limits.

Since version 2.0, Proxmox VE offers a high availability option for clusters based on the Corosync communication stack. Starting from the PVE v.6.0 Corosync v.3.x is in use (not compatible with the earlier PVE versions). Individual virtual servers can be configured for high availability, using the built-in ha-manager. If a Proxmox node becomes unavailable or fails, the virtual servers can be automatically moved to another node and restarted. The database and FUSE-based Proxmox Cluster filesystem (pmxcfs) makes it possible to perform the configuration of each cluster node via the Corosync communication stack with SQlite engine.

VM/containers migration

At least since 2012, in a HA cluster, live virtual machines can be moved from one physical host to another without downtime. Since Proxmox VE 1.0, released 29.10.2008 KVM and OpenVZ live migration is supported (starting from version 4.0 OpenVZ is replaced by LXC).

Related to migration, PVE support next VM/Container migration types:
* Offline
* Online (with some limitations related to LXC containers)

Also PVE supports VM/Container migration from local storage's (local to Host/Hypervisor) or from (using) a shared storages (e.g. NFS/iSCSI/Ceph).

Clones and templates

PVE supports clones and templates. Clones are exact copy of the VM/container and a template can be created from VM/Container. One can use the template(s) to clone them to the new VM/container.

Virtual appliances

Proxmox VE has pre-packaged server software appliances which can be downloaded via the GUI. It is possible to download and deploy appliances from the TurnKey Linux Virtual Appliance Library.

Data backup

PVE includes backup software, vzdump, which allows data compression and on-line operation (snapshot mode). Since 2020 there's also an advanced client-server software for that called Proxmox Backup Server (PBS), which offers deduplication, compression, authenticated encryption and incremental backups.

Backup can be made only on storages selected as a backup (type) of the storage, and they may be on:
* NFS, SMB/CIFS, iSCSI, Ceph RBD, Proxmox Backup Server (even on local disk)

There are a few levels of backup:
* Scheduled Backup job from the cluster view on selected VM/containers to selected backup storage
* Manual backup of only selected VM/container

There are also a few backup modes:
* Stop mode - provides highest consistency, it includes stopping VM/container, creating backup and when finished, starting it again.
* Suspend mode - provides a little bit lower consistency, it includes suspending VM/container, then starting snapshot mode and creating backup and when finished, starting it again (from suspend).
* Snapshot mode - provides lowest operation downtime, it includes snapshoting the VM/container, creating backup (live) without stopping the VM/container (usage of the guest agent is advised in this mode).

Snapshots

PVE also support creating the snapshot of the VM/container in time. You can create many snapshots (for example before updating the VM/container) and switch back to any one of them.

Integrated firewall

PVE includes integrated firewall in three levels:

* Cluster wide.
* Per PVE node (host/hypervisor).
* Per VM/Linux container (guests).

Known limitations

In Proxmox VE 7.2 (May 2022) there are known limitations:

* Host (physical server) system maximum RAM: 12 TB
* Host (physical server) system maximum CPU core count: 768 logical CPU cores
* Number of hosts (physical servers) in high availability cluster: 32+ (official, but more is possible).
* Maximum number of bridges per host (physical server): 4094.
** Maximum number of logical networks when using VLANs: 4094.
** Maximum number of logical networks when using VxLANs (with SDN): 16 777 216.
** Maximum number of logical networks when using QinQ (so called Stacked VLAN): 16 777 216.

See also

* Comparison of platform virtualization software
* Kernel-based Virtual Machine (KVM)
* OVirt

References

External links

*
Project wikiProject HOWTOsProject ForumProject Roadmap
- features by the version of the Proxmox VE
Project Bug TrackerProject API

{{Authority control

2008 software
Debian-based distributions
Free software programmed in Perl
Free virtualization software
Linux distributions
Perl software
Software using the GNU AGPL license