The Maildir

e-mail Electronic mail (usually shortened to email; alternatively hyphenated e-mail) is a method of transmitting and receiving Digital media, digital messages using electronics, electronic devices over a computer network. It was conceived in the ...

format is a common way of storing

email Electronic mail (usually shortened to email; alternatively hyphenated e-mail) is a method of transmitting and receiving Digital media, digital messages using electronics, electronic devices over a computer network. It was conceived in the ...

messages on a file system, rather than in a database. Each message is assigned a file with a unique name, and each mail folder is a file system directory containing these files. Maildir was designed by Daniel J. Bernstein circa 1995, with a major goal of eliminating the need for program code to handle file locking and unlocking through use of the local filesystem. Maildir design reflects the fact that the only operations valid for an email message is that it be created, deleted or have its status changed in some way. Maildir

__TOC__

Specifications

A Maildir directory (often named Maildir) usually has three subdirectories named tmp, new, and cur. * The tmp subdirectory temporarily stores e-mail messages that are in the process of being delivered. This subdirectory may also store other kinds of temporary files. * The new subdirectory stores messages that have been delivered, but have not yet been seen by any mail application. * The cur subdirectory stores messages that have already been seen by mail applications.

Maildir++

Sam Varshavchik, the author of the

Courier Mail Server The Courier Mail Server is a mail transfer agent (MTA) server that provides Simple Mail Transfer Protocol, SMTP, IMAP, POP3, Simple Mail Access Protocol, SMAP, webmail, and electronic mailing list, mailing list services with individual components ...

and other software, defined the Maildir++ extension to the Maildir format to support subfolders and mail quotas. Maildir++ directories contain subdirectories with names that start with a '.' (dot) which are also Maildir++ folders. The extension complies with the original Maildir specification, which allows for subdirectories in addition to ''tmp'', ''new'' and ''cur''.

Technical operation

mail delivery agent A message delivery agent (MDA), or mail delivery agent, is a computer software component that is responsible for the delivery of e-mail messages to a local recipient's mailbox. It is also called a local delivery agent (LDA). Within the Internet ...

is a program that delivers an email message into a Maildir. The mail delivery agent creates a new file with a unique filename in the tmp directory. At the time of its invention guaranteeing unique filenames efficiently was difficult. The original qmail algorithm for unique names was: # read the current

Unix time Unix time is a date and time representation widely used in computing. It measures time by the number of non-leap seconds that have elapsed since 00:00:00 Coordinated Universal Time, UTC on 1 January 1970, the Unix Epoch (computing), epoc ...

# read the current

process identifier In computing, the process identifier (a.k.a. process ID or PID) is a number used by most operating system kernel (operating system), kernels—such as those of Unix, macOS and Windows—to uniquely identify an active Process (computing), process. ...

(PID) # read the current

hostname In computer networking, a hostname (archaically nodename) is a label that is assigned to a device connected to a computer network and that is used to identify the device in various forms of electronic communication, such as the World Wide Web. Hos ...

# concatenate the above three values into a string separated by the period character; this is the new filename # if stat() reports that the filename exists, then wait two seconds # go to previous step until the filename does not exist # create a file with the unique filename and write the message contents to the new file By 2000, the qmail author recommended in an updated specification to append the value of a per-process counter to the PID, whose value should be incremented after each delivery. The rate-limiting recommendation to "wait two seconds" was dropped. By 2003, the recommendations had been further amended to require that instead of the PID and counter, the middle part of the filename should be created by "concatenating enough of the following strings to guarantee uniqueness" even in the face of multiple simultaneous deliveries to the same maildir from one or more processes:

* #''n'', where ''n'' is (in hexadecimal) the output of the operating system's unix_sequencenumber() system call, which returns a number that increases by 1 every time it is called, starting from 0 after reboot. * X''n'', where ''n'' is (in hexadecimal) the output of the operating system's unix_bootnumber() system call, which reports the number of times that the system has been booted. Together with #, this guarantees uniqueness; unfortunately, most operating systems don't support unix_sequencenumber() and unix_bootnumber(). * R''n'', where ''n'' is (in hexadecimal) the output of the operating system's unix_cryptorandomnumber() system call or an equivalent source, such as /dev/urandom. Unfortunately, some operating systems don't include cryptographic random number generators. * I''n'', where ''n'' is (in hexadecimal) the UNIX
inode An inode (index node) is a data structure in a Unix-style file system that describes a file-system object such as a file or a directory. Each inode stores the attributes and disk block locations of the object's data. File-system object attribu ...
number of this file. Unfortunately, inode numbers aren't always available through NFS. * V''n'', where ''n'' is (in hexadecimal) the UNIX device number of this file. Unfortunately, device numbers aren't always available through NFS. (Device numbers are also not helpful with the standard UNIX filesystem: a maildir has to be within a single UNIX device for link() and rename() to work.) * M''n'', where ''n'' is (in decimal) the microsecond counter from the same gettimeofday() used for the left part of the unique name. * P''n'', where ''n'' is (in decimal) the process ID. * Q''n'', where ''n'' is (in decimal) the number of deliveries made by this process.

This 2003 algorithm was criticised in 2006 as being unnecessarily complex by

Timo Sirainen Timo Sirainen, born 1979, is a Finnish programmer also known under the handles "cras" and "tss". Sirainen is the original author of the IRC-client Irssi and the POP/ IMAP server Dovecot. Sirainen lives in Helsinki, Finland. In 2006, Sirainen b ...

, the creator of Dovecot. As of November 2023, qmail author Daniel Bernstein had made no further changes to the 2003 filename generation recommendations. On modern POSIX systems,

temporary file A temporary file is a file created to store information temporarily, either for a program's intermediate use or for transfer to a permanent file when complete. It may be created by computer programs for a variety of purposes, such as when a progra ...

s can be safely created with the mkstemp C library function. The delivery process stores the message in the maildir by creating and writing to tmp/''uniquefilename'', and then moving this file to new/''uniquefilename''. The moving can be done using rename, which is atomic in many systems. Alternatively, it can be done by hard-linking the file to new and then unlinking the file from tmp. Any leftover file will eventually be deleted. This sequence guarantees that a maildir-reading program will not see a partially written message. There can be multiple programs reading a maildir at the same time. They range from

mail user agent The mail or post is a system for physically transporting postcards, letters, and parcels. A postal service can be private or public, though many governments place restrictions on private systems. Since the mid-19th century, national postal sy ...

s (MUAs), which access the server's file system directly, through

Internet Message Access Protocol In computing, the Internet Message Access Protocol (IMAP) is an Internet standard protocol used by email clients to retrieve email messages from a mail server over a TCP/IP connection. IMAP is defined by . IMAP was designed with the goal of per ...

Post Office Protocol In computing, the Post Office Protocol (POP) is an application-layer Internet standard protocol used by e-mail clients to retrieve e-mail from a mail server. Today, POP version 3 (POP3) is the most commonly used version. Together with IMAP, ...

servers acting on behalf of remote MUAs, to utilities such as biff and

rsync rsync (remote sync) is a utility for transferring and synchronizing files between a computer and a storage drive and across networked computers by comparing the modification times and sizes of files. It is commonly found on Unix-like opera ...

, which may or may not be aware of the maildir structure. Readers should never look in tmp. When a cognizant maildir-reading process (either a POP or

IMAP In computing, the Internet Message Access Protocol (IMAP) is an Internet standard protocol used by email clients to retrieve email messages from a mail server over a TCP/IP connection. IMAP is defined by . IMAP was designed with the goal of per ...

server, or a mail user agent acting locally) finds messages in the new directory, it ''must'' move them to cur. It is just a means to notify the user "you have ''X'' new messages". This moving needs to be done using the atomic filesystem rename(), as the alternative ''link-then-unlink'' technique is non-atomic and may result in duplicated messages. An informational suffix is appended to filenames at this stage. It consists of a colon (to separate the unique part of the filename from the actual information), a "2", a

comma The comma is a punctuation mark that appears in several variants in different languages. Some typefaces render it as a small line, slightly curved or straight, but inclined from the vertical; others give it the appearance of a miniature fille ...

and various

flag A flag is a piece of textile, fabric (most often rectangular) with distinctive colours and design. It is used as a symbol, a signalling device, or for decoration. The term ''flag'' is also used to refer to the graphic design employed, and fla ...

s. The "2" specifies the version of the information that follows the comma. "2" is the only currently officially specified version, "1" being an experimental version. The specification defines flags that show whether the message has been read, deleted and so on: the initial (capital) letter of "Passed", "Replied", "Seen", "Trashed", "Draft", and "Flagged". Applications often choose to supplement this very limited set of flags, for example notmuch offers flag synchronization in addition to arbitrary user-defined flags, while Dovecot uses lowercase letters to match 26 IMAP keywords, which may include keywords such as $ MDNSent or user-defined flags. Although Maildir was intended to allow lockless usage, in practice some software that uses Maildirs also uses locks, such as Dovecot.

File-system compatibility issues

The Maildir standard can only be implemented on systems that accept colons in filenames. Systems that don't allow colons in filenames (this includes

Microsoft Windows Windows is a Product lining, product line of Proprietary software, proprietary graphical user interface, graphical operating systems developed and marketed by Microsoft. It is grouped into families and subfamilies that cater to particular sec ...

and some configurations of Novell Storage Services) can use a non-standard alternative separator, such as ";" or "-". It is often trivial to patch

free and open-source software Free and open-source software (FOSS) is software available under a license that grants users the right to use, modify, and distribute the software modified or not to everyone free of charge. FOSS is an inclusive umbrella term encompassing free ...

to use a different separator. As there is currently no agreement on what character this alternative separator should be, there can be interoperability difficulties between different Maildir-supporting programs on these systems. However, not all Maildir-related software needs to know what the separator character is, because not all Maildir-related software needs to be able to read or modify the flags of a message ("read", "replied to" etc.); software that merely delivers to a Maildir or archives old messages from it based only on date, should work no matter what separator is in use. If only the MUA needs to read or modify message flags, and only one MUA is used, then non-standard alternative separators may be used without interoperability problems.

Software that supports Maildir directly

Mail servers

* Dovecot IMAP server *

SMTP and IMAP server, for which the Maildir++ format was invented *

Sendmail Sendmail is a general purpose internetwork email routing facility that supports many kinds of mail-transfer and delivery methods, including the Simple Mail Transfer Protocol (SMTP) used for email transport over the Internet. A descendant of t ...

The original SMTP server * Exim SMTP server * Postfix SMTP server * qmail SMTP server, for which the Maildir format was invented * MeTA1 SMTP server * OpenSMTPD SMTP server
Stalwart Mail Server
SMTP and IMAP server implemented in

Rust Rust is an iron oxide, a usually reddish-brown oxide formed by the reaction of iron and oxygen in the catalytic presence of water or air moisture. Rust consists of hydrous iron(III) oxides (Fe2O3·nH2O) and iron(III) oxide-hydroxide (FeO(OH) ...

Delivery agents

procmail procmail is an email server software component — specifically, a message delivery agent (MDA). It was one of the earliest mail filter programs. It is typically used in Unix-like mail systems, using the mbox and Maildir storage formats. pr ...

* Dovecot delivery agent * maildrop * getmail, a Maildir-aware mail-retrieval and delivery agent alternative to Fetchmail * fdm
muchsync
synchronisin
notmuch mail
mailboxes between any number of replicas * OfflineIMAP
isync
synchronises mailboxes, supporting Maildir and IMAP4
Attomail
a minimal Maildir-aware MDA implemented in Haskell

Mail readers

*aerc (efficient and extensible email client) *

Balsa ''Ochroma pyramidale'', commonly known as balsa, is a large, fast-growing tree native to the Americas. It is the sole member of the genus ''Ochroma'', and is classified in the subfamily Bombacoideae of the mallow family Malvaceae. The tree is fa ...

previously the official GNOME mail reader (prior to Evolution) *

Cone In geometry, a cone is a three-dimensional figure that tapers smoothly from a flat base (typically a circle) to a point not contained in the base, called the '' apex'' or '' vertex''. A cone is formed by a set of line segments, half-lines ...

a curses-based mail reader *

Evolution Evolution is the change in the heritable Phenotypic trait, characteristics of biological populations over successive generations. It occurs when evolutionary processes such as natural selection and genetic drift act on genetic variation, re ...

, official GNOME mail client * GNUMail *

Gnus Gnus (), or Gnus Network User Services, is a message reader which is part of GNU Emacs. It supports reading and composing both e-mail and news and can also act as an RSS reader, web processor, and directory browser for both local and remote file ...

* KMail, KDE mail reader *

mailx mailx is a Unix utility program for sending and receiving mail, also known as a Mail User Agent program. Being a console application with a command syntax similar to ed, it is the POSIX standardized variant of the Berkeley Mail utility. See als ...

* Mutt *Notmuch (fast, global-search and tag-based email system) * Pine/Alpine *

Mozilla Thunderbird Mozilla Thunderbird is a free and open-source email client that also functions as a personal information manager with a Digital calendar, calendar and contactbook, as well as an RSS feed reader, chat client (IRC/XMPP/Matrix (protocol), Matrix), ...

– experimental and “disabled by default because there are still many bugs”

Notes and references

{{reflist, refs= {{cite web , first=Daniel J. , last=Bernstein. , author-link=Daniel J. Bernstein , year=c. 2000 , url=http://cr.yp.to/proto/maildir.html , title=Using maildir format , archive-url=https://web.archive.org/web/20000902121438/http://cr.yp.to/proto/maildir.html , archive-date=2000-09-02 , orig-year=First published 2000 or earlier , url-status=live , access-date=2018-11-23 {{cite web , first=Daniel J. , last=Bernstein. , author-link=Daniel J. Bernstein , year=2003, url=http://cr.yp.to/proto/maildir.html , title=Using maildir format , archive-url=https://web.archive.org/web/20030401082238/https://cr.yp.to/proto/maildir.html , archive-date=2003-04-01 , orig-year=The earliest version of this document was first published in 2000 or earlier , url-status=live , access-date=2018-11-23 {{cite web , url=https://web.archive.org/web/20230503185934/cr.yp.to/proto/maildir.html , publisher=

Internet Archive The Internet Archive is an American 501(c)(3) organization, non-profit organization founded in 1996 by Brewster Kahle that runs a digital library website, archive.org. It provides free access to collections of digitized media including web ...

, access-date=2023-11-23 , title=Wayback Machine snapshots of cr.yp.to/proto/maildir.html , year=2023 {{cite web , url=http://www.courier-mta.org/maildir.html , title=maildir(5) , first=Sam , last=Varshavchik , year=2009 , access-date=2024-08-09 , url-status=live , archive-url=https://web.archive.org/web/20240417195248/https://www.courier-mta.org/maildir.html , archive-date=2024-04-17 {{cite mailing list , url=http://www.mail-archive.com/courier-users@lists.sourceforge.net/msg38512.html , title=Management of maildir structures , date=25 July 2016 , access-date=26 July 2016 , mailing-list=courier-users , author=Sam Varshavchik {{cite web , url=http://www.courier-mta.org/imap/README.maildirquota.html , title=Maildir++ , first=Sam , last=Varshavchik , year=2011 , access-date=2024-08-09 , url-status=live , archive-url=https://web.archive.org/web/20240529081454/https://www.courier-mta.org/imap/README.maildirquota.html , archive-date=2024-05-29 Dovecot Wiki: maildir format
/ref> mutt maildir support: workaround for filesystems that don't accept colons
/ref> {{cite web , url=https://doc.dovecot.org/admin_manual/mailbox_formats/maildir/#mail-delivery , title=Maildir Mailbox Format: Mail Delivery' , first=Timo , last=Sirainen , author-link=Timo Sirainen , access-date=2024-08-09 , orig-date=2006-12-05 , url-status=live , archive-url=https://web.archive.org/web/20240624123136/https://doc.dovecot.org/admin_manual/mailbox_formats/maildir/#mail-delivery , archive-date=2024-06-24 , quote="All this trouble is rather pointless. Only the first step is what really guarantees that the mails won’t get overwritten, the rest just sounds nice. Even though they might catch a problem once in a while, they give no guaranteed protection and will just as easily pass duplicate filenames through and overwrite existing mails.¶ Step 2 is pointless because there’s a race condition between steps 2 and 3. PID/host combination by itself should already guarantee that it never finds such a file. If it does, something’s broken and the stat() check won’t help since another process might be doing the same thing at the same time, and you end up writing to the same file in tmp/, causing the mail to get corrupted.¶ In step 4 the link() would fail if an identical file already existed in the Maildir, right? Wrong. The file may already have been moved to cur/ directory, and since it may contain any number of flags by then you can’t check with a simple stat() anymore if it exists or not.¶ Step 2 was pointed out to be useful if clock had moved backwards. However, this doesn’t give any actual safety guarantees because an identical base filename could already exist in cur/. Besides if the system was just rebooted, the file in tmp/ could probably be even overwritten safely (assuming it wasn’t already link()ed to new/).¶ So really, all that’s important in not getting mails overwritten in your Maildir is step 1: Always create filenames that are guaranteed to be unique. Forget about the 2 second waits and such that the Qmail’s man page talks about." {{cite web , url=https://doc.dovecot.org/admin_manual/mailbox_formats/maildir/#locking , title=Maildir Mailbox Format: Locking , first=Timo , last=Sirainen , author-link=Timo Sirainen , access-date=2024-08-09 , orig-date=2006-12-05 , url-status=live , archive-url=https://web.archive.org/web/20240624123136/https://doc.dovecot.org/admin_manual/mailbox_formats/maildir/#locking , archive-date=2024-06-24 {{cite web , url=http://pubs.opengroup.org/onlinepubs/9699919799/functions/rename.html , title=rename , author= , year=2013 , publisher=

The Open Group The Open Group is a global consortium that seeks to "enable the achievement of business objectives" by developing " open, vendor-neutral technology standards and certifications." It has 900+ member organizations and provides a number of services ...

, access-date=23 July 2016 , quote=That specification requires that the action of the function be atomic. {{cite web , archive-url=https://web.archive.org/web/19971012032244/http://www.qmail.org/qmail-manual-html/man5/maildir.html , url=http://www.qmail.org/qmail-manual-html/man5/maildir.html , first=Daniel J. , last=Bernstein , author-link=Daniel J. Bernstein , title=maildir(5) , archive-date=1997-10-12 , year=1995 , access-date=2018-11-23 , url-status=dead

External links

manual page for maildir

Email storage formats