UTF-EBCDIC

UTF-EBCDIC is a character encoding capable of encoding all 1,112,064 valid character code points in Unicode using one to five one-byte (8-bit) code units (in contrast to a maximum of four for UTF-8). It is meant to be EBCDIC-friendly, so that legacy EBCDIC applications on mainframes may process the characters without much difficulty. Its advantages for existing EBCDIC-based systems are similar to UTF-8's advantages for existing ASCII-based systems. Details on UTF-EBCDIC are defined in Unicode Technical Report #16.

To produce the UTF-EBCDIC encoded version of a series of Unicode code points, an encoding based on UTF-8 (known in the specification as UTF-8-Mod) is applied first (creating what the specification calls an I8 sequence). The main difference between this encoding and UTF-8 is that it allows Unicode code points U+0080 through U+009F (the C1 control codes) to be represented as a single byte and therefore later mapped to corresponding EBCDIC control codes. In order to achieve this, UTF-8-Mod uses 101XXXXX instead of 10XXXXXX as the format for trailing bytes in a multi-byte sequence. As this can only hold 5 bits rather than 6, the UTF-8-Mod encoding of codepoints above U+03FF are larger than the UTF-8 encoding.

The UTF-8-Mod transformation leaves the data in an ASCII-based format (for example, U+0041 "A" is still encoded as 01000001), so each byte is fed through a reversible (one-to-one) lookup table to produce the final UTF-EBCDIC encoding. For example, 01000001 in this table maps to 11000001; thus the UTF-EBCDIC encoding of U+0041 (Unicode's "A") is 0xC1 (EBCDIC's "A").

This encoding form is rarely used, even on the EBCDIC-based mainframes for which it was designed. IBM EBCDIC-based mainframe operating systems, such as z/OS, usually use UTF-16 for complete Unicode support. For example, IBM Db2, COBOL, PL/I, Java and the IBM XML toolkit support UTF-16 on IBM mainframes.

Codepage layout

There are 160 characters with single-byte encodings in UTF-EBCDIC (compared to 128 in UTF-8). As can be seen, the single-byte portion is similar to IBM-1047 instead of IBM-37 due to the location of the square brackets. CCSID 37 has [] at hex BA and BB instead of at hex AD and BD respectively.

Oracle UTFE

''Oracle UTFE'' is a Unicode 3.0 UTF-8 Oracle database variation, similar to the CESU-8 variant of UTF-8, where supplementary characters are encoded as two 4-byte characters rather than a single 4- or 5-byte character. It is used only on EBCDIC platforms.

Advantages:
* Only Unicode character set for EBCDIC.
* Length of SQL CHAR types can be specified in number of characters.
* Binary order of the SQL CHAR columns is same as binary order of the SQL NCHAR columns if the data consists of same supplementary characters. Consequently, these columns sort the same for identical strings.

Disadvantages:
* Supplementary characters occupy six bytes instead of four bytes only. Consequently, supplementary characters need to be converted.
* UTFE is not a Unicode standard encoding. Clients requiring UTF-8 encoding must convert data on retrieval and storage.

Mksh nega-UTF-8

Mksh is a POSIX shell with a “utf8-mode” that complicates porting to EBCDIC. Instead of implementing UTF-EBCDIC (which is “not used by anyone”), the developers found it easier to just use normal UTF-8 and convert it to EBCDIC by pretending it’s the “extended ASCII” encoding for the current code page. Doing so destroys EBCDIC control characters not found in ASCII, however.

See also

* UTF-1
* UTF-8
* BOCU-1

References

External links

* V.S. Umamaheswaran
Unicode Technical Report #16: the definition of UTF-EBCDIC
(2002-04-16)

{{DEFAULTSORT:Utf-Ebcdic
Character encoding
Unicode Transformation Formats