movq/msdos-encodings
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add the Windows ANSI family: 874 Thai and 1250-1258 as WindowsEncoding

Browse Source 
The ten single-byte ANSI pages join as a second enum alongside
DosEncoding, mirroring its API. Errors now carry a public Encoding
wrapper (Dos | Windows) so both families share the single-byte codec
functions; Encoding::from_code_page resolves a raw code page number
across families, DOS first.

data/ is reorganized into pc/, windows/, and bestfit/, and the
generated single-byte tables split into dos.rs and ansi.rs. Microsoft
publishes an identical Thai table for both families, so DosEncoding::
Cp874 and WindowsEncoding::Cp874 agree byte-for-byte (pinned by test).
This commit is contained in:
Claude Fable 5
2026-07-19 12:40:36 -05:00
parent 518250da2c
commit 2fe53fdee4
40 changed files with 5857 additions and 100 deletions
Download Patch File Download Diff File Expand all files Collapse all files

42
README.md
View File

@@ -1,15 +1,15 @@
# msdos-encodings # msdos-encodings
Bidirectional, emulator-grade conversion between MS-DOS code pages and Unicode — Bidirectional, emulator-grade conversion between Microsoft code pages and
without linking ICU. The crate owns the canonical Microsoft tables, so it stays Unicode — without linking ICU. The crate owns the canonical Microsoft tables,
small, self-contained, and cross-compiles cleanly. so it stays small, self-contained, and cross-compiles cleanly.
These are strictly **Microsoft's** mappings as published by the Unicode These are strictly **Microsoft's** mappings as published by the Unicode
Consortium (`VENDORS/MICSFT/PC/`) — not the IBM or Oracle variants, which Consortium (`VENDORS/MICSFT/`) — not the IBM or Oracle variants, which
differ in a handful of slots. differ in a handful of slots.
```rust ```rust
use msdos_encodings::DosEncoding; use msdos_encodings::{DosEncoding, Encoding, WindowsEncoding};
let enc = DosEncoding::default(); // code page 437 let enc = DosEncoding::default(); // code page 437
assert_eq!(enc.decode(b"Caf\x82").unwrap(), "Café"); assert_eq!(enc.decode(b"Caf\x82").unwrap(), "Café");
@@ -19,26 +19,36 @@ assert_eq!(enc.encode("Café").unwrap(), b"Caf\x82");
let enc = DosEncoding::from_code_page(866).unwrap(); let enc = DosEncoding::from_code_page(866).unwrap();
assert_eq!(enc.decode(&[0x80]).unwrap(), "А"); // DOS Cyrillic Russian assert_eq!(enc.decode(&[0x80]).unwrap(), "А"); // DOS Cyrillic Russian
// Four pages leave bytes undefined; decode is fallible, or go lossy: // The Windows ANSI family is a separate enum:
let enc = WindowsEncoding::Cp1252;
assert_eq!(enc.decode(&[0x93, 0x80, 0x94]).unwrap(), "\u{201C}€\u{201D}");
// Or resolve a raw code page number across both families:
assert_eq!(Encoding::from_code_page(1251).unwrap().name(), "Windows Cyrillic");
// Some pages leave bytes undefined; decode is fallible, or go lossy:
assert!(DosEncoding::Cp874.decode(&[0xFF]).is_err()); assert!(DosEncoding::Cp874.decode(&[0xFF]).is_err());
assert_eq!(DosEncoding::Cp874.decode_lossy(&[0xFF]), "\u{FFFD}"); assert_eq!(DosEncoding::Cp874.decode_lossy(&[0xFF]), "\u{FFFD}");
``` ```
## Status ## Status
- **All sixteen single-byte DOS pages** — implemented, decode + encode: - **All sixteen single-byte DOS (OEM) pages** — implemented, decode + encode:
437, 737, 775, 850, 852, 855, 857, 860, 861, 862, 863, 864, 865, 866, 869, 874. 437, 737, 775, 850, 852, 855, 857, 860, 861, 862, 863, 864, 865, 866, 869, 874.
- **All four double-byte CJK pages** — implemented, decode + encode: - **All four double-byte CJK pages** — implemented, decode + encode:
932 (Shift-JIS), 936 (GBK), 949 (Unified Hangul), 950 (Big5). They sit 932 (Shift-JIS), 936 (GBK), 949 (Unified Hangul), 950 (Big5). They sit
behind the on-by-default `dbcs` feature; build with behind the on-by-default `dbcs` feature; build with
`default-features = false` for a lean single-byte-only crate. `default-features = false` for a lean single-byte-only crate.
- **All ten single-byte Windows ANSI pages** — implemented, decode + encode:
874 (Thai) and 1250–1258, as `WindowsEncoding`.
## Fidelity notes ## Fidelity notes
- Every single-byte table is a verified bijection: anything a page decodes - Every single-byte table is a verified bijection: anything a page decodes
re-encodes to the identical bytes. re-encodes to the identical bytes.
- Code pages 857, 864, 869, and 874 leave some byte values undefined, so - Many pages leave byte values undefined (DOS 857/864/869/874; every ANSI
`decode` returns a `Result`; `decode_lossy` substitutes U+FFFD. page except 1256 has holes in `0x80..=0x9F`), so `decode` returns a
`Result`; `decode_lossy` substitutes U+FFFD.
- Microsoft's CP864 (DOS Arabic) maps `0x25` to ARABIC PERCENT SIGN — the low - Microsoft's CP864 (DOS Arabic) maps `0x25` to ARABIC PERCENT SIGN — the low
half is *not* pure ASCII, and `%` itself is unmappable. This is faithful to half is *not* pure ASCII, and `%` itself is unmappable. This is faithful to
the source table. the source table.
@@ -46,20 +56,23 @@ assert_eq!(DosEncoding::Cp874.decode_lossy(&[0xFF]), "\u{FFFD}");
character (932's NEC/IBM extension overlap, 950's duplicated box-drawing character (932's NEC/IBM extension overlap, 950's duplicated box-drawing
rows). All of them decode; re-encoding picks the code Windows picks, as rows). All of them decode; re-encoding picks the code Windows picks, as
resolved from Microsoft's own `bestfit932/950.txt` WCTABLE (vendored in resolved from Microsoft's own `bestfit932/950.txt` WCTABLE (vendored in
`data/`, used *only* for duplicate resolution — no lossy best-fit mappings `data/bestfit/`, used *only* for duplicate resolution — no lossy best-fit
are imported). mappings are imported).
- Double-byte decode adds two failure modes: a lead byte at end of input - Double-byte decode adds two failure modes: a lead byte at end of input
(`TruncatedPair`) and an undefined lead/trail pair (`UndefinedPair`). (`TruncatedPair`) and an undefined lead/trail pair (`UndefinedPair`).
`decode_lossy` emits one U+FFFD per failed pair. `decode_lossy` emits one U+FFFD per failed pair.
- The full decode direction of all four CJK pages agrees exactly with - The full decode direction of all four CJK pages agrees exactly with
Python's `cp932`/`cp936`/`cp949`/`cp950` codecs (59k+ pairs, verified Python's `cp932`/`cp936`/`cp949`/`cp950` codecs (59k+ pairs, verified
during development). during development).
- Microsoft publishes the identical Thai table for both families, so
`DosEncoding::Cp874` and `WindowsEncoding::Cp874` agree byte-for-byte
(pinned by a test).
## Regenerating tables ## Regenerating tables
Everything in `src/tables/` except `mod.rs` is generated from the vendored Everything in `src/tables/` except `mod.rs` is generated from the vendored
mapping files in `data/` mapping files in `data/` (`pc/` and `windows/` mapping tables, `bestfit/`
(needs Python ≥ 3.10 and `rustfmt` on `PATH`): for duplicate resolution; needs Python ≥ 3.10 and `rustfmt` on `PATH`):
```sh ```sh
uv run tools/gen_tables.py uv run tools/gen_tables.py
@@ -69,7 +82,8 @@ uv run tools/gen_tables.py
Text-encoding conversion only. Filename conventions (8.3 space padding, Text-encoding conversion only. Filename conventions (8.3 space padding,
case folding, the 0x05/0xE5 deleted-entry dance) are deliberately **out of case folding, the 0x05/0xE5 deleted-entry dance) are deliberately **out of
scope** and left to consumers. scope** and left to consumers, as is any Vietnamese (CP1258) combining-mark
normalization.
## Consumers ## Consumers

0
data/bestfit932.txt → data/bestfit/bestfit932.txt
View File

0
data/bestfit936.txt → data/bestfit/bestfit936.txt
View File

0
data/bestfit949.txt → data/bestfit/bestfit949.txt
View File

0
data/bestfit950.txt → data/bestfit/bestfit950.txt
View File

0
data/CP437.TXT → data/pc/CP437.TXT
View File

0
data/CP737.TXT → data/pc/CP737.TXT
View File

0
data/CP775.TXT → data/pc/CP775.TXT
View File

0
data/CP850.TXT → data/pc/CP850.TXT
View File

0
data/CP852.TXT → data/pc/CP852.TXT
View File

0
data/CP855.TXT → data/pc/CP855.TXT
View File

0
data/CP857.TXT → data/pc/CP857.TXT
View File

0
data/CP860.TXT → data/pc/CP860.TXT
View File

0
data/CP861.TXT → data/pc/CP861.TXT
View File

0
data/CP862.TXT → data/pc/CP862.TXT
View File

0
data/CP863.TXT → data/pc/CP863.TXT
View File

0
data/CP864.TXT → data/pc/CP864.TXT
View File

0
data/CP865.TXT → data/pc/CP865.TXT
View File

0
data/CP866.TXT → data/pc/CP866.TXT
View File

0
data/CP869.TXT → data/pc/CP869.TXT
View File

0
data/CP874.TXT → data/pc/CP874.TXT
View File

274
data/windows/CP1250.TXT Normal file
View File

@@ -0,0 +1,274 @@
#
# Name: cp1250 to Unicode table
# Unicode version: 2.0
# Table version: 2.01
# Table format: Format A
# Date: 04/15/98
#
# Contact: Shawn.Steele@microsoft.com
#
# General notes: none
#
# Format: Three tab-separated columns
# Column #1 is the cp1250 code (in hex)
# Column #2 is the Unicode (in hex as 0xXXXX)
# Column #3 is the Unicode name (follows a comment sign, '#')
#
# The entries are in cp1250 order
#
0x00 0x0000 #NULL
0x01 0x0001 #START OF HEADING
0x02 0x0002 #START OF TEXT
0x03 0x0003 #END OF TEXT
0x04 0x0004 #END OF TRANSMISSION
0x05 0x0005 #ENQUIRY
0x06 0x0006 #ACKNOWLEDGE
0x07 0x0007 #BELL
0x08 0x0008 #BACKSPACE
0x09 0x0009 #HORIZONTAL TABULATION
0x0A 0x000A #LINE FEED
0x0B 0x000B #VERTICAL TABULATION
0x0C 0x000C #FORM FEED
0x0D 0x000D #CARRIAGE RETURN
0x0E 0x000E #SHIFT OUT
0x0F 0x000F #SHIFT IN
0x10 0x0010 #DATA LINK ESCAPE
0x11 0x0011 #DEVICE CONTROL ONE
0x12 0x0012 #DEVICE CONTROL TWO
0x13 0x0013 #DEVICE CONTROL THREE
0x14 0x0014 #DEVICE CONTROL FOUR
0x15 0x0015 #NEGATIVE ACKNOWLEDGE
0x16 0x0016 #SYNCHRONOUS IDLE
0x17 0x0017 #END OF TRANSMISSION BLOCK
0x18 0x0018 #CANCEL
0x19 0x0019 #END OF MEDIUM
0x1A 0x001A #SUBSTITUTE
0x1B 0x001B #ESCAPE
0x1C 0x001C #FILE SEPARATOR
0x1D 0x001D #GROUP SEPARATOR
0x1E 0x001E #RECORD SEPARATOR
0x1F 0x001F #UNIT SEPARATOR
0x20 0x0020 #SPACE
0x21 0x0021 #EXCLAMATION MARK
0x22 0x0022 #QUOTATION MARK
0x23 0x0023 #NUMBER SIGN
0x24 0x0024 #DOLLAR SIGN
0x25 0x0025 #PERCENT SIGN
0x26 0x0026 #AMPERSAND
0x27 0x0027 #APOSTROPHE
0x28 0x0028 #LEFT PARENTHESIS
0x29 0x0029 #RIGHT PARENTHESIS
0x2A 0x002A #ASTERISK
0x2B 0x002B #PLUS SIGN
0x2C 0x002C #COMMA
0x2D 0x002D #HYPHEN-MINUS
0x2E 0x002E #FULL STOP
0x2F 0x002F #SOLIDUS
0x30 0x0030 #DIGIT ZERO
0x31 0x0031 #DIGIT ONE
0x32 0x0032 #DIGIT TWO
0x33 0x0033 #DIGIT THREE
0x34 0x0034 #DIGIT FOUR
0x35 0x0035 #DIGIT FIVE
0x36 0x0036 #DIGIT SIX
0x37 0x0037 #DIGIT SEVEN
0x38 0x0038 #DIGIT EIGHT
0x39 0x0039 #DIGIT NINE
0x3A 0x003A #COLON
0x3B 0x003B #SEMICOLON
0x3C 0x003C #LESS-THAN SIGN
0x3D 0x003D #EQUALS SIGN
0x3E 0x003E #GREATER-THAN SIGN
0x3F 0x003F #QUESTION MARK
0x40 0x0040 #COMMERCIAL AT
0x41 0x0041 #LATIN CAPITAL LETTER A
0x42 0x0042 #LATIN CAPITAL LETTER B
0x43 0x0043 #LATIN CAPITAL LETTER C
0x44 0x0044 #LATIN CAPITAL LETTER D
0x45 0x0045 #LATIN CAPITAL LETTER E
0x46 0x0046 #LATIN CAPITAL LETTER F
0x47 0x0047 #LATIN CAPITAL LETTER G
0x48 0x0048 #LATIN CAPITAL LETTER H
0x49 0x0049 #LATIN CAPITAL LETTER I
0x4A 0x004A #LATIN CAPITAL LETTER J
0x4B 0x004B #LATIN CAPITAL LETTER K
0x4C 0x004C #LATIN CAPITAL LETTER L
0x4D 0x004D #LATIN CAPITAL LETTER M
0x4E 0x004E #LATIN CAPITAL LETTER N
0x4F 0x004F #LATIN CAPITAL LETTER O
0x50 0x0050 #LATIN CAPITAL LETTER P
0x51 0x0051 #LATIN CAPITAL LETTER Q
0x52 0x0052 #LATIN CAPITAL LETTER R
0x53 0x0053 #LATIN CAPITAL LETTER S
0x54 0x0054 #LATIN CAPITAL LETTER T
0x55 0x0055 #LATIN CAPITAL LETTER U
0x56 0x0056 #LATIN CAPITAL LETTER V
0x57 0x0057 #LATIN CAPITAL LETTER W
0x58 0x0058 #LATIN CAPITAL LETTER X
0x59 0x0059 #LATIN CAPITAL LETTER Y
0x5A 0x005A #LATIN CAPITAL LETTER Z
0x5B 0x005B #LEFT SQUARE BRACKET
0x5C 0x005C #REVERSE SOLIDUS
0x5D 0x005D #RIGHT SQUARE BRACKET
0x5E 0x005E #CIRCUMFLEX ACCENT
0x5F 0x005F #LOW LINE
0x60 0x0060 #GRAVE ACCENT
0x61 0x0061 #LATIN SMALL LETTER A
0x62 0x0062 #LATIN SMALL LETTER B
0x63 0x0063 #LATIN SMALL LETTER C
0x64 0x0064 #LATIN SMALL LETTER D
0x65 0x0065 #LATIN SMALL LETTER E
0x66 0x0066 #LATIN SMALL LETTER F
0x67 0x0067 #LATIN SMALL LETTER G
0x68 0x0068 #LATIN SMALL LETTER H
0x69 0x0069 #LATIN SMALL LETTER I
0x6A 0x006A #LATIN SMALL LETTER J
0x6B 0x006B #LATIN SMALL LETTER K
0x6C 0x006C #LATIN SMALL LETTER L
0x6D 0x006D #LATIN SMALL LETTER M
0x6E 0x006E #LATIN SMALL LETTER N
0x6F 0x006F #LATIN SMALL LETTER O
0x70 0x0070 #LATIN SMALL LETTER P
0x71 0x0071 #LATIN SMALL LETTER Q
0x72 0x0072 #LATIN SMALL LETTER R
0x73 0x0073 #LATIN SMALL LETTER S
0x74 0x0074 #LATIN SMALL LETTER T
0x75 0x0075 #LATIN SMALL LETTER U
0x76 0x0076 #LATIN SMALL LETTER V
0x77 0x0077 #LATIN SMALL LETTER W
0x78 0x0078 #LATIN SMALL LETTER X
0x79 0x0079 #LATIN SMALL LETTER Y
0x7A 0x007A #LATIN SMALL LETTER Z
0x7B 0x007B #LEFT CURLY BRACKET
0x7C 0x007C #VERTICAL LINE
0x7D 0x007D #RIGHT CURLY BRACKET
0x7E 0x007E #TILDE
0x7F 0x007F #DELETE
0x80 0x20AC #EURO SIGN
0x81 #UNDEFINED
0x82 0x201A #SINGLE LOW-9 QUOTATION MARK
0x83 #UNDEFINED
0x84 0x201E #DOUBLE LOW-9 QUOTATION MARK
0x85 0x2026 #HORIZONTAL ELLIPSIS
0x86 0x2020 #DAGGER
0x87 0x2021 #DOUBLE DAGGER
0x88 #UNDEFINED
0x89 0x2030 #PER MILLE SIGN
0x8A 0x0160 #LATIN CAPITAL LETTER S WITH CARON
0x8B 0x2039 #SINGLE LEFT-POINTING ANGLE QUOTATION MARK
0x8C 0x015A #LATIN CAPITAL LETTER S WITH ACUTE
0x8D 0x0164 #LATIN CAPITAL LETTER T WITH CARON
0x8E 0x017D #LATIN CAPITAL LETTER Z WITH CARON
0x8F 0x0179 #LATIN CAPITAL LETTER Z WITH ACUTE
0x90 #UNDEFINED
0x91 0x2018 #LEFT SINGLE QUOTATION MARK
0x92 0x2019 #RIGHT SINGLE QUOTATION MARK
0x93 0x201C #LEFT DOUBLE QUOTATION MARK
0x94 0x201D #RIGHT DOUBLE QUOTATION MARK
0x95 0x2022 #BULLET
0x96 0x2013 #EN DASH
0x97 0x2014 #EM DASH
0x98 #UNDEFINED
0x99 0x2122 #TRADE MARK SIGN
0x9A 0x0161 #LATIN SMALL LETTER S WITH CARON
0x9B 0x203A #SINGLE RIGHT-POINTING ANGLE QUOTATION MARK
0x9C 0x015B #LATIN SMALL LETTER S WITH ACUTE
0x9D 0x0165 #LATIN SMALL LETTER T WITH CARON
0x9E 0x017E #LATIN SMALL LETTER Z WITH CARON
0x9F 0x017A #LATIN SMALL LETTER Z WITH ACUTE
0xA0 0x00A0 #NO-BREAK SPACE
0xA1 0x02C7 #CARON
0xA2 0x02D8 #BREVE
0xA3 0x0141 #LATIN CAPITAL LETTER L WITH STROKE
0xA4 0x00A4 #CURRENCY SIGN
0xA5 0x0104 #LATIN CAPITAL LETTER A WITH OGONEK
0xA6 0x00A6 #BROKEN BAR
0xA7 0x00A7 #SECTION SIGN
0xA8 0x00A8 #DIAERESIS
0xA9 0x00A9 #COPYRIGHT SIGN
0xAA 0x015E #LATIN CAPITAL LETTER S WITH CEDILLA
0xAB 0x00AB #LEFT-POINTING DOUBLE ANGLE QUOTATION MARK
0xAC 0x00AC #NOT SIGN
0xAD 0x00AD #SOFT HYPHEN
0xAE 0x00AE #REGISTERED SIGN
0xAF 0x017B #LATIN CAPITAL LETTER Z WITH DOT ABOVE
0xB0 0x00B0 #DEGREE SIGN
0xB1 0x00B1 #PLUS-MINUS SIGN
0xB2 0x02DB #OGONEK
0xB3 0x0142 #LATIN SMALL LETTER L WITH STROKE
0xB4 0x00B4 #ACUTE ACCENT
0xB5 0x00B5 #MICRO SIGN
0xB6 0x00B6 #PILCROW SIGN
0xB7 0x00B7 #MIDDLE DOT
0xB8 0x00B8 #CEDILLA
0xB9 0x0105 #LATIN SMALL LETTER A WITH OGONEK
0xBA 0x015F #LATIN SMALL LETTER S WITH CEDILLA
0xBB 0x00BB #RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK
0xBC 0x013D #LATIN CAPITAL LETTER L WITH CARON
0xBD 0x02DD #DOUBLE ACUTE ACCENT
0xBE 0x013E #LATIN SMALL LETTER L WITH CARON
0xBF 0x017C #LATIN SMALL LETTER Z WITH DOT ABOVE
0xC0 0x0154 #LATIN CAPITAL LETTER R WITH ACUTE
0xC1 0x00C1 #LATIN CAPITAL LETTER A WITH ACUTE
0xC2 0x00C2 #LATIN CAPITAL LETTER A WITH CIRCUMFLEX
0xC3 0x0102 #LATIN CAPITAL LETTER A WITH BREVE
0xC4 0x00C4 #LATIN CAPITAL LETTER A WITH DIAERESIS
0xC5 0x0139 #LATIN CAPITAL LETTER L WITH ACUTE
0xC6 0x0106 #LATIN CAPITAL LETTER C WITH ACUTE
0xC7 0x00C7 #LATIN CAPITAL LETTER C WITH CEDILLA
0xC8 0x010C #LATIN CAPITAL LETTER C WITH CARON
0xC9 0x00C9 #LATIN CAPITAL LETTER E WITH ACUTE
0xCA 0x0118 #LATIN CAPITAL LETTER E WITH OGONEK
0xCB 0x00CB #LATIN CAPITAL LETTER E WITH DIAERESIS
0xCC 0x011A #LATIN CAPITAL LETTER E WITH CARON
0xCD 0x00CD #LATIN CAPITAL LETTER I WITH ACUTE
0xCE 0x00CE #LATIN CAPITAL LETTER I WITH CIRCUMFLEX
0xCF 0x010E #LATIN CAPITAL LETTER D WITH CARON
0xD0 0x0110 #LATIN CAPITAL LETTER D WITH STROKE
0xD1 0x0143 #LATIN CAPITAL LETTER N WITH ACUTE
0xD2 0x0147 #LATIN CAPITAL LETTER N WITH CARON
0xD3 0x00D3 #LATIN CAPITAL LETTER O WITH ACUTE
0xD4 0x00D4 #LATIN CAPITAL LETTER O WITH CIRCUMFLEX
0xD5 0x0150 #LATIN CAPITAL LETTER O WITH DOUBLE ACUTE
0xD6 0x00D6 #LATIN CAPITAL LETTER O WITH DIAERESIS
0xD7 0x00D7 #MULTIPLICATION SIGN
0xD8 0x0158 #LATIN CAPITAL LETTER R WITH CARON
0xD9 0x016E #LATIN CAPITAL LETTER U WITH RING ABOVE
0xDA 0x00DA #LATIN CAPITAL LETTER U WITH ACUTE
0xDB 0x0170 #LATIN CAPITAL LETTER U WITH DOUBLE ACUTE
0xDC 0x00DC #LATIN CAPITAL LETTER U WITH DIAERESIS
0xDD 0x00DD #LATIN CAPITAL LETTER Y WITH ACUTE
0xDE 0x0162 #LATIN CAPITAL LETTER T WITH CEDILLA
0xDF 0x00DF #LATIN SMALL LETTER SHARP S
0xE0 0x0155 #LATIN SMALL LETTER R WITH ACUTE
0xE1 0x00E1 #LATIN SMALL LETTER A WITH ACUTE
0xE2 0x00E2 #LATIN SMALL LETTER A WITH CIRCUMFLEX
0xE3 0x0103 #LATIN SMALL LETTER A WITH BREVE
0xE4 0x00E4 #LATIN SMALL LETTER A WITH DIAERESIS
0xE5 0x013A #LATIN SMALL LETTER L WITH ACUTE
0xE6 0x0107 #LATIN SMALL LETTER C WITH ACUTE
0xE7 0x00E7 #LATIN SMALL LETTER C WITH CEDILLA
0xE8 0x010D #LATIN SMALL LETTER C WITH CARON
0xE9 0x00E9 #LATIN SMALL LETTER E WITH ACUTE
0xEA 0x0119 #LATIN SMALL LETTER E WITH OGONEK
0xEB 0x00EB #LATIN SMALL LETTER E WITH DIAERESIS
0xEC 0x011B #LATIN SMALL LETTER E WITH CARON
0xED 0x00ED #LATIN SMALL LETTER I WITH ACUTE
0xEE 0x00EE #LATIN SMALL LETTER I WITH CIRCUMFLEX
0xEF 0x010F #LATIN SMALL LETTER D WITH CARON
0xF0 0x0111 #LATIN SMALL LETTER D WITH STROKE
0xF1 0x0144 #LATIN SMALL LETTER N WITH ACUTE
0xF2 0x0148 #LATIN SMALL LETTER N WITH CARON
0xF3 0x00F3 #LATIN SMALL LETTER O WITH ACUTE
0xF4 0x00F4 #LATIN SMALL LETTER O WITH CIRCUMFLEX
0xF5 0x0151 #LATIN SMALL LETTER O WITH DOUBLE ACUTE
0xF6 0x00F6 #LATIN SMALL LETTER O WITH DIAERESIS
0xF7 0x00F7 #DIVISION SIGN
0xF8 0x0159 #LATIN SMALL LETTER R WITH CARON
0xF9 0x016F #LATIN SMALL LETTER U WITH RING ABOVE
0xFA 0x00FA #LATIN SMALL LETTER U WITH ACUTE
0xFB 0x0171 #LATIN SMALL LETTER U WITH DOUBLE ACUTE
0xFC 0x00FC #LATIN SMALL LETTER U WITH DIAERESIS
0xFD 0x00FD #LATIN SMALL LETTER Y WITH ACUTE
0xFE 0x0163 #LATIN SMALL LETTER T WITH CEDILLA
0xFF 0x02D9 #DOT ABOVE

274
data/windows/CP1251.TXT Normal file
View File

@@ -0,0 +1,274 @@
#
# Name: cp1251 to Unicode table
# Unicode version: 2.0
# Table version: 2.01
# Table format: Format A
# Date: 04/15/98
#
# Contact: Shawn.Steele@microsoft.com
#
# General notes: none
#
# Format: Three tab-separated columns
# Column #1 is the cp1251 code (in hex)
# Column #2 is the Unicode (in hex as 0xXXXX)
# Column #3 is the Unicode name (follows a comment sign, '#')
#
# The entries are in cp1251 order
#
0x00 0x0000 #NULL
0x01 0x0001 #START OF HEADING
0x02 0x0002 #START OF TEXT
0x03 0x0003 #END OF TEXT
0x04 0x0004 #END OF TRANSMISSION
0x05 0x0005 #ENQUIRY
0x06 0x0006 #ACKNOWLEDGE
0x07 0x0007 #BELL
0x08 0x0008 #BACKSPACE
0x09 0x0009 #HORIZONTAL TABULATION
0x0A 0x000A #LINE FEED
0x0B 0x000B #VERTICAL TABULATION
0x0C 0x000C #FORM FEED
0x0D 0x000D #CARRIAGE RETURN
0x0E 0x000E #SHIFT OUT
0x0F 0x000F #SHIFT IN
0x10 0x0010 #DATA LINK ESCAPE
0x11 0x0011 #DEVICE CONTROL ONE
0x12 0x0012 #DEVICE CONTROL TWO
0x13 0x0013 #DEVICE CONTROL THREE
0x14 0x0014 #DEVICE CONTROL FOUR
0x15 0x0015 #NEGATIVE ACKNOWLEDGE
0x16 0x0016 #SYNCHRONOUS IDLE
0x17 0x0017 #END OF TRANSMISSION BLOCK
0x18 0x0018 #CANCEL
0x19 0x0019 #END OF MEDIUM
0x1A 0x001A #SUBSTITUTE
0x1B 0x001B #ESCAPE
0x1C 0x001C #FILE SEPARATOR
0x1D 0x001D #GROUP SEPARATOR
0x1E 0x001E #RECORD SEPARATOR
0x1F 0x001F #UNIT SEPARATOR
0x20 0x0020 #SPACE
0x21 0x0021 #EXCLAMATION MARK
0x22 0x0022 #QUOTATION MARK
0x23 0x0023 #NUMBER SIGN
0x24 0x0024 #DOLLAR SIGN
0x25 0x0025 #PERCENT SIGN
0x26 0x0026 #AMPERSAND
0x27 0x0027 #APOSTROPHE
0x28 0x0028 #LEFT PARENTHESIS
0x29 0x0029 #RIGHT PARENTHESIS
0x2A 0x002A #ASTERISK
0x2B 0x002B #PLUS SIGN
0x2C 0x002C #COMMA
0x2D 0x002D #HYPHEN-MINUS
0x2E 0x002E #FULL STOP
0x2F 0x002F #SOLIDUS
0x30 0x0030 #DIGIT ZERO
0x31 0x0031 #DIGIT ONE
0x32 0x0032 #DIGIT TWO
0x33 0x0033 #DIGIT THREE
0x34 0x0034 #DIGIT FOUR
0x35 0x0035 #DIGIT FIVE
0x36 0x0036 #DIGIT SIX
0x37 0x0037 #DIGIT SEVEN
0x38 0x0038 #DIGIT EIGHT
0x39 0x0039 #DIGIT NINE
0x3A 0x003A #COLON
0x3B 0x003B #SEMICOLON
0x3C 0x003C #LESS-THAN SIGN
0x3D 0x003D #EQUALS SIGN
0x3E 0x003E #GREATER-THAN SIGN
0x3F 0x003F #QUESTION MARK
0x40 0x0040 #COMMERCIAL AT
0x41 0x0041 #LATIN CAPITAL LETTER A
0x42 0x0042 #LATIN CAPITAL LETTER B
0x43 0x0043 #LATIN CAPITAL LETTER C
0x44 0x0044 #LATIN CAPITAL LETTER D
0x45 0x0045 #LATIN CAPITAL LETTER E
0x46 0x0046 #LATIN CAPITAL LETTER F
0x47 0x0047 #LATIN CAPITAL LETTER G
0x48 0x0048 #LATIN CAPITAL LETTER H
0x49 0x0049 #LATIN CAPITAL LETTER I
0x4A 0x004A #LATIN CAPITAL LETTER J
0x4B 0x004B #LATIN CAPITAL LETTER K
0x4C 0x004C #LATIN CAPITAL LETTER L
0x4D 0x004D #LATIN CAPITAL LETTER M
0x4E 0x004E #LATIN CAPITAL LETTER N
0x4F 0x004F #LATIN CAPITAL LETTER O
0x50 0x0050 #LATIN CAPITAL LETTER P
0x51 0x0051 #LATIN CAPITAL LETTER Q
0x52 0x0052 #LATIN CAPITAL LETTER R
0x53 0x0053 #LATIN CAPITAL LETTER S
0x54 0x0054 #LATIN CAPITAL LETTER T
0x55 0x0055 #LATIN CAPITAL LETTER U
0x56 0x0056 #LATIN CAPITAL LETTER V
0x57 0x0057 #LATIN CAPITAL LETTER W
0x58 0x0058 #LATIN CAPITAL LETTER X
0x59 0x0059 #LATIN CAPITAL LETTER Y
0x5A 0x005A #LATIN CAPITAL LETTER Z
0x5B 0x005B #LEFT SQUARE BRACKET
0x5C 0x005C #REVERSE SOLIDUS
0x5D 0x005D #RIGHT SQUARE BRACKET
0x5E 0x005E #CIRCUMFLEX ACCENT
0x5F 0x005F #LOW LINE
0x60 0x0060 #GRAVE ACCENT
0x61 0x0061 #LATIN SMALL LETTER A
0x62 0x0062 #LATIN SMALL LETTER B
0x63 0x0063 #LATIN SMALL LETTER C
0x64 0x0064 #LATIN SMALL LETTER D
0x65 0x0065 #LATIN SMALL LETTER E
0x66 0x0066 #LATIN SMALL LETTER F
0x67 0x0067 #LATIN SMALL LETTER G
0x68 0x0068 #LATIN SMALL LETTER H
0x69 0x0069 #LATIN SMALL LETTER I
0x6A 0x006A #LATIN SMALL LETTER J
0x6B 0x006B #LATIN SMALL LETTER K
0x6C 0x006C #LATIN SMALL LETTER L
0x6D 0x006D #LATIN SMALL LETTER M
0x6E 0x006E #LATIN SMALL LETTER N
0x6F 0x006F #LATIN SMALL LETTER O
0x70 0x0070 #LATIN SMALL LETTER P
0x71 0x0071 #LATIN SMALL LETTER Q
0x72 0x0072 #LATIN SMALL LETTER R
0x73 0x0073 #LATIN SMALL LETTER S
0x74 0x0074 #LATIN SMALL LETTER T
0x75 0x0075 #LATIN SMALL LETTER U
0x76 0x0076 #LATIN SMALL LETTER V
0x77 0x0077 #LATIN SMALL LETTER W
0x78 0x0078 #LATIN SMALL LETTER X
0x79 0x0079 #LATIN SMALL LETTER Y
0x7A 0x007A #LATIN SMALL LETTER Z
0x7B 0x007B #LEFT CURLY BRACKET
0x7C 0x007C #VERTICAL LINE
0x7D 0x007D #RIGHT CURLY BRACKET
0x7E 0x007E #TILDE
0x7F 0x007F #DELETE
0x80 0x0402 #CYRILLIC CAPITAL LETTER DJE
0x81 0x0403 #CYRILLIC CAPITAL LETTER GJE
0x82 0x201A #SINGLE LOW-9 QUOTATION MARK
0x83 0x0453 #CYRILLIC SMALL LETTER GJE
0x84 0x201E #DOUBLE LOW-9 QUOTATION MARK
0x85 0x2026 #HORIZONTAL ELLIPSIS
0x86 0x2020 #DAGGER
0x87 0x2021 #DOUBLE DAGGER
0x88 0x20AC #EURO SIGN
0x89 0x2030 #PER MILLE SIGN
0x8A 0x0409 #CYRILLIC CAPITAL LETTER LJE
0x8B 0x2039 #SINGLE LEFT-POINTING ANGLE QUOTATION MARK
0x8C 0x040A #CYRILLIC CAPITAL LETTER NJE
0x8D 0x040C #CYRILLIC CAPITAL LETTER KJE
0x8E 0x040B #CYRILLIC CAPITAL LETTER TSHE
0x8F 0x040F #CYRILLIC CAPITAL LETTER DZHE
0x90 0x0452 #CYRILLIC SMALL LETTER DJE
0x91 0x2018 #LEFT SINGLE QUOTATION MARK
0x92 0x2019 #RIGHT SINGLE QUOTATION MARK
0x93 0x201C #LEFT DOUBLE QUOTATION MARK
0x94 0x201D #RIGHT DOUBLE QUOTATION MARK
0x95 0x2022 #BULLET
0x96 0x2013 #EN DASH
0x97 0x2014 #EM DASH
0x98 #UNDEFINED
0x99 0x2122 #TRADE MARK SIGN
0x9A 0x0459 #CYRILLIC SMALL LETTER LJE
0x9B 0x203A #SINGLE RIGHT-POINTING ANGLE QUOTATION MARK
0x9C 0x045A #CYRILLIC SMALL LETTER NJE
0x9D 0x045C #CYRILLIC SMALL LETTER KJE
0x9E 0x045B #CYRILLIC SMALL LETTER TSHE
0x9F 0x045F #CYRILLIC SMALL LETTER DZHE
0xA0 0x00A0 #NO-BREAK SPACE
0xA1 0x040E #CYRILLIC CAPITAL LETTER SHORT U
0xA2 0x045E #CYRILLIC SMALL LETTER SHORT U
0xA3 0x0408 #CYRILLIC CAPITAL LETTER JE
0xA4 0x00A4 #CURRENCY SIGN
0xA5 0x0490 #CYRILLIC CAPITAL LETTER GHE WITH UPTURN
0xA6 0x00A6 #BROKEN BAR
0xA7 0x00A7 #SECTION SIGN
0xA8 0x0401 #CYRILLIC CAPITAL LETTER IO
0xA9 0x00A9 #COPYRIGHT SIGN
0xAA 0x0404 #CYRILLIC CAPITAL LETTER UKRAINIAN IE
0xAB 0x00AB #LEFT-POINTING DOUBLE ANGLE QUOTATION MARK
0xAC 0x00AC #NOT SIGN
0xAD 0x00AD #SOFT HYPHEN
0xAE 0x00AE #REGISTERED SIGN
0xAF 0x0407 #CYRILLIC CAPITAL LETTER YI
0xB0 0x00B0 #DEGREE SIGN
0xB1 0x00B1 #PLUS-MINUS SIGN
0xB2 0x0406 #CYRILLIC CAPITAL LETTER BYELORUSSIAN-UKRAINIAN I
0xB3 0x0456 #CYRILLIC SMALL LETTER BYELORUSSIAN-UKRAINIAN I
0xB4 0x0491 #CYRILLIC SMALL LETTER GHE WITH UPTURN
0xB5 0x00B5 #MICRO SIGN
0xB6 0x00B6 #PILCROW SIGN
0xB7 0x00B7 #MIDDLE DOT
0xB8 0x0451 #CYRILLIC SMALL LETTER IO
0xB9 0x2116 #NUMERO SIGN
0xBA 0x0454 #CYRILLIC SMALL LETTER UKRAINIAN IE
0xBB 0x00BB #RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK
0xBC 0x0458 #CYRILLIC SMALL LETTER JE
0xBD 0x0405 #CYRILLIC CAPITAL LETTER DZE
0xBE 0x0455 #CYRILLIC SMALL LETTER DZE
0xBF 0x0457 #CYRILLIC SMALL LETTER YI
0xC0 0x0410 #CYRILLIC CAPITAL LETTER A
0xC1 0x0411 #CYRILLIC CAPITAL LETTER BE
0xC2 0x0412 #CYRILLIC CAPITAL LETTER VE
0xC3 0x0413 #CYRILLIC CAPITAL LETTER GHE
0xC4 0x0414 #CYRILLIC CAPITAL LETTER DE
0xC5 0x0415 #CYRILLIC CAPITAL LETTER IE
0xC6 0x0416 #CYRILLIC CAPITAL LETTER ZHE
0xC7 0x0417 #CYRILLIC CAPITAL LETTER ZE
0xC8 0x0418 #CYRILLIC CAPITAL LETTER I
0xC9 0x0419 #CYRILLIC CAPITAL LETTER SHORT I
0xCA 0x041A #CYRILLIC CAPITAL LETTER KA
0xCB 0x041B #CYRILLIC CAPITAL LETTER EL
0xCC 0x041C #CYRILLIC CAPITAL LETTER EM
0xCD 0x041D #CYRILLIC CAPITAL LETTER EN
0xCE 0x041E #CYRILLIC CAPITAL LETTER O
0xCF 0x041F #CYRILLIC CAPITAL LETTER PE
0xD0 0x0420 #CYRILLIC CAPITAL LETTER ER
0xD1 0x0421 #CYRILLIC CAPITAL LETTER ES
0xD2 0x0422 #CYRILLIC CAPITAL LETTER TE
0xD3 0x0423 #CYRILLIC CAPITAL LETTER U
0xD4 0x0424 #CYRILLIC CAPITAL LETTER EF
0xD5 0x0425 #CYRILLIC CAPITAL LETTER HA
0xD6 0x0426 #CYRILLIC CAPITAL LETTER TSE
0xD7 0x0427 #CYRILLIC CAPITAL LETTER CHE
0xD8 0x0428 #CYRILLIC CAPITAL LETTER SHA
0xD9 0x0429 #CYRILLIC CAPITAL LETTER SHCHA
0xDA 0x042A #CYRILLIC CAPITAL LETTER HARD SIGN
0xDB 0x042B #CYRILLIC CAPITAL LETTER YERU
0xDC 0x042C #CYRILLIC CAPITAL LETTER SOFT SIGN
0xDD 0x042D #CYRILLIC CAPITAL LETTER E
0xDE 0x042E #CYRILLIC CAPITAL LETTER YU
0xDF 0x042F #CYRILLIC CAPITAL LETTER YA
0xE0 0x0430 #CYRILLIC SMALL LETTER A
0xE1 0x0431 #CYRILLIC SMALL LETTER BE
0xE2 0x0432 #CYRILLIC SMALL LETTER VE
0xE3 0x0433 #CYRILLIC SMALL LETTER GHE
0xE4 0x0434 #CYRILLIC SMALL LETTER DE
0xE5 0x0435 #CYRILLIC SMALL LETTER IE
0xE6 0x0436 #CYRILLIC SMALL LETTER ZHE
0xE7 0x0437 #CYRILLIC SMALL LETTER ZE
0xE8 0x0438 #CYRILLIC SMALL LETTER I
0xE9 0x0439 #CYRILLIC SMALL LETTER SHORT I
0xEA 0x043A #CYRILLIC SMALL LETTER KA
0xEB 0x043B #CYRILLIC SMALL LETTER EL
0xEC 0x043C #CYRILLIC SMALL LETTER EM
0xED 0x043D #CYRILLIC SMALL LETTER EN
0xEE 0x043E #CYRILLIC SMALL LETTER O
0xEF 0x043F #CYRILLIC SMALL LETTER PE
0xF0 0x0440 #CYRILLIC SMALL LETTER ER
0xF1 0x0441 #CYRILLIC SMALL LETTER ES
0xF2 0x0442 #CYRILLIC SMALL LETTER TE
0xF3 0x0443 #CYRILLIC SMALL LETTER U
0xF4 0x0444 #CYRILLIC SMALL LETTER EF
0xF5 0x0445 #CYRILLIC SMALL LETTER HA
0xF6 0x0446 #CYRILLIC SMALL LETTER TSE
0xF7 0x0447 #CYRILLIC SMALL LETTER CHE
0xF8 0x0448 #CYRILLIC SMALL LETTER SHA
0xF9 0x0449 #CYRILLIC SMALL LETTER SHCHA
0xFA 0x044A #CYRILLIC SMALL LETTER HARD SIGN
0xFB 0x044B #CYRILLIC SMALL LETTER YERU
0xFC 0x044C #CYRILLIC SMALL LETTER SOFT SIGN
0xFD 0x044D #CYRILLIC SMALL LETTER E
0xFE 0x044E #CYRILLIC SMALL LETTER YU
0xFF 0x044F #CYRILLIC SMALL LETTER YA

274
data/windows/CP1252.TXT Normal file
View File

@@ -0,0 +1,274 @@
#
# Name: cp1252 to Unicode table
# Unicode version: 2.0
# Table version: 2.01
# Table format: Format A
# Date: 04/15/98
#
# Contact: Shawn.Steele@microsoft.com
#
# General notes: none
#
# Format: Three tab-separated columns
# Column #1 is the cp1252 code (in hex)
# Column #2 is the Unicode (in hex as 0xXXXX)
# Column #3 is the Unicode name (follows a comment sign, '#')
#
# The entries are in cp1252 order
#
0x00 0x0000 #NULL
0x01 0x0001 #START OF HEADING
0x02 0x0002 #START OF TEXT
0x03 0x0003 #END OF TEXT
0x04 0x0004 #END OF TRANSMISSION
0x05 0x0005 #ENQUIRY
0x06 0x0006 #ACKNOWLEDGE
0x07 0x0007 #BELL
0x08 0x0008 #BACKSPACE
0x09 0x0009 #HORIZONTAL TABULATION
0x0A 0x000A #LINE FEED
0x0B 0x000B #VERTICAL TABULATION
0x0C 0x000C #FORM FEED
0x0D 0x000D #CARRIAGE RETURN
0x0E 0x000E #SHIFT OUT
0x0F 0x000F #SHIFT IN
0x10 0x0010 #DATA LINK ESCAPE
0x11 0x0011 #DEVICE CONTROL ONE
0x12 0x0012 #DEVICE CONTROL TWO
0x13 0x0013 #DEVICE CONTROL THREE
0x14 0x0014 #DEVICE CONTROL FOUR
0x15 0x0015 #NEGATIVE ACKNOWLEDGE
0x16 0x0016 #SYNCHRONOUS IDLE
0x17 0x0017 #END OF TRANSMISSION BLOCK
0x18 0x0018 #CANCEL
0x19 0x0019 #END OF MEDIUM
0x1A 0x001A #SUBSTITUTE
0x1B 0x001B #ESCAPE
0x1C 0x001C #FILE SEPARATOR
0x1D 0x001D #GROUP SEPARATOR
0x1E 0x001E #RECORD SEPARATOR
0x1F 0x001F #UNIT SEPARATOR
0x20 0x0020 #SPACE
0x21 0x0021 #EXCLAMATION MARK
0x22 0x0022 #QUOTATION MARK
0x23 0x0023 #NUMBER SIGN
0x24 0x0024 #DOLLAR SIGN
0x25 0x0025 #PERCENT SIGN
0x26 0x0026 #AMPERSAND
0x27 0x0027 #APOSTROPHE
0x28 0x0028 #LEFT PARENTHESIS
0x29 0x0029 #RIGHT PARENTHESIS
0x2A 0x002A #ASTERISK
0x2B 0x002B #PLUS SIGN
0x2C 0x002C #COMMA
0x2D 0x002D #HYPHEN-MINUS
0x2E 0x002E #FULL STOP
0x2F 0x002F #SOLIDUS
0x30 0x0030 #DIGIT ZERO
0x31 0x0031 #DIGIT ONE
0x32 0x0032 #DIGIT TWO
0x33 0x0033 #DIGIT THREE
0x34 0x0034 #DIGIT FOUR
0x35 0x0035 #DIGIT FIVE
0x36 0x0036 #DIGIT SIX
0x37 0x0037 #DIGIT SEVEN
0x38 0x0038 #DIGIT EIGHT
0x39 0x0039 #DIGIT NINE
0x3A 0x003A #COLON
0x3B 0x003B #SEMICOLON
0x3C 0x003C #LESS-THAN SIGN
0x3D 0x003D #EQUALS SIGN
0x3E 0x003E #GREATER-THAN SIGN
0x3F 0x003F #QUESTION MARK
0x40 0x0040 #COMMERCIAL AT
0x41 0x0041 #LATIN CAPITAL LETTER A
0x42 0x0042 #LATIN CAPITAL LETTER B
0x43 0x0043 #LATIN CAPITAL LETTER C
0x44 0x0044 #LATIN CAPITAL LETTER D
0x45 0x0045 #LATIN CAPITAL LETTER E
0x46 0x0046 #LATIN CAPITAL LETTER F
0x47 0x0047 #LATIN CAPITAL LETTER G
0x48 0x0048 #LATIN CAPITAL LETTER H
0x49 0x0049 #LATIN CAPITAL LETTER I
0x4A 0x004A #LATIN CAPITAL LETTER J
0x4B 0x004B #LATIN CAPITAL LETTER K
0x4C 0x004C #LATIN CAPITAL LETTER L
0x4D 0x004D #LATIN CAPITAL LETTER M
0x4E 0x004E #LATIN CAPITAL LETTER N
0x4F 0x004F #LATIN CAPITAL LETTER O
0x50 0x0050 #LATIN CAPITAL LETTER P
0x51 0x0051 #LATIN CAPITAL LETTER Q
0x52 0x0052 #LATIN CAPITAL LETTER R
0x53 0x0053 #LATIN CAPITAL LETTER S
0x54 0x0054 #LATIN CAPITAL LETTER T
0x55 0x0055 #LATIN CAPITAL LETTER U
0x56 0x0056 #LATIN CAPITAL LETTER V
0x57 0x0057 #LATIN CAPITAL LETTER W
0x58 0x0058 #LATIN CAPITAL LETTER X
0x59 0x0059 #LATIN CAPITAL LETTER Y
0x5A 0x005A #LATIN CAPITAL LETTER Z
0x5B 0x005B #LEFT SQUARE BRACKET
0x5C 0x005C #REVERSE SOLIDUS
0x5D 0x005D #RIGHT SQUARE BRACKET
0x5E 0x005E #CIRCUMFLEX ACCENT
0x5F 0x005F #LOW LINE
0x60 0x0060 #GRAVE ACCENT
0x61 0x0061 #LATIN SMALL LETTER A
0x62 0x0062 #LATIN SMALL LETTER B
0x63 0x0063 #LATIN SMALL LETTER C
0x64 0x0064 #LATIN SMALL LETTER D
0x65 0x0065 #LATIN SMALL LETTER E
0x66 0x0066 #LATIN SMALL LETTER F
0x67 0x0067 #LATIN SMALL LETTER G
0x68 0x0068 #LATIN SMALL LETTER H
0x69 0x0069 #LATIN SMALL LETTER I
0x6A 0x006A #LATIN SMALL LETTER J
0x6B 0x006B #LATIN SMALL LETTER K
0x6C 0x006C #LATIN SMALL LETTER L
0x6D 0x006D #LATIN SMALL LETTER M
0x6E 0x006E #LATIN SMALL LETTER N
0x6F 0x006F #LATIN SMALL LETTER O
0x70 0x0070 #LATIN SMALL LETTER P
0x71 0x0071 #LATIN SMALL LETTER Q
0x72 0x0072 #LATIN SMALL LETTER R
0x73 0x0073 #LATIN SMALL LETTER S
0x74 0x0074 #LATIN SMALL LETTER T
0x75 0x0075 #LATIN SMALL LETTER U
0x76 0x0076 #LATIN SMALL LETTER V
0x77 0x0077 #LATIN SMALL LETTER W
0x78 0x0078 #LATIN SMALL LETTER X
0x79 0x0079 #LATIN SMALL LETTER Y
0x7A 0x007A #LATIN SMALL LETTER Z
0x7B 0x007B #LEFT CURLY BRACKET
0x7C 0x007C #VERTICAL LINE
0x7D 0x007D #RIGHT CURLY BRACKET
0x7E 0x007E #TILDE
0x7F 0x007F #DELETE
0x80 0x20AC #EURO SIGN
0x81 #UNDEFINED
0x82 0x201A #SINGLE LOW-9 QUOTATION MARK
0x83 0x0192 #LATIN SMALL LETTER F WITH HOOK
0x84 0x201E #DOUBLE LOW-9 QUOTATION MARK
0x85 0x2026 #HORIZONTAL ELLIPSIS
0x86 0x2020 #DAGGER
0x87 0x2021 #DOUBLE DAGGER
0x88 0x02C6 #MODIFIER LETTER CIRCUMFLEX ACCENT
0x89 0x2030 #PER MILLE SIGN
0x8A 0x0160 #LATIN CAPITAL LETTER S WITH CARON
0x8B 0x2039 #SINGLE LEFT-POINTING ANGLE QUOTATION MARK
0x8C 0x0152 #LATIN CAPITAL LIGATURE OE
0x8D #UNDEFINED
0x8E 0x017D #LATIN CAPITAL LETTER Z WITH CARON
0x8F #UNDEFINED
0x90 #UNDEFINED
0x91 0x2018 #LEFT SINGLE QUOTATION MARK
0x92 0x2019 #RIGHT SINGLE QUOTATION MARK
0x93 0x201C #LEFT DOUBLE QUOTATION MARK
0x94 0x201D #RIGHT DOUBLE QUOTATION MARK
0x95 0x2022 #BULLET
0x96 0x2013 #EN DASH
0x97 0x2014 #EM DASH
0x98 0x02DC #SMALL TILDE
0x99 0x2122 #TRADE MARK SIGN
0x9A 0x0161 #LATIN SMALL LETTER S WITH CARON
0x9B 0x203A #SINGLE RIGHT-POINTING ANGLE QUOTATION MARK
0x9C 0x0153 #LATIN SMALL LIGATURE OE
0x9D #UNDEFINED
0x9E 0x017E #LATIN SMALL LETTER Z WITH CARON
0x9F 0x0178 #LATIN CAPITAL LETTER Y WITH DIAERESIS
0xA0 0x00A0 #NO-BREAK SPACE
0xA1 0x00A1 #INVERTED EXCLAMATION MARK
0xA2 0x00A2 #CENT SIGN
0xA3 0x00A3 #POUND SIGN
0xA4 0x00A4 #CURRENCY SIGN
0xA5 0x00A5 #YEN SIGN
0xA6 0x00A6 #BROKEN BAR
0xA7 0x00A7 #SECTION SIGN
0xA8 0x00A8 #DIAERESIS
0xA9 0x00A9 #COPYRIGHT SIGN
0xAA 0x00AA #FEMININE ORDINAL INDICATOR
0xAB 0x00AB #LEFT-POINTING DOUBLE ANGLE QUOTATION MARK
0xAC 0x00AC #NOT SIGN
0xAD 0x00AD #SOFT HYPHEN
0xAE 0x00AE #REGISTERED SIGN
0xAF 0x00AF #MACRON
0xB0 0x00B0 #DEGREE SIGN
0xB1 0x00B1 #PLUS-MINUS SIGN
0xB2 0x00B2 #SUPERSCRIPT TWO
0xB3 0x00B3 #SUPERSCRIPT THREE
0xB4 0x00B4 #ACUTE ACCENT
0xB5 0x00B5 #MICRO SIGN
0xB6 0x00B6 #PILCROW SIGN
0xB7 0x00B7 #MIDDLE DOT
0xB8 0x00B8 #CEDILLA
0xB9 0x00B9 #SUPERSCRIPT ONE
0xBA 0x00BA #MASCULINE ORDINAL INDICATOR
0xBB 0x00BB #RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK
0xBC 0x00BC #VULGAR FRACTION ONE QUARTER
0xBD 0x00BD #VULGAR FRACTION ONE HALF
0xBE 0x00BE #VULGAR FRACTION THREE QUARTERS
0xBF 0x00BF #INVERTED QUESTION MARK
0xC0 0x00C0 #LATIN CAPITAL LETTER A WITH GRAVE
0xC1 0x00C1 #LATIN CAPITAL LETTER A WITH ACUTE
0xC2 0x00C2 #LATIN CAPITAL LETTER A WITH CIRCUMFLEX
0xC3 0x00C3 #LATIN CAPITAL LETTER A WITH TILDE
0xC4 0x00C4 #LATIN CAPITAL LETTER A WITH DIAERESIS
0xC5 0x00C5 #LATIN CAPITAL LETTER A WITH RING ABOVE
0xC6 0x00C6 #LATIN CAPITAL LETTER AE
0xC7 0x00C7 #LATIN CAPITAL LETTER C WITH CEDILLA
0xC8 0x00C8 #LATIN CAPITAL LETTER E WITH GRAVE
0xC9 0x00C9 #LATIN CAPITAL LETTER E WITH ACUTE
0xCA 0x00CA #LATIN CAPITAL LETTER E WITH CIRCUMFLEX
0xCB 0x00CB #LATIN CAPITAL LETTER E WITH DIAERESIS
0xCC 0x00CC #LATIN CAPITAL LETTER I WITH GRAVE
0xCD 0x00CD #LATIN CAPITAL LETTER I WITH ACUTE
0xCE 0x00CE #LATIN CAPITAL LETTER I WITH CIRCUMFLEX
0xCF 0x00CF #LATIN CAPITAL LETTER I WITH DIAERESIS
0xD0 0x00D0 #LATIN CAPITAL LETTER ETH
0xD1 0x00D1 #LATIN CAPITAL LETTER N WITH TILDE
0xD2 0x00D2 #LATIN CAPITAL LETTER O WITH GRAVE
0xD3 0x00D3 #LATIN CAPITAL LETTER O WITH ACUTE
0xD4 0x00D4 #LATIN CAPITAL LETTER O WITH CIRCUMFLEX
0xD5 0x00D5 #LATIN CAPITAL LETTER O WITH TILDE
0xD6 0x00D6 #LATIN CAPITAL LETTER O WITH DIAERESIS
0xD7 0x00D7 #MULTIPLICATION SIGN
0xD8 0x00D8 #LATIN CAPITAL LETTER O WITH STROKE
0xD9 0x00D9 #LATIN CAPITAL LETTER U WITH GRAVE
0xDA 0x00DA #LATIN CAPITAL LETTER U WITH ACUTE
0xDB 0x00DB #LATIN CAPITAL LETTER U WITH CIRCUMFLEX
0xDC 0x00DC #LATIN CAPITAL LETTER U WITH DIAERESIS
0xDD 0x00DD #LATIN CAPITAL LETTER Y WITH ACUTE
0xDE 0x00DE #LATIN CAPITAL LETTER THORN
0xDF 0x00DF #LATIN SMALL LETTER SHARP S
0xE0 0x00E0 #LATIN SMALL LETTER A WITH GRAVE
0xE1 0x00E1 #LATIN SMALL LETTER A WITH ACUTE
0xE2 0x00E2 #LATIN SMALL LETTER A WITH CIRCUMFLEX
0xE3 0x00E3 #LATIN SMALL LETTER A WITH TILDE
0xE4 0x00E4 #LATIN SMALL LETTER A WITH DIAERESIS
0xE5 0x00E5 #LATIN SMALL LETTER A WITH RING ABOVE
0xE6 0x00E6 #LATIN SMALL LETTER AE
0xE7 0x00E7 #LATIN SMALL LETTER C WITH CEDILLA
0xE8 0x00E8 #LATIN SMALL LETTER E WITH GRAVE
0xE9 0x00E9 #LATIN SMALL LETTER E WITH ACUTE
0xEA 0x00EA #LATIN SMALL LETTER E WITH CIRCUMFLEX
0xEB 0x00EB #LATIN SMALL LETTER E WITH DIAERESIS
0xEC 0x00EC #LATIN SMALL LETTER I WITH GRAVE
0xED 0x00ED #LATIN SMALL LETTER I WITH ACUTE
0xEE 0x00EE #LATIN SMALL LETTER I WITH CIRCUMFLEX
0xEF 0x00EF #LATIN SMALL LETTER I WITH DIAERESIS
0xF0 0x00F0 #LATIN SMALL LETTER ETH
0xF1 0x00F1 #LATIN SMALL LETTER N WITH TILDE
0xF2 0x00F2 #LATIN SMALL LETTER O WITH GRAVE
0xF3 0x00F3 #LATIN SMALL LETTER O WITH ACUTE
0xF4 0x00F4 #LATIN SMALL LETTER O WITH CIRCUMFLEX
0xF5 0x00F5 #LATIN SMALL LETTER O WITH TILDE
0xF6 0x00F6 #LATIN SMALL LETTER O WITH DIAERESIS
0xF7 0x00F7 #DIVISION SIGN
0xF8 0x00F8 #LATIN SMALL LETTER O WITH STROKE
0xF9 0x00F9 #LATIN SMALL LETTER U WITH GRAVE
0xFA 0x00FA #LATIN SMALL LETTER U WITH ACUTE
0xFB 0x00FB #LATIN SMALL LETTER U WITH CIRCUMFLEX
0xFC 0x00FC #LATIN SMALL LETTER U WITH DIAERESIS
0xFD 0x00FD #LATIN SMALL LETTER Y WITH ACUTE
0xFE 0x00FE #LATIN SMALL LETTER THORN
0xFF 0x00FF #LATIN SMALL LETTER Y WITH DIAERESIS

274
data/windows/CP1253.TXT Normal file
View File

@@ -0,0 +1,274 @@
#
# Name: cp1253 to Unicode table
# Unicode version: 2.0
# Table version: 2.01
# Table format: Format A
# Date: 04/15/98
#
# Contact: Shawn.Steele@microsoft.com
#
# General notes: none
#
# Format: Three tab-separated columns
# Column #1 is the cp1253 code (in hex)
# Column #2 is the Unicode (in hex as 0xXXXX)
# Column #3 is the Unicode name (follows a comment sign, '#')
#
# The entries are in cp1253 order
#
0x00 0x0000 #NULL
0x01 0x0001 #START OF HEADING
0x02 0x0002 #START OF TEXT
0x03 0x0003 #END OF TEXT
0x04 0x0004 #END OF TRANSMISSION
0x05 0x0005 #ENQUIRY
0x06 0x0006 #ACKNOWLEDGE
0x07 0x0007 #BELL
0x08 0x0008 #BACKSPACE
0x09 0x0009 #HORIZONTAL TABULATION
0x0A 0x000A #LINE FEED
0x0B 0x000B #VERTICAL TABULATION
0x0C 0x000C #FORM FEED
0x0D 0x000D #CARRIAGE RETURN
0x0E 0x000E #SHIFT OUT
0x0F 0x000F #SHIFT IN
0x10 0x0010 #DATA LINK ESCAPE
0x11 0x0011 #DEVICE CONTROL ONE
0x12 0x0012 #DEVICE CONTROL TWO
0x13 0x0013 #DEVICE CONTROL THREE
0x14 0x0014 #DEVICE CONTROL FOUR
0x15 0x0015 #NEGATIVE ACKNOWLEDGE
0x16 0x0016 #SYNCHRONOUS IDLE
0x17 0x0017 #END OF TRANSMISSION BLOCK
0x18 0x0018 #CANCEL
0x19 0x0019 #END OF MEDIUM
0x1A 0x001A #SUBSTITUTE
0x1B 0x001B #ESCAPE
0x1C 0x001C #FILE SEPARATOR
0x1D 0x001D #GROUP SEPARATOR
0x1E 0x001E #RECORD SEPARATOR
0x1F 0x001F #UNIT SEPARATOR
0x20 0x0020 #SPACE
0x21 0x0021 #EXCLAMATION MARK
0x22 0x0022 #QUOTATION MARK
0x23 0x0023 #NUMBER SIGN
0x24 0x0024 #DOLLAR SIGN
0x25 0x0025 #PERCENT SIGN
0x26 0x0026 #AMPERSAND
0x27 0x0027 #APOSTROPHE
0x28 0x0028 #LEFT PARENTHESIS
0x29 0x0029 #RIGHT PARENTHESIS
0x2A 0x002A #ASTERISK
0x2B 0x002B #PLUS SIGN
0x2C 0x002C #COMMA
0x2D 0x002D #HYPHEN-MINUS
0x2E 0x002E #FULL STOP
0x2F 0x002F #SOLIDUS
0x30 0x0030 #DIGIT ZERO
0x31 0x0031 #DIGIT ONE
0x32 0x0032 #DIGIT TWO
0x33 0x0033 #DIGIT THREE
0x34 0x0034 #DIGIT FOUR
0x35 0x0035 #DIGIT FIVE
0x36 0x0036 #DIGIT SIX
0x37 0x0037 #DIGIT SEVEN
0x38 0x0038 #DIGIT EIGHT
0x39 0x0039 #DIGIT NINE
0x3A 0x003A #COLON
0x3B 0x003B #SEMICOLON
0x3C 0x003C #LESS-THAN SIGN
0x3D 0x003D #EQUALS SIGN
0x3E 0x003E #GREATER-THAN SIGN
0x3F 0x003F #QUESTION MARK
0x40 0x0040 #COMMERCIAL AT
0x41 0x0041 #LATIN CAPITAL LETTER A
0x42 0x0042 #LATIN CAPITAL LETTER B
0x43 0x0043 #LATIN CAPITAL LETTER C
0x44 0x0044 #LATIN CAPITAL LETTER D
0x45 0x0045 #LATIN CAPITAL LETTER E
0x46 0x0046 #LATIN CAPITAL LETTER F
0x47 0x0047 #LATIN CAPITAL LETTER G
0x48 0x0048 #LATIN CAPITAL LETTER H
0x49 0x0049 #LATIN CAPITAL LETTER I
0x4A 0x004A #LATIN CAPITAL LETTER J
0x4B 0x004B #LATIN CAPITAL LETTER K
0x4C 0x004C #LATIN CAPITAL LETTER L
0x4D 0x004D #LATIN CAPITAL LETTER M
0x4E 0x004E #LATIN CAPITAL LETTER N
0x4F 0x004F #LATIN CAPITAL LETTER O
0x50 0x0050 #LATIN CAPITAL LETTER P
0x51 0x0051 #LATIN CAPITAL LETTER Q
0x52 0x0052 #LATIN CAPITAL LETTER R
0x53 0x0053 #LATIN CAPITAL LETTER S
0x54 0x0054 #LATIN CAPITAL LETTER T
0x55 0x0055 #LATIN CAPITAL LETTER U
0x56 0x0056 #LATIN CAPITAL LETTER V
0x57 0x0057 #LATIN CAPITAL LETTER W
0x58 0x0058 #LATIN CAPITAL LETTER X
0x59 0x0059 #LATIN CAPITAL LETTER Y
0x5A 0x005A #LATIN CAPITAL LETTER Z
0x5B 0x005B #LEFT SQUARE BRACKET
0x5C 0x005C #REVERSE SOLIDUS
0x5D 0x005D #RIGHT SQUARE BRACKET
0x5E 0x005E #CIRCUMFLEX ACCENT
0x5F 0x005F #LOW LINE
0x60 0x0060 #GRAVE ACCENT
0x61 0x0061 #LATIN SMALL LETTER A
0x62 0x0062 #LATIN SMALL LETTER B
0x63 0x0063 #LATIN SMALL LETTER C
0x64 0x0064 #LATIN SMALL LETTER D
0x65 0x0065 #LATIN SMALL LETTER E
0x66 0x0066 #LATIN SMALL LETTER F
0x67 0x0067 #LATIN SMALL LETTER G
0x68 0x0068 #LATIN SMALL LETTER H
0x69 0x0069 #LATIN SMALL LETTER I
0x6A 0x006A #LATIN SMALL LETTER J
0x6B 0x006B #LATIN SMALL LETTER K
0x6C 0x006C #LATIN SMALL LETTER L
0x6D 0x006D #LATIN SMALL LETTER M
0x6E 0x006E #LATIN SMALL LETTER N
0x6F 0x006F #LATIN SMALL LETTER O
0x70 0x0070 #LATIN SMALL LETTER P
0x71 0x0071 #LATIN SMALL LETTER Q
0x72 0x0072 #LATIN SMALL LETTER R
0x73 0x0073 #LATIN SMALL LETTER S
0x74 0x0074 #LATIN SMALL LETTER T
0x75 0x0075 #LATIN SMALL LETTER U
0x76 0x0076 #LATIN SMALL LETTER V
0x77 0x0077 #LATIN SMALL LETTER W
0x78 0x0078 #LATIN SMALL LETTER X
0x79 0x0079 #LATIN SMALL LETTER Y
0x7A 0x007A #LATIN SMALL LETTER Z
0x7B 0x007B #LEFT CURLY BRACKET
0x7C 0x007C #VERTICAL LINE
0x7D 0x007D #RIGHT CURLY BRACKET
0x7E 0x007E #TILDE
0x7F 0x007F #DELETE
0x80 0x20AC #EURO SIGN
0x81 #UNDEFINED
0x82 0x201A #SINGLE LOW-9 QUOTATION MARK
0x83 0x0192 #LATIN SMALL LETTER F WITH HOOK
0x84 0x201E #DOUBLE LOW-9 QUOTATION MARK
0x85 0x2026 #HORIZONTAL ELLIPSIS
0x86 0x2020 #DAGGER
0x87 0x2021 #DOUBLE DAGGER
0x88 #UNDEFINED
0x89 0x2030 #PER MILLE SIGN
0x8A #UNDEFINED
0x8B 0x2039 #SINGLE LEFT-POINTING ANGLE QUOTATION MARK
0x8C #UNDEFINED
0x8D #UNDEFINED
0x8E #UNDEFINED
0x8F #UNDEFINED
0x90 #UNDEFINED
0x91 0x2018 #LEFT SINGLE QUOTATION MARK
0x92 0x2019 #RIGHT SINGLE QUOTATION MARK
0x93 0x201C #LEFT DOUBLE QUOTATION MARK
0x94 0x201D #RIGHT DOUBLE QUOTATION MARK
0x95 0x2022 #BULLET
0x96 0x2013 #EN DASH
0x97 0x2014 #EM DASH
0x98 #UNDEFINED
0x99 0x2122 #TRADE MARK SIGN
0x9A #UNDEFINED
0x9B 0x203A #SINGLE RIGHT-POINTING ANGLE QUOTATION MARK
0x9C #UNDEFINED
0x9D #UNDEFINED
0x9E #UNDEFINED
0x9F #UNDEFINED
0xA0 0x00A0 #NO-BREAK SPACE
0xA1 0x0385 #GREEK DIALYTIKA TONOS
0xA2 0x0386 #GREEK CAPITAL LETTER ALPHA WITH TONOS
0xA3 0x00A3 #POUND SIGN
0xA4 0x00A4 #CURRENCY SIGN
0xA5 0x00A5 #YEN SIGN
0xA6 0x00A6 #BROKEN BAR
0xA7 0x00A7 #SECTION SIGN
0xA8 0x00A8 #DIAERESIS
0xA9 0x00A9 #COPYRIGHT SIGN
0xAA #UNDEFINED
0xAB 0x00AB #LEFT-POINTING DOUBLE ANGLE QUOTATION MARK
0xAC 0x00AC #NOT SIGN
0xAD 0x00AD #SOFT HYPHEN
0xAE 0x00AE #REGISTERED SIGN
0xAF 0x2015 #HORIZONTAL BAR
0xB0 0x00B0 #DEGREE SIGN
0xB1 0x00B1 #PLUS-MINUS SIGN
0xB2 0x00B2 #SUPERSCRIPT TWO
0xB3 0x00B3 #SUPERSCRIPT THREE
0xB4 0x0384 #GREEK TONOS
0xB5 0x00B5 #MICRO SIGN
0xB6 0x00B6 #PILCROW SIGN
0xB7 0x00B7 #MIDDLE DOT
0xB8 0x0388 #GREEK CAPITAL LETTER EPSILON WITH TONOS
0xB9 0x0389 #GREEK CAPITAL LETTER ETA WITH TONOS
0xBA 0x038A #GREEK CAPITAL LETTER IOTA WITH TONOS
0xBB 0x00BB #RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK
0xBC 0x038C #GREEK CAPITAL LETTER OMICRON WITH TONOS
0xBD 0x00BD #VULGAR FRACTION ONE HALF
0xBE 0x038E #GREEK CAPITAL LETTER UPSILON WITH TONOS
0xBF 0x038F #GREEK CAPITAL LETTER OMEGA WITH TONOS
0xC0 0x0390 #GREEK SMALL LETTER IOTA WITH DIALYTIKA AND TONOS
0xC1 0x0391 #GREEK CAPITAL LETTER ALPHA
0xC2 0x0392 #GREEK CAPITAL LETTER BETA
0xC3 0x0393 #GREEK CAPITAL LETTER GAMMA
0xC4 0x0394 #GREEK CAPITAL LETTER DELTA
0xC5 0x0395 #GREEK CAPITAL LETTER EPSILON
0xC6 0x0396 #GREEK CAPITAL LETTER ZETA
0xC7 0x0397 #GREEK CAPITAL LETTER ETA
0xC8 0x0398 #GREEK CAPITAL LETTER THETA
0xC9 0x0399 #GREEK CAPITAL LETTER IOTA
0xCA 0x039A #GREEK CAPITAL LETTER KAPPA
0xCB 0x039B #GREEK CAPITAL LETTER LAMDA
0xCC 0x039C #GREEK CAPITAL LETTER MU
0xCD 0x039D #GREEK CAPITAL LETTER NU
0xCE 0x039E #GREEK CAPITAL LETTER XI
0xCF 0x039F #GREEK CAPITAL LETTER OMICRON
0xD0 0x03A0 #GREEK CAPITAL LETTER PI
0xD1 0x03A1 #GREEK CAPITAL LETTER RHO
0xD2 #UNDEFINED
0xD3 0x03A3 #GREEK CAPITAL LETTER SIGMA
0xD4 0x03A4 #GREEK CAPITAL LETTER TAU
0xD5 0x03A5 #GREEK CAPITAL LETTER UPSILON
0xD6 0x03A6 #GREEK CAPITAL LETTER PHI
0xD7 0x03A7 #GREEK CAPITAL LETTER CHI
0xD8 0x03A8 #GREEK CAPITAL LETTER PSI
0xD9 0x03A9 #GREEK CAPITAL LETTER OMEGA
0xDA 0x03AA #GREEK CAPITAL LETTER IOTA WITH DIALYTIKA
0xDB 0x03AB #GREEK CAPITAL LETTER UPSILON WITH DIALYTIKA
0xDC 0x03AC #GREEK SMALL LETTER ALPHA WITH TONOS
0xDD 0x03AD #GREEK SMALL LETTER EPSILON WITH TONOS
0xDE 0x03AE #GREEK SMALL LETTER ETA WITH TONOS
0xDF 0x03AF #GREEK SMALL LETTER IOTA WITH TONOS
0xE0 0x03B0 #GREEK SMALL LETTER UPSILON WITH DIALYTIKA AND TONOS
0xE1 0x03B1 #GREEK SMALL LETTER ALPHA
0xE2 0x03B2 #GREEK SMALL LETTER BETA
0xE3 0x03B3 #GREEK SMALL LETTER GAMMA
0xE4 0x03B4 #GREEK SMALL LETTER DELTA
0xE5 0x03B5 #GREEK SMALL LETTER EPSILON
0xE6 0x03B6 #GREEK SMALL LETTER ZETA
0xE7 0x03B7 #GREEK SMALL LETTER ETA
0xE8 0x03B8 #GREEK SMALL LETTER THETA
0xE9 0x03B9 #GREEK SMALL LETTER IOTA
0xEA 0x03BA #GREEK SMALL LETTER KAPPA
0xEB 0x03BB #GREEK SMALL LETTER LAMDA
0xEC 0x03BC #GREEK SMALL LETTER MU
0xED 0x03BD #GREEK SMALL LETTER NU
0xEE 0x03BE #GREEK SMALL LETTER XI
0xEF 0x03BF #GREEK SMALL LETTER OMICRON
0xF0 0x03C0 #GREEK SMALL LETTER PI
0xF1 0x03C1 #GREEK SMALL LETTER RHO
0xF2 0x03C2 #GREEK SMALL LETTER FINAL SIGMA
0xF3 0x03C3 #GREEK SMALL LETTER SIGMA
0xF4 0x03C4 #GREEK SMALL LETTER TAU
0xF5 0x03C5 #GREEK SMALL LETTER UPSILON
0xF6 0x03C6 #GREEK SMALL LETTER PHI
0xF7 0x03C7 #GREEK SMALL LETTER CHI
0xF8 0x03C8 #GREEK SMALL LETTER PSI
0xF9 0x03C9 #GREEK SMALL LETTER OMEGA
0xFA 0x03CA #GREEK SMALL LETTER IOTA WITH DIALYTIKA
0xFB 0x03CB #GREEK SMALL LETTER UPSILON WITH DIALYTIKA
0xFC 0x03CC #GREEK SMALL LETTER OMICRON WITH TONOS
0xFD 0x03CD #GREEK SMALL LETTER UPSILON WITH TONOS
0xFE 0x03CE #GREEK SMALL LETTER OMEGA WITH TONOS
0xFF #UNDEFINED

274
data/windows/CP1254.TXT Normal file
View File

@@ -0,0 +1,274 @@
#
# Name: cp1254 to Unicode table
# Unicode version: 2.0
# Table version: 2.01
# Table format: Format A
# Date: 04/15/98
#
# Contact: Shawn.Steele@microsoft.com
#
# General notes: none
#
# Format: Three tab-separated columns
# Column #1 is the cp1254 code (in hex)
# Column #2 is the Unicode (in hex as 0xXXXX)
# Column #3 is the Unicode name (follows a comment sign, '#')
#
# The entries are in cp1254 order
#
0x00 0x0000 #NULL
0x01 0x0001 #START OF HEADING
0x02 0x0002 #START OF TEXT
0x03 0x0003 #END OF TEXT
0x04 0x0004 #END OF TRANSMISSION
0x05 0x0005 #ENQUIRY
0x06 0x0006 #ACKNOWLEDGE
0x07 0x0007 #BELL
0x08 0x0008 #BACKSPACE
0x09 0x0009 #HORIZONTAL TABULATION
0x0A 0x000A #LINE FEED
0x0B 0x000B #VERTICAL TABULATION
0x0C 0x000C #FORM FEED
0x0D 0x000D #CARRIAGE RETURN
0x0E 0x000E #SHIFT OUT
0x0F 0x000F #SHIFT IN
0x10 0x0010 #DATA LINK ESCAPE
0x11 0x0011 #DEVICE CONTROL ONE
0x12 0x0012 #DEVICE CONTROL TWO
0x13 0x0013 #DEVICE CONTROL THREE
0x14 0x0014 #DEVICE CONTROL FOUR
0x15 0x0015 #NEGATIVE ACKNOWLEDGE
0x16 0x0016 #SYNCHRONOUS IDLE
0x17 0x0017 #END OF TRANSMISSION BLOCK
0x18 0x0018 #CANCEL
0x19 0x0019 #END OF MEDIUM
0x1A 0x001A #SUBSTITUTE
0x1B 0x001B #ESCAPE
0x1C 0x001C #FILE SEPARATOR
0x1D 0x001D #GROUP SEPARATOR
0x1E 0x001E #RECORD SEPARATOR
0x1F 0x001F #UNIT SEPARATOR
0x20 0x0020 #SPACE
0x21 0x0021 #EXCLAMATION MARK
0x22 0x0022 #QUOTATION MARK
0x23 0x0023 #NUMBER SIGN
0x24 0x0024 #DOLLAR SIGN
0x25 0x0025 #PERCENT SIGN
0x26 0x0026 #AMPERSAND
0x27 0x0027 #APOSTROPHE
0x28 0x0028 #LEFT PARENTHESIS
0x29 0x0029 #RIGHT PARENTHESIS
0x2A 0x002A #ASTERISK
0x2B 0x002B #PLUS SIGN
0x2C 0x002C #COMMA
0x2D 0x002D #HYPHEN-MINUS
0x2E 0x002E #FULL STOP
0x2F 0x002F #SOLIDUS
0x30 0x0030 #DIGIT ZERO
0x31 0x0031 #DIGIT ONE
0x32 0x0032 #DIGIT TWO
0x33 0x0033 #DIGIT THREE
0x34 0x0034 #DIGIT FOUR
0x35 0x0035 #DIGIT FIVE
0x36 0x0036 #DIGIT SIX
0x37 0x0037 #DIGIT SEVEN
0x38 0x0038 #DIGIT EIGHT
0x39 0x0039 #DIGIT NINE
0x3A 0x003A #COLON
0x3B 0x003B #SEMICOLON
0x3C 0x003C #LESS-THAN SIGN
0x3D 0x003D #EQUALS SIGN
0x3E 0x003E #GREATER-THAN SIGN
0x3F 0x003F #QUESTION MARK
0x40 0x0040 #COMMERCIAL AT
0x41 0x0041 #LATIN CAPITAL LETTER A
0x42 0x0042 #LATIN CAPITAL LETTER B
0x43 0x0043 #LATIN CAPITAL LETTER C
0x44 0x0044 #LATIN CAPITAL LETTER D
0x45 0x0045 #LATIN CAPITAL LETTER E
0x46 0x0046 #LATIN CAPITAL LETTER F
0x47 0x0047 #LATIN CAPITAL LETTER G
0x48 0x0048 #LATIN CAPITAL LETTER H
0x49 0x0049 #LATIN CAPITAL LETTER I
0x4A 0x004A #LATIN CAPITAL LETTER J
0x4B 0x004B #LATIN CAPITAL LETTER K
0x4C 0x004C #LATIN CAPITAL LETTER L
0x4D 0x004D #LATIN CAPITAL LETTER M
0x4E 0x004E #LATIN CAPITAL LETTER N
0x4F 0x004F #LATIN CAPITAL LETTER O
0x50 0x0050 #LATIN CAPITAL LETTER P
0x51 0x0051 #LATIN CAPITAL LETTER Q
0x52 0x0052 #LATIN CAPITAL LETTER R
0x53 0x0053 #LATIN CAPITAL LETTER S
0x54 0x0054 #LATIN CAPITAL LETTER T
0x55 0x0055 #LATIN CAPITAL LETTER U
0x56 0x0056 #LATIN CAPITAL LETTER V
0x57 0x0057 #LATIN CAPITAL LETTER W
0x58 0x0058 #LATIN CAPITAL LETTER X
0x59 0x0059 #LATIN CAPITAL LETTER Y
0x5A 0x005A #LATIN CAPITAL LETTER Z
0x5B 0x005B #LEFT SQUARE BRACKET
0x5C 0x005C #REVERSE SOLIDUS
0x5D 0x005D #RIGHT SQUARE BRACKET
0x5E 0x005E #CIRCUMFLEX ACCENT
0x5F 0x005F #LOW LINE
0x60 0x0060 #GRAVE ACCENT
0x61 0x0061 #LATIN SMALL LETTER A
0x62 0x0062 #LATIN SMALL LETTER B
0x63 0x0063 #LATIN SMALL LETTER C
0x64 0x0064 #LATIN SMALL LETTER D
0x65 0x0065 #LATIN SMALL LETTER E
0x66 0x0066 #LATIN SMALL LETTER F
0x67 0x0067 #LATIN SMALL LETTER G
0x68 0x0068 #LATIN SMALL LETTER H
0x69 0x0069 #LATIN SMALL LETTER I
0x6A 0x006A #LATIN SMALL LETTER J
0x6B 0x006B #LATIN SMALL LETTER K
0x6C 0x006C #LATIN SMALL LETTER L
0x6D 0x006D #LATIN SMALL LETTER M
0x6E 0x006E #LATIN SMALL LETTER N
0x6F 0x006F #LATIN SMALL LETTER O
0x70 0x0070 #LATIN SMALL LETTER P
0x71 0x0071 #LATIN SMALL LETTER Q
0x72 0x0072 #LATIN SMALL LETTER R
0x73 0x0073 #LATIN SMALL LETTER S
0x74 0x0074 #LATIN SMALL LETTER T
0x75 0x0075 #LATIN SMALL LETTER U
0x76 0x0076 #LATIN SMALL LETTER V
0x77 0x0077 #LATIN SMALL LETTER W
0x78 0x0078 #LATIN SMALL LETTER X
0x79 0x0079 #LATIN SMALL LETTER Y
0x7A 0x007A #LATIN SMALL LETTER Z
0x7B 0x007B #LEFT CURLY BRACKET
0x7C 0x007C #VERTICAL LINE
0x7D 0x007D #RIGHT CURLY BRACKET
0x7E 0x007E #TILDE
0x7F 0x007F #DELETE
0x80 0x20AC #EURO SIGN
0x81 #UNDEFINED
0x82 0x201A #SINGLE LOW-9 QUOTATION MARK
0x83 0x0192 #LATIN SMALL LETTER F WITH HOOK
0x84 0x201E #DOUBLE LOW-9 QUOTATION MARK
0x85 0x2026 #HORIZONTAL ELLIPSIS
0x86 0x2020 #DAGGER
0x87 0x2021 #DOUBLE DAGGER
0x88 0x02C6 #MODIFIER LETTER CIRCUMFLEX ACCENT
0x89 0x2030 #PER MILLE SIGN
0x8A 0x0160 #LATIN CAPITAL LETTER S WITH CARON
0x8B 0x2039 #SINGLE LEFT-POINTING ANGLE QUOTATION MARK
0x8C 0x0152 #LATIN CAPITAL LIGATURE OE
0x8D #UNDEFINED
0x8E #UNDEFINED
0x8F #UNDEFINED
0x90 #UNDEFINED
0x91 0x2018 #LEFT SINGLE QUOTATION MARK
0x92 0x2019 #RIGHT SINGLE QUOTATION MARK
0x93 0x201C #LEFT DOUBLE QUOTATION MARK
0x94 0x201D #RIGHT DOUBLE QUOTATION MARK
0x95 0x2022 #BULLET
0x96 0x2013 #EN DASH
0x97 0x2014 #EM DASH
0x98 0x02DC #SMALL TILDE
0x99 0x2122 #TRADE MARK SIGN
0x9A 0x0161 #LATIN SMALL LETTER S WITH CARON
0x9B 0x203A #SINGLE RIGHT-POINTING ANGLE QUOTATION MARK
0x9C 0x0153 #LATIN SMALL LIGATURE OE
0x9D #UNDEFINED
0x9E #UNDEFINED
0x9F 0x0178 #LATIN CAPITAL LETTER Y WITH DIAERESIS
0xA0 0x00A0 #NO-BREAK SPACE
0xA1 0x00A1 #INVERTED EXCLAMATION MARK
0xA2 0x00A2 #CENT SIGN
0xA3 0x00A3 #POUND SIGN
0xA4 0x00A4 #CURRENCY SIGN
0xA5 0x00A5 #YEN SIGN
0xA6 0x00A6 #BROKEN BAR
0xA7 0x00A7 #SECTION SIGN
0xA8 0x00A8 #DIAERESIS
0xA9 0x00A9 #COPYRIGHT SIGN
0xAA 0x00AA #FEMININE ORDINAL INDICATOR
0xAB 0x00AB #LEFT-POINTING DOUBLE ANGLE QUOTATION MARK
0xAC 0x00AC #NOT SIGN
0xAD 0x00AD #SOFT HYPHEN
0xAE 0x00AE #REGISTERED SIGN
0xAF 0x00AF #MACRON
0xB0 0x00B0 #DEGREE SIGN
0xB1 0x00B1 #PLUS-MINUS SIGN
0xB2 0x00B2 #SUPERSCRIPT TWO
0xB3 0x00B3 #SUPERSCRIPT THREE
0xB4 0x00B4 #ACUTE ACCENT
0xB5 0x00B5 #MICRO SIGN
0xB6 0x00B6 #PILCROW SIGN
0xB7 0x00B7 #MIDDLE DOT
0xB8 0x00B8 #CEDILLA
0xB9 0x00B9 #SUPERSCRIPT ONE
0xBA 0x00BA #MASCULINE ORDINAL INDICATOR
0xBB 0x00BB #RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK
0xBC 0x00BC #VULGAR FRACTION ONE QUARTER
0xBD 0x00BD #VULGAR FRACTION ONE HALF
0xBE 0x00BE #VULGAR FRACTION THREE QUARTERS
0xBF 0x00BF #INVERTED QUESTION MARK
0xC0 0x00C0 #LATIN CAPITAL LETTER A WITH GRAVE
0xC1 0x00C1 #LATIN CAPITAL LETTER A WITH ACUTE
0xC2 0x00C2 #LATIN CAPITAL LETTER A WITH CIRCUMFLEX
0xC3 0x00C3 #LATIN CAPITAL LETTER A WITH TILDE
0xC4 0x00C4 #LATIN CAPITAL LETTER A WITH DIAERESIS
0xC5 0x00C5 #LATIN CAPITAL LETTER A WITH RING ABOVE
0xC6 0x00C6 #LATIN CAPITAL LETTER AE
0xC7 0x00C7 #LATIN CAPITAL LETTER C WITH CEDILLA
0xC8 0x00C8 #LATIN CAPITAL LETTER E WITH GRAVE
0xC9 0x00C9 #LATIN CAPITAL LETTER E WITH ACUTE
0xCA 0x00CA #LATIN CAPITAL LETTER E WITH CIRCUMFLEX
0xCB 0x00CB #LATIN CAPITAL LETTER E WITH DIAERESIS
0xCC 0x00CC #LATIN CAPITAL LETTER I WITH GRAVE
0xCD 0x00CD #LATIN CAPITAL LETTER I WITH ACUTE
0xCE 0x00CE #LATIN CAPITAL LETTER I WITH CIRCUMFLEX
0xCF 0x00CF #LATIN CAPITAL LETTER I WITH DIAERESIS
0xD0 0x011E #LATIN CAPITAL LETTER G WITH BREVE
0xD1 0x00D1 #LATIN CAPITAL LETTER N WITH TILDE
0xD2 0x00D2 #LATIN CAPITAL LETTER O WITH GRAVE
0xD3 0x00D3 #LATIN CAPITAL LETTER O WITH ACUTE
0xD4 0x00D4 #LATIN CAPITAL LETTER O WITH CIRCUMFLEX
0xD5 0x00D5 #LATIN CAPITAL LETTER O WITH TILDE
0xD6 0x00D6 #LATIN CAPITAL LETTER O WITH DIAERESIS
0xD7 0x00D7 #MULTIPLICATION SIGN
0xD8 0x00D8 #LATIN CAPITAL LETTER O WITH STROKE
0xD9 0x00D9 #LATIN CAPITAL LETTER U WITH GRAVE
0xDA 0x00DA #LATIN CAPITAL LETTER U WITH ACUTE
0xDB 0x00DB #LATIN CAPITAL LETTER U WITH CIRCUMFLEX
0xDC 0x00DC #LATIN CAPITAL LETTER U WITH DIAERESIS
0xDD 0x0130 #LATIN CAPITAL LETTER I WITH DOT ABOVE
0xDE 0x015E #LATIN CAPITAL LETTER S WITH CEDILLA
0xDF 0x00DF #LATIN SMALL LETTER SHARP S
0xE0 0x00E0 #LATIN SMALL LETTER A WITH GRAVE
0xE1 0x00E1 #LATIN SMALL LETTER A WITH ACUTE
0xE2 0x00E2 #LATIN SMALL LETTER A WITH CIRCUMFLEX
0xE3 0x00E3 #LATIN SMALL LETTER A WITH TILDE
0xE4 0x00E4 #LATIN SMALL LETTER A WITH DIAERESIS
0xE5 0x00E5 #LATIN SMALL LETTER A WITH RING ABOVE
0xE6 0x00E6 #LATIN SMALL LETTER AE
0xE7 0x00E7 #LATIN SMALL LETTER C WITH CEDILLA
0xE8 0x00E8 #LATIN SMALL LETTER E WITH GRAVE
0xE9 0x00E9 #LATIN SMALL LETTER E WITH ACUTE
0xEA 0x00EA #LATIN SMALL LETTER E WITH CIRCUMFLEX
0xEB 0x00EB #LATIN SMALL LETTER E WITH DIAERESIS
0xEC 0x00EC #LATIN SMALL LETTER I WITH GRAVE
0xED 0x00ED #LATIN SMALL LETTER I WITH ACUTE
0xEE 0x00EE #LATIN SMALL LETTER I WITH CIRCUMFLEX
0xEF 0x00EF #LATIN SMALL LETTER I WITH DIAERESIS
0xF0 0x011F #LATIN SMALL LETTER G WITH BREVE
0xF1 0x00F1 #LATIN SMALL LETTER N WITH TILDE
0xF2 0x00F2 #LATIN SMALL LETTER O WITH GRAVE
0xF3 0x00F3 #LATIN SMALL LETTER O WITH ACUTE
0xF4 0x00F4 #LATIN SMALL LETTER O WITH CIRCUMFLEX
0xF5 0x00F5 #LATIN SMALL LETTER O WITH TILDE
0xF6 0x00F6 #LATIN SMALL LETTER O WITH DIAERESIS
0xF7 0x00F7 #DIVISION SIGN
0xF8 0x00F8 #LATIN SMALL LETTER O WITH STROKE
0xF9 0x00F9 #LATIN SMALL LETTER U WITH GRAVE
0xFA 0x00FA #LATIN SMALL LETTER U WITH ACUTE
0xFB 0x00FB #LATIN SMALL LETTER U WITH CIRCUMFLEX
0xFC 0x00FC #LATIN SMALL LETTER U WITH DIAERESIS
0xFD 0x0131 #LATIN SMALL LETTER DOTLESS I
0xFE 0x015F #LATIN SMALL LETTER S WITH CEDILLA
0xFF 0x00FF #LATIN SMALL LETTER Y WITH DIAERESIS

274
data/windows/CP1255.TXT Normal file
View File

@@ -0,0 +1,274 @@
#
# Name: cp1255 to Unicode table
# Unicode version: 2.0
# Table version: 2.01
# Table format: Format A
# Date: 1/7/2000
#
# Contact: Shawn.Steele@microsoft.com
#
# General notes: none
#
# Format: Three tab-separated columns
# Column #1 is the cp1255 code (in hex)
# Column #2 is the Unicode (in hex as 0xXXXX)
# Column #3 is the Unicode name (follows a comment sign, '#')
#
# The entries are in cp1255 order
#
0x00 0x0000 #NULL
0x01 0x0001 #START OF HEADING
0x02 0x0002 #START OF TEXT
0x03 0x0003 #END OF TEXT
0x04 0x0004 #END OF TRANSMISSION
0x05 0x0005 #ENQUIRY
0x06 0x0006 #ACKNOWLEDGE
0x07 0x0007 #BELL
0x08 0x0008 #BACKSPACE
0x09 0x0009 #HORIZONTAL TABULATION
0x0A 0x000A #LINE FEED
0x0B 0x000B #VERTICAL TABULATION
0x0C 0x000C #FORM FEED
0x0D 0x000D #CARRIAGE RETURN
0x0E 0x000E #SHIFT OUT
0x0F 0x000F #SHIFT IN
0x10 0x0010 #DATA LINK ESCAPE
0x11 0x0011 #DEVICE CONTROL ONE
0x12 0x0012 #DEVICE CONTROL TWO
0x13 0x0013 #DEVICE CONTROL THREE
0x14 0x0014 #DEVICE CONTROL FOUR
0x15 0x0015 #NEGATIVE ACKNOWLEDGE
0x16 0x0016 #SYNCHRONOUS IDLE
0x17 0x0017 #END OF TRANSMISSION BLOCK
0x18 0x0018 #CANCEL
0x19 0x0019 #END OF MEDIUM
0x1A 0x001A #SUBSTITUTE
0x1B 0x001B #ESCAPE
0x1C 0x001C #FILE SEPARATOR
0x1D 0x001D #GROUP SEPARATOR
0x1E 0x001E #RECORD SEPARATOR
0x1F 0x001F #UNIT SEPARATOR
0x20 0x0020 #SPACE
0x21 0x0021 #EXCLAMATION MARK
0x22 0x0022 #QUOTATION MARK
0x23 0x0023 #NUMBER SIGN
0x24 0x0024 #DOLLAR SIGN
0x25 0x0025 #PERCENT SIGN
0x26 0x0026 #AMPERSAND
0x27 0x0027 #APOSTROPHE
0x28 0x0028 #LEFT PARENTHESIS
0x29 0x0029 #RIGHT PARENTHESIS
0x2A 0x002A #ASTERISK
0x2B 0x002B #PLUS SIGN
0x2C 0x002C #COMMA
0x2D 0x002D #HYPHEN-MINUS
0x2E 0x002E #FULL STOP
0x2F 0x002F #SOLIDUS
0x30 0x0030 #DIGIT ZERO
0x31 0x0031 #DIGIT ONE
0x32 0x0032 #DIGIT TWO
0x33 0x0033 #DIGIT THREE
0x34 0x0034 #DIGIT FOUR
0x35 0x0035 #DIGIT FIVE
0x36 0x0036 #DIGIT SIX
0x37 0x0037 #DIGIT SEVEN
0x38 0x0038 #DIGIT EIGHT
0x39 0x0039 #DIGIT NINE
0x3A 0x003A #COLON
0x3B 0x003B #SEMICOLON
0x3C 0x003C #LESS-THAN SIGN
0x3D 0x003D #EQUALS SIGN
0x3E 0x003E #GREATER-THAN SIGN
0x3F 0x003F #QUESTION MARK
0x40 0x0040 #COMMERCIAL AT
0x41 0x0041 #LATIN CAPITAL LETTER A
0x42 0x0042 #LATIN CAPITAL LETTER B
0x43 0x0043 #LATIN CAPITAL LETTER C
0x44 0x0044 #LATIN CAPITAL LETTER D
0x45 0x0045 #LATIN CAPITAL LETTER E
0x46 0x0046 #LATIN CAPITAL LETTER F
0x47 0x0047 #LATIN CAPITAL LETTER G
0x48 0x0048 #LATIN CAPITAL LETTER H
0x49 0x0049 #LATIN CAPITAL LETTER I
0x4A 0x004A #LATIN CAPITAL LETTER J
0x4B 0x004B #LATIN CAPITAL LETTER K
0x4C 0x004C #LATIN CAPITAL LETTER L
0x4D 0x004D #LATIN CAPITAL LETTER M
0x4E 0x004E #LATIN CAPITAL LETTER N
0x4F 0x004F #LATIN CAPITAL LETTER O
0x50 0x0050 #LATIN CAPITAL LETTER P
0x51 0x0051 #LATIN CAPITAL LETTER Q
0x52 0x0052 #LATIN CAPITAL LETTER R
0x53 0x0053 #LATIN CAPITAL LETTER S
0x54 0x0054 #LATIN CAPITAL LETTER T
0x55 0x0055 #LATIN CAPITAL LETTER U
0x56 0x0056 #LATIN CAPITAL LETTER V
0x57 0x0057 #LATIN CAPITAL LETTER W
0x58 0x0058 #LATIN CAPITAL LETTER X
0x59 0x0059 #LATIN CAPITAL LETTER Y
0x5A 0x005A #LATIN CAPITAL LETTER Z
0x5B 0x005B #LEFT SQUARE BRACKET
0x5C 0x005C #REVERSE SOLIDUS
0x5D 0x005D #RIGHT SQUARE BRACKET
0x5E 0x005E #CIRCUMFLEX ACCENT
0x5F 0x005F #LOW LINE
0x60 0x0060 #GRAVE ACCENT
0x61 0x0061 #LATIN SMALL LETTER A
0x62 0x0062 #LATIN SMALL LETTER B
0x63 0x0063 #LATIN SMALL LETTER C
0x64 0x0064 #LATIN SMALL LETTER D
0x65 0x0065 #LATIN SMALL LETTER E
0x66 0x0066 #LATIN SMALL LETTER F
0x67 0x0067 #LATIN SMALL LETTER G
0x68 0x0068 #LATIN SMALL LETTER H
0x69 0x0069 #LATIN SMALL LETTER I
0x6A 0x006A #LATIN SMALL LETTER J
0x6B 0x006B #LATIN SMALL LETTER K
0x6C 0x006C #LATIN SMALL LETTER L
0x6D 0x006D #LATIN SMALL LETTER M
0x6E 0x006E #LATIN SMALL LETTER N
0x6F 0x006F #LATIN SMALL LETTER O
0x70 0x0070 #LATIN SMALL LETTER P
0x71 0x0071 #LATIN SMALL LETTER Q
0x72 0x0072 #LATIN SMALL LETTER R
0x73 0x0073 #LATIN SMALL LETTER S
0x74 0x0074 #LATIN SMALL LETTER T
0x75 0x0075 #LATIN SMALL LETTER U
0x76 0x0076 #LATIN SMALL LETTER V
0x77 0x0077 #LATIN SMALL LETTER W
0x78 0x0078 #LATIN SMALL LETTER X
0x79 0x0079 #LATIN SMALL LETTER Y
0x7A 0x007A #LATIN SMALL LETTER Z
0x7B 0x007B #LEFT CURLY BRACKET
0x7C 0x007C #VERTICAL LINE
0x7D 0x007D #RIGHT CURLY BRACKET
0x7E 0x007E #TILDE
0x7F 0x007F #DELETE
0x80 0x20AC #EURO SIGN
0x81 #UNDEFINED
0x82 0x201A #SINGLE LOW-9 QUOTATION MARK
0x83 0x0192 #LATIN SMALL LETTER F WITH HOOK
0x84 0x201E #DOUBLE LOW-9 QUOTATION MARK
0x85 0x2026 #HORIZONTAL ELLIPSIS
0x86 0x2020 #DAGGER
0x87 0x2021 #DOUBLE DAGGER
0x88 0x02C6 #MODIFIER LETTER CIRCUMFLEX ACCENT
0x89 0x2030 #PER MILLE SIGN
0x8A #UNDEFINED
0x8B 0x2039 #SINGLE LEFT-POINTING ANGLE QUOTATION MARK
0x8C #UNDEFINED
0x8D #UNDEFINED
0x8E #UNDEFINED
0x8F #UNDEFINED
0x90 #UNDEFINED
0x91 0x2018 #LEFT SINGLE QUOTATION MARK
0x92 0x2019 #RIGHT SINGLE QUOTATION MARK
0x93 0x201C #LEFT DOUBLE QUOTATION MARK
0x94 0x201D #RIGHT DOUBLE QUOTATION MARK
0x95 0x2022 #BULLET
0x96 0x2013 #EN DASH
0x97 0x2014 #EM DASH
0x98 0x02DC #SMALL TILDE
0x99 0x2122 #TRADE MARK SIGN
0x9A #UNDEFINED
0x9B 0x203A #SINGLE RIGHT-POINTING ANGLE QUOTATION MARK
0x9C #UNDEFINED
0x9D #UNDEFINED
0x9E #UNDEFINED
0x9F #UNDEFINED
0xA0 0x00A0 #NO-BREAK SPACE
0xA1 0x00A1 #INVERTED EXCLAMATION MARK
0xA2 0x00A2 #CENT SIGN
0xA3 0x00A3 #POUND SIGN
0xA4 0x20AA #NEW SHEQEL SIGN
0xA5 0x00A5 #YEN SIGN
0xA6 0x00A6 #BROKEN BAR
0xA7 0x00A7 #SECTION SIGN
0xA8 0x00A8 #DIAERESIS
0xA9 0x00A9 #COPYRIGHT SIGN
0xAA 0x00D7 #MULTIPLICATION SIGN
0xAB 0x00AB #LEFT-POINTING DOUBLE ANGLE QUOTATION MARK
0xAC 0x00AC #NOT SIGN
0xAD 0x00AD #SOFT HYPHEN
0xAE 0x00AE #REGISTERED SIGN
0xAF 0x00AF #MACRON
0xB0 0x00B0 #DEGREE SIGN
0xB1 0x00B1 #PLUS-MINUS SIGN
0xB2 0x00B2 #SUPERSCRIPT TWO
0xB3 0x00B3 #SUPERSCRIPT THREE
0xB4 0x00B4 #ACUTE ACCENT
0xB5 0x00B5 #MICRO SIGN
0xB6 0x00B6 #PILCROW SIGN
0xB7 0x00B7 #MIDDLE DOT
0xB8 0x00B8 #CEDILLA
0xB9 0x00B9 #SUPERSCRIPT ONE
0xBA 0x00F7 #DIVISION SIGN
0xBB 0x00BB #RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK
0xBC 0x00BC #VULGAR FRACTION ONE QUARTER
0xBD 0x00BD #VULGAR FRACTION ONE HALF
0xBE 0x00BE #VULGAR FRACTION THREE QUARTERS
0xBF 0x00BF #INVERTED QUESTION MARK
0xC0 0x05B0 #HEBREW POINT SHEVA
0xC1 0x05B1 #HEBREW POINT HATAF SEGOL
0xC2 0x05B2 #HEBREW POINT HATAF PATAH
0xC3 0x05B3 #HEBREW POINT HATAF QAMATS
0xC4 0x05B4 #HEBREW POINT HIRIQ
0xC5 0x05B5 #HEBREW POINT TSERE
0xC6 0x05B6 #HEBREW POINT SEGOL
0xC7 0x05B7 #HEBREW POINT PATAH
0xC8 0x05B8 #HEBREW POINT QAMATS
0xC9 0x05B9 #HEBREW POINT HOLAM
0xCA #UNDEFINED
0xCB 0x05BB #HEBREW POINT QUBUTS
0xCC 0x05BC #HEBREW POINT DAGESH OR MAPIQ
0xCD 0x05BD #HEBREW POINT METEG
0xCE 0x05BE #HEBREW PUNCTUATION MAQAF
0xCF 0x05BF #HEBREW POINT RAFE
0xD0 0x05C0 #HEBREW PUNCTUATION PASEQ
0xD1 0x05C1 #HEBREW POINT SHIN DOT
0xD2 0x05C2 #HEBREW POINT SIN DOT
0xD3 0x05C3 #HEBREW PUNCTUATION SOF PASUQ
0xD4 0x05F0 #HEBREW LIGATURE YIDDISH DOUBLE VAV
0xD5 0x05F1 #HEBREW LIGATURE YIDDISH VAV YOD
0xD6 0x05F2 #HEBREW LIGATURE YIDDISH DOUBLE YOD
0xD7 0x05F3 #HEBREW PUNCTUATION GERESH
0xD8 0x05F4 #HEBREW PUNCTUATION GERSHAYIM
0xD9 #UNDEFINED
0xDA #UNDEFINED
0xDB #UNDEFINED
0xDC #UNDEFINED
0xDD #UNDEFINED
0xDE #UNDEFINED
0xDF #UNDEFINED
0xE0 0x05D0 #HEBREW LETTER ALEF
0xE1 0x05D1 #HEBREW LETTER BET
0xE2 0x05D2 #HEBREW LETTER GIMEL
0xE3 0x05D3 #HEBREW LETTER DALET
0xE4 0x05D4 #HEBREW LETTER HE
0xE5 0x05D5 #HEBREW LETTER VAV
0xE6 0x05D6 #HEBREW LETTER ZAYIN
0xE7 0x05D7 #HEBREW LETTER HET
0xE8 0x05D8 #HEBREW LETTER TET
0xE9 0x05D9 #HEBREW LETTER YOD
0xEA 0x05DA #HEBREW LETTER FINAL KAF
0xEB 0x05DB #HEBREW LETTER KAF
0xEC 0x05DC #HEBREW LETTER LAMED
0xED 0x05DD #HEBREW LETTER FINAL MEM
0xEE 0x05DE #HEBREW LETTER MEM
0xEF 0x05DF #HEBREW LETTER FINAL NUN
0xF0 0x05E0 #HEBREW LETTER NUN
0xF1 0x05E1 #HEBREW LETTER SAMEKH
0xF2 0x05E2 #HEBREW LETTER AYIN
0xF3 0x05E3 #HEBREW LETTER FINAL PE
0xF4 0x05E4 #HEBREW LETTER PE
0xF5 0x05E5 #HEBREW LETTER FINAL TSADI
0xF6 0x05E6 #HEBREW LETTER TSADI
0xF7 0x05E7 #HEBREW LETTER QOF
0xF8 0x05E8 #HEBREW LETTER RESH
0xF9 0x05E9 #HEBREW LETTER SHIN
0xFA 0x05EA #HEBREW LETTER TAV
0xFB #UNDEFINED
0xFC #UNDEFINED
0xFD 0x200E #LEFT-TO-RIGHT MARK
0xFE 0x200F #RIGHT-TO-LEFT MARK
0xFF #UNDEFINED

274
data/windows/CP1256.TXT Normal file
View File

@@ -0,0 +1,274 @@
#
# Name: cp1256 to Unicode table
# Unicode version: 2.1
# Table version: 2.01
# Table format: Format A
# Date: 01/5/99
#
# Contact: Shawn.Steele@microsoft.com
#
# General notes: none
#
# Format: Three tab-separated columns
# Column #1 is the cp1256 code (in hex)
# Column #2 is the Unicode (in hex as 0xXXXX)
# Column #3 is the Unicode name (follows a comment sign, '#')
#
# The entries are in cp1256 order
#
0x00 0x0000 #NULL
0x01 0x0001 #START OF HEADING
0x02 0x0002 #START OF TEXT
0x03 0x0003 #END OF TEXT
0x04 0x0004 #END OF TRANSMISSION
0x05 0x0005 #ENQUIRY
0x06 0x0006 #ACKNOWLEDGE
0x07 0x0007 #BELL
0x08 0x0008 #BACKSPACE
0x09 0x0009 #HORIZONTAL TABULATION
0x0A 0x000A #LINE FEED
0x0B 0x000B #VERTICAL TABULATION
0x0C 0x000C #FORM FEED
0x0D 0x000D #CARRIAGE RETURN
0x0E 0x000E #SHIFT OUT
0x0F 0x000F #SHIFT IN
0x10 0x0010 #DATA LINK ESCAPE
0x11 0x0011 #DEVICE CONTROL ONE
0x12 0x0012 #DEVICE CONTROL TWO
0x13 0x0013 #DEVICE CONTROL THREE
0x14 0x0014 #DEVICE CONTROL FOUR
0x15 0x0015 #NEGATIVE ACKNOWLEDGE
0x16 0x0016 #SYNCHRONOUS IDLE
0x17 0x0017 #END OF TRANSMISSION BLOCK
0x18 0x0018 #CANCEL
0x19 0x0019 #END OF MEDIUM
0x1A 0x001A #SUBSTITUTE
0x1B 0x001B #ESCAPE
0x1C 0x001C #FILE SEPARATOR
0x1D 0x001D #GROUP SEPARATOR
0x1E 0x001E #RECORD SEPARATOR
0x1F 0x001F #UNIT SEPARATOR
0x20 0x0020 #SPACE
0x21 0x0021 #EXCLAMATION MARK
0x22 0x0022 #QUOTATION MARK
0x23 0x0023 #NUMBER SIGN
0x24 0x0024 #DOLLAR SIGN
0x25 0x0025 #PERCENT SIGN
0x26 0x0026 #AMPERSAND
0x27 0x0027 #APOSTROPHE
0x28 0x0028 #LEFT PARENTHESIS
0x29 0x0029 #RIGHT PARENTHESIS
0x2A 0x002A #ASTERISK
0x2B 0x002B #PLUS SIGN
0x2C 0x002C #COMMA
0x2D 0x002D #HYPHEN-MINUS
0x2E 0x002E #FULL STOP
0x2F 0x002F #SOLIDUS
0x30 0x0030 #DIGIT ZERO
0x31 0x0031 #DIGIT ONE
0x32 0x0032 #DIGIT TWO
0x33 0x0033 #DIGIT THREE
0x34 0x0034 #DIGIT FOUR
0x35 0x0035 #DIGIT FIVE
0x36 0x0036 #DIGIT SIX
0x37 0x0037 #DIGIT SEVEN
0x38 0x0038 #DIGIT EIGHT
0x39 0x0039 #DIGIT NINE
0x3A 0x003A #COLON
0x3B 0x003B #SEMICOLON
0x3C 0x003C #LESS-THAN SIGN
0x3D 0x003D #EQUALS SIGN
0x3E 0x003E #GREATER-THAN SIGN
0x3F 0x003F #QUESTION MARK
0x40 0x0040 #COMMERCIAL AT
0x41 0x0041 #LATIN CAPITAL LETTER A
0x42 0x0042 #LATIN CAPITAL LETTER B
0x43 0x0043 #LATIN CAPITAL LETTER C
0x44 0x0044 #LATIN CAPITAL LETTER D
0x45 0x0045 #LATIN CAPITAL LETTER E
0x46 0x0046 #LATIN CAPITAL LETTER F
0x47 0x0047 #LATIN CAPITAL LETTER G
0x48 0x0048 #LATIN CAPITAL LETTER H
0x49 0x0049 #LATIN CAPITAL LETTER I
0x4A 0x004A #LATIN CAPITAL LETTER J
0x4B 0x004B #LATIN CAPITAL LETTER K
0x4C 0x004C #LATIN CAPITAL LETTER L
0x4D 0x004D #LATIN CAPITAL LETTER M
0x4E 0x004E #LATIN CAPITAL LETTER N
0x4F 0x004F #LATIN CAPITAL LETTER O
0x50 0x0050 #LATIN CAPITAL LETTER P
0x51 0x0051 #LATIN CAPITAL LETTER Q
0x52 0x0052 #LATIN CAPITAL LETTER R
0x53 0x0053 #LATIN CAPITAL LETTER S
0x54 0x0054 #LATIN CAPITAL LETTER T
0x55 0x0055 #LATIN CAPITAL LETTER U
0x56 0x0056 #LATIN CAPITAL LETTER V
0x57 0x0057 #LATIN CAPITAL LETTER W
0x58 0x0058 #LATIN CAPITAL LETTER X
0x59 0x0059 #LATIN CAPITAL LETTER Y
0x5A 0x005A #LATIN CAPITAL LETTER Z
0x5B 0x005B #LEFT SQUARE BRACKET
0x5C 0x005C #REVERSE SOLIDUS
0x5D 0x005D #RIGHT SQUARE BRACKET
0x5E 0x005E #CIRCUMFLEX ACCENT
0x5F 0x005F #LOW LINE
0x60 0x0060 #GRAVE ACCENT
0x61 0x0061 #LATIN SMALL LETTER A
0x62 0x0062 #LATIN SMALL LETTER B
0x63 0x0063 #LATIN SMALL LETTER C
0x64 0x0064 #LATIN SMALL LETTER D
0x65 0x0065 #LATIN SMALL LETTER E
0x66 0x0066 #LATIN SMALL LETTER F
0x67 0x0067 #LATIN SMALL LETTER G
0x68 0x0068 #LATIN SMALL LETTER H
0x69 0x0069 #LATIN SMALL LETTER I
0x6A 0x006A #LATIN SMALL LETTER J
0x6B 0x006B #LATIN SMALL LETTER K
0x6C 0x006C #LATIN SMALL LETTER L
0x6D 0x006D #LATIN SMALL LETTER M
0x6E 0x006E #LATIN SMALL LETTER N
0x6F 0x006F #LATIN SMALL LETTER O
0x70 0x0070 #LATIN SMALL LETTER P
0x71 0x0071 #LATIN SMALL LETTER Q
0x72 0x0072 #LATIN SMALL LETTER R
0x73 0x0073 #LATIN SMALL LETTER S
0x74 0x0074 #LATIN SMALL LETTER T
0x75 0x0075 #LATIN SMALL LETTER U
0x76 0x0076 #LATIN SMALL LETTER V
0x77 0x0077 #LATIN SMALL LETTER W
0x78 0x0078 #LATIN SMALL LETTER X
0x79 0x0079 #LATIN SMALL LETTER Y
0x7A 0x007A #LATIN SMALL LETTER Z
0x7B 0x007B #LEFT CURLY BRACKET
0x7C 0x007C #VERTICAL LINE
0x7D 0x007D #RIGHT CURLY BRACKET
0x7E 0x007E #TILDE
0x7F 0x007F #DELETE
0x80 0x20AC #EURO SIGN
0x81 0x067E #ARABIC LETTER PEH
0x82 0x201A #SINGLE LOW-9 QUOTATION MARK
0x83 0x0192 #LATIN SMALL LETTER F WITH HOOK
0x84 0x201E #DOUBLE LOW-9 QUOTATION MARK
0x85 0x2026 #HORIZONTAL ELLIPSIS
0x86 0x2020 #DAGGER
0x87 0x2021 #DOUBLE DAGGER
0x88 0x02C6 #MODIFIER LETTER CIRCUMFLEX ACCENT
0x89 0x2030 #PER MILLE SIGN
0x8A 0x0679 #ARABIC LETTER TTEH
0x8B 0x2039 #SINGLE LEFT-POINTING ANGLE QUOTATION MARK
0x8C 0x0152 #LATIN CAPITAL LIGATURE OE
0x8D 0x0686 #ARABIC LETTER TCHEH
0x8E 0x0698 #ARABIC LETTER JEH
0x8F 0x0688 #ARABIC LETTER DDAL
0x90 0x06AF #ARABIC LETTER GAF
0x91 0x2018 #LEFT SINGLE QUOTATION MARK
0x92 0x2019 #RIGHT SINGLE QUOTATION MARK
0x93 0x201C #LEFT DOUBLE QUOTATION MARK
0x94 0x201D #RIGHT DOUBLE QUOTATION MARK
0x95 0x2022 #BULLET
0x96 0x2013 #EN DASH
0x97 0x2014 #EM DASH
0x98 0x06A9 #ARABIC LETTER KEHEH
0x99 0x2122 #TRADE MARK SIGN
0x9A 0x0691 #ARABIC LETTER RREH
0x9B 0x203A #SINGLE RIGHT-POINTING ANGLE QUOTATION MARK
0x9C 0x0153 #LATIN SMALL LIGATURE OE
0x9D 0x200C #ZERO WIDTH NON-JOINER
0x9E 0x200D #ZERO WIDTH JOINER
0x9F 0x06BA #ARABIC LETTER NOON GHUNNA
0xA0 0x00A0 #NO-BREAK SPACE
0xA1 0x060C #ARABIC COMMA
0xA2 0x00A2 #CENT SIGN
0xA3 0x00A3 #POUND SIGN
0xA4 0x00A4 #CURRENCY SIGN
0xA5 0x00A5 #YEN SIGN
0xA6 0x00A6 #BROKEN BAR
0xA7 0x00A7 #SECTION SIGN
0xA8 0x00A8 #DIAERESIS
0xA9 0x00A9 #COPYRIGHT SIGN
0xAA 0x06BE #ARABIC LETTER HEH DOACHASHMEE
0xAB 0x00AB #LEFT-POINTING DOUBLE ANGLE QUOTATION MARK
0xAC 0x00AC #NOT SIGN
0xAD 0x00AD #SOFT HYPHEN
0xAE 0x00AE #REGISTERED SIGN
0xAF 0x00AF #MACRON
0xB0 0x00B0 #DEGREE SIGN
0xB1 0x00B1 #PLUS-MINUS SIGN
0xB2 0x00B2 #SUPERSCRIPT TWO
0xB3 0x00B3 #SUPERSCRIPT THREE
0xB4 0x00B4 #ACUTE ACCENT
0xB5 0x00B5 #MICRO SIGN
0xB6 0x00B6 #PILCROW SIGN
0xB7 0x00B7 #MIDDLE DOT
0xB8 0x00B8 #CEDILLA
0xB9 0x00B9 #SUPERSCRIPT ONE
0xBA 0x061B #ARABIC SEMICOLON
0xBB 0x00BB #RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK
0xBC 0x00BC #VULGAR FRACTION ONE QUARTER
0xBD 0x00BD #VULGAR FRACTION ONE HALF
0xBE 0x00BE #VULGAR FRACTION THREE QUARTERS
0xBF 0x061F #ARABIC QUESTION MARK
0xC0 0x06C1 #ARABIC LETTER HEH GOAL
0xC1 0x0621 #ARABIC LETTER HAMZA
0xC2 0x0622 #ARABIC LETTER ALEF WITH MADDA ABOVE
0xC3 0x0623 #ARABIC LETTER ALEF WITH HAMZA ABOVE
0xC4 0x0624 #ARABIC LETTER WAW WITH HAMZA ABOVE
0xC5 0x0625 #ARABIC LETTER ALEF WITH HAMZA BELOW
0xC6 0x0626 #ARABIC LETTER YEH WITH HAMZA ABOVE
0xC7 0x0627 #ARABIC LETTER ALEF
0xC8 0x0628 #ARABIC LETTER BEH
0xC9 0x0629 #ARABIC LETTER TEH MARBUTA
0xCA 0x062A #ARABIC LETTER TEH
0xCB 0x062B #ARABIC LETTER THEH
0xCC 0x062C #ARABIC LETTER JEEM
0xCD 0x062D #ARABIC LETTER HAH
0xCE 0x062E #ARABIC LETTER KHAH
0xCF 0x062F #ARABIC LETTER DAL
0xD0 0x0630 #ARABIC LETTER THAL
0xD1 0x0631 #ARABIC LETTER REH
0xD2 0x0632 #ARABIC LETTER ZAIN
0xD3 0x0633 #ARABIC LETTER SEEN
0xD4 0x0634 #ARABIC LETTER SHEEN
0xD5 0x0635 #ARABIC LETTER SAD
0xD6 0x0636 #ARABIC LETTER DAD
0xD7 0x00D7 #MULTIPLICATION SIGN
0xD8 0x0637 #ARABIC LETTER TAH
0xD9 0x0638 #ARABIC LETTER ZAH
0xDA 0x0639 #ARABIC LETTER AIN
0xDB 0x063A #ARABIC LETTER GHAIN
0xDC 0x0640 #ARABIC TATWEEL
0xDD 0x0641 #ARABIC LETTER FEH
0xDE 0x0642 #ARABIC LETTER QAF
0xDF 0x0643 #ARABIC LETTER KAF
0xE0 0x00E0 #LATIN SMALL LETTER A WITH GRAVE
0xE1 0x0644 #ARABIC LETTER LAM
0xE2 0x00E2 #LATIN SMALL LETTER A WITH CIRCUMFLEX
0xE3 0x0645 #ARABIC LETTER MEEM
0xE4 0x0646 #ARABIC LETTER NOON
0xE5 0x0647 #ARABIC LETTER HEH
0xE6 0x0648 #ARABIC LETTER WAW
0xE7 0x00E7 #LATIN SMALL LETTER C WITH CEDILLA
0xE8 0x00E8 #LATIN SMALL LETTER E WITH GRAVE
0xE9 0x00E9 #LATIN SMALL LETTER E WITH ACUTE
0xEA 0x00EA #LATIN SMALL LETTER E WITH CIRCUMFLEX
0xEB 0x00EB #LATIN SMALL LETTER E WITH DIAERESIS
0xEC 0x0649 #ARABIC LETTER ALEF MAKSURA
0xED 0x064A #ARABIC LETTER YEH
0xEE 0x00EE #LATIN SMALL LETTER I WITH CIRCUMFLEX
0xEF 0x00EF #LATIN SMALL LETTER I WITH DIAERESIS
0xF0 0x064B #ARABIC FATHATAN
0xF1 0x064C #ARABIC DAMMATAN
0xF2 0x064D #ARABIC KASRATAN
0xF3 0x064E #ARABIC FATHA
0xF4 0x00F4 #LATIN SMALL LETTER O WITH CIRCUMFLEX
0xF5 0x064F #ARABIC DAMMA
0xF6 0x0650 #ARABIC KASRA
0xF7 0x00F7 #DIVISION SIGN
0xF8 0x0651 #ARABIC SHADDA
0xF9 0x00F9 #LATIN SMALL LETTER U WITH GRAVE
0xFA 0x0652 #ARABIC SUKUN
0xFB 0x00FB #LATIN SMALL LETTER U WITH CIRCUMFLEX
0xFC 0x00FC #LATIN SMALL LETTER U WITH DIAERESIS
0xFD 0x200E #LEFT-TO-RIGHT MARK
0xFE 0x200F #RIGHT-TO-LEFT MARK
0xFF 0x06D2 #ARABIC LETTER YEH BARREE

274
data/windows/CP1257.TXT Normal file
View File

@@ -0,0 +1,274 @@
#
# Name: cp1257 to Unicode table
# Unicode version: 2.0
# Table version: 2.01
# Table format: Format A
# Date: 04/15/98
#
# Contact: Shawn.Steele@microsoft.com
#
# General notes: none
#
# Format: Three tab-separated columns
# Column #1 is the cp1257 code (in hex)
# Column #2 is the Unicode (in hex as 0xXXXX)
# Column #3 is the Unicode name (follows a comment sign, '#')
#
# The entries are in cp1257 order
#
0x00 0x0000 #NULL
0x01 0x0001 #START OF HEADING
0x02 0x0002 #START OF TEXT
0x03 0x0003 #END OF TEXT
0x04 0x0004 #END OF TRANSMISSION
0x05 0x0005 #ENQUIRY
0x06 0x0006 #ACKNOWLEDGE
0x07 0x0007 #BELL
0x08 0x0008 #BACKSPACE
0x09 0x0009 #HORIZONTAL TABULATION
0x0A 0x000A #LINE FEED
0x0B 0x000B #VERTICAL TABULATION
0x0C 0x000C #FORM FEED
0x0D 0x000D #CARRIAGE RETURN
0x0E 0x000E #SHIFT OUT
0x0F 0x000F #SHIFT IN
0x10 0x0010 #DATA LINK ESCAPE
0x11 0x0011 #DEVICE CONTROL ONE
0x12 0x0012 #DEVICE CONTROL TWO
0x13 0x0013 #DEVICE CONTROL THREE
0x14 0x0014 #DEVICE CONTROL FOUR
0x15 0x0015 #NEGATIVE ACKNOWLEDGE
0x16 0x0016 #SYNCHRONOUS IDLE
0x17 0x0017 #END OF TRANSMISSION BLOCK
0x18 0x0018 #CANCEL
0x19 0x0019 #END OF MEDIUM
0x1A 0x001A #SUBSTITUTE
0x1B 0x001B #ESCAPE
0x1C 0x001C #FILE SEPARATOR
0x1D 0x001D #GROUP SEPARATOR
0x1E 0x001E #RECORD SEPARATOR
0x1F 0x001F #UNIT SEPARATOR
0x20 0x0020 #SPACE
0x21 0x0021 #EXCLAMATION MARK
0x22 0x0022 #QUOTATION MARK
0x23 0x0023 #NUMBER SIGN
0x24 0x0024 #DOLLAR SIGN
0x25 0x0025 #PERCENT SIGN
0x26 0x0026 #AMPERSAND
0x27 0x0027 #APOSTROPHE
0x28 0x0028 #LEFT PARENTHESIS
0x29 0x0029 #RIGHT PARENTHESIS
0x2A 0x002A #ASTERISK
0x2B 0x002B #PLUS SIGN
0x2C 0x002C #COMMA
0x2D 0x002D #HYPHEN-MINUS
0x2E 0x002E #FULL STOP
0x2F 0x002F #SOLIDUS
0x30 0x0030 #DIGIT ZERO
0x31 0x0031 #DIGIT ONE
0x32 0x0032 #DIGIT TWO
0x33 0x0033 #DIGIT THREE
0x34 0x0034 #DIGIT FOUR
0x35 0x0035 #DIGIT FIVE
0x36 0x0036 #DIGIT SIX
0x37 0x0037 #DIGIT SEVEN
0x38 0x0038 #DIGIT EIGHT
0x39 0x0039 #DIGIT NINE
0x3A 0x003A #COLON
0x3B 0x003B #SEMICOLON
0x3C 0x003C #LESS-THAN SIGN
0x3D 0x003D #EQUALS SIGN
0x3E 0x003E #GREATER-THAN SIGN
0x3F 0x003F #QUESTION MARK
0x40 0x0040 #COMMERCIAL AT
0x41 0x0041 #LATIN CAPITAL LETTER A
0x42 0x0042 #LATIN CAPITAL LETTER B
0x43 0x0043 #LATIN CAPITAL LETTER C
0x44 0x0044 #LATIN CAPITAL LETTER D
0x45 0x0045 #LATIN CAPITAL LETTER E
0x46 0x0046 #LATIN CAPITAL LETTER F
0x47 0x0047 #LATIN CAPITAL LETTER G
0x48 0x0048 #LATIN CAPITAL LETTER H
0x49 0x0049 #LATIN CAPITAL LETTER I
0x4A 0x004A #LATIN CAPITAL LETTER J
0x4B 0x004B #LATIN CAPITAL LETTER K
0x4C 0x004C #LATIN CAPITAL LETTER L
0x4D 0x004D #LATIN CAPITAL LETTER M
0x4E 0x004E #LATIN CAPITAL LETTER N
0x4F 0x004F #LATIN CAPITAL LETTER O
0x50 0x0050 #LATIN CAPITAL LETTER P
0x51 0x0051 #LATIN CAPITAL LETTER Q
0x52 0x0052 #LATIN CAPITAL LETTER R
0x53 0x0053 #LATIN CAPITAL LETTER S
0x54 0x0054 #LATIN CAPITAL LETTER T
0x55 0x0055 #LATIN CAPITAL LETTER U
0x56 0x0056 #LATIN CAPITAL LETTER V
0x57 0x0057 #LATIN CAPITAL LETTER W
0x58 0x0058 #LATIN CAPITAL LETTER X
0x59 0x0059 #LATIN CAPITAL LETTER Y
0x5A 0x005A #LATIN CAPITAL LETTER Z
0x5B 0x005B #LEFT SQUARE BRACKET
0x5C 0x005C #REVERSE SOLIDUS
0x5D 0x005D #RIGHT SQUARE BRACKET
0x5E 0x005E #CIRCUMFLEX ACCENT
0x5F 0x005F #LOW LINE
0x60 0x0060 #GRAVE ACCENT
0x61 0x0061 #LATIN SMALL LETTER A
0x62 0x0062 #LATIN SMALL LETTER B
0x63 0x0063 #LATIN SMALL LETTER C
0x64 0x0064 #LATIN SMALL LETTER D
0x65 0x0065 #LATIN SMALL LETTER E
0x66 0x0066 #LATIN SMALL LETTER F
0x67 0x0067 #LATIN SMALL LETTER G
0x68 0x0068 #LATIN SMALL LETTER H
0x69 0x0069 #LATIN SMALL LETTER I
0x6A 0x006A #LATIN SMALL LETTER J
0x6B 0x006B #LATIN SMALL LETTER K
0x6C 0x006C #LATIN SMALL LETTER L
0x6D 0x006D #LATIN SMALL LETTER M
0x6E 0x006E #LATIN SMALL LETTER N
0x6F 0x006F #LATIN SMALL LETTER O
0x70 0x0070 #LATIN SMALL LETTER P
0x71 0x0071 #LATIN SMALL LETTER Q
0x72 0x0072 #LATIN SMALL LETTER R
0x73 0x0073 #LATIN SMALL LETTER S
0x74 0x0074 #LATIN SMALL LETTER T
0x75 0x0075 #LATIN SMALL LETTER U
0x76 0x0076 #LATIN SMALL LETTER V
0x77 0x0077 #LATIN SMALL LETTER W
0x78 0x0078 #LATIN SMALL LETTER X
0x79 0x0079 #LATIN SMALL LETTER Y
0x7A 0x007A #LATIN SMALL LETTER Z
0x7B 0x007B #LEFT CURLY BRACKET
0x7C 0x007C #VERTICAL LINE
0x7D 0x007D #RIGHT CURLY BRACKET
0x7E 0x007E #TILDE
0x7F 0x007F #DELETE
0x80 0x20AC #EURO SIGN
0x81 #UNDEFINED
0x82 0x201A #SINGLE LOW-9 QUOTATION MARK
0x83 #UNDEFINED
0x84 0x201E #DOUBLE LOW-9 QUOTATION MARK
0x85 0x2026 #HORIZONTAL ELLIPSIS
0x86 0x2020 #DAGGER
0x87 0x2021 #DOUBLE DAGGER
0x88 #UNDEFINED
0x89 0x2030 #PER MILLE SIGN
0x8A #UNDEFINED
0x8B 0x2039 #SINGLE LEFT-POINTING ANGLE QUOTATION MARK
0x8C #UNDEFINED
0x8D 0x00A8 #DIAERESIS
0x8E 0x02C7 #CARON
0x8F 0x00B8 #CEDILLA
0x90 #UNDEFINED
0x91 0x2018 #LEFT SINGLE QUOTATION MARK
0x92 0x2019 #RIGHT SINGLE QUOTATION MARK
0x93 0x201C #LEFT DOUBLE QUOTATION MARK
0x94 0x201D #RIGHT DOUBLE QUOTATION MARK
0x95 0x2022 #BULLET
0x96 0x2013 #EN DASH
0x97 0x2014 #EM DASH
0x98 #UNDEFINED
0x99 0x2122 #TRADE MARK SIGN
0x9A #UNDEFINED
0x9B 0x203A #SINGLE RIGHT-POINTING ANGLE QUOTATION MARK
0x9C #UNDEFINED
0x9D 0x00AF #MACRON
0x9E 0x02DB #OGONEK
0x9F #UNDEFINED
0xA0 0x00A0 #NO-BREAK SPACE
0xA1 #UNDEFINED
0xA2 0x00A2 #CENT SIGN
0xA3 0x00A3 #POUND SIGN
0xA4 0x00A4 #CURRENCY SIGN
0xA5 #UNDEFINED
0xA6 0x00A6 #BROKEN BAR
0xA7 0x00A7 #SECTION SIGN
0xA8 0x00D8 #LATIN CAPITAL LETTER O WITH STROKE
0xA9 0x00A9 #COPYRIGHT SIGN
0xAA 0x0156 #LATIN CAPITAL LETTER R WITH CEDILLA
0xAB 0x00AB #LEFT-POINTING DOUBLE ANGLE QUOTATION MARK
0xAC 0x00AC #NOT SIGN
0xAD 0x00AD #SOFT HYPHEN
0xAE 0x00AE #REGISTERED SIGN
0xAF 0x00C6 #LATIN CAPITAL LETTER AE
0xB0 0x00B0 #DEGREE SIGN
0xB1 0x00B1 #PLUS-MINUS SIGN
0xB2 0x00B2 #SUPERSCRIPT TWO
0xB3 0x00B3 #SUPERSCRIPT THREE
0xB4 0x00B4 #ACUTE ACCENT
0xB5 0x00B5 #MICRO SIGN
0xB6 0x00B6 #PILCROW SIGN
0xB7 0x00B7 #MIDDLE DOT
0xB8 0x00F8 #LATIN SMALL LETTER O WITH STROKE
0xB9 0x00B9 #SUPERSCRIPT ONE
0xBA 0x0157 #LATIN SMALL LETTER R WITH CEDILLA
0xBB 0x00BB #RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK
0xBC 0x00BC #VULGAR FRACTION ONE QUARTER
0xBD 0x00BD #VULGAR FRACTION ONE HALF
0xBE 0x00BE #VULGAR FRACTION THREE QUARTERS
0xBF 0x00E6 #LATIN SMALL LETTER AE
0xC0 0x0104 #LATIN CAPITAL LETTER A WITH OGONEK
0xC1 0x012E #LATIN CAPITAL LETTER I WITH OGONEK
0xC2 0x0100 #LATIN CAPITAL LETTER A WITH MACRON
0xC3 0x0106 #LATIN CAPITAL LETTER C WITH ACUTE
0xC4 0x00C4 #LATIN CAPITAL LETTER A WITH DIAERESIS
0xC5 0x00C5 #LATIN CAPITAL LETTER A WITH RING ABOVE
0xC6 0x0118 #LATIN CAPITAL LETTER E WITH OGONEK
0xC7 0x0112 #LATIN CAPITAL LETTER E WITH MACRON
0xC8 0x010C #LATIN CAPITAL LETTER C WITH CARON
0xC9 0x00C9 #LATIN CAPITAL LETTER E WITH ACUTE
0xCA 0x0179 #LATIN CAPITAL LETTER Z WITH ACUTE
0xCB 0x0116 #LATIN CAPITAL LETTER E WITH DOT ABOVE
0xCC 0x0122 #LATIN CAPITAL LETTER G WITH CEDILLA
0xCD 0x0136 #LATIN CAPITAL LETTER K WITH CEDILLA
0xCE 0x012A #LATIN CAPITAL LETTER I WITH MACRON
0xCF 0x013B #LATIN CAPITAL LETTER L WITH CEDILLA
0xD0 0x0160 #LATIN CAPITAL LETTER S WITH CARON
0xD1 0x0143 #LATIN CAPITAL LETTER N WITH ACUTE
0xD2 0x0145 #LATIN CAPITAL LETTER N WITH CEDILLA
0xD3 0x00D3 #LATIN CAPITAL LETTER O WITH ACUTE
0xD4 0x014C #LATIN CAPITAL LETTER O WITH MACRON
0xD5 0x00D5 #LATIN CAPITAL LETTER O WITH TILDE
0xD6 0x00D6 #LATIN CAPITAL LETTER O WITH DIAERESIS
0xD7 0x00D7 #MULTIPLICATION SIGN
0xD8 0x0172 #LATIN CAPITAL LETTER U WITH OGONEK
0xD9 0x0141 #LATIN CAPITAL LETTER L WITH STROKE
0xDA 0x015A #LATIN CAPITAL LETTER S WITH ACUTE
0xDB 0x016A #LATIN CAPITAL LETTER U WITH MACRON
0xDC 0x00DC #LATIN CAPITAL LETTER U WITH DIAERESIS
0xDD 0x017B #LATIN CAPITAL LETTER Z WITH DOT ABOVE
0xDE 0x017D #LATIN CAPITAL LETTER Z WITH CARON
0xDF 0x00DF #LATIN SMALL LETTER SHARP S
0xE0 0x0105 #LATIN SMALL LETTER A WITH OGONEK
0xE1 0x012F #LATIN SMALL LETTER I WITH OGONEK
0xE2 0x0101 #LATIN SMALL LETTER A WITH MACRON
0xE3 0x0107 #LATIN SMALL LETTER C WITH ACUTE
0xE4 0x00E4 #LATIN SMALL LETTER A WITH DIAERESIS
0xE5 0x00E5 #LATIN SMALL LETTER A WITH RING ABOVE
0xE6 0x0119 #LATIN SMALL LETTER E WITH OGONEK
0xE7 0x0113 #LATIN SMALL LETTER E WITH MACRON
0xE8 0x010D #LATIN SMALL LETTER C WITH CARON
0xE9 0x00E9 #LATIN SMALL LETTER E WITH ACUTE
0xEA 0x017A #LATIN SMALL LETTER Z WITH ACUTE
0xEB 0x0117 #LATIN SMALL LETTER E WITH DOT ABOVE
0xEC 0x0123 #LATIN SMALL LETTER G WITH CEDILLA
0xED 0x0137 #LATIN SMALL LETTER K WITH CEDILLA
0xEE 0x012B #LATIN SMALL LETTER I WITH MACRON
0xEF 0x013C #LATIN SMALL LETTER L WITH CEDILLA
0xF0 0x0161 #LATIN SMALL LETTER S WITH CARON
0xF1 0x0144 #LATIN SMALL LETTER N WITH ACUTE
0xF2 0x0146 #LATIN SMALL LETTER N WITH CEDILLA
0xF3 0x00F3 #LATIN SMALL LETTER O WITH ACUTE
0xF4 0x014D #LATIN SMALL LETTER O WITH MACRON
0xF5 0x00F5 #LATIN SMALL LETTER O WITH TILDE
0xF6 0x00F6 #LATIN SMALL LETTER O WITH DIAERESIS
0xF7 0x00F7 #DIVISION SIGN
0xF8 0x0173 #LATIN SMALL LETTER U WITH OGONEK
0xF9 0x0142 #LATIN SMALL LETTER L WITH STROKE
0xFA 0x015B #LATIN SMALL LETTER S WITH ACUTE
0xFB 0x016B #LATIN SMALL LETTER U WITH MACRON
0xFC 0x00FC #LATIN SMALL LETTER U WITH DIAERESIS
0xFD 0x017C #LATIN SMALL LETTER Z WITH DOT ABOVE
0xFE 0x017E #LATIN SMALL LETTER Z WITH CARON
0xFF 0x02D9 #DOT ABOVE

274
data/windows/CP1258.TXT Normal file
View File

@@ -0,0 +1,274 @@
#
# Name: cp1258 to Unicode table
# Unicode version: 2.0
# Table version: 2.01
# Table format: Format A
# Date: 04/15/98
#
# Contact: Shawn.Steele@microsoft.com
#
# General notes: none
#
# Format: Three tab-separated columns
# Column #1 is the cp1258 code (in hex)
# Column #2 is the Unicode (in hex as 0xXXXX)
# Column #3 is the Unicode name (follows a comment sign, '#')
#
# The entries are in cp1258 order
#
0x00 0x0000 #NULL
0x01 0x0001 #START OF HEADING
0x02 0x0002 #START OF TEXT
0x03 0x0003 #END OF TEXT
0x04 0x0004 #END OF TRANSMISSION
0x05 0x0005 #ENQUIRY
0x06 0x0006 #ACKNOWLEDGE
0x07 0x0007 #BELL
0x08 0x0008 #BACKSPACE
0x09 0x0009 #HORIZONTAL TABULATION
0x0A 0x000A #LINE FEED
0x0B 0x000B #VERTICAL TABULATION
0x0C 0x000C #FORM FEED
0x0D 0x000D #CARRIAGE RETURN
0x0E 0x000E #SHIFT OUT
0x0F 0x000F #SHIFT IN
0x10 0x0010 #DATA LINK ESCAPE
0x11 0x0011 #DEVICE CONTROL ONE
0x12 0x0012 #DEVICE CONTROL TWO
0x13 0x0013 #DEVICE CONTROL THREE
0x14 0x0014 #DEVICE CONTROL FOUR
0x15 0x0015 #NEGATIVE ACKNOWLEDGE
0x16 0x0016 #SYNCHRONOUS IDLE
0x17 0x0017 #END OF TRANSMISSION BLOCK
0x18 0x0018 #CANCEL
0x19 0x0019 #END OF MEDIUM
0x1A 0x001A #SUBSTITUTE
0x1B 0x001B #ESCAPE
0x1C 0x001C #FILE SEPARATOR
0x1D 0x001D #GROUP SEPARATOR
0x1E 0x001E #RECORD SEPARATOR
0x1F 0x001F #UNIT SEPARATOR
0x20 0x0020 #SPACE
0x21 0x0021 #EXCLAMATION MARK
0x22 0x0022 #QUOTATION MARK
0x23 0x0023 #NUMBER SIGN
0x24 0x0024 #DOLLAR SIGN
0x25 0x0025 #PERCENT SIGN
0x26 0x0026 #AMPERSAND
0x27 0x0027 #APOSTROPHE
0x28 0x0028 #LEFT PARENTHESIS
0x29 0x0029 #RIGHT PARENTHESIS
0x2A 0x002A #ASTERISK
0x2B 0x002B #PLUS SIGN
0x2C 0x002C #COMMA
0x2D 0x002D #HYPHEN-MINUS
0x2E 0x002E #FULL STOP
0x2F 0x002F #SOLIDUS
0x30 0x0030 #DIGIT ZERO
0x31 0x0031 #DIGIT ONE
0x32 0x0032 #DIGIT TWO
0x33 0x0033 #DIGIT THREE
0x34 0x0034 #DIGIT FOUR
0x35 0x0035 #DIGIT FIVE
0x36 0x0036 #DIGIT SIX
0x37 0x0037 #DIGIT SEVEN
0x38 0x0038 #DIGIT EIGHT
0x39 0x0039 #DIGIT NINE
0x3A 0x003A #COLON
0x3B 0x003B #SEMICOLON
0x3C 0x003C #LESS-THAN SIGN
0x3D 0x003D #EQUALS SIGN
0x3E 0x003E #GREATER-THAN SIGN
0x3F 0x003F #QUESTION MARK
0x40 0x0040 #COMMERCIAL AT
0x41 0x0041 #LATIN CAPITAL LETTER A
0x42 0x0042 #LATIN CAPITAL LETTER B
0x43 0x0043 #LATIN CAPITAL LETTER C
0x44 0x0044 #LATIN CAPITAL LETTER D
0x45 0x0045 #LATIN CAPITAL LETTER E
0x46 0x0046 #LATIN CAPITAL LETTER F
0x47 0x0047 #LATIN CAPITAL LETTER G
0x48 0x0048 #LATIN CAPITAL LETTER H
0x49 0x0049 #LATIN CAPITAL LETTER I
0x4A 0x004A #LATIN CAPITAL LETTER J
0x4B 0x004B #LATIN CAPITAL LETTER K
0x4C 0x004C #LATIN CAPITAL LETTER L
0x4D 0x004D #LATIN CAPITAL LETTER M
0x4E 0x004E #LATIN CAPITAL LETTER N
0x4F 0x004F #LATIN CAPITAL LETTER O
0x50 0x0050 #LATIN CAPITAL LETTER P
0x51 0x0051 #LATIN CAPITAL LETTER Q
0x52 0x0052 #LATIN CAPITAL LETTER R
0x53 0x0053 #LATIN CAPITAL LETTER S
0x54 0x0054 #LATIN CAPITAL LETTER T
0x55 0x0055 #LATIN CAPITAL LETTER U
0x56 0x0056 #LATIN CAPITAL LETTER V
0x57 0x0057 #LATIN CAPITAL LETTER W
0x58 0x0058 #LATIN CAPITAL LETTER X
0x59 0x0059 #LATIN CAPITAL LETTER Y
0x5A 0x005A #LATIN CAPITAL LETTER Z
0x5B 0x005B #LEFT SQUARE BRACKET
0x5C 0x005C #REVERSE SOLIDUS
0x5D 0x005D #RIGHT SQUARE BRACKET
0x5E 0x005E #CIRCUMFLEX ACCENT
0x5F 0x005F #LOW LINE
0x60 0x0060 #GRAVE ACCENT
0x61 0x0061 #LATIN SMALL LETTER A
0x62 0x0062 #LATIN SMALL LETTER B
0x63 0x0063 #LATIN SMALL LETTER C
0x64 0x0064 #LATIN SMALL LETTER D
0x65 0x0065 #LATIN SMALL LETTER E
0x66 0x0066 #LATIN SMALL LETTER F
0x67 0x0067 #LATIN SMALL LETTER G
0x68 0x0068 #LATIN SMALL LETTER H
0x69 0x0069 #LATIN SMALL LETTER I
0x6A 0x006A #LATIN SMALL LETTER J
0x6B 0x006B #LATIN SMALL LETTER K
0x6C 0x006C #LATIN SMALL LETTER L
0x6D 0x006D #LATIN SMALL LETTER M
0x6E 0x006E #LATIN SMALL LETTER N
0x6F 0x006F #LATIN SMALL LETTER O
0x70 0x0070 #LATIN SMALL LETTER P
0x71 0x0071 #LATIN SMALL LETTER Q
0x72 0x0072 #LATIN SMALL LETTER R
0x73 0x0073 #LATIN SMALL LETTER S
0x74 0x0074 #LATIN SMALL LETTER T
0x75 0x0075 #LATIN SMALL LETTER U
0x76 0x0076 #LATIN SMALL LETTER V
0x77 0x0077 #LATIN SMALL LETTER W
0x78 0x0078 #LATIN SMALL LETTER X
0x79 0x0079 #LATIN SMALL LETTER Y
0x7A 0x007A #LATIN SMALL LETTER Z
0x7B 0x007B #LEFT CURLY BRACKET
0x7C 0x007C #VERTICAL LINE
0x7D 0x007D #RIGHT CURLY BRACKET
0x7E 0x007E #TILDE
0x7F 0x007F #DELETE
0x80 0x20AC #EURO SIGN
0x81 #UNDEFINED
0x82 0x201A #SINGLE LOW-9 QUOTATION MARK
0x83 0x0192 #LATIN SMALL LETTER F WITH HOOK
0x84 0x201E #DOUBLE LOW-9 QUOTATION MARK
0x85 0x2026 #HORIZONTAL ELLIPSIS
0x86 0x2020 #DAGGER
0x87 0x2021 #DOUBLE DAGGER
0x88 0x02C6 #MODIFIER LETTER CIRCUMFLEX ACCENT
0x89 0x2030 #PER MILLE SIGN
0x8A #UNDEFINED
0x8B 0x2039 #SINGLE LEFT-POINTING ANGLE QUOTATION MARK
0x8C 0x0152 #LATIN CAPITAL LIGATURE OE
0x8D #UNDEFINED
0x8E #UNDEFINED
0x8F #UNDEFINED
0x90 #UNDEFINED
0x91 0x2018 #LEFT SINGLE QUOTATION MARK
0x92 0x2019 #RIGHT SINGLE QUOTATION MARK
0x93 0x201C #LEFT DOUBLE QUOTATION MARK
0x94 0x201D #RIGHT DOUBLE QUOTATION MARK
0x95 0x2022 #BULLET
0x96 0x2013 #EN DASH
0x97 0x2014 #EM DASH
0x98 0x02DC #SMALL TILDE
0x99 0x2122 #TRADE MARK SIGN
0x9A #UNDEFINED
0x9B 0x203A #SINGLE RIGHT-POINTING ANGLE QUOTATION MARK
0x9C 0x0153 #LATIN SMALL LIGATURE OE
0x9D #UNDEFINED
0x9E #UNDEFINED
0x9F 0x0178 #LATIN CAPITAL LETTER Y WITH DIAERESIS
0xA0 0x00A0 #NO-BREAK SPACE
0xA1 0x00A1 #INVERTED EXCLAMATION MARK
0xA2 0x00A2 #CENT SIGN
0xA3 0x00A3 #POUND SIGN
0xA4 0x00A4 #CURRENCY SIGN
0xA5 0x00A5 #YEN SIGN
0xA6 0x00A6 #BROKEN BAR
0xA7 0x00A7 #SECTION SIGN
0xA8 0x00A8 #DIAERESIS
0xA9 0x00A9 #COPYRIGHT SIGN
0xAA 0x00AA #FEMININE ORDINAL INDICATOR
0xAB 0x00AB #LEFT-POINTING DOUBLE ANGLE QUOTATION MARK
0xAC 0x00AC #NOT SIGN
0xAD 0x00AD #SOFT HYPHEN
0xAE 0x00AE #REGISTERED SIGN
0xAF 0x00AF #MACRON
0xB0 0x00B0 #DEGREE SIGN
0xB1 0x00B1 #PLUS-MINUS SIGN
0xB2 0x00B2 #SUPERSCRIPT TWO
0xB3 0x00B3 #SUPERSCRIPT THREE
0xB4 0x00B4 #ACUTE ACCENT
0xB5 0x00B5 #MICRO SIGN
0xB6 0x00B6 #PILCROW SIGN
0xB7 0x00B7 #MIDDLE DOT
0xB8 0x00B8 #CEDILLA
0xB9 0x00B9 #SUPERSCRIPT ONE
0xBA 0x00BA #MASCULINE ORDINAL INDICATOR
0xBB 0x00BB #RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK
0xBC 0x00BC #VULGAR FRACTION ONE QUARTER
0xBD 0x00BD #VULGAR FRACTION ONE HALF
0xBE 0x00BE #VULGAR FRACTION THREE QUARTERS
0xBF 0x00BF #INVERTED QUESTION MARK
0xC0 0x00C0 #LATIN CAPITAL LETTER A WITH GRAVE
0xC1 0x00C1 #LATIN CAPITAL LETTER A WITH ACUTE
0xC2 0x00C2 #LATIN CAPITAL LETTER A WITH CIRCUMFLEX
0xC3 0x0102 #LATIN CAPITAL LETTER A WITH BREVE
0xC4 0x00C4 #LATIN CAPITAL LETTER A WITH DIAERESIS
0xC5 0x00C5 #LATIN CAPITAL LETTER A WITH RING ABOVE
0xC6 0x00C6 #LATIN CAPITAL LETTER AE
0xC7 0x00C7 #LATIN CAPITAL LETTER C WITH CEDILLA
0xC8 0x00C8 #LATIN CAPITAL LETTER E WITH GRAVE
0xC9 0x00C9 #LATIN CAPITAL LETTER E WITH ACUTE
0xCA 0x00CA #LATIN CAPITAL LETTER E WITH CIRCUMFLEX
0xCB 0x00CB #LATIN CAPITAL LETTER E WITH DIAERESIS
0xCC 0x0300 #COMBINING GRAVE ACCENT
0xCD 0x00CD #LATIN CAPITAL LETTER I WITH ACUTE
0xCE 0x00CE #LATIN CAPITAL LETTER I WITH CIRCUMFLEX
0xCF 0x00CF #LATIN CAPITAL LETTER I WITH DIAERESIS
0xD0 0x0110 #LATIN CAPITAL LETTER D WITH STROKE
0xD1 0x00D1 #LATIN CAPITAL LETTER N WITH TILDE
0xD2 0x0309 #COMBINING HOOK ABOVE
0xD3 0x00D3 #LATIN CAPITAL LETTER O WITH ACUTE
0xD4 0x00D4 #LATIN CAPITAL LETTER O WITH CIRCUMFLEX
0xD5 0x01A0 #LATIN CAPITAL LETTER O WITH HORN
0xD6 0x00D6 #LATIN CAPITAL LETTER O WITH DIAERESIS
0xD7 0x00D7 #MULTIPLICATION SIGN
0xD8 0x00D8 #LATIN CAPITAL LETTER O WITH STROKE
0xD9 0x00D9 #LATIN CAPITAL LETTER U WITH GRAVE
0xDA 0x00DA #LATIN CAPITAL LETTER U WITH ACUTE
0xDB 0x00DB #LATIN CAPITAL LETTER U WITH CIRCUMFLEX
0xDC 0x00DC #LATIN CAPITAL LETTER U WITH DIAERESIS
0xDD 0x01AF #LATIN CAPITAL LETTER U WITH HORN
0xDE 0x0303 #COMBINING TILDE
0xDF 0x00DF #LATIN SMALL LETTER SHARP S
0xE0 0x00E0 #LATIN SMALL LETTER A WITH GRAVE
0xE1 0x00E1 #LATIN SMALL LETTER A WITH ACUTE
0xE2 0x00E2 #LATIN SMALL LETTER A WITH CIRCUMFLEX
0xE3 0x0103 #LATIN SMALL LETTER A WITH BREVE
0xE4 0x00E4 #LATIN SMALL LETTER A WITH DIAERESIS
0xE5 0x00E5 #LATIN SMALL LETTER A WITH RING ABOVE
0xE6 0x00E6 #LATIN SMALL LETTER AE
0xE7 0x00E7 #LATIN SMALL LETTER C WITH CEDILLA
0xE8 0x00E8 #LATIN SMALL LETTER E WITH GRAVE
0xE9 0x00E9 #LATIN SMALL LETTER E WITH ACUTE
0xEA 0x00EA #LATIN SMALL LETTER E WITH CIRCUMFLEX
0xEB 0x00EB #LATIN SMALL LETTER E WITH DIAERESIS
0xEC 0x0301 #COMBINING ACUTE ACCENT
0xED 0x00ED #LATIN SMALL LETTER I WITH ACUTE
0xEE 0x00EE #LATIN SMALL LETTER I WITH CIRCUMFLEX
0xEF 0x00EF #LATIN SMALL LETTER I WITH DIAERESIS
0xF0 0x0111 #LATIN SMALL LETTER D WITH STROKE
0xF1 0x00F1 #LATIN SMALL LETTER N WITH TILDE
0xF2 0x0323 #COMBINING DOT BELOW
0xF3 0x00F3 #LATIN SMALL LETTER O WITH ACUTE
0xF4 0x00F4 #LATIN SMALL LETTER O WITH CIRCUMFLEX
0xF5 0x01A1 #LATIN SMALL LETTER O WITH HORN
0xF6 0x00F6 #LATIN SMALL LETTER O WITH DIAERESIS
0xF7 0x00F7 #DIVISION SIGN
0xF8 0x00F8 #LATIN SMALL LETTER O WITH STROKE
0xF9 0x00F9 #LATIN SMALL LETTER U WITH GRAVE
0xFA 0x00FA #LATIN SMALL LETTER U WITH ACUTE
0xFB 0x00FB #LATIN SMALL LETTER U WITH CIRCUMFLEX
0xFC 0x00FC #LATIN SMALL LETTER U WITH DIAERESIS
0xFD 0x01B0 #LATIN SMALL LETTER U WITH HORN
0xFE 0x20AB #DONG SIGN
0xFF 0x00FF #LATIN SMALL LETTER Y WITH DIAERESIS

274
data/windows/CP874.TXT Normal file
View File

@@ -0,0 +1,274 @@
#
# Name: cp874 to Unicode table
# Unicode version: 2.0
# Table version: 2.01
# Table format: Format A
# Date: 02/28/98
#
# Contact: Shawn.Steele@microsoft.com
#
# General notes: none
#
# Format: Three tab-separated columns
# Column #1 is the cp874 code (in hex)
# Column #2 is the Unicode (in hex as 0xXXXX)
# Column #3 is the Unicode name (follows a comment sign, '#')
#
# The entries are in cp874 order
#
0x00 0x0000 #NULL
0x01 0x0001 #START OF HEADING
0x02 0x0002 #START OF TEXT
0x03 0x0003 #END OF TEXT
0x04 0x0004 #END OF TRANSMISSION
0x05 0x0005 #ENQUIRY
0x06 0x0006 #ACKNOWLEDGE
0x07 0x0007 #BELL
0x08 0x0008 #BACKSPACE
0x09 0x0009 #HORIZONTAL TABULATION
0x0A 0x000A #LINE FEED
0x0B 0x000B #VERTICAL TABULATION
0x0C 0x000C #FORM FEED
0x0D 0x000D #CARRIAGE RETURN
0x0E 0x000E #SHIFT OUT
0x0F 0x000F #SHIFT IN
0x10 0x0010 #DATA LINK ESCAPE
0x11 0x0011 #DEVICE CONTROL ONE
0x12 0x0012 #DEVICE CONTROL TWO
0x13 0x0013 #DEVICE CONTROL THREE
0x14 0x0014 #DEVICE CONTROL FOUR
0x15 0x0015 #NEGATIVE ACKNOWLEDGE
0x16 0x0016 #SYNCHRONOUS IDLE
0x17 0x0017 #END OF TRANSMISSION BLOCK
0x18 0x0018 #CANCEL
0x19 0x0019 #END OF MEDIUM
0x1A 0x001A #SUBSTITUTE
0x1B 0x001B #ESCAPE
0x1C 0x001C #FILE SEPARATOR
0x1D 0x001D #GROUP SEPARATOR
0x1E 0x001E #RECORD SEPARATOR
0x1F 0x001F #UNIT SEPARATOR
0x20 0x0020 #SPACE
0x21 0x0021 #EXCLAMATION MARK
0x22 0x0022 #QUOTATION MARK
0x23 0x0023 #NUMBER SIGN
0x24 0x0024 #DOLLAR SIGN
0x25 0x0025 #PERCENT SIGN
0x26 0x0026 #AMPERSAND
0x27 0x0027 #APOSTROPHE
0x28 0x0028 #LEFT PARENTHESIS
0x29 0x0029 #RIGHT PARENTHESIS
0x2A 0x002A #ASTERISK
0x2B 0x002B #PLUS SIGN
0x2C 0x002C #COMMA
0x2D 0x002D #HYPHEN-MINUS
0x2E 0x002E #FULL STOP
0x2F 0x002F #SOLIDUS
0x30 0x0030 #DIGIT ZERO
0x31 0x0031 #DIGIT ONE
0x32 0x0032 #DIGIT TWO
0x33 0x0033 #DIGIT THREE
0x34 0x0034 #DIGIT FOUR
0x35 0x0035 #DIGIT FIVE
0x36 0x0036 #DIGIT SIX
0x37 0x0037 #DIGIT SEVEN
0x38 0x0038 #DIGIT EIGHT
0x39 0x0039 #DIGIT NINE
0x3A 0x003A #COLON
0x3B 0x003B #SEMICOLON
0x3C 0x003C #LESS-THAN SIGN
0x3D 0x003D #EQUALS SIGN
0x3E 0x003E #GREATER-THAN SIGN
0x3F 0x003F #QUESTION MARK
0x40 0x0040 #COMMERCIAL AT
0x41 0x0041 #LATIN CAPITAL LETTER A
0x42 0x0042 #LATIN CAPITAL LETTER B
0x43 0x0043 #LATIN CAPITAL LETTER C
0x44 0x0044 #LATIN CAPITAL LETTER D
0x45 0x0045 #LATIN CAPITAL LETTER E
0x46 0x0046 #LATIN CAPITAL LETTER F
0x47 0x0047 #LATIN CAPITAL LETTER G
0x48 0x0048 #LATIN CAPITAL LETTER H
0x49 0x0049 #LATIN CAPITAL LETTER I
0x4A 0x004A #LATIN CAPITAL LETTER J
0x4B 0x004B #LATIN CAPITAL LETTER K
0x4C 0x004C #LATIN CAPITAL LETTER L
0x4D 0x004D #LATIN CAPITAL LETTER M
0x4E 0x004E #LATIN CAPITAL LETTER N
0x4F 0x004F #LATIN CAPITAL LETTER O
0x50 0x0050 #LATIN CAPITAL LETTER P
0x51 0x0051 #LATIN CAPITAL LETTER Q
0x52 0x0052 #LATIN CAPITAL LETTER R
0x53 0x0053 #LATIN CAPITAL LETTER S
0x54 0x0054 #LATIN CAPITAL LETTER T
0x55 0x0055 #LATIN CAPITAL LETTER U
0x56 0x0056 #LATIN CAPITAL LETTER V
0x57 0x0057 #LATIN CAPITAL LETTER W
0x58 0x0058 #LATIN CAPITAL LETTER X
0x59 0x0059 #LATIN CAPITAL LETTER Y
0x5A 0x005A #LATIN CAPITAL LETTER Z
0x5B 0x005B #LEFT SQUARE BRACKET
0x5C 0x005C #REVERSE SOLIDUS
0x5D 0x005D #RIGHT SQUARE BRACKET
0x5E 0x005E #CIRCUMFLEX ACCENT
0x5F 0x005F #LOW LINE
0x60 0x0060 #GRAVE ACCENT
0x61 0x0061 #LATIN SMALL LETTER A
0x62 0x0062 #LATIN SMALL LETTER B
0x63 0x0063 #LATIN SMALL LETTER C
0x64 0x0064 #LATIN SMALL LETTER D
0x65 0x0065 #LATIN SMALL LETTER E
0x66 0x0066 #LATIN SMALL LETTER F
0x67 0x0067 #LATIN SMALL LETTER G
0x68 0x0068 #LATIN SMALL LETTER H
0x69 0x0069 #LATIN SMALL LETTER I
0x6A 0x006A #LATIN SMALL LETTER J
0x6B 0x006B #LATIN SMALL LETTER K
0x6C 0x006C #LATIN SMALL LETTER L
0x6D 0x006D #LATIN SMALL LETTER M
0x6E 0x006E #LATIN SMALL LETTER N
0x6F 0x006F #LATIN SMALL LETTER O
0x70 0x0070 #LATIN SMALL LETTER P
0x71 0x0071 #LATIN SMALL LETTER Q
0x72 0x0072 #LATIN SMALL LETTER R
0x73 0x0073 #LATIN SMALL LETTER S
0x74 0x0074 #LATIN SMALL LETTER T
0x75 0x0075 #LATIN SMALL LETTER U
0x76 0x0076 #LATIN SMALL LETTER V
0x77 0x0077 #LATIN SMALL LETTER W
0x78 0x0078 #LATIN SMALL LETTER X
0x79 0x0079 #LATIN SMALL LETTER Y
0x7A 0x007A #LATIN SMALL LETTER Z
0x7B 0x007B #LEFT CURLY BRACKET
0x7C 0x007C #VERTICAL LINE
0x7D 0x007D #RIGHT CURLY BRACKET
0x7E 0x007E #TILDE
0x7F 0x007F #DELETE
0x80 0x20AC #EURO SIGN
0x81 #UNDEFINED
0x82 #UNDEFINED
0x83 #UNDEFINED
0x84 #UNDEFINED
0x85 0x2026 #HORIZONTAL ELLIPSIS
0x86 #UNDEFINED
0x87 #UNDEFINED
0x88 #UNDEFINED
0x89 #UNDEFINED
0x8A #UNDEFINED
0x8B #UNDEFINED
0x8C #UNDEFINED
0x8D #UNDEFINED
0x8E #UNDEFINED
0x8F #UNDEFINED
0x90 #UNDEFINED
0x91 0x2018 #LEFT SINGLE QUOTATION MARK
0x92 0x2019 #RIGHT SINGLE QUOTATION MARK
0x93 0x201C #LEFT DOUBLE QUOTATION MARK
0x94 0x201D #RIGHT DOUBLE QUOTATION MARK
0x95 0x2022 #BULLET
0x96 0x2013 #EN DASH
0x97 0x2014 #EM DASH
0x98 #UNDEFINED
0x99 #UNDEFINED
0x9A #UNDEFINED
0x9B #UNDEFINED
0x9C #UNDEFINED
0x9D #UNDEFINED
0x9E #UNDEFINED
0x9F #UNDEFINED
0xA0 0x00A0 #NO-BREAK SPACE
0xA1 0x0E01 #THAI CHARACTER KO KAI
0xA2 0x0E02 #THAI CHARACTER KHO KHAI
0xA3 0x0E03 #THAI CHARACTER KHO KHUAT
0xA4 0x0E04 #THAI CHARACTER KHO KHWAI
0xA5 0x0E05 #THAI CHARACTER KHO KHON
0xA6 0x0E06 #THAI CHARACTER KHO RAKHANG
0xA7 0x0E07 #THAI CHARACTER NGO NGU
0xA8 0x0E08 #THAI CHARACTER CHO CHAN
0xA9 0x0E09 #THAI CHARACTER CHO CHING
0xAA 0x0E0A #THAI CHARACTER CHO CHANG
0xAB 0x0E0B #THAI CHARACTER SO SO
0xAC 0x0E0C #THAI CHARACTER CHO CHOE
0xAD 0x0E0D #THAI CHARACTER YO YING
0xAE 0x0E0E #THAI CHARACTER DO CHADA
0xAF 0x0E0F #THAI CHARACTER TO PATAK
0xB0 0x0E10 #THAI CHARACTER THO THAN
0xB1 0x0E11 #THAI CHARACTER THO NANGMONTHO
0xB2 0x0E12 #THAI CHARACTER THO PHUTHAO
0xB3 0x0E13 #THAI CHARACTER NO NEN
0xB4 0x0E14 #THAI CHARACTER DO DEK
0xB5 0x0E15 #THAI CHARACTER TO TAO
0xB6 0x0E16 #THAI CHARACTER THO THUNG
0xB7 0x0E17 #THAI CHARACTER THO THAHAN
0xB8 0x0E18 #THAI CHARACTER THO THONG
0xB9 0x0E19 #THAI CHARACTER NO NU
0xBA 0x0E1A #THAI CHARACTER BO BAIMAI
0xBB 0x0E1B #THAI CHARACTER PO PLA
0xBC 0x0E1C #THAI CHARACTER PHO PHUNG
0xBD 0x0E1D #THAI CHARACTER FO FA
0xBE 0x0E1E #THAI CHARACTER PHO PHAN
0xBF 0x0E1F #THAI CHARACTER FO FAN
0xC0 0x0E20 #THAI CHARACTER PHO SAMPHAO
0xC1 0x0E21 #THAI CHARACTER MO MA
0xC2 0x0E22 #THAI CHARACTER YO YAK
0xC3 0x0E23 #THAI CHARACTER RO RUA
0xC4 0x0E24 #THAI CHARACTER RU
0xC5 0x0E25 #THAI CHARACTER LO LING
0xC6 0x0E26 #THAI CHARACTER LU
0xC7 0x0E27 #THAI CHARACTER WO WAEN
0xC8 0x0E28 #THAI CHARACTER SO SALA
0xC9 0x0E29 #THAI CHARACTER SO RUSI
0xCA 0x0E2A #THAI CHARACTER SO SUA
0xCB 0x0E2B #THAI CHARACTER HO HIP
0xCC 0x0E2C #THAI CHARACTER LO CHULA
0xCD 0x0E2D #THAI CHARACTER O ANG
0xCE 0x0E2E #THAI CHARACTER HO NOKHUK
0xCF 0x0E2F #THAI CHARACTER PAIYANNOI
0xD0 0x0E30 #THAI CHARACTER SARA A
0xD1 0x0E31 #THAI CHARACTER MAI HAN-AKAT
0xD2 0x0E32 #THAI CHARACTER SARA AA
0xD3 0x0E33 #THAI CHARACTER SARA AM
0xD4 0x0E34 #THAI CHARACTER SARA I
0xD5 0x0E35 #THAI CHARACTER SARA II
0xD6 0x0E36 #THAI CHARACTER SARA UE
0xD7 0x0E37 #THAI CHARACTER SARA UEE
0xD8 0x0E38 #THAI CHARACTER SARA U
0xD9 0x0E39 #THAI CHARACTER SARA UU
0xDA 0x0E3A #THAI CHARACTER PHINTHU
0xDB #UNDEFINED
0xDC #UNDEFINED
0xDD #UNDEFINED
0xDE #UNDEFINED
0xDF 0x0E3F #THAI CURRENCY SYMBOL BAHT
0xE0 0x0E40 #THAI CHARACTER SARA E
0xE1 0x0E41 #THAI CHARACTER SARA AE
0xE2 0x0E42 #THAI CHARACTER SARA O
0xE3 0x0E43 #THAI CHARACTER SARA AI MAIMUAN
0xE4 0x0E44 #THAI CHARACTER SARA AI MAIMALAI
0xE5 0x0E45 #THAI CHARACTER LAKKHANGYAO
0xE6 0x0E46 #THAI CHARACTER MAIYAMOK
0xE7 0x0E47 #THAI CHARACTER MAITAIKHU
0xE8 0x0E48 #THAI CHARACTER MAI EK
0xE9 0x0E49 #THAI CHARACTER MAI THO
0xEA 0x0E4A #THAI CHARACTER MAI TRI
0xEB 0x0E4B #THAI CHARACTER MAI CHATTAWA
0xEC 0x0E4C #THAI CHARACTER THANTHAKHAT
0xED 0x0E4D #THAI CHARACTER NIKHAHIT
0xEE 0x0E4E #THAI CHARACTER YAMAKKAN
0xEF 0x0E4F #THAI CHARACTER FONGMAN
0xF0 0x0E50 #THAI DIGIT ZERO
0xF1 0x0E51 #THAI DIGIT ONE
0xF2 0x0E52 #THAI DIGIT TWO
0xF3 0x0E53 #THAI DIGIT THREE
0xF4 0x0E54 #THAI DIGIT FOUR
0xF5 0x0E55 #THAI DIGIT FIVE
0xF6 0x0E56 #THAI DIGIT SIX
0xF7 0x0E57 #THAI DIGIT SEVEN
0xF8 0x0E58 #THAI DIGIT EIGHT
0xF9 0x0E59 #THAI DIGIT NINE
0xFA 0x0E5A #THAI CHARACTER ANGKHANKHU
0xFB 0x0E5B #THAI CHARACTER KHOMUT
0xFC #UNDEFINED
0xFD #UNDEFINED
0xFE #UNDEFINED
0xFF #UNDEFINED

0
data/CP932.TXT → data/windows/CP932.TXT
View File

0
data/CP936.TXT → data/windows/CP936.TXT
View File

0
data/CP949.TXT → data/windows/CP949.TXT
View File

0
data/CP950.TXT → data/windows/CP950.TXT
View File

515
src/lib.rs
View File

@@ -1,31 +1,39 @@
//! Bidirectional conversion between MS-DOS code pages and Unicode. //! Bidirectional conversion between Microsoft code pages and Unicode.
//! //!
//! MS-DOS and the FAT family of filesystems stored text in one of a family of //! MS-DOS and the FAT family of filesystems stored text in one of a family of
//! OEM code pages selected by country/locale. This crate converts those bytes //! OEM code pages selected by country/locale, and Windows-era files add the
//! to and from Unicode with emulator-grade fidelity: it owns the canonical //! ANSI ("125x") family on top. This crate converts those bytes to and from
//! tables rather than linking ICU, so it stays small, self-contained, and //! Unicode with emulator-grade fidelity: it owns the canonical tables rather
//! cross-compiles cleanly. //! than linking ICU, so it stays small, self-contained, and cross-compiles
//! cleanly.
//! //!
//! The tables are **Microsoft's** mappings as published by the Unicode //! The tables are **Microsoft's** mappings as published by the Unicode
//! Consortium (`VENDORS/MICSFT/`) — not the IBM or Oracle variants, which //! Consortium (`VENDORS/MICSFT/`) — not the IBM or Oracle variants, which
//! differ in a handful of slots. All sixteen single-byte DOS pages are always //! differ in a handful of slots. Two families are exposed:
//! available; the double-byte CJK pages (932 Shift-JIS, 936 GBK, 949 Korean, //!
//! 950 Big5) sit behind the on-by-default `dbcs` feature because their tables //! - [`DosEncoding`] — the sixteen single-byte OEM pages, always available,
//! are large. //! plus the double-byte CJK pages (932 Shift-JIS, 936 GBK, 949 Korean,
//! 950 Big5) behind the on-by-default `dbcs` feature (their tables are
//! large).
//! - [`WindowsEncoding`] — the ten single-byte ANSI pages (874 Thai and
//! 1250–1258), always available.
//! //!
//! Decoding is fallible: several pages leave byte values undefined, and the //! Decoding is fallible: several pages leave byte values undefined, and the
//! double-byte pages add truncated/undefined pair failures. //! double-byte pages add truncated/undefined pair failures. The `decode_lossy`
//! [`DosEncoding::decode_lossy`] substitutes U+FFFD instead. Anything a page //! methods substitute U+FFFD instead. Anything a page decodes re-encodes to
//! decodes re-encodes to the same bytes, except the handful of CP932/CP950 //! the same bytes, except the handful of CP932/CP950 codes that share a
//! codes that share a scalar — those re-encode to Microsoft's preferred code, //! scalar — those re-encode to Microsoft's preferred code, exactly as Windows
//! exactly as Windows does. //! does.
//! //!
//! ``` //! ```
//! use msdos_encodings::DosEncoding; //! use msdos_encodings::{DosEncoding, WindowsEncoding};
//! //!
//! let enc = DosEncoding::default(); // code page 437 //! let enc = DosEncoding::default(); // code page 437
//! assert_eq!(enc.decode(b"Caf\x82").unwrap(), "Café"); //! assert_eq!(enc.decode(b"Caf\x82").unwrap(), "Café");
//! assert_eq!(enc.encode("Café").unwrap(), b"Caf\x82"); //! assert_eq!(enc.encode("Café").unwrap(), b"Caf\x82");
//!
//! let enc = WindowsEncoding::default(); // code page 1252
//! assert_eq!(enc.decode(b"Caf\xE9").unwrap(), "Café");
//! ``` //! ```
#![forbid(unsafe_code)] #![forbid(unsafe_code)]
#![warn(missing_docs)] #![warn(missing_docs)]
@@ -213,15 +221,7 @@ impl DosEncoding {
/// [`DecodeError::TruncatedPair`] and [`DecodeError::UndefinedPair`]. /// [`DecodeError::TruncatedPair`] and [`DecodeError::UndefinedPair`].
pub fn decode(self, bytes: &[u8]) -> Result<String, DecodeError> { pub fn decode(self, bytes: &[u8]) -> Result<String, DecodeError> {
match self.codec() { match self.codec() {
Codec::Single(table) => bytes Codec::Single(table) => decode_single(table, bytes, self.into()),
.iter()
.map(|&byte| {
table[usize::from(byte)].ok_or(DecodeError::Undefined {
byte,
encoding: self,
})
})
.collect(),
#[cfg(feature = "dbcs")] #[cfg(feature = "dbcs")]
Codec::Dbcs(page) => self.decode_dbcs(page, bytes), Codec::Dbcs(page) => self.decode_dbcs(page, bytes),
} }
@@ -234,10 +234,7 @@ impl DosEncoding {
#[must_use] #[must_use]
pub fn decode_lossy(self, bytes: &[u8]) -> String { pub fn decode_lossy(self, bytes: &[u8]) -> String {
match self.codec() { match self.codec() {
Codec::Single(table) => bytes Codec::Single(table) => decode_single_lossy(table, bytes),
.iter()
.map(|&byte| table[usize::from(byte)].unwrap_or(REPLACEMENT))
.collect(),
#[cfg(feature = "dbcs")] #[cfg(feature = "dbcs")]
Codec::Dbcs(page) => Self::decode_dbcs_lossy(page, bytes), Codec::Dbcs(page) => Self::decode_dbcs_lossy(page, bytes),
} }
@@ -251,8 +248,7 @@ impl DosEncoding {
/// ///
/// Returns [`DecodeError`] as [`decode`](Self::decode) does. /// Returns [`DecodeError`] as [`decode`](Self::decode) does.
pub fn decode_cstr(self, bytes: &[u8]) -> Result<String, DecodeError> { pub fn decode_cstr(self, bytes: &[u8]) -> Result<String, DecodeError> {
let end = bytes.iter().position(|&b| b == 0).unwrap_or(bytes.len()); self.decode(cstr_prefix(bytes))
self.decode(&bytes[..end])
} }
/// Encode `text` back to this code page's bytes. /// Encode `text` back to this code page's bytes.
@@ -264,16 +260,7 @@ impl DosEncoding {
/// under DOS Arabic, whose `0x25` is ARABIC PERCENT SIGN). /// under DOS Arabic, whose `0x25` is ARABIC PERCENT SIGN).
pub fn encode(self, text: &str) -> Result<Vec<u8>, EncodeError> { pub fn encode(self, text: &str) -> Result<Vec<u8>, EncodeError> {
match self.codec() { match self.codec() {
Codec::Single(table) => text Codec::Single(table) => encode_single(table, text, self.into()),
.chars()
.map(|ch| {
table
.iter()
.position(|&slot| slot == Some(ch))
.map(|i| i as u8)
.ok_or(EncodeError::Unmappable { ch, encoding: self })
})
.collect(),
#[cfg(feature = "dbcs")] #[cfg(feature = "dbcs")]
Codec::Dbcs(page) => self.encode_dbcs(page, text), Codec::Dbcs(page) => self.encode_dbcs(page, text),
} }
@@ -293,21 +280,21 @@ impl DosEncoding {
SingleEntry::Undefined => { SingleEntry::Undefined => {
return Err(DecodeError::Undefined { return Err(DecodeError::Undefined {
byte: *byte, byte: *byte,
encoding: self, encoding: self.into(),
}) })
} }
SingleEntry::Lead => { SingleEntry::Lead => {
let [trail, tail @ ..] = tail else { let [trail, tail @ ..] = tail else {
return Err(DecodeError::TruncatedPair { return Err(DecodeError::TruncatedPair {
lead: *byte, lead: *byte,
encoding: self, encoding: self.into(),
}); });
}; };
let ch = let ch =
lookup_pair(page, *byte, *trail).ok_or(DecodeError::UndefinedPair { lookup_pair(page, *byte, *trail).ok_or(DecodeError::UndefinedPair {
lead: *byte, lead: *byte,
trail: *trail, trail: *trail,
encoding: self, encoding: self.into(),
})?; })?;
out.push(ch); out.push(ch);
rest = tail; rest = tail;
@@ -361,7 +348,10 @@ impl DosEncoding {
let (_, code) = page.encode[i]; let (_, code) = page.encode[i];
out.extend_from_slice(&code.to_be_bytes()); out.extend_from_slice(&code.to_be_bytes());
} else { } else {
return Err(EncodeError::Unmappable { ch, encoding: self }); return Err(EncodeError::Unmappable {
ch,
encoding: self.into(),
});
} }
} }
Ok(out) Ok(out)
@@ -370,22 +360,22 @@ impl DosEncoding {
/// The lookup machinery for this code page. /// The lookup machinery for this code page.
fn codec(self) -> Codec { fn codec(self) -> Codec {
match self { match self {
Self::Cp437 => Codec::Single(&tables::single::CP437), Self::Cp437 => Codec::Single(&tables::dos::CP437),
Self::Cp737 => Codec::Single(&tables::single::CP737), Self::Cp737 => Codec::Single(&tables::dos::CP737),
Self::Cp775 => Codec::Single(&tables::single::CP775), Self::Cp775 => Codec::Single(&tables::dos::CP775),
Self::Cp850 => Codec::Single(&tables::single::CP850), Self::Cp850 => Codec::Single(&tables::dos::CP850),
Self::Cp852 => Codec::Single(&tables::single::CP852), Self::Cp852 => Codec::Single(&tables::dos::CP852),
Self::Cp855 => Codec::Single(&tables::single::CP855), Self::Cp855 => Codec::Single(&tables::dos::CP855),
Self::Cp857 => Codec::Single(&tables::single::CP857), Self::Cp857 => Codec::Single(&tables::dos::CP857),
Self::Cp860 => Codec::Single(&tables::single::CP860), Self::Cp860 => Codec::Single(&tables::dos::CP860),
Self::Cp861 => Codec::Single(&tables::single::CP861), Self::Cp861 => Codec::Single(&tables::dos::CP861),
Self::Cp862 => Codec::Single(&tables::single::CP862), Self::Cp862 => Codec::Single(&tables::dos::CP862),
Self::Cp863 => Codec::Single(&tables::single::CP863), Self::Cp863 => Codec::Single(&tables::dos::CP863),
Self::Cp864 => Codec::Single(&tables::single::CP864), Self::Cp864 => Codec::Single(&tables::dos::CP864),
Self::Cp865 => Codec::Single(&tables::single::CP865), Self::Cp865 => Codec::Single(&tables::dos::CP865),
Self::Cp866 => Codec::Single(&tables::single::CP866), Self::Cp866 => Codec::Single(&tables::dos::CP866),
Self::Cp869 => Codec::Single(&tables::single::CP869), Self::Cp869 => Codec::Single(&tables::dos::CP869),
Self::Cp874 => Codec::Single(&tables::single::CP874), Self::Cp874 => Codec::Single(&tables::dos::CP874),
#[cfg(feature = "dbcs")] #[cfg(feature = "dbcs")]
Self::Cp932 => Codec::Dbcs(&tables::cp932::CP932), Self::Cp932 => Codec::Dbcs(&tables::cp932::CP932),
#[cfg(feature = "dbcs")] #[cfg(feature = "dbcs")]
@@ -398,9 +388,258 @@ impl DosEncoding {
} }
} }
/// A Windows ANSI code page.
///
/// These are the single-byte "125x" pages Windows uses for non-console text
/// (the `GetACP` family), as distinct from the OEM pages in [`DosEncoding`].
///
/// Non-exhaustive: more code pages may be added without it being a breaking
/// change, so external matches must include a wildcard arm.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Default)]
#[non_exhaustive]
pub enum WindowsEncoding {
/// Code page 874 — Windows Thai. Microsoft publishes the identical table
/// for DOS and Windows Thai, so this equals [`DosEncoding::Cp874`].
Cp874,
/// Code page 1250 — Windows Central European.
Cp1250,
/// Code page 1251 — Windows Cyrillic.
Cp1251,
/// Code page 1252 — Windows Western, the near-superset of Latin-1.
#[default]
Cp1252,
/// Code page 1253 — Windows Greek.
Cp1253,
/// Code page 1254 — Windows Turkish.
Cp1254,
/// Code page 1255 — Windows Hebrew.
Cp1255,
/// Code page 1256 — Windows Arabic.
Cp1256,
/// Code page 1257 — Windows Baltic.
Cp1257,
/// Code page 1258 — Windows Vietnamese (uses combining diacritics).
Cp1258,
}
impl WindowsEncoding {
/// Every Windows code page this crate implements, in numeric order.
pub const ALL: &'static [WindowsEncoding] = &[
Self::Cp874,
Self::Cp1250,
Self::Cp1251,
Self::Cp1252,
Self::Cp1253,
Self::Cp1254,
Self::Cp1255,
Self::Cp1256,
Self::Cp1257,
Self::Cp1258,
];
/// Select an encoding from its numeric code page identifier (an ACP value
/// from the `GetACP` family), or `None` if this crate does not implement
/// it.
#[must_use]
pub fn from_code_page(code_page: u16) -> Option<Self> {
Self::ALL
.iter()
.copied()
.find(|enc| enc.code_page() == code_page)
}
/// The numeric code page identifier (1252, 874, …).
#[must_use]
pub fn code_page(self) -> u16 {
match self {
Self::Cp874 => 874,
Self::Cp1250 => 1250,
Self::Cp1251 => 1251,
Self::Cp1252 => 1252,
Self::Cp1253 => 1253,
Self::Cp1254 => 1254,
Self::Cp1255 => 1255,
Self::Cp1256 => 1256,
Self::Cp1257 => 1257,
Self::Cp1258 => 1258,
}
}
/// A short human-readable name, e.g. "Windows Western".
#[must_use]
pub fn name(self) -> &'static str {
match self {
Self::Cp874 => "Windows Thai",
Self::Cp1250 => "Windows Central European",
Self::Cp1251 => "Windows Cyrillic",
Self::Cp1252 => "Windows Western",
Self::Cp1253 => "Windows Greek",
Self::Cp1254 => "Windows Turkish",
Self::Cp1255 => "Windows Hebrew",
Self::Cp1256 => "Windows Arabic",
Self::Cp1257 => "Windows Baltic",
Self::Cp1258 => "Windows Vietnamese",
}
}
/// Decode every byte of `bytes` to Unicode, faithfully (control bytes and
/// a trailing `NUL` included). Use [`decode_cstr`](Self::decode_cstr) for
/// `NUL`-terminated fixed-width fields.
///
/// # Errors
///
/// Returns [`DecodeError::Undefined`] for a byte value the code page
/// leaves unmapped (every page here except 1256 has a few undefined
/// slots in the `0x80..=0x9F` region).
pub fn decode(self, bytes: &[u8]) -> Result<String, DecodeError> {
decode_single(self.table(), bytes, self.into())
}
/// Decode like [`decode`](Self::decode), substituting U+FFFD REPLACEMENT
/// CHARACTER for any byte the code page leaves undefined.
#[must_use]
pub fn decode_lossy(self, bytes: &[u8]) -> String {
decode_single_lossy(self.table(), bytes)
}
/// Decode up to (but not including) the first `NUL` byte.
///
/// # Errors
///
/// Returns [`DecodeError`] as [`decode`](Self::decode) does.
pub fn decode_cstr(self, bytes: &[u8]) -> Result<String, DecodeError> {
self.decode(cstr_prefix(bytes))
}
/// Encode `text` back to this code page's bytes.
///
/// # Errors
///
/// Returns [`EncodeError::Unmappable`] for the first character with no
/// representation in this code page.
pub fn encode(self, text: &str) -> Result<Vec<u8>, EncodeError> {
encode_single(self.table(), text, self.into())
}
/// The full byte-to-scalar table for this code page.
fn table(self) -> &'static [Option<char>; 256] {
match self {
Self::Cp874 => &tables::ansi::CP874,
Self::Cp1250 => &tables::ansi::CP1250,
Self::Cp1251 => &tables::ansi::CP1251,
Self::Cp1252 => &tables::ansi::CP1252,
Self::Cp1253 => &tables::ansi::CP1253,
Self::Cp1254 => &tables::ansi::CP1254,
Self::Cp1255 => &tables::ansi::CP1255,
Self::Cp1256 => &tables::ansi::CP1256,
Self::Cp1257 => &tables::ansi::CP1257,
Self::Cp1258 => &tables::ansi::CP1258,
}
}
}
/// A code page from either Microsoft family, as carried in errors.
///
/// Non-exhaustive: more families may be added without it being a breaking
/// change, so external matches must include a wildcard arm.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
#[non_exhaustive]
pub enum Encoding {
/// An OEM (DOS) code page.
Dos(DosEncoding),
/// A Windows ANSI code page.
Windows(WindowsEncoding),
}
impl Encoding {
/// Select an encoding from its numeric code page identifier, searching
/// the DOS family first, then Windows. The only overlap is 874, whose
/// two tables are identical in Microsoft's published data.
#[must_use]
pub fn from_code_page(code_page: u16) -> Option<Self> {
DosEncoding::from_code_page(code_page)
.map(Self::Dos)
.or_else(|| WindowsEncoding::from_code_page(code_page).map(Self::Windows))
}
/// The numeric code page identifier.
#[must_use]
pub fn code_page(self) -> u16 {
match self {
Self::Dos(enc) => enc.code_page(),
Self::Windows(enc) => enc.code_page(),
}
}
/// A short human-readable name.
#[must_use]
pub fn name(self) -> &'static str {
match self {
Self::Dos(enc) => enc.name(),
Self::Windows(enc) => enc.name(),
}
}
}
impl From<DosEncoding> for Encoding {
fn from(enc: DosEncoding) -> Self {
Self::Dos(enc)
}
}
impl From<WindowsEncoding> for Encoding {
fn from(enc: WindowsEncoding) -> Self {
Self::Windows(enc)
}
}
/// U+FFFD REPLACEMENT CHARACTER, used by the lossy decoders. /// U+FFFD REPLACEMENT CHARACTER, used by the lossy decoders.
const REPLACEMENT: char = '\u{FFFD}'; const REPLACEMENT: char = '\u{FFFD}';
/// Everything before the first `NUL` byte (or all of `bytes` if there is
/// none) — the shared trim behind the `decode_cstr` methods.
fn cstr_prefix(bytes: &[u8]) -> &[u8] {
let end = bytes.iter().position(|&b| b == 0).unwrap_or(bytes.len());
&bytes[..end]
}
/// Decode a single-byte page strictly.
fn decode_single(
table: &[Option<char>; 256],
bytes: &[u8],
encoding: Encoding,
) -> Result<String, DecodeError> {
bytes
.iter()
.map(|&byte| table[usize::from(byte)].ok_or(DecodeError::Undefined { byte, encoding }))
.collect()
}
/// Decode a single-byte page, substituting U+FFFD for undefined bytes.
fn decode_single_lossy(table: &[Option<char>; 256], bytes: &[u8]) -> String {
bytes
.iter()
.map(|&byte| table[usize::from(byte)].unwrap_or(REPLACEMENT))
.collect()
}
/// Encode to a single-byte page by scanning the table for each character.
fn encode_single(
table: &[Option<char>; 256],
text: &str,
encoding: Encoding,
) -> Result<Vec<u8>, EncodeError> {
text.chars()
.map(|ch| {
table
.iter()
.position(|&slot| slot == Some(ch))
.map(|i| i as u8)
.ok_or(EncodeError::Unmappable { ch, encoding })
})
.collect()
}
/// Look up a lead/trail pair in a DBCS page's sorted decode table. /// Look up a lead/trail pair in a DBCS page's sorted decode table.
#[cfg(feature = "dbcs")] #[cfg(feature = "dbcs")]
fn lookup_pair(page: &DbcsPage, lead: u8, trail: u8) -> Option<char> { fn lookup_pair(page: &DbcsPage, lead: u8, trail: u8) -> Option<char> {
@@ -417,6 +656,18 @@ impl fmt::Display for DosEncoding {
} }
} }
impl fmt::Display for WindowsEncoding {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "code page {} ({})", self.code_page(), self.name())
}
}
impl fmt::Display for Encoding {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "code page {} ({})", self.code_page(), self.name())
}
}
/// A byte sequence could not be decoded from the source code page. /// A byte sequence could not be decoded from the source code page.
/// ///
/// Non-exhaustive so future variants are not a breaking change. /// Non-exhaustive so future variants are not a breaking change.
@@ -429,7 +680,7 @@ pub enum DecodeError {
/// The offending byte value. /// The offending byte value.
byte: u8, byte: u8,
/// The code page that leaves it undefined. /// The code page that leaves it undefined.
encoding: DosEncoding, encoding: Encoding,
}, },
/// A lead byte of a double-byte page appeared with no trail byte after it. /// A lead byte of a double-byte page appeared with no trail byte after it.
#[error("lead byte {lead:#04x} at end of input in {encoding}")] #[error("lead byte {lead:#04x} at end of input in {encoding}")]
@@ -437,7 +688,7 @@ pub enum DecodeError {
/// The dangling lead byte. /// The dangling lead byte.
lead: u8, lead: u8,
/// The double-byte code page being decoded. /// The double-byte code page being decoded.
encoding: DosEncoding, encoding: Encoding,
}, },
/// A lead/trail pair is not a defined code of the page. /// A lead/trail pair is not a defined code of the page.
#[error("byte pair {lead:#04x} {trail:#04x} is undefined in {encoding}")] #[error("byte pair {lead:#04x} {trail:#04x} is undefined in {encoding}")]
@@ -447,7 +698,7 @@ pub enum DecodeError {
/// The trail byte of the pair. /// The trail byte of the pair.
trail: u8, trail: u8,
/// The double-byte code page being decoded. /// The double-byte code page being decoded.
encoding: DosEncoding, encoding: Encoding,
}, },
} }
@@ -463,7 +714,7 @@ pub enum EncodeError {
/// The offending character. /// The offending character.
ch: char, ch: char,
/// The code page that lacks a mapping for it. /// The code page that lacks a mapping for it.
encoding: DosEncoding, encoding: Encoding,
}, },
} }
@@ -520,7 +771,7 @@ mod tests {
enc.decode(b"a\xFFb"), enc.decode(b"a\xFFb"),
Err(DecodeError::Undefined { Err(DecodeError::Undefined {
byte: 0xFF, byte: 0xFF,
encoding: enc encoding: enc.into()
}) })
); );
assert_eq!(enc.decode_lossy(b"a\xFFb"), "a\u{FFFD}b"); assert_eq!(enc.decode_lossy(b"a\xFFb"), "a\u{FFFD}b");
@@ -629,6 +880,134 @@ mod tests {
} }
} }
#[cfg(test)]
mod ansi_tests {
use super::*;
#[test]
fn golden_ansi_mappings() {
// Golden values straight from the Microsoft tables.
assert_eq!(WindowsEncoding::Cp1252.decode(&[0x80]).unwrap(), "€");
assert_eq!(WindowsEncoding::Cp1252.decode(&[0x93]).unwrap(), "\u{201C}");
assert_eq!(WindowsEncoding::Cp1251.decode(&[0xC0]).unwrap(), "А");
assert_eq!(WindowsEncoding::Cp1255.decode(&[0xE0]).unwrap(), "א");
assert_eq!(WindowsEncoding::Cp1253.decode(&[0xDF]).unwrap(), "ί");
// 1258 spells Vietnamese with combining diacritics.
assert_eq!(WindowsEncoding::Cp1258.decode(&[0xCC]).unwrap(), "\u{0300}");
}
#[test]
fn smart_quotes_round_trip() {
let text = "\u{201C}Hello,\u{201D} she said \u{2014} twice\u{2026}";
let bytes = WindowsEncoding::Cp1252.encode(text).unwrap();
assert_eq!(WindowsEncoding::Cp1252.decode(&bytes).unwrap(), text);
}
#[test]
fn undefined_c1_slots_error_and_lossy_substitutes() {
// 0x98 is the lone undefined byte in CP1251.
let enc = WindowsEncoding::Cp1251;
assert_eq!(
enc.decode(b"a\x98b"),
Err(DecodeError::Undefined {
byte: 0x98,
encoding: enc.into()
})
);
assert_eq!(enc.decode_lossy(b"a\x98b"), "a\u{FFFD}b");
}
#[test]
fn cp1256_is_fully_defined() {
for byte in 0..=0xFFu8 {
assert!(WindowsEncoding::Cp1256.decode(&[byte]).is_ok());
}
}
#[test]
fn every_ansi_page_round_trips_every_byte() {
for &enc in WindowsEncoding::ALL {
for byte in 0..=0xFFu8 {
match enc.decode(&[byte]) {
Ok(decoded) => assert_eq!(
enc.encode(&decoded).unwrap(),
vec![byte],
"byte {byte:#04x} failed to round-trip under {enc}"
),
Err(DecodeError::Undefined { .. }) => {
assert_eq!(enc.decode_lossy(&[byte]), "\u{FFFD}");
}
Err(other) => panic!("unexpected error for {byte:#04x}: {other}"),
}
}
}
}
#[test]
fn every_ansi_table_is_a_bijection() {
for &enc in WindowsEncoding::ALL {
let mut seen = std::collections::HashSet::new();
for slot in enc.table().iter().flatten() {
assert!(seen.insert(*slot), "duplicate scalar {slot:?} in {enc}");
}
}
}
#[test]
fn windows_thai_equals_dos_thai() {
// Microsoft publishes the identical 874 table for both families.
for byte in 0..=0xFFu8 {
assert_eq!(
WindowsEncoding::Cp874.decode(&[byte]).ok(),
DosEncoding::Cp874.decode(&[byte]).ok(),
);
}
}
#[test]
fn from_code_page_families() {
for &enc in WindowsEncoding::ALL {
assert_eq!(WindowsEncoding::from_code_page(enc.code_page()), Some(enc));
}
assert_eq!(WindowsEncoding::from_code_page(437), None);
assert_eq!(DosEncoding::from_code_page(1252), None);
// The unified lookup searches DOS first, then Windows.
assert_eq!(
Encoding::from_code_page(437),
Some(Encoding::Dos(DosEncoding::Cp437))
);
assert_eq!(
Encoding::from_code_page(1252),
Some(Encoding::Windows(WindowsEncoding::Cp1252))
);
assert_eq!(
Encoding::from_code_page(874),
Some(Encoding::Dos(DosEncoding::Cp874))
);
assert_eq!(Encoding::from_code_page(65001), None); // UTF-8 is not ours
}
#[test]
fn ansi_decode_cstr_stops_at_nul() {
let enc = WindowsEncoding::Cp1252;
assert_eq!(enc.decode_cstr(b"caf\xE9\x00junk").unwrap(), "café");
assert_eq!(enc.decode_cstr(b"caf\xE9").unwrap(), "café");
}
#[test]
fn display_and_error_name_the_family() {
assert_eq!(
WindowsEncoding::Cp1252.to_string(),
"code page 1252 (Windows Western)"
);
let err = WindowsEncoding::Cp1253.encode("é").unwrap_err();
assert_eq!(
err.to_string(),
"character 'é' has no mapping in code page 1253 (Windows Greek)"
);
}
}
#[cfg(all(test, not(feature = "dbcs")))] #[cfg(all(test, not(feature = "dbcs")))]
mod no_dbcs_tests { mod no_dbcs_tests {
use super::*; use super::*;
@@ -722,7 +1101,7 @@ mod dbcs_tests {
enc.decode(b"abc\x93"), enc.decode(b"abc\x93"),
Err(DecodeError::TruncatedPair { Err(DecodeError::TruncatedPair {
lead: 0x93, lead: 0x93,
encoding: enc encoding: enc.into()
}) })
); );
assert_eq!(enc.decode_lossy(b"abc\x93"), "abc\u{FFFD}"); assert_eq!(enc.decode_lossy(b"abc\x93"), "abc\u{FFFD}");
@@ -737,7 +1116,7 @@ mod dbcs_tests {
Err(DecodeError::UndefinedPair { Err(DecodeError::UndefinedPair {
lead: 0x81, lead: 0x81,
trail: 0x00, trail: 0x00,
encoding: enc encoding: enc.into()
}) })
); );
assert_eq!(enc.decode_lossy(b"\x81\x00A"), "\u{FFFD}A"); assert_eq!(enc.decode_lossy(b"\x81\x00A"), "\u{FFFD}A");
@@ -751,7 +1130,7 @@ mod dbcs_tests {
enc.decode(&[0x80]), enc.decode(&[0x80]),
Err(DecodeError::Undefined { Err(DecodeError::Undefined {
byte: 0x80, byte: 0x80,
encoding: enc encoding: enc.into()
}) })
); );
assert_eq!(enc.decode_lossy(&[0x80]), "\u{FFFD}"); assert_eq!(enc.decode_lossy(&[0x80]), "\u{FFFD}");

2604
src/tables/ansi.rs Normal file
View File

File diff suppressed because it is too large Load Diff

0
src/tables/single.rs → src/tables/dos.rs
View File

3
src/tables/mod.rs
View File

@@ -2,7 +2,8 @@
//! `tools/gen_tables.py` from the vendored Microsoft mapping tables in //! `tools/gen_tables.py` from the vendored Microsoft mapping tables in
//! `data/` — regenerate rather than editing by hand. //! `data/` — regenerate rather than editing by hand.
pub(crate) mod single; pub(crate) mod ansi;
pub(crate) mod dos;
#[cfg(feature = "dbcs")] #[cfg(feature = "dbcs")]
pub(crate) mod cp932; pub(crate) mod cp932;

53
tools/gen_tables.py
View File

@@ -5,14 +5,15 @@
"""Generate src/tables/ from the Microsoft mapping tables in data/. """Generate src/tables/ from the Microsoft mapping tables in data/.
The inputs are the canonical Microsoft code page tables published by the The inputs are the canonical Microsoft code page tables published by the
Unicode Consortium (https://www.unicode.org/Public/MAPPINGS/VENDORS/MICSFT/, Unicode Consortium (https://www.unicode.org/Public/MAPPINGS/VENDORS/MICSFT/):
``PC/`` for the single-byte DOS pages and ``WINDOWS/`` for the double-byte ``PC/`` (vendored in ``data/pc/``) for the single-byte DOS pages and
CJK pages). ``WINDOWS/`` (vendored in ``data/windows/``) for the Windows ANSI pages and
the double-byte CJK pages.
Single-byte pages become full 256-entry ``[Option<char>; 256]`` tables: Single-byte pages become full 256-entry ``[Option<char>; 256]`` tables:
MS-DOS pages are not uniformly ASCII in the low half (CP864 maps 0x25 to MS-DOS pages are not uniformly ASCII in the low half (CP864 maps 0x25 to
ARABIC PERCENT SIGN) and several pages leave byte values undefined ARABIC PERCENT SIGN) and several pages leave byte values undefined
(CP857/864/869/874). (CP857/864/869/874, most of the ANSI pages).
Double-byte pages become a 256-entry single-byte/lead-byte table plus a pair Double-byte pages become a 256-entry single-byte/lead-byte table plus a pair
of sorted ``(code, scalar)`` arrays for binary search. Where several codes of sorted ``(code, scalar)`` arrays for binary search. Where several codes
@@ -33,13 +34,19 @@ from collections.abc import Sequence
from dataclasses import dataclass from dataclasses import dataclass
from pathlib import Path from pathlib import Path
SINGLE_BYTE_PAGES: Sequence[int] = ( DOS_PAGES: Sequence[int] = (
437, 737, 775, 850, 852, 855, 857, 860, 437, 737, 775, 850, 852, 855, 857, 860,
861, 862, 863, 864, 865, 866, 869, 874, 861, 862, 863, 864, 865, 866, 869, 874,
) )
ANSI_PAGES: Sequence[int] = (
874, 1250, 1251, 1252, 1253, 1254, 1255, 1256, 1257, 1258,
)
DOUBLE_BYTE_PAGES: Sequence[int] = (932, 936, 949, 950) DOUBLE_BYTE_PAGES: Sequence[int] = (932, 936, 949, 950)
DATA_DIR = Path(__file__).resolve().parent.parent / "data" DATA_DIR = Path(__file__).resolve().parent.parent / "data"
PC_DIR = DATA_DIR / "pc"
WINDOWS_DIR = DATA_DIR / "windows"
BESTFIT_DIR = DATA_DIR / "bestfit"
OUTPUT_DIR = Path(__file__).resolve().parent.parent / "src" / "tables" OUTPUT_DIR = Path(__file__).resolve().parent.parent / "src" / "tables"
GENERATED_NOTE = """\ GENERATED_NOTE = """\
@@ -316,27 +323,39 @@ def render_double_byte_page(
) )
def main() -> None: def generate_single_byte_module(
"""Regenerate src/tables/ from every configured code page.""" output_name: str, source_dir: Path, code_pages: Sequence[int]
outputs: list[Path] = [] ) -> Path:
"""Render a family of single-byte pages into one tables module.
single_rendered = [GENERATED_NOTE] Raises:
for code_page in SINGLE_BYTE_PAGES: TableError: If any page skips a byte value or contains DBCS content.
table = parse_table(DATA_DIR / f"CP{code_page}.TXT") """
rendered = [GENERATED_NOTE]
for code_page in code_pages:
table = parse_table(source_dir / f"CP{code_page}.TXT")
if table.listed_bytes != set(range(256)): if table.listed_bytes != set(range(256)):
missing = sorted(set(range(256)) - table.listed_bytes) missing = sorted(set(range(256)) - table.listed_bytes)
raise TableError(f"CP{code_page}: bytes never listed: {missing}") raise TableError(f"CP{code_page}: bytes never listed: {missing}")
if table.leads or table.pairs: if table.leads or table.pairs:
raise TableError(f"CP{code_page}: unexpected DBCS content") raise TableError(f"CP{code_page}: unexpected DBCS content")
check_bijection(f"CP{code_page}", table) check_bijection(f"CP{code_page}", table)
single_rendered.append(render_single_byte_page(code_page, table)) rendered.append(render_single_byte_page(code_page, table))
single_path = OUTPUT_DIR / "single.rs" path = OUTPUT_DIR / output_name
single_path.write_text("\n".join(single_rendered), encoding="utf-8") path.write_text("\n".join(rendered), encoding="utf-8")
outputs.append(single_path) return path
def main() -> None:
"""Regenerate src/tables/ from every configured code page."""
outputs = [
generate_single_byte_module("dos.rs", PC_DIR, DOS_PAGES),
generate_single_byte_module("ansi.rs", WINDOWS_DIR, ANSI_PAGES),
]
for code_page in DOUBLE_BYTE_PAGES: for code_page in DOUBLE_BYTE_PAGES:
table = parse_table(DATA_DIR / f"CP{code_page}.TXT") table = parse_table(WINDOWS_DIR / f"CP{code_page}.TXT")
wctable = parse_wctable(DATA_DIR / f"bestfit{code_page}.txt") wctable = parse_wctable(BESTFIT_DIR / f"bestfit{code_page}.txt")
encode = resolve_encode(f"CP{code_page}", table, wctable) encode = resolve_encode(f"CP{code_page}", table, wctable)
page_path = OUTPUT_DIR / f"cp{code_page}.rs" page_path = OUTPUT_DIR / f"cp{code_page}.rs"
page_path.write_text( page_path.write_text(