67 lines
2.2 KiB
Markdown
67 lines
2.2 KiB
Markdown
|
# msdos-encodings
|
|||
|
|
|
|||
|
|
Bidirectional, emulator-grade conversion between MS-DOS code pages and Unicode —
|
|||
|
|
without linking ICU. The crate owns the canonical Microsoft tables, so it stays
|
|||
|
|
small, self-contained, and cross-compiles cleanly.
|
|||
|
|
|
|||
|
|
These are strictly **Microsoft's** mappings as published by the Unicode
|
|||
|
|
Consortium (`VENDORS/MICSFT/PC/`) — not the IBM or Oracle variants, which
|
|||
|
|
differ in a handful of slots.
|
|||
|
|
|
|||
|
|
```rust
|
|||
|
|
use msdos_encodings::DosEncoding;
|
|||
|
|
|
|||
|
|
let enc = DosEncoding::default(); // code page 437
|
|||
|
|
assert_eq!(enc.decode(b"Caf\x82").unwrap(), "Café");
|
|||
|
|
assert_eq!(enc.encode("Café").unwrap(), b"Caf\x82");
|
|||
|
|
|
|||
|
|
// Pick by numeric code page (e.g. from CHCP or locale data):
|
|||
|
|
let enc = DosEncoding::from_code_page(866).unwrap();
|
|||
|
|
assert_eq!(enc.decode(&[0x80]).unwrap(), "А"); // DOS Cyrillic Russian
|
|||
|
|
|
|||
|
|
// Four pages leave bytes undefined; decode is fallible, or go lossy:
|
|||
|
|
assert!(DosEncoding::Cp874.decode(&[0xFF]).is_err());
|
|||
|
|
assert_eq!(DosEncoding::Cp874.decode_lossy(&[0xFF]), "\u{FFFD}");
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## Status
|
|||
|
|
|
|||
|
|
- **All sixteen single-byte DOS pages** — implemented, decode + encode:
|
|||
|
|
437, 737, 775, 850, 852, 855, 857, 860, 861, 862, 863, 864, 865, 866, 869, 874.
|
|||
|
|
- **Double-byte CJK pages** (932 Shift-JIS, 936 GBK, 949 Korean, 950 Big5) —
|
|||
|
|
planned, to be codegen'd the same way and feature-gated.
|
|||
|
|
|
|||
|
|
## Fidelity notes
|
|||
|
|
|
|||
|
|
- Every table is a verified bijection: anything a page decodes re-encodes to
|
|||
|
|
the identical bytes.
|
|||
|
|
- Code pages 857, 864, 869, and 874 leave some byte values undefined, so
|
|||
|
|
`decode` returns a `Result`; `decode_lossy` substitutes U+FFFD.
|
|||
|
|
- Microsoft's CP864 (DOS Arabic) maps `0x25` to ARABIC PERCENT SIGN — the low
|
|||
|
|
half is *not* pure ASCII, and `%` itself is unmappable. This is faithful to
|
|||
|
|
the source table.
|
|||
|
|
|
|||
|
|
## Regenerating tables
|
|||
|
|
|
|||
|
|
`src/tables.rs` is generated from the vendored mapping files in `data/`
|
|||
|
|
(needs Python ≥ 3.10 and `rustfmt` on `PATH`):
|
|||
|
|
|
|||
|
|
```sh
|
|||
|
|
uv run tools/gen_tables.py
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## Scope
|
|||
|
|
|
|||
|
|
Text-encoding conversion only. Filename conventions (8.3 space padding,
|
|||
|
|
case folding, the 0x05/0xE5 deleted-entry dance) are deliberately **out of
|
|||
|
|
scope** and left to consumers.
|
|||
|
|
|
|||
|
|
## Consumers
|
|||
|
|
|
|||
|
|
Designed to be shared by `fsinspect` and friends via path (and later git)
|
|||
|
|
dependency.
|
|||
|
|
|
|||
|
|
## License
|
|||
|
|
|
|||
|
|
MIT
|