apple-encodings/README.md

apple-encodings

Bidirectional, emulator-grade conversion between classic Mac OS text encodings and Unicode — without linking ICU. The crate owns the canonical Apple tables, so it stays small, self-contained, and cross-compiles cleanly.

These are strictly Apple's mappings as published by the Unicode Consortium (VENDORS/APPLE/), vendored in data/apple/.

use apple_encodings::{AppleEncoding, Revision};

let enc = AppleEncoding::default(); // Mac OS Roman, modern revision
assert_eq!(enc.decode(b"Caf\x8e").unwrap(), "Café");
assert_eq!(enc.encode("Café").unwrap(), b"Caf\x8e");

// Pick by the HFS+ `textEncoding` hint (kTextEncodingMac… base values):
let enc = AppleEncoding::from_text_encoding(7).unwrap();
assert_eq!(enc.decode(&[0x80]).unwrap(), "А"); // Mac OS Cyrillic

// Or by the raw HFS Finder Info `fdScript` byte (bit 7 flags validity;
// a Cyrillic system writes 0x87). Script system, not table — the Roman
// regional variants all report script 0:
let enc = AppleEncoding::from_fd_script(0x87).unwrap();
assert_eq!(enc.name(), "Mac OS Cyrillic");

// Classic Mac name fields are Pascal strings (Str27/Str31):
let name = enc.decode_pstr(b"\x08\x84\xEE\xEA\xF3\xEC\xE5\xED\xF2").unwrap();
assert_eq!(name, "Документ");

// The Euro rollout revised several tables, at different bytes per encoding;
// pick the pre-Euro table for older-system fidelity:
let classic = AppleEncoding::MacRoman(Revision::Classic);
assert_eq!(classic.decode(&[0xDB]).unwrap(), "¤"); // pre-8.5, not €

// Decode is fallible for API uniformity (Greek pre-9.2.2 leaves 0xFF
// undefined), or go lossy:
let greek = AppleEncoding::MacGreek(Revision::Classic);
assert!(greek.decode(&[0xFF]).is_err());
assert_eq!(greek.decode_lossy(&[0xFF]), "\u{FFFD}");

Status

• Twelve single-byte encodings — implemented, decode + encode: Roman, Greek, Cyrillic, Central European Roman, Turkish, Croatian, Icelandic, Romanian, Celtic, Gaelic, Ukrainian (the pre-9.0 Cyrillic variant), and Inuit.
• Pre-Euro revisions — implemented for the eight encodings Apple revised: Roman/Croatian/Icelandic/Romanian/Celtic/Gaelic (0xDB, Mac OS 8.5), Cyrillic (0xA2/0xB6/0xFF, Mac OS 9.0), Greek (0x9C/0xFF, Mac OS 9.2.2).
• CJK double-byte (Japanese, Chinese, Korean) and the bidirectional/complex scripts (Arabic, Hebrew, Thai, Indic — which need multi-scalar decode) — planned, behind the same API; the double-byte tables will be feature-gated.

Fidelity notes

• Every table is a verified bijection: anything an encoding decodes re-encodes to the identical bytes.
• Every table is total except Mac OS Greek at Revision::Classic, whose 0xFF was undefined before Mac OS 9.2.2 — so decode returns a Result and decode_lossy substitutes U+FFFD, matching the msdos-encodings sister crate.
• Apple's corporate Private Use Area mappings are preserved, not normalized: the Apple logo (0xF0 → U+F8FF in Roman) and Mac OS Turkish's explicit "undefined character" slot (0xF5 → U+F8A0) round-trip exactly.
• The pre-Euro revision tables are generated from per-encoding deltas taken from each mapping file's own change history, with invariant checks that fail regeneration loudly if the vendored data ever disagrees — the Euro landed at different bytes per encoding, so nothing is hand-placed.
• Mac OS Ukrainian is derived from CYRILLIC.TXT plus its documented delta (UKRAINE.TXT upstream is a notes-only stub: Mac OS 9.0 retired the separate Ukrainian character set).

Regenerating tables

Everything in src/tables/ except mod.rs is generated from the vendored mapping files in data/apple/ (needs Python ≥ 3.10 and rustfmt on PATH):

uv run tools/gen_tables.py

Scope

Text-encoding conversion only. Unicode normalization concerns (e.g. HFS+'s NFD-ish decomposition) are deliberately out of scope and left to consumers.

Consumers

Designed to be shared by ad-decoder/adx and fsinspect via path (and later git) dependency.

License

MIT