bbf6d6b2b9
Rework the crate on the msdos-encodings design: vendored Unicode Consortium VENDORS/APPLE mapping files in data/apple/, a tools/gen_tables.py generator (PEP 723 / uv run / rustfmt) emitting full [Option<char>; 256] tables, and a fallible decode / decode_lossy / encode API with non_exhaustive thiserror errors. Encodings: Roman, Greek, Cyrillic, Central European Roman, Turkish, Croatian, Icelandic, Romanian, Celtic, Gaelic, Ukrainian (pre-9.0 Cyrillic variant), Inuit. The eight encodings Apple revised for the Euro carry a Revision payload (Classic = pre-8.5/9.0/9.2.2), generated from per-encoding deltas with invariant checks that fail regeneration if the vendored data drifts. Apple-specific API points beyond the msdos shape: from_text_encoding (kTextEncodingMac… base values, the HFS+ hint), from_script_code plus from_fd_script (bit-7 validity flag on the raw Finder Info byte), and decode_pstr for length-prefixed Str27/Str31 name fields with a TruncatedPascalString error for corrupt lengths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
99 lines
4.0 KiB
Markdown
99 lines
4.0 KiB
Markdown
# apple-encodings
|
||
|
||
Bidirectional, emulator-grade conversion between classic Mac OS text encodings
|
||
and Unicode — without linking ICU. The crate owns the canonical Apple tables, so
|
||
it stays small, self-contained, and cross-compiles cleanly.
|
||
|
||
These are strictly **Apple's** mappings as published by the Unicode Consortium
|
||
(`VENDORS/APPLE/`), vendored in `data/apple/`.
|
||
|
||
```rust
|
||
use apple_encodings::{AppleEncoding, Revision};
|
||
|
||
let enc = AppleEncoding::default(); // Mac OS Roman, modern revision
|
||
assert_eq!(enc.decode(b"Caf\x8e").unwrap(), "Café");
|
||
assert_eq!(enc.encode("Café").unwrap(), b"Caf\x8e");
|
||
|
||
// Pick by the HFS+ `textEncoding` hint (kTextEncodingMac… base values):
|
||
let enc = AppleEncoding::from_text_encoding(7).unwrap();
|
||
assert_eq!(enc.decode(&[0x80]).unwrap(), "А"); // Mac OS Cyrillic
|
||
|
||
// Or by the raw HFS Finder Info `fdScript` byte (bit 7 flags validity;
|
||
// a Cyrillic system writes 0x87). Script system, not table — the Roman
|
||
// regional variants all report script 0:
|
||
let enc = AppleEncoding::from_fd_script(0x87).unwrap();
|
||
assert_eq!(enc.name(), "Mac OS Cyrillic");
|
||
|
||
// Classic Mac name fields are Pascal strings (Str27/Str31):
|
||
let name = enc.decode_pstr(b"\x08\x84\xEE\xEA\xF3\xEC\xE5\xED\xF2").unwrap();
|
||
assert_eq!(name, "Документ");
|
||
|
||
// The Euro rollout revised several tables, at different bytes per encoding;
|
||
// pick the pre-Euro table for older-system fidelity:
|
||
let classic = AppleEncoding::MacRoman(Revision::Classic);
|
||
assert_eq!(classic.decode(&[0xDB]).unwrap(), "¤"); // pre-8.5, not €
|
||
|
||
// Decode is fallible for API uniformity (Greek pre-9.2.2 leaves 0xFF
|
||
// undefined), or go lossy:
|
||
let greek = AppleEncoding::MacGreek(Revision::Classic);
|
||
assert!(greek.decode(&[0xFF]).is_err());
|
||
assert_eq!(greek.decode_lossy(&[0xFF]), "\u{FFFD}");
|
||
```
|
||
|
||
## Status
|
||
|
||
- **Twelve single-byte encodings** — implemented, decode + encode: Roman,
|
||
Greek, Cyrillic, Central European Roman, Turkish, Croatian, Icelandic,
|
||
Romanian, Celtic, Gaelic, Ukrainian (the pre-9.0 Cyrillic variant), and
|
||
Inuit.
|
||
- **Pre-Euro revisions** — implemented for the eight encodings Apple revised:
|
||
Roman/Croatian/Icelandic/Romanian/Celtic/Gaelic (`0xDB`, Mac OS 8.5),
|
||
Cyrillic (`0xA2`/`0xB6`/`0xFF`, Mac OS 9.0), Greek (`0x9C`/`0xFF`,
|
||
Mac OS 9.2.2).
|
||
- **CJK double-byte** (Japanese, Chinese, Korean) and the
|
||
**bidirectional/complex scripts** (Arabic, Hebrew, Thai, Indic — which need
|
||
multi-scalar decode) — planned, behind the same API; the double-byte tables
|
||
will be feature-gated.
|
||
|
||
## Fidelity notes
|
||
|
||
- Every table is a verified bijection: anything an encoding decodes re-encodes
|
||
to the identical bytes.
|
||
- Every table is total except Mac OS Greek at `Revision::Classic`, whose
|
||
`0xFF` was undefined before Mac OS 9.2.2 — so `decode` returns a `Result`
|
||
and `decode_lossy` substitutes U+FFFD, matching the `msdos-encodings`
|
||
sister crate.
|
||
- Apple's corporate Private Use Area mappings are preserved, not normalized:
|
||
the Apple logo (`0xF0` → `U+F8FF` in Roman) and Mac OS Turkish's explicit
|
||
"undefined character" slot (`0xF5` → `U+F8A0`) round-trip exactly.
|
||
- The pre-Euro revision tables are generated from per-encoding deltas taken
|
||
from each mapping file's own change history, with invariant checks that
|
||
fail regeneration loudly if the vendored data ever disagrees — the Euro
|
||
landed at different bytes per encoding, so nothing is hand-placed.
|
||
- Mac OS Ukrainian is derived from `CYRILLIC.TXT` plus its documented delta
|
||
(`UKRAINE.TXT` upstream is a notes-only stub: Mac OS 9.0 retired the
|
||
separate Ukrainian character set).
|
||
|
||
## Regenerating tables
|
||
|
||
Everything in `src/tables/` except `mod.rs` is generated from the vendored
|
||
mapping files in `data/apple/` (needs Python ≥ 3.10 and `rustfmt` on `PATH`):
|
||
|
||
```sh
|
||
uv run tools/gen_tables.py
|
||
```
|
||
|
||
## Scope
|
||
|
||
Text-encoding conversion only. Unicode normalization concerns (e.g. HFS+'s
|
||
NFD-ish decomposition) are deliberately **out of scope** and left to consumers.
|
||
|
||
## Consumers
|
||
|
||
Designed to be shared by `ad-decoder`/`adx` and `fsinspect` via path (and later
|
||
git) dependency.
|
||
|
||
## License
|
||
|
||
MIT
|