dragon-hd/docs/DEVLOG.md

# DEVLOG

Working log for the *The I of the Dragon* localization / HD project.
Newest entries on top.

## 2026-09-20 — Center option titles and shorten tutorial exit prompt

- User confirmed the HUD tooltip fix in game.
- Original tutorial text was `\aOK - quit, Cancel - continue tutorial.`;
  replaced the verbose literal Chinese translation with `\a退出教程？`.
- Audio, game, video and controls titles all used fixed left x=620 before
  resolution scaling. Their panel/button center is reference x=640. At
  1280px, the 24px Chinese controls title was consequently 20px left of the
  Bind button center. Title-only draw wrappers now measure actual text width
  and subtract half from the transformed x=640 anchor, after screen scaling.
- `test_cjk_titles.py` executes all four original title call sites with three
  text widths plus 1280/1920 screen-width cases. Both title tests and all 14
  tooltip/layout tests pass. Game source patcher and translated Strings.dat
  are deployed locally; the user confirmed both fixes in game.

## 2026-09-20 — Fix Chinese HUD tooltip overlap

- Reproduced the reported world-map/minimap tooltip bug in the game's actual
  x86 wrapped-text routine (`0x4ED380`) using Unicorn. It builds a one-byte
  temporary string before measuring each character, so the existing DBCS
  metrics hook measured isolated GB2312 bytes as width/height zero. This
  miscentered Chinese lines and advanced the next line by only the 5px line
  distance, overlapping the 12px Chinese artwork.
- Added pair-aware read, append, and fit caves to `tools/apply_cjk.py`.
  GB2312 pairs now reach the measurer together, are appended/consumed together,
  and become complete-character wrap checkpoints. Initialized empty-line
  checkpoints correctly so mixed ASCII/CJK wraps retain the prior line height.
  A glyph wider than the available box is emitted intact rather than losing
  its lead byte; existing caller clipping remains responsible for box bounds.
- Added `tools/test_cjk_layout.py`: executes the real layout/measurement
  instructions and captures only the Direct3D draw boundary. Fourteen tests
  cover screenshot tooltip strings, shortcuts, centering, control codes,
  measure-only height, mixed text, whitespace, narrow boxes, and oversize glyphs.
  The original overlap case changes from `(Chinese x=24,y=0; S x=20,y=5)` to
  `(Chinese x=12,y=0; S x=20,y=17)` in a 48px box.
- No atlas, translation, font-size, or mission-script changes are needed.
  Original exe and patcher snapshots: `backup_game/font-layout-20260920/`.
  Local deployed program: `F:\steam\steamapps\common\The I of the Dragon\TheIOfTheDragon.exe`.

Validation (requires `unicorn`):
`python tools/test_cjk_layout.py --exe build/cjk-layout-fixed.exe -v`

Progress audit: local HEAD matched Gitea `ca57488` before this fix; 1884 texture
entries / 1882 preprocessed PNGs, 1397 geometry entries, and 479/496 translated
Strings.dat IDs. Mission scripts, description tables, baked text, and HD
upscaling remain separate unfinished work. Other-font CJK coverage is also
still limited as documented in patch-notes.md.

## 2026-09-20 — Full `Strings.dat` translation

- Translated the `English` column for **479 / 496** entries (the rest are pure-format
  strings, empty/whitespace values, the tutorial texture *path*, and a few odd key labels).
- `tools/check_translations.py` verifies the printf specifier sequence (`%s`/`%d`/`%i`/`%%`)
  matches the English source exactly (0 mismatches) and that every value GB2312-encodes.
- `tools/dump_all_strings.py` dumps the source table to `build/strings_en.json`.

### Known layout caveats (engine UI is tuned for the small Latin pixel font)

- Some engine UI positions tab/button labels against a box sized for English, so short
  Chinese labels can look left/right-aligned; a couple of centred dialogs jitter while a
  teletype reveal is running.
- In-game HUD button labels wrap and can overlap the shortcut hint because the boxes are
  small; to revisit via a smaller CJK cell/advance if needed.

## 2026-09-20 — Chinese in the main menu (first playable)

- Built `tools/build_cjk.py`: renders the full GB2312 set (7445 double-byte codes) as **12x12**
  glyphs with simsun (crisp/binary, matching the game's pixel font) into a **2048x1024** TGA.
  The original 256x128 ASCII artwork is copied to (0,0), so all existing `Fonts.dat` rects keep
  working unchanged (verified byte-identical). Also emits `cjk_glyph.bin` / `cjk_metric.bin`.
  Advance/height = 12 (the UI font cap height is ~11-12px), drawn 1px up to fit the cell.
- `tools/cn_strings.py` + `tools/make_cn_strings.py`: translate the `English` column of
  `Strings.dat` to GB2312 (46 entries; escape-aware so `\"%s\"` formats survive).
- `tools/install_cjk.py`: one-shot deploy/restore (backs up exe/res/strings; rebuilds `Textures.res`
  with the new atlas, drops the loose atlas, patches the exe and the strings).
- **Result**: the main menu renders Chinese — 开始新游戏 / 载入游戏 / 选项 / 教程 / 制作人员 / 退出 /
  载入上次存档："…" — confirmed by the user on screen.

### Fixed a byte-order bug in the metrics cave

The draw cave decoded the code as `(first<<8)|second`, but the metrics cave used
`movzx eax,al; mov ah,[ebx+1]` which produced `(second<<8)|first`. Every CJK width measured as 0,
so centred labels (the "Start New Game" button) were flushed right. Fixed to
`movzx eax,al; shl eax,8; mov al,[ebx+1]`. Centring is now correct.

### Notes / next

- The menu button labels *are* dynamic (`Strings.dat`) — the earlier suspicion of baked button
  images was wrong; the layer confusion came from the animated particle background.
- Remaining UI text that is baked into textures (credits, tutorial pics) still needs redrawing.
- Next: translate the rest of `Strings.dat`, scripts and `*Description.dat`; then HD textures.

## 2026-09-20 — Double-byte font patch proven

- Reverse-engineered the whole text pipeline (see `patch-notes.md`).
- Built `tools/add_section.py` (PE section injection) and verified the patched exe still runs.
- Built `tools/apply_cjk.py`: assembles two code caves with keystone, adds a `.cjk` section,
  hooks the draw loop (`0x4EACB0`) and the metrics loop (`0x4E8026`).
- **Verified on screen**: a GB2312-coded string `0xC1FA` (龙) in a menu label rendered a 16x16
  glyph at the mapped atlas position (green control test). Double-byte decode + render works.
- Game files restored afterwards (exe SHA1 `9EDB9B99...`, `Textures.res` byte-identical).

### Next
1. Big atlas (2048x2048): ASCII glyphs (copied) + CJK glyphs; re-point `Fonts.dat` `Texture` + ASCII rects.
2. Generator for `cjk_glyph` / `cjk_metric` tables from a chosen charset.
3. Convert `Strings.dat`, scripts, and `*Description.dat` display text to GB2312.
4. Redraw baked-in text textures (menu buttons, credits, tutorial).
5. Package: patched exe + atlas + data files.

## 2026-09-20 — HD texture preprocessing

- Cracked the `.res` archive format and built `res_unpack.py` / `res_repack.py`
  (byte-identical round-trip once 16-byte alignment was found).
- Extracted `Textures.res` (1884) and `Geometry.res` (1397).
- Built `preprocess.py` (TGA/DDS→PNG, classification) and `make_plan.py`.
- Result: 63.6 Mpix, 1882 PNG, 0 failures. Plan: 1195 upscale / 499 tiling / 182 review / 8 skip
  (6 font atlases + 2 non-textures).

## 2026-09-20 — Cracking the `.dat` obfuscation

- `Fonts.dat` / `UnitDescription.dat` / `AIDescription.dat` decoded:
  `b"==" + XOR(plaintext, "GBDFYTNE")`; key at `0x620D18`, decoder at `0x4AE1D0`.
- `Fonts.dat` parsed: 4 font blocks, per-char atlas rects; parser keeps only the first byte of `Code`
  (`0x4EC168`), so the stock engine is single-byte.

## 2026-09-20 — Initial findings

- Engine has a data-driven multi-language system: per-language folders for scripts/speech/video/UI
  textures and language columns in `Strings.dat`.
- Text is externalized: `Strings.dat` (499 entries, plain) and plain-text scripts.
- First font test: replaced the `e` glyph in `Shrift_gb_Germany.TGA` and confirmed the atlas override
  works in game (rendered 龙 where `e` was).
- Investigated the "pirate build": PROPHET 2009 Topware release, languages EN/DE/PL/CZ/HU, **no
  Chinese**; its exe is identical to the Steam exe (no engine patch), so the Chinese build the user
  remembers was a different repack (not recoverable from it).

## Decisions

- Do the localization ourselves (route B): patch the engine for double-byte text, build our own font
  atlas, translate only display text (never Ids / event names / unit types) to avoid trigger bugs.
- Keep the repository text-only; regenerate binaries from scripts (see `.gitignore`).