dragon-hd/docs/DEVLOG.md

# DEVLOG

Working log for the *The I of the Dragon* localization / HD project.
Newest entries on top.

## 2026-09-20 — Accepted fixes and next-step plan

- User confirmed the tutorial prompt and option-title follow-up in game.
- Pushed fix commit `253cc47` to `codex/fix-cjk-tooltip-layout` on Gitea.
- Added a prioritized implementation plan: tutorial display-text translation,
  an eight-texture 2x pilot, then validated 20–50 texture batches and chapter-by-
  chapter mission translation. The next-stage work is planned, not implemented.
- Plan: `docs/superpowers/plans/2026-09-20-high-impact-simple-first.md`.

## 2026-09-20 — Center option titles and shorten tutorial exit prompt

- User confirmed the HUD tooltip fix in game.
- Original tutorial text was `\aOK - quit, Cancel - continue tutorial.`;
  replaced the verbose literal Chinese translation with `\a退出教程？`.
- Audio, game, video and controls titles all used fixed left x=620 before
  resolution scaling. Their panel/button center is reference x=640. At
  1280px, the 24px Chinese controls title was consequently 20px left of the
  Bind button center. Title-only draw wrappers now measure actual text width
  and subtract half from the transformed x=640 anchor, after screen scaling.
- `test_cjk_titles.py` executes all four original title call sites with three
  text widths plus 1280/1920 screen-width cases. Both title tests and all 14
  tooltip/layout tests pass. Game source patcher and translated Strings.dat
  are deployed locally; the user confirmed both fixes in game.

## 2026-09-20 — Fix Chinese HUD tooltip overlap

- Reproduced the reported world-map/minimap tooltip bug in the game's actual
  x86 wrapped-text routine (`0x4ED380`) using Unicorn. It builds a one-byte
  temporary string before measuring each character, so the existing DBCS
  metrics hook measured isolated GB2312 bytes as width/height zero. This
  miscentered Chinese lines and advanced the next line by only the 5px line
  distance, overlapping the 12px Chinese artwork.
- Added pair-aware read, append, and fit caves to `tools/apply_cjk.py`.
  GB2312 pairs now reach the measurer together, are appended/consumed together,
  and become complete-character wrap checkpoints. Initialized empty-line
  checkpoints correctly so mixed ASCII/CJK wraps retain the prior line height.
  A glyph wider than the available box is emitted intact rather than losing
  its lead byte; existing caller clipping remains responsible for box bounds.
- Added `tools/test_cjk_layout.py`: executes the real layout/measurement
  instructions and captures only the Direct3D draw boundary. Fourteen tests
  cover screenshot tooltip strings, shortcuts, centering, control codes,
  measure-only height, mixed text, whitespace, narrow boxes, and oversize glyphs.
  The original overlap case changes from `(Chinese x=24,y=0; S x=20,y=5)` to
  `(Chinese x=12,y=0; S x=20,y=17)` in a 48px box.
- No atlas, translation, font-size, or mission-script changes are needed.
  Original exe and patcher snapshots: `backup_game/font-layout-20260920/`.
  Local deployed program: `F:\steam\steamapps\common\The I of the Dragon\TheIOfTheDragon.exe`.

Validation (requires `unicorn`):
`python tools/test_cjk_layout.py --exe build/cjk-layout-fixed.exe -v`

Progress audit: local HEAD matched Gitea `ca57488` before this fix; 1884 texture
entries / 1882 preprocessed PNGs, 1397 geometry entries, and 479/496 translated
Strings.dat IDs. Mission scripts, description tables, baked text, and HD
upscaling remain separate unfinished work. Other-font CJK coverage is also
still limited as documented in patch-notes.md.

## 2026-09-20 — Full `Strings.dat` translation

- Translated the `English` column for **479 / 496** entries (the rest are pure-format
  strings, empty/whitespace values, the tutorial texture *path*, and a few odd key labels).
- `tools/check_translations.py` verifies the printf specifier sequence (`%s`/`%d`/`%i`/`%%`)
  matches the English source exactly (0 mismatches) and that every value GB2312-encodes.
- `tools/dump_all_strings.py` dumps the source table to `build/strings_en.json`.

### Known layout caveats (engine UI is tuned for the small Latin pixel font)

- Some engine UI positions tab/button labels against a box sized for English, so short
  Chinese labels can look left/right-aligned; a couple of centred dialogs jitter while a
  teletype reveal is running.
- In-game HUD button labels wrap and can overlap the shortcut hint because the boxes are
  small; to revisit via a smaller CJK cell/advance if needed.

## 2026-09-20 — Chinese in the main menu (first playable)

- Built `tools/build_cjk.py`: renders the full GB2312 set (7445 double-byte codes) as **12x12**
  glyphs with simsun (crisp/binary, matching the game's pixel font) into a **2048x1024** TGA.
  The original 256x128 ASCII artwork is copied to (0,0), so all existing `Fonts.dat` rects keep
  working unchanged (verified byte-identical). Also emits `cjk_glyph.bin` / `cjk_metric.bin`.
  Advance/height = 12 (the UI font cap height is ~11-12px), drawn 1px up to fit the cell.
- `tools/cn_strings.py` + `tools/make_cn_strings.py`: translate the `English` column of
  `Strings.dat` to GB2312 (46 entries; escape-aware so `\"%s\"` formats survive).
- `tools/install_cjk.py`: one-shot deploy/restore (backs up exe/res/strings; rebuilds `Textures.res`
  with the new atlas, drops the loose atlas, patches the exe and the strings).
- **Result**: the main menu renders Chinese — 开始新游戏 / 载入游戏 / 选项 / 教程 / 制作人员 / 退出 /
  载入上次存档："…" — confirmed by the user on screen.

### Fixed a byte-order bug in the metrics cave

The draw cave decoded the code as `(first<<8)|second`, but the metrics cave used
`movzx eax,al; mov ah,[ebx+1]` which produced `(second<<8)|first`. Every CJK width measured as 0,
so centred labels (the "Start New Game" button) were flushed right. Fixed to
`movzx eax,al; shl eax,8; mov al,[ebx+1]`. Centring is now correct.

### Notes / next

- The menu button labels *are* dynamic (`Strings.dat`) — the earlier suspicion of baked button
  images was wrong; the layer confusion came from the animated particle background.
- Remaining UI text that is baked into textures (credits, tutorial pics) still needs redrawing.
- Next: translate the rest of `Strings.dat`, scripts and `*Description.dat`; then HD textures.

## 2026-09-20 — Double-byte font patch proven

- Reverse-engineered the whole text pipeline (see `patch-notes.md`).
- Built `tools/add_section.py` (PE section injection) and verified the patched exe still runs.
- Built `tools/apply_cjk.py`: assembles two code caves with keystone, adds a `.cjk` section,
  hooks the draw loop (`0x4EACB0`) and the metrics loop (`0x4E8026`).
- **Verified on screen**: a GB2312-coded string `0xC1FA` (龙) in a menu label rendered a 16x16
  glyph at the mapped atlas position (green control test). Double-byte decode + render works.
- Game files restored afterwards (exe SHA1 `9EDB9B99...`, `Textures.res` byte-identical).

### Next
1. Big atlas (2048x2048): ASCII glyphs (copied) + CJK glyphs; re-point `Fonts.dat` `Texture` + ASCII rects.
2. Generator for `cjk_glyph` / `cjk_metric` tables from a chosen charset.
3. Convert `Strings.dat`, scripts, and `*Description.dat` display text to GB2312.
4. Redraw baked-in text textures (menu buttons, credits, tutorial).
5. Package: patched exe + atlas + data files.

## 2026-09-20 — HD texture preprocessing

- Cracked the `.res` archive format and built `res_unpack.py` / `res_repack.py`
  (byte-identical round-trip once 16-byte alignment was found).
- Extracted `Textures.res` (1884) and `Geometry.res` (1397).
- Built `preprocess.py` (TGA/DDS→PNG, classification) and `make_plan.py`.
- Result: 63.6 Mpix, 1882 PNG, 0 failures. Plan: 1195 upscale / 499 tiling / 182 review / 8 skip
  (6 font atlases + 2 non-textures).

## 2026-09-20 — Cracking the `.dat` obfuscation

- `Fonts.dat` / `UnitDescription.dat` / `AIDescription.dat` decoded:
  `b"==" + XOR(plaintext, "GBDFYTNE")`; key at `0x620D18`, decoder at `0x4AE1D0`.
- `Fonts.dat` parsed: 4 font blocks, per-char atlas rects; parser keeps only the first byte of `Code`
  (`0x4EC168`), so the stock engine is single-byte.

## 2026-09-20 — Initial findings

- Engine has a data-driven multi-language system: per-language folders for scripts/speech/video/UI
  textures and language columns in `Strings.dat`.
- Text is externalized: `Strings.dat` (499 entries, plain) and plain-text scripts.
- First font test: replaced the `e` glyph in `Shrift_gb_Germany.TGA` and confirmed the atlas override
  works in game (rendered 龙 where `e` was).
- Investigated the "pirate build": PROPHET 2009 Topware release, languages EN/DE/PL/CZ/HU, **no
  Chinese**; its exe is identical to the Steam exe (no engine patch), so the Chinese build the user
  remembers was a different repack (not recoverable from it).

## Decisions

- Do the localization ourselves (route B): patch the engine for double-byte text, build our own font
  atlas, translate only display text (never Ids / event names / unit types) to avoid trigger bugs.
- Keep the repository text-only; regenerate binaries from scripts (see `.gitignore`).

## 2026-09-20 — Tutorial Chinese trial, batch 1

- Branch: `codex/tutorial-step1`. Translate only introduction, camera rotation, camera centering and zoom: 9 mappings replace 12 of 55 direct display literals. Flight and later stages remain unchanged.
- Added `tools/script_text.py`, a byte-span scanner restricted to direct `DisplayMessage` / `SetMissionDescription` string arguments; preserves controls, printf parameters and all non-display bytes. JSON contains decoded strings; translated output uses GB2312.
- Added 14 scanner tests and `tools/verify_tutorial_step1.py` for the pinned source, changed-span coverage and full-byte baseline/rollback checks. All 14 scanner + 16 existing font regression tests pass.
- Original SHA256: `bd959758223b976ffac35b4ef7d94661f62060c1134316eddb559772d544e02c`. Translated SHA256: `c0e9d1d0fe2e8f3eb656907e5c2e1c44222614b8ecdb6e2ef148ccf897ed40e0`.
- Original backup: `E:/DragonHD/backup_game/tutorial-step1-20260920/Tutorial.dsc`; build: `E:/DragonHD/build/tutorial-step1/Tutorial.dsc`. Deployed to the Steam English script directory after checking the game was stopped and the original hash still matched. Re-read deployed file and passed the same byte-span verification.
- Git Bash rollback script was executed on a separate translated copy; restored bytes/hash match the original exactly. The deployed tutorial and deliverable remain translated.
- In-game acceptance PENDING: start a fresh tutorial, read intro, rotate camera, press numpad 5, zoom with wheel, and check each next-step trigger. Then translate flight in the next small batch. No claim yet about native script loading or rendered layout.

## 2026-09-20 — Intro paragraph layout correction

- User confirmed the subsequent tutorial pages work; opening screenshot showed orphan comma and inconsistent alignment. Engine replay at width 400 reproduced the first-line 15px indent from `\f`, unindented wrapped continuation, and centered final paragraph from `\a`.
- Replaced only intro message/mission paragraphs (literal indexes 1 and 2 relative to the previous trial) with concise text and `\b` left alignment. Added explicit `--allow-alignment-changes` build opt-in; default control validation remains strict, opt-in still preserves control order/count and newlines.
- New regression failed on original mapping (x=15, expected 0), then passed at widths 360/400/480. All 32 tests pass. All other script bytes unchanged relative to previous Chinese trial.
- Built and deployed after process and source-hash checks. SHA256: `15a7a24a236d01af4f5f3bcc89901dac5ab0211ed51357dfda5ef78d0026c423`. Backup of previous trial: `E:/DragonHD/backup_game/tutorial-intro-layout-20260920/Tutorial.dsc`. Rollback executed on separate copy and restored previous hash.
- Rebuild: `python tools/script_text.py --source E:/DragonHD/backup_game/tutorial-step1-20260920/Tutorial.dsc --mapping tools/tutorial_cn.json --out E:/DragonHD/build/tutorial-intro-layout/Tutorial.dsc --allow-alignment-changes`.
- Opening page visual recheck pending. Later pages retained exactly as user-tested.

## 2026-09-20 — Correct opening page to centered alignment

- User rejected left alignment. The prior fix interpreted the visual preference incorrectly. Changed all four intro paragraphs to centered alignment, preserving the concise text and all later pages.
- Exactly eight escape bytes changed (`\b` to `\a`) across the intro message and repeated mission description.
- Center-coordinate test first failed on prior left-aligned mapping (0 != 66). New mapping passes at widths 360/400/480 using engine font metrics including 5px spaces. All 32 tests pass; rollback restores the previous copy exactly.
- User exited game; deployed centered file and reread/verified it. SHA256: `941fb4899d37304a304a985743c18cfd4b7dfed7613d831f8359c4d6b6972d6d`. Visual acceptance pending.

## 2026-09-20 — Chinese full-stop baseline correction

- Inspected live 12px SimSun atlas: U+3002 ink bounding box `(2,5,6,9)` in a 12x12 cell leaves three empty rows below, reproducing the user's raised punctuation.
- Anchored only U+3002 to the cell bottom: bbox `(2,8,6,12)`, same eight pixels and shape. All other atlas pixels and bytes remain identical. Updated builder to retain this placement on future generation; `fix_full_stop.py` patches existing TGA without rebuilding other glyphs.
- Prepared matching loose atlas and in-place same-size `Textures.res` entry; bytes before and after that entry remain identical. No executable, metrics, UV table, script, or other punctuation changes.
- 34 tests pass. Checked original/fixed glyph bounds and exact shape preservation; rollback of both files tested against backups in an isolated directory. Comparison PNG uses actual atlas pixels enlarged 8x.
- Backup: `E:/DragonHD/backup_game/full-stop-20260920`. User explicitly authorized terminating the running game for installation. Real in-game appearance pending.

## 2026-09-20 — Flight tutorial trial

- User accepted full-stop correction. Continued with flight only: title, message and mission description (literal indexes 12–14). Two concise centered paragraphs retain mouse destination, altitude globe and keyboard movement instructions.
- All other text and non-display bytes preserved; the original movement threshold (>15 units along either horizontal axis within the timed loop) and next-button gating remain unchanged. Flight height/attack are next batches.
- 35 tests pass, including engine layout at widths 360/400/480; original/modified byte verification and rollback passed. Backup: `E:/DragonHD/backup_game/tutorial-flight-20260920/Tutorial.dsc`.
- Game stopped using user's prior termination authorization; deployed file reread and matched output. SHA256: `a41caad224f748222489f8d062526ae663bb04201c593f59d6af02d69a74f604`.
- In-game acceptance pending: enter Flight after Zoom, use left click or arrow keys to move some distance, then check Continue Tutorial becomes available and Flight height starts.