Initial commit: RE tooling and docs for The I of the Dragon localization/HD
This commit is contained in:
DragonHD
@@ -0,0 +1,54 @@
|
||||
# DEVLOG
|
||||
|
||||
Working log for the *The I of the Dragon* localization / HD project.
|
||||
Newest entries on top.
|
||||
|
||||
## 2026-09-20 — Double-byte font patch proven
|
||||
|
||||
- Reverse-engineered the whole text pipeline (see `patch-notes.md`).
|
||||
- Built `tools/add_section.py` (PE section injection) and verified the patched exe still runs.
|
||||
- Built `tools/apply_cjk.py`: assembles two code caves with keystone, adds a `.cjk` section,
|
||||
hooks the draw loop (`0x4EACB0`) and the metrics loop (`0x4E8026`).
|
||||
- **Verified on screen**: a GB2312-coded string `0xC1FA` (龙) in a menu label rendered a 16x16
|
||||
glyph at the mapped atlas position (green control test). Double-byte decode + render works.
|
||||
- Game files restored afterwards (exe SHA1 `9EDB9B99...`, `Textures.res` byte-identical).
|
||||
|
||||
### Next
|
||||
1. Big atlas (2048x2048): ASCII glyphs (copied) + CJK glyphs; re-point `Fonts.dat` `Texture` + ASCII rects.
|
||||
2. Generator for `cjk_glyph` / `cjk_metric` tables from a chosen charset.
|
||||
3. Convert `Strings.dat`, scripts, and `*Description.dat` display text to GB2312.
|
||||
4. Redraw baked-in text textures (menu buttons, credits, tutorial).
|
||||
5. Package: patched exe + atlas + data files.
|
||||
|
||||
## 2026-09-20 — HD texture preprocessing
|
||||
|
||||
- Cracked the `.res` archive format and built `res_unpack.py` / `res_repack.py`
|
||||
(byte-identical round-trip once 16-byte alignment was found).
|
||||
- Extracted `Textures.res` (1884) and `Geometry.res` (1397).
|
||||
- Built `preprocess.py` (TGA/DDS→PNG, classification) and `make_plan.py`.
|
||||
- Result: 63.6 Mpix, 1882 PNG, 0 failures. Plan: 1195 upscale / 499 tiling / 182 review / 8 skip
|
||||
(6 font atlases + 2 non-textures).
|
||||
|
||||
## 2026-09-20 — Cracking the `.dat` obfuscation
|
||||
|
||||
- `Fonts.dat` / `UnitDescription.dat` / `AIDescription.dat` decoded:
|
||||
`b"==" + XOR(plaintext, "GBDFYTNE")`; key at `0x620D18`, decoder at `0x4AE1D0`.
|
||||
- `Fonts.dat` parsed: 4 font blocks, per-char atlas rects; parser keeps only the first byte of `Code`
|
||||
(`0x4EC168`), so the stock engine is single-byte.
|
||||
|
||||
## 2026-09-20 — Initial findings
|
||||
|
||||
- Engine has a data-driven multi-language system: per-language folders for scripts/speech/video/UI
|
||||
textures and language columns in `Strings.dat`.
|
||||
- Text is externalized: `Strings.dat` (499 entries, plain) and plain-text scripts.
|
||||
- First font test: replaced the `e` glyph in `Shrift_gb_Germany.TGA` and confirmed the atlas override
|
||||
works in game (rendered 龙 where `e` was).
|
||||
- Investigated the "pirate build": PROPHET 2009 Topware release, languages EN/DE/PL/CZ/HU, **no
|
||||
Chinese**; its exe is identical to the Steam exe (no engine patch), so the Chinese build the user
|
||||
remembers was a different repack (not recoverable from it).
|
||||
|
||||
## Decisions
|
||||
|
||||
- Do the localization ourselves (route B): patch the engine for double-byte text, build our own font
|
||||
atlas, translate only display text (never Ids / event names / unit types) to avoid trigger bugs.
|
||||
- Keep the repository text-only; regenerate binaries from scripts (see `.gitignore`).
|
||||
@@ -0,0 +1,84 @@
|
||||
# File formats
|
||||
|
||||
All addresses are absolute VAs in `TheIOfTheDragon.exe` (image base `0x400000`).
|
||||
|
||||
## 1. `Data\Misc\Strings.dat` — plain text
|
||||
|
||||
Same syntax as the other `.dat` files, but **not** obfuscated. Blocks look like:
|
||||
|
||||
```
|
||||
String
|
||||
{
|
||||
Id = "WindowCaption"
|
||||
English = "..."
|
||||
Russian = "..."
|
||||
German = "..."
|
||||
}
|
||||
```
|
||||
|
||||
- 499 blocks in the Steam build. Keys are `String{ Id, English, Russian, German }`
|
||||
(retail multilanguage builds also carry `Polish`, `Czech`, `Hungarian`, `Hungarian/Russian` variants
|
||||
split across separate installed files).
|
||||
- **Translate only the language values.** Never touch `Id`.
|
||||
- The file is single-byte code page text (Russian is CP1251).
|
||||
|
||||
## 2. `Data\Misc\Fonts.dat`, `UnitDescription.dat`, `AIDescription.dat` — XOR-obfuscated text
|
||||
|
||||
Layout: `b"==" + XOR(plaintext, repeating key)`.
|
||||
|
||||
- Magic `==` at VA `0x620D24`.
|
||||
- Repeating XOR key `GBDFYTNE` (8 bytes) at VA `0x620D18`.
|
||||
- Decoder/encoder routine at `0x4AE1D0`.
|
||||
|
||||
Decoded, these are the same text format as `Strings.dat`:
|
||||
|
||||
- `Fonts.dat`: `Font { Name; Language; Texture; SpaceWidth; LineDistance; Char { Code; X1;Y1;X2;Y2 } ... }`
|
||||
- `UnitDescription.dat`: unit/enemy definitions (Russian comments in CP1251 + English keys).
|
||||
- `AIDescription.dat`: `Brain { AI = ...; TypeAI = ...; ... }`.
|
||||
|
||||
Round-trip is exact; see `tools/decode_dat.py`.
|
||||
|
||||
## 3. `res` archives (`Textures.res`, `Geometry.res`)
|
||||
|
||||
```
|
||||
uint32 count
|
||||
repeat count:
|
||||
uint32 nameLen
|
||||
char[] name (backslash separated)
|
||||
uint32 offset (absolute file offset)
|
||||
uint32 size (exact blob size)
|
||||
pad the index to 16 bytes
|
||||
data region:
|
||||
for each entry: write blob, then pad to 16-byte boundary
|
||||
(the last blob is NOT padded)
|
||||
```
|
||||
|
||||
- All offsets are 16-byte aligned.
|
||||
- `Textures.res` holds 1884 entries: 947 TGA + 935 DDS (DXT1/DXT5) + 2 misc.
|
||||
- `res_unpack.py` / `res_repack.py` round-trip is **byte identical** (verified by SHA1).
|
||||
|
||||
## 4. Scripts
|
||||
|
||||
- `Data\Scripts\<Language>\MainMission.dsc`, `Tutorial.dsc` — plain text, **loaded by the engine**
|
||||
(path template `Data\Scripts\%s\...`; `.dsc` is referenced, `.csc` is not).
|
||||
- Block-structured scripting language. Quoted strings are of two kinds:
|
||||
- **display text** (translate): `SetMissionDescription("...")`
|
||||
- **logic identifiers** (never translate): event names `EnableEvent("TimePassed")`,
|
||||
`LastEvent_Type == "TimePassed"`, commands `EntityCommand("FaceThePlayer",...)`,
|
||||
modes `SetGlobalAIMode("Disable")`, unit types `CreateEntity(..., "Human", ...)`, etc.
|
||||
- Translating logic identifiers is what breaks mission triggers ("完成任务不触发剧情").
|
||||
|
||||
## 5. Language mechanism
|
||||
|
||||
- Current language: registry `HKCU\Software\Primal\Dragon` value `language`.
|
||||
- Assets are per-language by folder: `Data\Scripts\<L>`, `Data\Sounds\Speech\<L>`,
|
||||
`Data\Video\<L>`, `Data\Textures\Ui\...\<L>`.
|
||||
- `Fonts.dat` / `Strings.dat` are installed per language group (observed: `[en de] [pl] [cs] [hu]`;
|
||||
Steam build ships a single English variant with `English/Russian/German`).
|
||||
- Loose files under `Data\Textures\...` override the packed copies.
|
||||
|
||||
## 6. Textures
|
||||
|
||||
- TGA: uncompressed 24/32 bpp and RLE 32 bpp.
|
||||
- DDS: DXT1 (535) and DXT5 (400), plus dimensions from 8x8 up to 1024x1024.
|
||||
- Total 63.6 Mpix across 1884 textures.
|
||||
@@ -0,0 +1,80 @@
|
||||
# Font rendering map + double-byte patch
|
||||
|
||||
## Font object layout (size `0x191C`, embedded by value in a manager struct)
|
||||
|
||||
Two UI font objects live at `manager + 0x3A8` and `manager + 0x1CC4` (difference = `0x191C`).
|
||||
|
||||
```
|
||||
+0x00 texture pointer (D3D texture object)
|
||||
+0x04 max width
|
||||
+0x08 max height
|
||||
+0x0C SpaceWidth
|
||||
+0x10 LineDistance
|
||||
+0x14 CharDistance
|
||||
+0x18 flag (1 = built)
|
||||
+0x1C 256 glyph entries x 16 bytes = 4 floats (u1,v1,u2,v2) normalized UV
|
||||
+0x101C 256 metric entries x 8 bytes = { int advance, int height }
|
||||
+0x181C 256 "defined" bytes
|
||||
```
|
||||
|
||||
The table stride differs (16 vs 8 vs 1), which is why the tables cannot be enlarged in place and a
|
||||
side table is used for CJK.
|
||||
|
||||
## Key code
|
||||
|
||||
| VA | role |
|
||||
|---|---|
|
||||
| `0x4EBF90` | parse/load `Fonts.dat` (generic config reader) |
|
||||
| `0x4E7DC0` | set font texture |
|
||||
| `0x4E7E30` | `AddChar(code, x1,y1,x2,y2)` — normalizes rect → stores glyph float[4] + metric[2] |
|
||||
| `0x4EC168` | `mov cl,[ebp]` — parser takes only the **first byte** of `Code` |
|
||||
| `0x4E7FD0` | measure text; loop body `0x4E7FF0`; metric lookups `0x4E802C`, `0x4E8033` |
|
||||
| `0x4EAC40` | draw text; loop head |
|
||||
| `0x4EACB0` | `movzx ecx,al` — glyph lookup site in draw loop |
|
||||
| `0x4EACDC`,`0x4EAE9A` | metric lookups in draw loop |
|
||||
| `0x4E56A0` | `IsCharDefined(font, code)` |
|
||||
|
||||
Texture is bound **once per draw call** (`[font]` + vtable `+0xF4`), so ASCII and CJK glyphs must
|
||||
share one texture atlas.
|
||||
|
||||
## Patch design (implemented)
|
||||
|
||||
A new PE section `.cjk` holds two code caves and two big tables:
|
||||
|
||||
```
|
||||
.cjk RVA 0x566000 (VA 0x966000), size 0x1C0000
|
||||
0x00 draw_cave
|
||||
0x0100 metrics_cave
|
||||
0x0200 cjk_glyph : 65536 x 16 bytes (u1,v1,u2,v2 floats)
|
||||
0x100200 cjk_metric : 65536 x 8 bytes (advance,height ints)
|
||||
```
|
||||
|
||||
Hooks (5-byte `jmp rel32`):
|
||||
|
||||
- `0x4EACB0 -> draw_cave` (was `movzx ecx,al`)
|
||||
- `0x4E8026 -> metrics_cave` (was `mov ebp,[ecx+0x14]`)
|
||||
|
||||
Behaviour of each cave:
|
||||
|
||||
- `al < 0x80`: execute the original instructions, jump back to the next instruction.
|
||||
- `al >= 0x80`: decode a 2-byte code `(al<<8)|[ptr+1]`; copy the glyph float[4] into font glyph slot
|
||||
`0xFF` and the metric into slot `0xFF`; set `ecx = 0xFF`, advance the string pointer by one extra
|
||||
byte; jump back into the engine's unchanged quad-building code.
|
||||
|
||||
Slot `0xFF` glyph lives at `font+0x100C`, its metric at `font+0x1814`; these do not collide with the
|
||||
real tables.
|
||||
|
||||
## Requirements / limits
|
||||
|
||||
- The font texture must contain **both** ASCII and CJK glyphs (one texture per draw).
|
||||
Plan: replace the UI font `Texture` with a 2048x2048 atlas and re-point the ASCII `Char` rects.
|
||||
- Lead bytes >= `0x80` are reserved for DBCS. ASCII is untouched.
|
||||
- Confirmed end to end: a GB2312-coded menu string rendered the mapped glyph on screen.
|
||||
|
||||
## Tools
|
||||
|
||||
- `tools/add_section.py` — append an executable PE section.
|
||||
- `tools/apply_cjk.py` — assemble the caves (keystone), add `.cjk`, write hooks and tables.
|
||||
- `tools/decode_dat.py` — decode/encode the XOR `.dat` files.
|
||||
- `tools/fontparse.py` — parse `Fonts.dat` into blocks + glyph rects.
|
||||
- `tools/xrefs.py`, `tools/disasm.py`, `tools/scan_disp.py` — RE helpers.
|
||||
Reference in New Issue
Block a user