From 2b626fb53e6a6d56505434f27c21e1408e1bed5c Mon Sep 17 00:00:00 2001 From: Luke Street Date: Mon, 4 Mar 2024 23:30:38 -0700 Subject: [PATCH] Minor documentation updates --- README.example.md | 31 +++++++++++++++++++++---------- README.md | 7 ++++++- config/GAMEID/config.example.yml | 5 +++++ config/GAMEID/config.yml | 4 +++- docs/comment_section.md | 15 ++++++++++----- docs/common_bss.md | 2 +- docs/dependencies.md | 16 +++++++++++----- docs/getting_started.md | 18 +++++++++++++----- docs/splits.md | 4 ++-- docs/symbols.md | 3 ++- 10 files changed, 74 insertions(+), 31 deletions(-) diff --git a/README.example.md b/README.example.md index fbe8164..2d8e688 100644 --- a/README.example.md +++ b/README.example.md @@ -42,7 +42,7 @@ Supported versions: Dependencies ============ -Windows: +Windows -------- On Windows, it's **highly recommended** to use native tooling. WSL or msys2 are **not** required. @@ -53,24 +53,30 @@ When running under WSL, [objdiff](#diffing) is unable to get filesystem notifica - Download [ninja](https://github.com/ninja-build/ninja/releases) and add it to `%PATH%`. - Quick install via pip: `pip install ninja` -macOS: +macOS ------ + - Install [ninja](https://github.com/ninja-build/ninja/wiki/Pre-built-Ninja-packages): - ``` + + ```sh brew install ninja ``` + - Install [wine-crossover](https://github.com/Gcenx/homebrew-wine): - ``` + + ```sh brew install --cask --no-quarantine gcenx/wine/wine-crossover ``` After OS upgrades, if macOS complains about `Wine Crossover.app` being unverified, you can unquarantine it using: + ```sh sudo xattr -rd com.apple.quarantine '/Applications/Wine Crossover.app' ``` -Linux: +Linux ------ + - Install [ninja](https://github.com/ninja-build/ninja/wiki/Pre-built-Ninja-packages). - For non-x86(_64) platforms: Install wine from your package manager. - For x86(_64), [wibo](https://github.com/decompals/wibo), a minimal 32-bit Windows binary wrapper, will be automatically downloaded and used. @@ -79,21 +85,26 @@ Building ======== - Clone the repository: - ``` + + ```sh git clone https://github.com/my/repo.git ``` + - Using [Dolphin Emulator](https://dolphin-emu.org/), extract your game to `orig/GAMEID`. ![](assets/dolphin-extract.png) - To save space, the only necessary files are the following. Any others can be deleted. - `sys/main.dol` - `files/rels/*.rel` - Configure: - ``` + + ```sh python configure.py ``` + To use a version other than `GAMEID` (USA), specify it with `--version`. - Build: - ``` + + ```sh ninja ``` @@ -105,9 +116,9 @@ If desired, use the recommended Visual Studio Code settings by renaming the `.vs Diffing ======= -Once the initial build succeeds, an `objdiff.json` should exist in the project root. +Once the initial build succeeds, an `objdiff.json` should exist in the project root. -Download the latest release from [encounter/objdiff](https://github.com/encounter/objdiff). Under project settings, set `Project directory`. The configuration should be loaded automatically. +Download the latest release from [encounter/objdiff](https://github.com/encounter/objdiff). Under project settings, set `Project directory`. The configuration should be loaded automatically. Select an object from the left sidebar to begin diffing. Changes to the project will rebuild automatically: changes to source files, headers, `configure.py`, `splits.txt` or `symbols.txt`. diff --git a/README.md b/README.md index e730d54..6967840 100644 --- a/README.md +++ b/README.md @@ -14,6 +14,7 @@ Documentation - [`splits.txt`](docs/splits.md) General: + - [Common BSS](docs/common_bss.md) - [`.comment` section](docs/comment_section.md) @@ -28,11 +29,13 @@ References - [sjiswrap](https://github.com/encounter/sjiswrap) (UTF-8 to Shift JIS wrapper) Projects using this structure: + - [zeldaret/tww](https://github.com/zeldaret/tww) - [PrimeDecomp/prime](https://github.com/PrimeDecomp/prime) - [PrimeDecomp/echoes](https://github.com/PrimeDecomp/echoes) - [DarkRTA/rb3](https://github.com/DarkRTA/rb3) -- [InputEvelution/sadx-dtk](https://github.com/InputEvelution/sadx-dtk) +- [doldecomp/melee](https://github.com/doldecomp/melee) +- [doldecomp/sadx](https://github.com/doldecomp/sadx) - [InputEvelution/wp](https://github.com/InputEvelution/wp) - [lepelog/ss-dtk](https://github.com/lepelog/ss-dtk) - [NWPlayer123/AnimalCrossing-dtk](https://github.com/NWPlayer123/AnimalCrossing-dtk) @@ -42,6 +45,7 @@ Projects using this structure: Features -------- + - Few external dependencies: Just `python` for the generator and `ninja` for the build system. See [Dependencies](docs/dependencies.md). - Simple configuration: Everything lives in `config.yml`, `symbols.txt`, and `splits.txt`. - Multi-version support: Separate configurations for each game version, and a `configure.py --version` flag to switch between them. @@ -66,6 +70,7 @@ Project structure - `tools/` - Scripts shared between projects. Temporary, delete when done: + - `config/GAMEID/config.example.yml` - Example configuration file and documentation. - `docs/` - Documentation for decomp-toolkit configuration. - `README.md` - This file, replace with your own. For a template, see [`README.example.md`](README.example.md). diff --git a/config/GAMEID/config.example.yml b/config/GAMEID/config.example.yml index 0af0bc5..4d7e219 100644 --- a/config/GAMEID/config.example.yml +++ b/config/GAMEID/config.example.yml @@ -26,6 +26,11 @@ common_start: 0x80001234 # (optional) Version used to generate `.comment` sections in the split objects. # If not specified, no `.comment` sections will be generated. # See docs/comment_section.md for more information. +# Known versions: +# 8 - CodeWarrior for GameCube 1.0+ +# 10 - CodeWarrior for GameCube 1.3.2+ +# 11 - CodeWarrior for GameCube 2.7+ +# 14 - CodeWarrior for GameCube 3.0a3+ mw_comment_version: 8 # (optional) Path to `selfile.sel` for Wii games with RSO files. diff --git a/config/GAMEID/config.yml b/config/GAMEID/config.yml index e48180d..e578d2b 100644 --- a/config/GAMEID/config.yml +++ b/config/GAMEID/config.yml @@ -3,7 +3,9 @@ object: orig/GAMEID/sys/main.dol hash: 0123456789abcdef0123456789abcdef01234567 symbols: config/GAMEID/symbols.txt splits: config/GAMEID/splits.txt -mw_comment_version: 8 +# Change this to match the linker verison. +# See config.example.yml for a list. +mw_comment_version: 11 modules: - object: orig/GAMEID/files/module.rel diff --git a/docs/comment_section.md b/docs/comment_section.md index 2e63b2e..590d15d 100644 --- a/docs/comment_section.md +++ b/docs/comment_section.md @@ -20,6 +20,7 @@ If missing, `mwld` will **not** adjust the alignment of symbols or remove any un This behavior is quite useful in some cases. When we split our program into objects, we're working from the final post-aligned, post-stripped result, and don't want the linker to make any changes. Most decompilation projects rely on this behavior unintentionally, since their generated objects don't contain a `.comment` section. (For example, objects built with `powerpc-eabi-as`.) However, we need the `.comment` section for some purposes: + - Reproducing the [common BSS inflation bug](common_bss.md#inflation-bug) requires the `.comment` section present, due to the above. The linker inflates the size of the first common BSS symbol in a TU, but won't actually move any data around unless the `.comment` section is present. - In newer versions of the linker, using common BSS at all _without_ a valid `.comment` section will cause an internal linker error. @@ -40,10 +41,11 @@ The contents of this section follow a very simple format: It's not known whether this field actually affects `mwld` in any way, but it's configurable for completeness sake. (See `mw_comment_version` in [`config.example.yml`](/config/GAMEID/config.example.yml).) Known values: -- `08` - CodeWarrior for GameCube 1.0+ -- `0A` - CodeWarrior for GameCube 1.3.2+ -- `0B`, `0C` - CodeWarrior for GameCube 2.7+ (difference unknown) -- `0E`, `0F` - CodeWarrior for GameCube 3.0a3+ (difference unknown) + +- `08` (8) - CodeWarrior for GameCube 1.0+ +- `0A` (10) - CodeWarrior for GameCube 1.3.2+ +- `0B` (11), `0C` (12) - CodeWarrior for GameCube 2.7+ (difference unknown) +- `0E` (14), `0F` (15) - CodeWarrior for GameCube 3.0a3+ (difference unknown) `[0xC size: 0x4]` Compiler version: `XX XX XX XX` @@ -71,6 +73,7 @@ Often the `.exe`'s properties (which `--help` reads from) and the internal versi `[0x15 size: 1]` "Quirk" flags: `XX` Bitfield of miscellaneous flags. Known flags: + - `01` - "Incompatible return small structs" - `02` - "Incompatible SFPE double params" - `04` - "Unsafe global reg vars" @@ -88,6 +91,7 @@ This includes the "null" ELF symbol, so the first entry will be all 0's. `[0x4 size: 1]` Visibility flags(?): `XX` Known values: + - `00` - Default - `0D` - Weak - `0E` - Unknown, also weak? @@ -95,6 +99,7 @@ Known values: `[0x5 size: 1]` Active flags(?): `XX` Known values: + - `00` - Default - `08` - Force active / export. Prevents the symbol from being deadstripped. When applied on a section symbol, the entire section is kept as-is. This is used @@ -103,4 +108,4 @@ Known values: - `10` - Unknown - `20` - Unknown -`[0x6 size: 2]` Padding(?): `00 00` \ No newline at end of file +`[0x6 size: 2]` Padding(?): `00 00` diff --git a/docs/common_bss.md b/docs/common_bss.md index cd88b3c..f00ddd9 100644 --- a/docs/common_bss.md +++ b/docs/common_bss.md @@ -15,7 +15,7 @@ With `-common off`, `foo` would be defined as a **global** symbol, and the linke In `splits.txt`, common BSS can be defined with the `common` attribute: -``` +```yaml foo.cpp: .text start:0x80047E5C end:0x8004875C .ctors start:0x803A54C4 end:0x803A54C8 diff --git a/docs/dependencies.md b/docs/dependencies.md index 56ab897..f6d1a30 100644 --- a/docs/dependencies.md +++ b/docs/dependencies.md @@ -1,7 +1,7 @@ Dependencies ============ -Windows: +Windows -------- On Windows, it's **highly recommended** to use native tooling. WSL or msys2 are **not** required. @@ -12,24 +12,30 @@ When running under WSL, [objdiff](#diffing) is unable to get filesystem notifica - Download [ninja](https://github.com/ninja-build/ninja/releases) and add it to `%PATH%`. - Quick install via pip: `pip install ninja` -macOS: +macOS ------ + - Install [ninja](https://github.com/ninja-build/ninja/wiki/Pre-built-Ninja-packages): - ``` + + ```sh brew install ninja ``` + - Install [wine-crossover](https://github.com/Gcenx/homebrew-wine): - ``` + + ```sh brew install --cask --no-quarantine gcenx/wine/wine-crossover ``` After OS upgrades, if macOS complains about `Wine Crossover.app` being unverified, you can unquarantine it using: + ```sh sudo xattr -rd com.apple.quarantine '/Applications/Wine Crossover.app' ``` -Linux: +Linux ------ + - Install [ninja](https://github.com/ninja-build/ninja/wiki/Pre-built-Ninja-packages). - For non-x86(_64) platforms: Install wine from your package manager. - For x86(_64), [wibo](https://github.com/decompals/wibo), a minimal 32-bit Windows binary wrapper, will be automatically downloaded and used. diff --git a/docs/getting_started.md b/docs/getting_started.md index b084c5e..c3c3628 100644 --- a/docs/getting_started.md +++ b/docs/getting_started.md @@ -12,8 +12,8 @@ Rename `config/GAMEID` to the game's ID and modify `config/[GAMEID]/config.yml` Generate a `config/[GAMEID]/build.sha1` file for verification. This file is a list of SHA-1 hashes for each build artifact. One possible way: -```shell -$ dtk shasum orig/[GAMEID]/sys/main.dol orig/[GAMEID]/files/*.rel > config/[GAMEID]/build.sha1 +```sh +dtk shasum orig/[GAMEID]/sys/main.dol orig/[GAMEID]/files/*.rel > config/[GAMEID]/build.sha1 ``` Then, modify the paths in `config/[GAMEID]/build.sha1` to point to the `build` directory instead of `orig`. The DOL will be built at `build/[GAMEID]/main.dol`, and modules will be built at `build/[GAMEID]/[module_name]/[module_name].rel`. @@ -49,7 +49,7 @@ Often indicated by the following error: # '__init_cpp_exceptions.cpp' both need to be updated to latest version. ``` -### GC 1.0 - 2.6 linkers: +### GC 1.0 - 2.6 linkers ```yaml # splits.txt @@ -62,11 +62,13 @@ Runtime.PPCEABI.H/__init_cpp_exceptions.cpp: `.text`: Find the following symbols in `symbols.txt`: + ``` GetR2__Fv = .text:0x803294EC; // type:function size:0x8 scope:local align:4 __fini_cpp_exceptions = .text:0x803294F4; // type:function size:0x34 scope:global align:4 __init_cpp_exceptions = .text:0x80329528; // type:function size:0x40 scope:global align:4 ``` + The split end is the address of `__init_cpp_exceptions` + size. `.ctors`: @@ -79,12 +81,14 @@ If `__fini_cpp_exceptions_reference` is present, it's size 8, otherwise size 4 `.sdata`: Find the following symbol in `symbols.txt`: + ``` fragmentID = .sdata:0x803F67F0; // type:object size:0x4 scope:local align:4 data:4byte ``` + The split end includes any inter-TU padding, so it's usually size 8. -### GC 2.7+ and Wii linkers: +### GC 2.7+ and Wii linkers ```yaml # splits.txt @@ -98,10 +102,12 @@ Runtime.PPCEABI.H/__init_cpp_exceptions.cpp: `.text`: Find the following symbols in `symbols.txt`: + ``` __fini_cpp_exceptions = .text:0x80345C34; // type:function size:0x34 scope:global __init_cpp_exceptions = .text:0x80345C68; // type:function size:0x3C scope:global ``` + The split end is the address of `__init_cpp_exceptions` + size. `.ctors$10`: @@ -118,7 +124,9 @@ Always size 4. `.sdata`: Find the following symbol in `symbols.txt`: + ``` fragmentID = .sdata:0x80418CA8; // type:object size:0x4 scope:local data:4byte ``` -The split end includes any inter-TU padding, so it's usually size 8. \ No newline at end of file + +The split end includes any inter-TU padding, so it's usually size 8. diff --git a/docs/splits.md b/docs/splits.md index 650d887..484e892 100644 --- a/docs/splits.md +++ b/docs/splits.md @@ -4,7 +4,7 @@ This file contains file splits for a module. Example: -``` +```yaml path/to/file.cpp: .text start:0x80047E5C end:0x8004875C .ctors start:0x803A54C4 end:0x803A54C8 @@ -15,7 +15,7 @@ path/to/file.cpp: ## Format -``` +```yaml path/to/file.cpp: [file attributes] section [section attributes] ... diff --git a/docs/symbols.md b/docs/symbols.md index 5c0d12b..5ce4b25 100644 --- a/docs/symbols.md +++ b/docs/symbols.md @@ -3,6 +3,7 @@ This file contains all symbols for a module, one per line. Example line: + ``` __dt__13mDoExt_bckAnmFv = .text:0x800DD2EC; // type:function size:0x5C scope:global align:4 ``` @@ -34,4 +35,4 @@ All attributes are optional, and are separated by spaces. - `force_active` Marked as ["exported"](comment_section.md) in the generated object, and added to `FORCEACTIVE` in the generated `ldscript.lcf`. Prevents the symbol from being deadstripped. - `noreloc` Prevents the _contents_ of the symbol from being interpreted as addresses. Used for objects containing data that look like pointers, but aren't. - `noexport` When `export_all` is enabled, this excludes the symbol from being exported. Used for symbols that are affected by linker `-code_merging`. -- `stripped` Indicates a symbol that was stripped by the linker. Used for symbols that affect the [common BSS inflation bug](common_bss.md), despite not existing in the final binary. \ No newline at end of file +- `stripped` Indicates a symbol that was stripped by the linker. Used for symbols that affect the [common BSS inflation bug](common_bss.md), despite not existing in the final binary.