Minor documentation updates

This commit is contained in:
Luke Street 2024-03-04 23:30:38 -07:00
parent 0be47a04f7
commit 2b626fb53e
10 changed files with 74 additions and 31 deletions

View File

@ -42,7 +42,7 @@ Supported versions:
Dependencies Dependencies
============ ============
Windows: Windows
-------- --------
On Windows, it's **highly recommended** to use native tooling. WSL or msys2 are **not** required. On Windows, it's **highly recommended** to use native tooling. WSL or msys2 are **not** required.
@ -53,24 +53,30 @@ When running under WSL, [objdiff](#diffing) is unable to get filesystem notifica
- Download [ninja](https://github.com/ninja-build/ninja/releases) and add it to `%PATH%`. - Download [ninja](https://github.com/ninja-build/ninja/releases) and add it to `%PATH%`.
- Quick install via pip: `pip install ninja` - Quick install via pip: `pip install ninja`
macOS: macOS
------ ------
- Install [ninja](https://github.com/ninja-build/ninja/wiki/Pre-built-Ninja-packages): - Install [ninja](https://github.com/ninja-build/ninja/wiki/Pre-built-Ninja-packages):
```
```sh
brew install ninja brew install ninja
``` ```
- Install [wine-crossover](https://github.com/Gcenx/homebrew-wine): - Install [wine-crossover](https://github.com/Gcenx/homebrew-wine):
```
```sh
brew install --cask --no-quarantine gcenx/wine/wine-crossover brew install --cask --no-quarantine gcenx/wine/wine-crossover
``` ```
After OS upgrades, if macOS complains about `Wine Crossover.app` being unverified, you can unquarantine it using: After OS upgrades, if macOS complains about `Wine Crossover.app` being unverified, you can unquarantine it using:
```sh ```sh
sudo xattr -rd com.apple.quarantine '/Applications/Wine Crossover.app' sudo xattr -rd com.apple.quarantine '/Applications/Wine Crossover.app'
``` ```
Linux: Linux
------ ------
- Install [ninja](https://github.com/ninja-build/ninja/wiki/Pre-built-Ninja-packages). - Install [ninja](https://github.com/ninja-build/ninja/wiki/Pre-built-Ninja-packages).
- For non-x86(_64) platforms: Install wine from your package manager. - For non-x86(_64) platforms: Install wine from your package manager.
- For x86(_64), [wibo](https://github.com/decompals/wibo), a minimal 32-bit Windows binary wrapper, will be automatically downloaded and used. - For x86(_64), [wibo](https://github.com/decompals/wibo), a minimal 32-bit Windows binary wrapper, will be automatically downloaded and used.
@ -79,21 +85,26 @@ Building
======== ========
- Clone the repository: - Clone the repository:
```
```sh
git clone https://github.com/my/repo.git git clone https://github.com/my/repo.git
``` ```
- Using [Dolphin Emulator](https://dolphin-emu.org/), extract your game to `orig/GAMEID`. - Using [Dolphin Emulator](https://dolphin-emu.org/), extract your game to `orig/GAMEID`.
![](assets/dolphin-extract.png) ![](assets/dolphin-extract.png)
- To save space, the only necessary files are the following. Any others can be deleted. - To save space, the only necessary files are the following. Any others can be deleted.
- `sys/main.dol` - `sys/main.dol`
- `files/rels/*.rel` - `files/rels/*.rel`
- Configure: - Configure:
```
```sh
python configure.py python configure.py
``` ```
To use a version other than `GAMEID` (USA), specify it with `--version`. To use a version other than `GAMEID` (USA), specify it with `--version`.
- Build: - Build:
```
```sh
ninja ninja
``` ```
@ -105,9 +116,9 @@ If desired, use the recommended Visual Studio Code settings by renaming the `.vs
Diffing Diffing
======= =======
Once the initial build succeeds, an `objdiff.json` should exist in the project root. Once the initial build succeeds, an `objdiff.json` should exist in the project root.
Download the latest release from [encounter/objdiff](https://github.com/encounter/objdiff). Under project settings, set `Project directory`. The configuration should be loaded automatically. Download the latest release from [encounter/objdiff](https://github.com/encounter/objdiff). Under project settings, set `Project directory`. The configuration should be loaded automatically.
Select an object from the left sidebar to begin diffing. Changes to the project will rebuild automatically: changes to source files, headers, `configure.py`, `splits.txt` or `symbols.txt`. Select an object from the left sidebar to begin diffing. Changes to the project will rebuild automatically: changes to source files, headers, `configure.py`, `splits.txt` or `symbols.txt`.

View File

@ -14,6 +14,7 @@ Documentation
- [`splits.txt`](docs/splits.md) - [`splits.txt`](docs/splits.md)
General: General:
- [Common BSS](docs/common_bss.md) - [Common BSS](docs/common_bss.md)
- [`.comment` section](docs/comment_section.md) - [`.comment` section](docs/comment_section.md)
@ -28,11 +29,13 @@ References
- [sjiswrap](https://github.com/encounter/sjiswrap) (UTF-8 to Shift JIS wrapper) - [sjiswrap](https://github.com/encounter/sjiswrap) (UTF-8 to Shift JIS wrapper)
Projects using this structure: Projects using this structure:
- [zeldaret/tww](https://github.com/zeldaret/tww) - [zeldaret/tww](https://github.com/zeldaret/tww)
- [PrimeDecomp/prime](https://github.com/PrimeDecomp/prime) - [PrimeDecomp/prime](https://github.com/PrimeDecomp/prime)
- [PrimeDecomp/echoes](https://github.com/PrimeDecomp/echoes) - [PrimeDecomp/echoes](https://github.com/PrimeDecomp/echoes)
- [DarkRTA/rb3](https://github.com/DarkRTA/rb3) - [DarkRTA/rb3](https://github.com/DarkRTA/rb3)
- [InputEvelution/sadx-dtk](https://github.com/InputEvelution/sadx-dtk) - [doldecomp/melee](https://github.com/doldecomp/melee)
- [doldecomp/sadx](https://github.com/doldecomp/sadx)
- [InputEvelution/wp](https://github.com/InputEvelution/wp) - [InputEvelution/wp](https://github.com/InputEvelution/wp)
- [lepelog/ss-dtk](https://github.com/lepelog/ss-dtk) - [lepelog/ss-dtk](https://github.com/lepelog/ss-dtk)
- [NWPlayer123/AnimalCrossing-dtk](https://github.com/NWPlayer123/AnimalCrossing-dtk) - [NWPlayer123/AnimalCrossing-dtk](https://github.com/NWPlayer123/AnimalCrossing-dtk)
@ -42,6 +45,7 @@ Projects using this structure:
Features Features
-------- --------
- Few external dependencies: Just `python` for the generator and `ninja` for the build system. See [Dependencies](docs/dependencies.md). - Few external dependencies: Just `python` for the generator and `ninja` for the build system. See [Dependencies](docs/dependencies.md).
- Simple configuration: Everything lives in `config.yml`, `symbols.txt`, and `splits.txt`. - Simple configuration: Everything lives in `config.yml`, `symbols.txt`, and `splits.txt`.
- Multi-version support: Separate configurations for each game version, and a `configure.py --version` flag to switch between them. - Multi-version support: Separate configurations for each game version, and a `configure.py --version` flag to switch between them.
@ -66,6 +70,7 @@ Project structure
- `tools/` - Scripts shared between projects. - `tools/` - Scripts shared between projects.
Temporary, delete when done: Temporary, delete when done:
- `config/GAMEID/config.example.yml` - Example configuration file and documentation. - `config/GAMEID/config.example.yml` - Example configuration file and documentation.
- `docs/` - Documentation for decomp-toolkit configuration. - `docs/` - Documentation for decomp-toolkit configuration.
- `README.md` - This file, replace with your own. For a template, see [`README.example.md`](README.example.md). - `README.md` - This file, replace with your own. For a template, see [`README.example.md`](README.example.md).

View File

@ -26,6 +26,11 @@ common_start: 0x80001234
# (optional) Version used to generate `.comment` sections in the split objects. # (optional) Version used to generate `.comment` sections in the split objects.
# If not specified, no `.comment` sections will be generated. # If not specified, no `.comment` sections will be generated.
# See docs/comment_section.md for more information. # See docs/comment_section.md for more information.
# Known versions:
# 8 - CodeWarrior for GameCube 1.0+
# 10 - CodeWarrior for GameCube 1.3.2+
# 11 - CodeWarrior for GameCube 2.7+
# 14 - CodeWarrior for GameCube 3.0a3+
mw_comment_version: 8 mw_comment_version: 8
# (optional) Path to `selfile.sel` for Wii games with RSO files. # (optional) Path to `selfile.sel` for Wii games with RSO files.

View File

@ -3,7 +3,9 @@ object: orig/GAMEID/sys/main.dol
hash: 0123456789abcdef0123456789abcdef01234567 hash: 0123456789abcdef0123456789abcdef01234567
symbols: config/GAMEID/symbols.txt symbols: config/GAMEID/symbols.txt
splits: config/GAMEID/splits.txt splits: config/GAMEID/splits.txt
mw_comment_version: 8 # Change this to match the linker verison.
# See config.example.yml for a list.
mw_comment_version: 11
modules: modules:
- object: orig/GAMEID/files/module.rel - object: orig/GAMEID/files/module.rel

View File

@ -20,6 +20,7 @@ If missing, `mwld` will **not** adjust the alignment of symbols or remove any un
This behavior is quite useful in some cases. When we split our program into objects, we're working from the final post-aligned, post-stripped result, and don't want the linker to make any changes. Most decompilation projects rely on this behavior unintentionally, since their generated objects don't contain a `.comment` section. (For example, objects built with `powerpc-eabi-as`.) This behavior is quite useful in some cases. When we split our program into objects, we're working from the final post-aligned, post-stripped result, and don't want the linker to make any changes. Most decompilation projects rely on this behavior unintentionally, since their generated objects don't contain a `.comment` section. (For example, objects built with `powerpc-eabi-as`.)
However, we need the `.comment` section for some purposes: However, we need the `.comment` section for some purposes:
- Reproducing the [common BSS inflation bug](common_bss.md#inflation-bug) requires the `.comment` section present, due to the above. The linker inflates the size of the first common BSS symbol in a TU, but won't actually move any data around unless the `.comment` section is present. - Reproducing the [common BSS inflation bug](common_bss.md#inflation-bug) requires the `.comment` section present, due to the above. The linker inflates the size of the first common BSS symbol in a TU, but won't actually move any data around unless the `.comment` section is present.
- In newer versions of the linker, using common BSS at all _without_ a valid `.comment` section will cause an internal linker error. - In newer versions of the linker, using common BSS at all _without_ a valid `.comment` section will cause an internal linker error.
@ -40,10 +41,11 @@ The contents of this section follow a very simple format:
It's not known whether this field actually affects `mwld` in any way, but it's configurable for completeness sake. (See `mw_comment_version` in [`config.example.yml`](/config/GAMEID/config.example.yml).) It's not known whether this field actually affects `mwld` in any way, but it's configurable for completeness sake. (See `mw_comment_version` in [`config.example.yml`](/config/GAMEID/config.example.yml).)
Known values: Known values:
- `08` - CodeWarrior for GameCube 1.0+
- `0A` - CodeWarrior for GameCube 1.3.2+ - `08` (8) - CodeWarrior for GameCube 1.0+
- `0B`, `0C` - CodeWarrior for GameCube 2.7+ (difference unknown) - `0A` (10) - CodeWarrior for GameCube 1.3.2+
- `0E`, `0F` - CodeWarrior for GameCube 3.0a3+ (difference unknown) - `0B` (11), `0C` (12) - CodeWarrior for GameCube 2.7+ (difference unknown)
- `0E` (14), `0F` (15) - CodeWarrior for GameCube 3.0a3+ (difference unknown)
`[0xC size: 0x4]` Compiler version: `XX XX XX XX` `[0xC size: 0x4]` Compiler version: `XX XX XX XX`
@ -71,6 +73,7 @@ Often the `.exe`'s properties (which `--help` reads from) and the internal versi
`[0x15 size: 1]` "Quirk" flags: `XX` `[0x15 size: 1]` "Quirk" flags: `XX`
Bitfield of miscellaneous flags. Known flags: Bitfield of miscellaneous flags. Known flags:
- `01` - "Incompatible return small structs" - `01` - "Incompatible return small structs"
- `02` - "Incompatible SFPE double params" - `02` - "Incompatible SFPE double params"
- `04` - "Unsafe global reg vars" - `04` - "Unsafe global reg vars"
@ -88,6 +91,7 @@ This includes the "null" ELF symbol, so the first entry will be all 0's.
`[0x4 size: 1]` Visibility flags(?): `XX` `[0x4 size: 1]` Visibility flags(?): `XX`
Known values: Known values:
- `00` - Default - `00` - Default
- `0D` - Weak - `0D` - Weak
- `0E` - Unknown, also weak? - `0E` - Unknown, also weak?
@ -95,6 +99,7 @@ Known values:
`[0x5 size: 1]` Active flags(?): `XX` `[0x5 size: 1]` Active flags(?): `XX`
Known values: Known values:
- `00` - Default - `00` - Default
- `08` - Force active / export. Prevents the symbol from being deadstripped. - `08` - Force active / export. Prevents the symbol from being deadstripped.
When applied on a section symbol, the entire section is kept as-is. This is used When applied on a section symbol, the entire section is kept as-is. This is used
@ -103,4 +108,4 @@ Known values:
- `10` - Unknown - `10` - Unknown
- `20` - Unknown - `20` - Unknown
`[0x6 size: 2]` Padding(?): `00 00` `[0x6 size: 2]` Padding(?): `00 00`

View File

@ -15,7 +15,7 @@ With `-common off`, `foo` would be defined as a **global** symbol, and the linke
In `splits.txt`, common BSS can be defined with the `common` attribute: In `splits.txt`, common BSS can be defined with the `common` attribute:
``` ```yaml
foo.cpp: foo.cpp:
.text start:0x80047E5C end:0x8004875C .text start:0x80047E5C end:0x8004875C
.ctors start:0x803A54C4 end:0x803A54C8 .ctors start:0x803A54C4 end:0x803A54C8

View File

@ -1,7 +1,7 @@
Dependencies Dependencies
============ ============
Windows: Windows
-------- --------
On Windows, it's **highly recommended** to use native tooling. WSL or msys2 are **not** required. On Windows, it's **highly recommended** to use native tooling. WSL or msys2 are **not** required.
@ -12,24 +12,30 @@ When running under WSL, [objdiff](#diffing) is unable to get filesystem notifica
- Download [ninja](https://github.com/ninja-build/ninja/releases) and add it to `%PATH%`. - Download [ninja](https://github.com/ninja-build/ninja/releases) and add it to `%PATH%`.
- Quick install via pip: `pip install ninja` - Quick install via pip: `pip install ninja`
macOS: macOS
------ ------
- Install [ninja](https://github.com/ninja-build/ninja/wiki/Pre-built-Ninja-packages): - Install [ninja](https://github.com/ninja-build/ninja/wiki/Pre-built-Ninja-packages):
```
```sh
brew install ninja brew install ninja
``` ```
- Install [wine-crossover](https://github.com/Gcenx/homebrew-wine): - Install [wine-crossover](https://github.com/Gcenx/homebrew-wine):
```
```sh
brew install --cask --no-quarantine gcenx/wine/wine-crossover brew install --cask --no-quarantine gcenx/wine/wine-crossover
``` ```
After OS upgrades, if macOS complains about `Wine Crossover.app` being unverified, you can unquarantine it using: After OS upgrades, if macOS complains about `Wine Crossover.app` being unverified, you can unquarantine it using:
```sh ```sh
sudo xattr -rd com.apple.quarantine '/Applications/Wine Crossover.app' sudo xattr -rd com.apple.quarantine '/Applications/Wine Crossover.app'
``` ```
Linux: Linux
------ ------
- Install [ninja](https://github.com/ninja-build/ninja/wiki/Pre-built-Ninja-packages). - Install [ninja](https://github.com/ninja-build/ninja/wiki/Pre-built-Ninja-packages).
- For non-x86(_64) platforms: Install wine from your package manager. - For non-x86(_64) platforms: Install wine from your package manager.
- For x86(_64), [wibo](https://github.com/decompals/wibo), a minimal 32-bit Windows binary wrapper, will be automatically downloaded and used. - For x86(_64), [wibo](https://github.com/decompals/wibo), a minimal 32-bit Windows binary wrapper, will be automatically downloaded and used.

View File

@ -12,8 +12,8 @@ Rename `config/GAMEID` to the game's ID and modify `config/[GAMEID]/config.yml`
Generate a `config/[GAMEID]/build.sha1` file for verification. This file is a list of SHA-1 hashes for each build artifact. One possible way: Generate a `config/[GAMEID]/build.sha1` file for verification. This file is a list of SHA-1 hashes for each build artifact. One possible way:
```shell ```sh
$ dtk shasum orig/[GAMEID]/sys/main.dol orig/[GAMEID]/files/*.rel > config/[GAMEID]/build.sha1 dtk shasum orig/[GAMEID]/sys/main.dol orig/[GAMEID]/files/*.rel > config/[GAMEID]/build.sha1
``` ```
Then, modify the paths in `config/[GAMEID]/build.sha1` to point to the `build` directory instead of `orig`. The DOL will be built at `build/[GAMEID]/main.dol`, and modules will be built at `build/[GAMEID]/[module_name]/[module_name].rel`. Then, modify the paths in `config/[GAMEID]/build.sha1` to point to the `build` directory instead of `orig`. The DOL will be built at `build/[GAMEID]/main.dol`, and modules will be built at `build/[GAMEID]/[module_name]/[module_name].rel`.
@ -49,7 +49,7 @@ Often indicated by the following error:
# '__init_cpp_exceptions.cpp' both need to be updated to latest version. # '__init_cpp_exceptions.cpp' both need to be updated to latest version.
``` ```
### GC 1.0 - 2.6 linkers: ### GC 1.0 - 2.6 linkers
```yaml ```yaml
# splits.txt # splits.txt
@ -62,11 +62,13 @@ Runtime.PPCEABI.H/__init_cpp_exceptions.cpp:
`.text`: `.text`:
Find the following symbols in `symbols.txt`: Find the following symbols in `symbols.txt`:
``` ```
GetR2__Fv = .text:0x803294EC; // type:function size:0x8 scope:local align:4 GetR2__Fv = .text:0x803294EC; // type:function size:0x8 scope:local align:4
__fini_cpp_exceptions = .text:0x803294F4; // type:function size:0x34 scope:global align:4 __fini_cpp_exceptions = .text:0x803294F4; // type:function size:0x34 scope:global align:4
__init_cpp_exceptions = .text:0x80329528; // type:function size:0x40 scope:global align:4 __init_cpp_exceptions = .text:0x80329528; // type:function size:0x40 scope:global align:4
``` ```
The split end is the address of `__init_cpp_exceptions` + size. The split end is the address of `__init_cpp_exceptions` + size.
`.ctors`: `.ctors`:
@ -79,12 +81,14 @@ If `__fini_cpp_exceptions_reference` is present, it's size 8, otherwise size 4
`.sdata`: `.sdata`:
Find the following symbol in `symbols.txt`: Find the following symbol in `symbols.txt`:
``` ```
fragmentID = .sdata:0x803F67F0; // type:object size:0x4 scope:local align:4 data:4byte fragmentID = .sdata:0x803F67F0; // type:object size:0x4 scope:local align:4 data:4byte
``` ```
The split end includes any inter-TU padding, so it's usually size 8. The split end includes any inter-TU padding, so it's usually size 8.
### GC 2.7+ and Wii linkers: ### GC 2.7+ and Wii linkers
```yaml ```yaml
# splits.txt # splits.txt
@ -98,10 +102,12 @@ Runtime.PPCEABI.H/__init_cpp_exceptions.cpp:
`.text`: `.text`:
Find the following symbols in `symbols.txt`: Find the following symbols in `symbols.txt`:
``` ```
__fini_cpp_exceptions = .text:0x80345C34; // type:function size:0x34 scope:global __fini_cpp_exceptions = .text:0x80345C34; // type:function size:0x34 scope:global
__init_cpp_exceptions = .text:0x80345C68; // type:function size:0x3C scope:global __init_cpp_exceptions = .text:0x80345C68; // type:function size:0x3C scope:global
``` ```
The split end is the address of `__init_cpp_exceptions` + size. The split end is the address of `__init_cpp_exceptions` + size.
`.ctors$10`: `.ctors$10`:
@ -118,7 +124,9 @@ Always size 4.
`.sdata`: `.sdata`:
Find the following symbol in `symbols.txt`: Find the following symbol in `symbols.txt`:
``` ```
fragmentID = .sdata:0x80418CA8; // type:object size:0x4 scope:local data:4byte fragmentID = .sdata:0x80418CA8; // type:object size:0x4 scope:local data:4byte
``` ```
The split end includes any inter-TU padding, so it's usually size 8.
The split end includes any inter-TU padding, so it's usually size 8.

View File

@ -4,7 +4,7 @@ This file contains file splits for a module.
Example: Example:
``` ```yaml
path/to/file.cpp: path/to/file.cpp:
.text start:0x80047E5C end:0x8004875C .text start:0x80047E5C end:0x8004875C
.ctors start:0x803A54C4 end:0x803A54C8 .ctors start:0x803A54C4 end:0x803A54C8
@ -15,7 +15,7 @@ path/to/file.cpp:
## Format ## Format
``` ```yaml
path/to/file.cpp: [file attributes] path/to/file.cpp: [file attributes]
section [section attributes] section [section attributes]
... ...

View File

@ -3,6 +3,7 @@
This file contains all symbols for a module, one per line. This file contains all symbols for a module, one per line.
Example line: Example line:
``` ```
__dt__13mDoExt_bckAnmFv = .text:0x800DD2EC; // type:function size:0x5C scope:global align:4 __dt__13mDoExt_bckAnmFv = .text:0x800DD2EC; // type:function size:0x5C scope:global align:4
``` ```
@ -34,4 +35,4 @@ All attributes are optional, and are separated by spaces.
- `force_active` Marked as ["exported"](comment_section.md) in the generated object, and added to `FORCEACTIVE` in the generated `ldscript.lcf`. Prevents the symbol from being deadstripped. - `force_active` Marked as ["exported"](comment_section.md) in the generated object, and added to `FORCEACTIVE` in the generated `ldscript.lcf`. Prevents the symbol from being deadstripped.
- `noreloc` Prevents the _contents_ of the symbol from being interpreted as addresses. Used for objects containing data that look like pointers, but aren't. - `noreloc` Prevents the _contents_ of the symbol from being interpreted as addresses. Used for objects containing data that look like pointers, but aren't.
- `noexport` When `export_all` is enabled, this excludes the symbol from being exported. Used for symbols that are affected by linker `-code_merging`. - `noexport` When `export_all` is enabled, this excludes the symbol from being exported. Used for symbols that are affected by linker `-code_merging`.
- `stripped` Indicates a symbol that was stripped by the linker. Used for symbols that affect the [common BSS inflation bug](common_bss.md), despite not existing in the final binary. - `stripped` Indicates a symbol that was stripped by the linker. Used for symbols that affect the [common BSS inflation bug](common_bss.md), despite not existing in the final binary.