85 Commits

Author SHA1 Message Date
6c3887c7e6 Improve split cyclic dependency errors
When a link order is unresolvable, decomp-toolkit
now prints out all of the TUs in a cycle, easing
debugging.

Example:
```
Cyclic dependency encountered while resolving link order: Dolphin/os/__start.c -> Dolphin/os/__ppc_eabi_init.cpp -> Dolphin/pad/PadClamp.c -> Dolphin/pad/pad.c
```

Thanks @simonlindholm for the toposort impl.
2025-01-27 18:06:25 -07:00
04b60d319c vfs ls: Fix column sizing with Unicode chars 2025-01-27 17:54:06 -07:00
1cc38ad621 Add WAD support to VFS & wad commands 2024-11-07 08:43:20 -07:00
146c4d2f8c Add header_type and custom_type to extract config
Extract configuration is now emitted in the output config, so
tooling can load and perform their own tasks on extracted assets
without having to parse YAML.

`header_type`:
- `symbol` (default): Emit a full symbol declaration.
- `raw`: Emit raw array data (for wrapping in your own declaration)
- `none`: Don't emit a header at all. (For custom processing)

`custom_type`/`custom_data`: Passed through to the output config
as-is for custom tasks.
2024-10-28 17:44:07 -06:00
3ada073da1 Update dependencies 2024-10-18 00:19:06 -06:00
bee4570a4c Always check for extracted files in object resolution
Fixes an issue where extracted files would not be found after
removing the disc image from the orig dir.
2024-10-13 13:36:01 -06:00
4fe2608e07 Make selfile relative to object_base 2024-10-12 23:31:41 -06:00
2e524e6806 Use typed-path in place of std Path/PathBuf
This allows handling path conversions in a more structured way,
as well as avoiding needless UTF-8 checks. All argument inputs
use `Utf8NativePathBuf`, while all config entries use
`Utf8UnixPathBuf`, ensuring that we deserialize/serialize using
forward slashes. We can omit `.display()` and lossy UTF-8
conversions since all paths are known valid UTF-8.
2024-10-04 23:38:15 -06:00
4611a4b501 Vendor nintendo-lz crate to fix issues & avoid old deps 2024-10-04 21:02:04 -06:00
1f4b452bd5 Fix disc VFS layout & update nod 2024-10-04 20:33:46 -06:00
ef7e0db095 VFS fixes and improvements, update README.md
`vfs ls`: Now displays size, detected file format, and decompressed
size (if applicable). `-r`/`--recursive` lists files recursively.
`-s`/`--short` prints only filenames.

`vfs cp`: Copies files recursively when the source is a directory.
`--no-decompress` disables automatic decompression for Yay0/Yaz0.
`-q` disables printing copied files.

`rarc` and `u8` commands are now thin wrappers over `vfs ls` and
`vfs cp`. For example, `rarc list file.arc` is now equivalent to
`vfs ls file.arc:`. `rarc extract file.arc -o out` is equivalent
to `vfs cp file.arc: out`.
2024-10-04 18:15:24 -06:00
281b0f7104 Decode extab entries as comment in assembly output 2024-10-03 22:33:21 -06:00
71701b5667 Update all dependencies 2024-10-03 22:24:54 -06:00
f91c2a1474 Load objects from disc image & vfs module
Revamps support for container paths and centralizes logic into a VFS (virtual file system) module.
The new VFS architecture supports disc images and any layer of nesting.

For example, the following command works:
`dtk dol info 'Interactive Multi-Game Demo Disc - July 2002 (USA).rvz:files/zz_StarFox051702_e3.tgc:files/default.dol'`
This opens a TGC file inside an RVZ disc image, then reads `default.dol` in the FST.

Another example:
`dtk rel info 'Legend of Zelda, The - The Wind Waker (USA).rvz:files/RELS.arc:mmem/f_pc_profile_lst.rel'`
This opens a RARC archive inside an RVZ disc image, loads the Yaz0-compressed REL and
decompresses it on the fly.

This all operates in memory with minimal overhead, with no need to extract temporary files.

Supported container formats:
- Disc images (ISO/GCM, WIA/RVZ, WBFS, CISO, NFS, GCZ, TGC)
- RARC/SZS and U8 (.arc)

Supported compression formats:
- Yaz0 (SZS)
- Yay0 (SZP)
- NLZSS (.lz)

Additionally, projects can utilize a new configuration key `object_base`:
```
object: orig/GZLE01/sys/main.dol
modules:
- object: orig/GZLE01/files/RELS.arc:rels/mmem/f_pc_profile_lst.rel
```
becomes
```
object_base: orig/GZLE01
object: sys/main.dol
modules:
- object: files/RELS.arc:mmem/f_pc_profile_lst.rel
```
When loading the objects, decomp-toolkit will automatically check the `object_base`
directory for any disc images. (They can be named anything, but must be in the folder
root) If one is found, all objects will be fetched from the disc image itself, rather
than having to extract the files manually.

While still a work in progress, two new `vfs` commands were added: `vfs ls` and `vfs cp`.
These commands are very barebones currently, but allow listing directory contents and
extracting files from decomp-toolkit's vfs representation:
```
❯ dtk vfs ls disc.rvz:
files
sys

❯ dtk vfs ls disc.rvz:sys
boot.bin
bi2.bin
apploader.bin
fst.bin
main.dol

❯ dtk vfs cp disc.rvz:sys/main.dol .
```
2024-10-03 21:50:35 -06:00
Amber Brault
26f52f65b7
Automatically check for invalid extab relocations (#75)
* Begin work on extab reloc analysis code

* Refactoring

* Make format happy

* Address feedback + improvements
2024-10-03 01:13:23 -06:00
c106123877 Use mimalloc when targeting musl
Also removes the armv7 linux build.
If you were using this, let me know!

Updates all dependencies
2024-09-29 12:20:28 -06:00
8b793b5616 Update CI workflow & all dependencies 2024-09-09 20:36:18 -06:00
9dfdbb9301 Fix v1-2 REL alignment regression
Alignment after section data and
before relocations / import table
is exclusive to REL v3.
2024-09-05 00:26:14 -06:00
c403931f0f Update nod for TGC disc support 2024-09-04 20:09:56 -06:00
b6a29fa910 Add split order attribute for manual reordering
Example in splits.txt:
```
file1.cpp: order:0
  ...

file2.cpp: order:1
  ...

file3.cpp: order:2
  ...
```

This ensures that file2.cpp is always
anchored in between 1 and 3 when resolving
the final link order.
2024-08-11 20:38:11 -06:00
c484952912 Support generating RELs with non-sequential module IDs
Uses the module ID from the configuration, if present,
falling back to the standard behavior of sequential
module IDs.
2024-06-24 17:43:33 -06:00
4d039140f2 Version 0.9.2 2024-06-10 17:41:05 -06:00
9c12efa64e Update orthrus-ncompress 2024-06-10 17:40:46 -06:00
8813a2c9e4 Version 0.9.1 2024-06-10 00:43:44 -06:00
46cf0be183 Add U8 (newer .arc) support
Supports the U8 .arc format, just
like the older RARC format.

`u8 list`, `u8 extract` and support
for U8 archive paths in config.yml
2024-06-03 20:34:35 -06:00
255123796e Instruction disassembly in dol diff
When a function diff is detected in
`dol diff`, objdiff-core is used to
print a detailed view highlighting
any differences.

Resolves #28
2024-06-03 20:31:06 -06:00
d9770c48b2 Version 0.8.3 2024-05-19 23:09:16 -06:00
bf0d1a823f Version 0.8.2: Fix asm output 2024-05-17 07:12:05 -06:00
0bcc539d36 Version 0.8.1: Fix .note.split generation 2024-05-16 18:31:27 -06:00
cb1a765041 Version 0.8.0 2024-05-15 23:16:26 -06:00
4dacf2f39a Add disc commands: info, extract, convert, verify
Utilizing https://github.com/encounter/nod-rs
2024-05-01 00:12:20 -06:00
963425793d Remove unused deps 2024-04-30 20:48:53 -06:00
c45f37eb10 Update ppc750cl (10x faster!) & upgrade deps 2024-04-30 20:40:14 -06:00
989293a477 Add Yay0/Yaz0 compression & decompression
Uses orthrus-ncompress

Fixes #6
2024-04-30 18:03:00 -06:00
3497d38ea3 Version 0.7.6 2024-04-03 00:13:42 -06:00
7fd039722d Format & use syntect regex-fancy
The onig crate no longer builds
in CI with cargo-zigbuild.
2024-03-13 18:30:15 -06:00
d727110262 Update all dependencies & support new demangle flag
- Support `--mw-extensions` flag for demangle
- Fix relocation handling for object crate upgrade
2024-03-13 18:23:09 -06:00
b829e15438 Rework .splitmeta, now .note.split
Uses actual ELF .note format, which is
more standard and handled better by mwld.
2024-03-04 18:12:20 -07:00
4f8a9e6fab Write .splitmeta section in split objects
This enables showing the original address of
symbols in objdiff, as well as `elf disasm`
on split objects retaining the original
addresses.
2024-02-28 22:27:38 -07:00
a41745fb97 Version 0.7.4 2024-02-28 19:39:13 -07:00
9a6cb70ff8 Add ar extract command 2024-02-09 15:00:09 -07:00
7b7c8ce251 Add noexport attr to symbols.txt
Rename `auto_force_active` to `export_all`.
This is a better solution to code_merging, as
individual functions can be marked `noexport`.
2024-01-14 13:58:18 -07:00
968f50ebed Add auto_force_active option (disable to support -code_merging)
Fixes #13
2024-01-13 21:54:10 -07:00
58b6d7c7d5 Avoid duplicate unit names when auto-splitting
Fixes #20
2024-01-06 21:57:42 -07:00
fdf421a5d8 Update dependencies (advisory fix) 2023-12-20 20:43:39 -07:00
d470f7aec6 Version 0.6.5 2023-12-20 20:30:48 -07:00
227153193e Sanitize auto-function-split names
Fixes #16
2023-12-20 20:28:30 -07:00
0cfc5df20b Overhauled common BSS support & more
- With a map, attempts to detect and handle common BSS automatically
- With a map, attempts to detect and correct inflated common BSS bug (< GC 2.7 linker)
- Support for "stripped" symbols, sometimes required to match inflated common BSS sizes
- Warns on duplicated TUs in a map (other than common BSS)
- Automatically adds `comment:0` to `.s` TUs from a map (avoids linker crash)
2023-11-29 18:31:35 -05:00
5128ff67b2 bin2c: Honor symbol alignment 2023-11-26 01:12:34 -05:00
dd60128ba0 Extract embedded assets to binary and C header
Adds an "extract" list to project configuration:
```
extract:
- symbol: SomeData
  binary: Lib/SomeData.bin
  header: Lib/SomeData.inc
```
This example extracts the data of symbol
`SomeData` to `out_dir/bin/Lib/SomeData.bin`,
and a C array representation to
`out_dir/include/Lib/SomeData.inc`.

Resolves #11
2023-11-26 00:35:05 -05:00