Commit Graph

80 Commits

Author SHA1 Message Date
Luke Street bee4570a4c Always check for extracted files in object resolution
Fixes an issue where extracted files would not be found after
removing the disc image from the orig dir.
2024-10-13 13:36:01 -06:00
Luke Street 4fe2608e07 Make `selfile` relative to `object_base` 2024-10-12 23:31:41 -06:00
Luke Street 2e524e6806 Use typed-path in place of std Path/PathBuf
This allows handling path conversions in a more structured way,
as well as avoiding needless UTF-8 checks. All argument inputs
use `Utf8NativePathBuf`, while all config entries use
`Utf8UnixPathBuf`, ensuring that we deserialize/serialize using
forward slashes. We can omit `.display()` and lossy UTF-8
conversions since all paths are known valid UTF-8.
2024-10-04 23:38:15 -06:00
Luke Street 4611a4b501 Vendor nintendo-lz crate to fix issues & avoid old deps 2024-10-04 21:02:04 -06:00
Luke Street 1f4b452bd5 Fix disc VFS layout & update nod 2024-10-04 20:33:46 -06:00
Luke Street ef7e0db095 VFS fixes and improvements, update README.md
`vfs ls`: Now displays size, detected file format, and decompressed
size (if applicable). `-r`/`--recursive` lists files recursively.
`-s`/`--short` prints only filenames.

`vfs cp`: Copies files recursively when the source is a directory.
`--no-decompress` disables automatic decompression for Yay0/Yaz0.
`-q` disables printing copied files.

`rarc` and `u8` commands are now thin wrappers over `vfs ls` and
`vfs cp`. For example, `rarc list file.arc` is now equivalent to
`vfs ls file.arc:`. `rarc extract file.arc -o out` is equivalent
to `vfs cp file.arc: out`.
2024-10-04 18:15:24 -06:00
Luke Street 281b0f7104 Decode extab entries as comment in assembly output 2024-10-03 22:33:21 -06:00
Luke Street 71701b5667 Update all dependencies 2024-10-03 22:24:54 -06:00
Luke Street f91c2a1474 Load objects from disc image & `vfs` module
Revamps support for container paths and centralizes logic into a VFS (virtual file system) module.
The new VFS architecture supports disc images and any layer of nesting.

For example, the following command works:
`dtk dol info 'Interactive Multi-Game Demo Disc - July 2002 (USA).rvz:files/zz_StarFox051702_e3.tgc:files/default.dol'`
This opens a TGC file inside an RVZ disc image, then reads `default.dol` in the FST.

Another example:
`dtk rel info 'Legend of Zelda, The - The Wind Waker (USA).rvz:files/RELS.arc:mmem/f_pc_profile_lst.rel'`
This opens a RARC archive inside an RVZ disc image, loads the Yaz0-compressed REL and
decompresses it on the fly.

This all operates in memory with minimal overhead, with no need to extract temporary files.

Supported container formats:
- Disc images (ISO/GCM, WIA/RVZ, WBFS, CISO, NFS, GCZ, TGC)
- RARC/SZS and U8 (.arc)

Supported compression formats:
- Yaz0 (SZS)
- Yay0 (SZP)
- NLZSS (.lz)

Additionally, projects can utilize a new configuration key `object_base`:
```
object: orig/GZLE01/sys/main.dol
modules:
- object: orig/GZLE01/files/RELS.arc:rels/mmem/f_pc_profile_lst.rel
```
becomes
```
object_base: orig/GZLE01
object: sys/main.dol
modules:
- object: files/RELS.arc:mmem/f_pc_profile_lst.rel
```
When loading the objects, decomp-toolkit will automatically check the `object_base`
directory for any disc images. (They can be named anything, but must be in the folder
root) If one is found, all objects will be fetched from the disc image itself, rather
than having to extract the files manually.

While still a work in progress, two new `vfs` commands were added: `vfs ls` and `vfs cp`.
These commands are very barebones currently, but allow listing directory contents and
extracting files from decomp-toolkit's vfs representation:
```
❯ dtk vfs ls disc.rvz:
files
sys

❯ dtk vfs ls disc.rvz:sys
boot.bin
bi2.bin
apploader.bin
fst.bin
main.dol

❯ dtk vfs cp disc.rvz:sys/main.dol .
```
2024-10-03 21:50:35 -06:00
Amber Brault 26f52f65b7
Automatically check for invalid extab relocations (#75)
* Begin work on extab reloc analysis code

* Refactoring

* Make format happy

* Address feedback + improvements
2024-10-03 01:13:23 -06:00
Luke Street c106123877 Use mimalloc when targeting musl
Also removes the armv7 linux build.
If you were using this, let me know!

Updates all dependencies
2024-09-29 12:20:28 -06:00
Luke Street 8b793b5616 Update CI workflow & all dependencies 2024-09-09 20:36:18 -06:00
Luke Street 9dfdbb9301 Fix v1-2 REL alignment regression
Alignment after section data and
before relocations / import table
is exclusive to REL v3.
2024-09-05 00:26:14 -06:00
Luke Street c403931f0f Update nod for TGC disc support 2024-09-04 20:09:56 -06:00
Luke Street b6a29fa910 Add split `order` attribute for manual reordering
Example in splits.txt:
```
file1.cpp: order:0
  ...

file2.cpp: order:1
  ...

file3.cpp: order:2
  ...
```

This ensures that file2.cpp is always
anchored in between 1 and 3 when resolving
the final link order.
2024-08-11 20:38:11 -06:00
Luke Street c484952912 Support generating RELs with non-sequential module IDs
Uses the module ID from the configuration, if present,
falling back to the standard behavior of sequential
module IDs.
2024-06-24 17:43:33 -06:00
Luke Street 4d039140f2 Version 0.9.2 2024-06-10 17:41:05 -06:00
Luke Street 9c12efa64e Update orthrus-ncompress 2024-06-10 17:40:46 -06:00
Luke Street 8813a2c9e4 Version 0.9.1 2024-06-10 00:43:44 -06:00
Luke Street 46cf0be183 Add U8 (newer .arc) support
Supports the U8 .arc format, just
like the older RARC format.

`u8 list`, `u8 extract` and support
for U8 archive paths in config.yml
2024-06-03 20:34:35 -06:00
Luke Street 255123796e Instruction disassembly in `dol diff`
When a function diff is detected in
`dol diff`, objdiff-core is used to
print a detailed view highlighting
any differences.

Resolves #28
2024-06-03 20:31:06 -06:00
Luke Street d9770c48b2 Version 0.8.3 2024-05-19 23:09:16 -06:00
Luke Street bf0d1a823f Version 0.8.2: Fix asm output 2024-05-17 07:12:05 -06:00
Luke Street 0bcc539d36 Version 0.8.1: Fix .note.split generation 2024-05-16 18:31:27 -06:00
Luke Street cb1a765041 Version 0.8.0 2024-05-15 23:16:26 -06:00
Luke Street 4dacf2f39a Add disc commands: info, extract, convert, verify
Utilizing https://github.com/encounter/nod-rs
2024-05-01 00:12:20 -06:00
Luke Street 963425793d Remove unused deps 2024-04-30 20:48:53 -06:00
Luke Street c45f37eb10 Update ppc750cl (10x faster!) & upgrade deps 2024-04-30 20:40:14 -06:00
Luke Street 989293a477 Add Yay0/Yaz0 compression & decompression
Uses orthrus-ncompress

Fixes #6
2024-04-30 18:03:00 -06:00
Luke Street 3497d38ea3 Version 0.7.6 2024-04-03 00:13:42 -06:00
Luke Street 7fd039722d Format & use syntect regex-fancy
The onig crate no longer builds
in CI with cargo-zigbuild.
2024-03-13 18:30:15 -06:00
Luke Street d727110262 Update all dependencies & support new demangle flag
- Support `--mw-extensions` flag for demangle
- Fix relocation handling for object crate upgrade
2024-03-13 18:23:09 -06:00
Luke Street b829e15438 Rework .splitmeta, now .note.split
Uses actual ELF .note format, which is
more standard and handled better by mwld.
2024-03-04 18:12:20 -07:00
Luke Street 4f8a9e6fab Write .splitmeta section in split objects
This enables showing the original address of
symbols in objdiff, as well as `elf disasm`
on split objects retaining the original
addresses.
2024-02-28 22:27:38 -07:00
Luke Street a41745fb97 Version 0.7.4 2024-02-28 19:39:13 -07:00
Luke Street 9a6cb70ff8 Add `ar extract` command 2024-02-09 15:00:09 -07:00
Luke Street 7b7c8ce251 Add `noexport` attr to symbols.txt
Rename `auto_force_active` to `export_all`.
This is a better solution to code_merging, as
individual functions can be marked `noexport`.
2024-01-14 13:58:18 -07:00
Luke Street 968f50ebed Add `auto_force_active` option (disable to support `-code_merging`)
Fixes #13
2024-01-13 21:54:10 -07:00
Luke Street 58b6d7c7d5 Avoid duplicate unit names when auto-splitting
Fixes #20
2024-01-06 21:57:42 -07:00
Luke Street fdf421a5d8 Update dependencies (advisory fix) 2023-12-20 20:43:39 -07:00
Luke Street d470f7aec6 Version 0.6.5 2023-12-20 20:30:48 -07:00
Luke Street 227153193e Sanitize auto-function-split names
Fixes #16
2023-12-20 20:28:30 -07:00
Luke Street 0cfc5df20b Overhauled common BSS support & more
- With a map, attempts to detect and handle common BSS automatically
- With a map, attempts to detect and correct inflated common BSS bug (< GC 2.7 linker)
- Support for "stripped" symbols, sometimes required to match inflated common BSS sizes
- Warns on duplicated TUs in a map (other than common BSS)
- Automatically adds `comment:0` to `.s` TUs from a map (avoids linker crash)
2023-11-29 18:31:35 -05:00
Luke Street 5128ff67b2 bin2c: Honor symbol alignment 2023-11-26 01:12:34 -05:00
Luke Street dd60128ba0 Extract embedded assets to binary and C header
Adds an "extract" list to project configuration:
```
extract:
- symbol: SomeData
  binary: Lib/SomeData.bin
  header: Lib/SomeData.inc
```
This example extracts the data of symbol
`SomeData` to `out_dir/bin/Lib/SomeData.bin`,
and a C array representation to
`out_dir/include/Lib/SomeData.inc`.

Resolves #11
2023-11-26 00:35:05 -05:00
Luke Street 038354a37e Version 0.6.1 2023-11-22 13:01:42 -05:00
Luke Street 7168d2632f Version 0.6.0 2023-11-18 23:38:43 -05:00
Luke Street 363fa2ac72 Use custom syntect packs to reduce binary size
- Custom syntax set containing only C/C++
- Custom themeset containing only Solarized (Dark)
- Disable default syntect features
2023-11-18 16:12:03 -05:00
Luke Street 8f559c8921 Update dependencies & CI changes
- Use separate release-lto profile for CI
- A couple of clippy fixes
2023-11-18 15:03:24 -05:00
Luke Street 5e13998e93 Major `dwarf dump` rework
- Supports games with C++ DWARF info
- Syntax highlighting when printing to console (disable with `--no-color`)
- Overall improvements to parsing and output
2023-11-18 13:56:58 -05:00