32 Commits

Author SHA1 Message Date
88d0e6b789 cargo clippy --fix 2025-06-01 16:42:00 -06:00
dbalatoni13
614d4f2efc
Add new options to rel make (#94) 2025-03-31 17:15:16 -06:00
2e524e6806 Use typed-path in place of std Path/PathBuf
This allows handling path conversions in a more structured way,
as well as avoiding needless UTF-8 checks. All argument inputs
use `Utf8NativePathBuf`, while all config entries use
`Utf8UnixPathBuf`, ensuring that we deserialize/serialize using
forward slashes. We can omit `.display()` and lossy UTF-8
conversions since all paths are known valid UTF-8.
2024-10-04 23:38:15 -06:00
b184fee73f Migrate SectionIndex/SymbolIndex to u32
This halves the size of structs like SectionAddress.
2024-10-04 20:40:50 -06:00
ef7e0db095 VFS fixes and improvements, update README.md
`vfs ls`: Now displays size, detected file format, and decompressed
size (if applicable). `-r`/`--recursive` lists files recursively.
`-s`/`--short` prints only filenames.

`vfs cp`: Copies files recursively when the source is a directory.
`--no-decompress` disables automatic decompression for Yay0/Yaz0.
`-q` disables printing copied files.

`rarc` and `u8` commands are now thin wrappers over `vfs ls` and
`vfs cp`. For example, `rarc list file.arc` is now equivalent to
`vfs ls file.arc:`. `rarc extract file.arc -o out` is equivalent
to `vfs cp file.arc: out`.
2024-10-04 18:15:24 -06:00
f91c2a1474 Load objects from disc image & vfs module
Revamps support for container paths and centralizes logic into a VFS (virtual file system) module.
The new VFS architecture supports disc images and any layer of nesting.

For example, the following command works:
`dtk dol info 'Interactive Multi-Game Demo Disc - July 2002 (USA).rvz:files/zz_StarFox051702_e3.tgc:files/default.dol'`
This opens a TGC file inside an RVZ disc image, then reads `default.dol` in the FST.

Another example:
`dtk rel info 'Legend of Zelda, The - The Wind Waker (USA).rvz:files/RELS.arc:mmem/f_pc_profile_lst.rel'`
This opens a RARC archive inside an RVZ disc image, loads the Yaz0-compressed REL and
decompresses it on the fly.

This all operates in memory with minimal overhead, with no need to extract temporary files.

Supported container formats:
- Disc images (ISO/GCM, WIA/RVZ, WBFS, CISO, NFS, GCZ, TGC)
- RARC/SZS and U8 (.arc)

Supported compression formats:
- Yaz0 (SZS)
- Yay0 (SZP)
- NLZSS (.lz)

Additionally, projects can utilize a new configuration key `object_base`:
```
object: orig/GZLE01/sys/main.dol
modules:
- object: orig/GZLE01/files/RELS.arc:rels/mmem/f_pc_profile_lst.rel
```
becomes
```
object_base: orig/GZLE01
object: sys/main.dol
modules:
- object: files/RELS.arc:mmem/f_pc_profile_lst.rel
```
When loading the objects, decomp-toolkit will automatically check the `object_base`
directory for any disc images. (They can be named anything, but must be in the folder
root) If one is found, all objects will be fetched from the disc image itself, rather
than having to extract the files manually.

While still a work in progress, two new `vfs` commands were added: `vfs ls` and `vfs cp`.
These commands are very barebones currently, but allow listing directory contents and
extracting files from decomp-toolkit's vfs representation:
```
❯ dtk vfs ls disc.rvz:
files
sys

❯ dtk vfs ls disc.rvz:sys
boot.bin
bi2.bin
apploader.bin
fst.bin
main.dol

❯ dtk vfs cp disc.rvz:sys/main.dol .
```
2024-10-03 21:50:35 -06:00
c484952912 Support generating RELs with non-sequential module IDs
Uses the module ID from the configuration, if present,
falling back to the standard behavior of sequential
module IDs.
2024-06-24 17:43:33 -06:00
761a940f9e Match original "exec" for REL sections
mwld writes empty code sections as
NULL type in the PLF, but sometimes
the original REL has the exec flag
set for these sections. Match the
original value.
2024-06-10 17:39:40 -06:00
4ea4ec86d0 Partially revert "Rework section alignment handling"
It turns out we can't actually
trust mwld to put our alignment
values in the PLF. Sad.
2024-06-10 00:37:17 -06:00
dc7e307c44 Rework section alignment handling
- Honor splits.txt alignment values when writing ldscript.lcf
- Add alignment values to ldscript_partial.lcf, remove hacky code from rel make
- Guess alignment values in DOL loader

Fixes #27
2024-04-30 18:04:17 -06:00
d0f39f1d82 Support global labels in REL creation (_savegpr, etc) 2024-04-03 00:13:15 -06:00
d727110262 Update all dependencies & support new demangle flag
- Support `--mw-extensions` flag for demangle
- Fix relocation handling for object crate upgrade
2024-03-13 18:23:09 -06:00
4f8a9e6fab Write .splitmeta section in split objects
This enables showing the original address of
symbols in objdiff, as well as `elf disasm`
on split objects retaining the original
addresses.
2024-02-28 22:27:38 -07:00
968f50ebed Add auto_force_active option (disable to support -code_merging)
Fixes #13
2024-01-13 21:54:10 -07:00
28af4872ab rel make: Add --quiet/-q, --names/-n args
`-n` is a somewhat ugly hack to only select certain modules from `config.yml`, for games with overlapping module IDs
2023-11-18 23:34:44 -05:00
4935708b61 Fix REL v2 creation (and v1, hopefully)
- Adjusts `write_rel` to use the proper ordering for relocations and imports based on the REL version.
- Adds `-r`/`--relocations` switch to `rel info` that prints (very) verbose relocation information.
2023-11-09 01:16:37 -05:00
dfb569b883 Add alf support (NVIDIA Shield TV binaries)
- Supports .alf files in all places .dol files are accepted.
- Adds `hash` and `dhash` to symbols config.
2023-10-07 02:03:09 -04:00
c354c6da4b REL section alignment fixes
- Honor section alignment from splits.txt when writing RELs
- Better heuristic for determining REL section alignment on initial analysis
2023-09-26 13:06:57 -04:00
42217a339e Update cargo deny.toml 2023-09-13 02:10:47 -04:00
d4ef1ce16a Analyzer fixes galore
- Transparent NLZSS decompression (add `:nlzss` to path)
- Overhaul portions of the analyzer to support more games
- Reject some invalid data relocations automatically
- Jump table analysis fixes
2023-09-13 02:08:51 -04:00
e3857d3212 Improvements to REL & map support
- Fix symbols.txt align attribute
- Fully support nested RARC files & transparent Yaz0 decompression
- Guess symbol visibility for maps without link map
- Add module name config
- Add manual force_active config
- Quiet option for shasum
- `symbols_known` and `fill_gaps` config
- Allow disabling .comment generation per-unit (`comment:0`)
- Various minor fixes
- Add `rarc` and `yaz0` commands
2023-09-05 17:22:22 -04:00
f9f7fb2e1e Working rel make & more
- Added `elf info`
- Improved `rel info`
- Colored output for `shasum`
- Fix section `rename` in RELs
- Added padding symbols to avoid linker issues
- Automatically set symbols to "active" in .comment output
2023-08-23 23:13:12 -04:00
3f63f1ef47 A lot more section-address-aware refactoring 2023-08-23 23:13:12 -04:00
5843ee021e Begin REL analysis & rework lots of code to be section-address aware 2023-08-17 22:09:45 -04:00
d9e1ae2777 MW 2.7 LD compatibility fixes & more
- More robust .comment section handling
- Auto-create .comment section for objects with common symbols (MW 2.7+ hack)
- Support loading REL modules in `dol split` (currently only for references)
- Add `dol diff` for quick diffing between linked ELF and expected symbols
- Add `dol apply` for applying linked ELF symbols to symbol config file
2023-08-08 23:34:00 -04:00
636213fdfd Remove unused imports 2023-08-03 19:19:05 -04:00
5c04800964 Run apply_signatures_post appropriately 2023-08-03 19:05:40 -04:00
f06a6ffbdd Ignore ABS symbols in ObjSymbols::for_range
- Add ObjSymbols::iter_ordered for all symbols (including ABS)
- Extract duplicated logic into ObjSymbols::for_relocation
2023-08-03 19:04:17 -04:00
bd0422e92a Migrate argh to argp, topological-sort to petgraph 2023-08-03 18:57:08 -04:00
0fa0aafaea 6 months of occasional work I guess 2023-07-21 17:59:07 -04:00
830f7b172f Reorganize files; start RSO support; config & split updates 2023-01-27 23:15:52 -05:00
d864b0e395 Very experimental analyzer, DOL & REL support, function signatures 2023-01-18 12:52:38 -08:00