Compare commits

53 Commits

Author SHA1 Message Date
6febcc2c79 Fetch nowide on Windows 2025-04-04 17:29:38 -06:00
9584303083 Replace logvisor with spdlog 2025-04-03 21:06:08 -06:00
b513a7f4e0 Update logvisor 2022-08-03 18:16:55 -04:00
258076c298 Fix OSUTF32To8 and OSUTF16To32 2022-02-28 22:04:40 -08:00
72893dcacb Refactor to remove <Windows.h> from headers 2022-02-22 00:48:22 -05:00
30697375ad update logvisor 2022-01-10 23:28:51 -08:00
2f53b21740 Stop trying to make <ranges> happen, it's not going to happen 2021-10-25 22:34:12 -04:00
45f56d21f1 DirectoryEnumerator: Only include <ranges> on Windows 2021-10-25 19:34:50 -04:00
27da856af3 Update logvisor 2021-10-25 19:10:27 -04:00
2288e71605 Fix OSUTF8to32 2021-07-10 19:40:39 -07:00
0396cf467b Restore C++17 compatibility 2021-06-30 17:00:42 -04:00
0985c63958 Fix compile errors/typos; update logvisor 2021-06-30 16:22:50 -04:00
c1635245b8 Use UTF-8 exclusively internally; replace iconv with OSUTF
Filenames are now unconditionally encoded/decoded as
Shift-JIS, which provides a reasonable fallback to
7-bit ASCII.
2021-06-30 13:47:10 -04:00
a525f60775 Assume CP_USA_ASCII by default in Node as well 2021-06-27 10:56:43 -07:00
8127bddb97 Merge branch 'Minty-Meeo-Shift-JIS-fixes' 2021-06-27 10:46:57 -07:00
Minty-Meeo
3c25647b6e The Encoding Update
While Nintendo's own documents claim GameCube and Wii disc file symbol tables only support 7-bit ASCII, this is far from the truth.  Indeed, even some first-party Nintendo games shipped with Shift-JIS encoded file symbol tables.  My guess?  The locale of whatever Windows machine mastered a GameCube or Wii disc influenced how wide character strings (UCS-2) were converted to narrow character strings.  To account for all possibilites, this update adds extensible multi-byte character set options to NOD-Tool.

A rundown of notable changes:
 - "-c XXXXX" option added to set the encoding of the GameCube / Wii ISO(s) being processed.
 - "SystemStringConv" renamed to "DiscLocToSystemConv"
 - "SystemUTF8Conv" renamed to "SystemToDiscLocConv"
 - Help message updated with new info.
 - Bugfix: AddBuildName had a logic error wherein the length of the SystemString was being used instead of length of the disc locale string.  This would corrupt the File Symbol Table if the disc locale string's length was greater than the SystemString's length.
 - Bugfix: recursiveMergeFST was not keeping track of parent indexes at all, meaning nested folders and their contents would be corrupted.  I simply copied the way recursiveBuildFST did things to fix this.
 - Bugfix (Windows): On Windows, for some reason, Sstat was a typedef for _stat (32-bit) instead of _stat64 (64-bit).  This is confounding, because untrimmed Wii ISOs will always be larger than the unsigned 32-bit integer limit (4,699,979,776 bytes vs 4,294,967,295 bytes), meaning the MergeWii errand has never worked for untrimmed ISOs on Windows.  Was this never tested??
 - Bugfix (Windows): Did you know Windows Command Prompt fully supports Unicode?  Stdio streams are now in _O_U16TEXT mode for Windows only.  Previously, attempting to print any character that could not be narrowed to your locale's encoding would either silently fail (std functions), or throw an exception (fmt functions).  As a minor drawback, narrow character print functions can no longer be used when stdio is in _O_U16TEXT mode, necessitating my PR for Logvisor here: (AxioDL/logvisor#7)
 - ExtractionContext::progressCB now uses SystemStringView because widechar printing works correctly on Windows now.
 - progFunc lambda no longer throws exceptions when printing unicode because widechar printing works correctly on Windows now.
 - Top-level constructors and functions with a Codepage_t parameter have also signatures that default to the US-ASCII codepage.
    - DiscGCN constructor
    - DiscBuilderGCN constructor
    - DiscBuilderGCN::CalculateTotalSizeRequired
    - DiscMergerGCN constructor
    - DiscMergerGCN::CalculateTotalSizeRequired
    - DiscWii constructor
    - DiscBuilderWii constructor
    - DiscBuilderWii::CalculateTotalSizeRequired
    - DiscMergerWii constructor
    - DiscMergerWii::CalculateTotalSizeRequired
    - OpenDiscFromImage
 - Conversion between system encoding and disc locale encoding has checks in place to warn the user if string conversion goes awry.
2021-06-27 03:26:20 -05:00
3eccde013d Update logvisor 2021-06-19 14:44:24 -07:00
Henrique Gemignani Passos Lima
ab8f4c3990 Update logvisor 2021-06-19 13:53:45 +03:00
6a231004b1 Update logvisor 2021-05-07 23:21:10 -07:00
d14b798b5f Don't install within add_subdirectory; update logvisor 2021-04-13 10:34:57 -04:00
1f110f3549 Update logvisor 2021-04-12 15:54:07 -04:00
311d20532e Update logvisor 2021-04-06 17:57:07 -04:00
8fdc893c86 Update logvisor; set submodule tracking branch 2021-04-06 12:48:38 -04:00
fdc8be487d Update logvisor 2021-04-05 16:31:21 -04:00
3a21961a4e Update logvisor 2021-04-05 13:26:39 -04:00
da399b5b67 Update logvisor 2021-04-05 12:06:30 -04:00
11c734be47 Update logvisor 2021-04-04 18:23:33 -04:00
364787604d Update logivisor 2021-04-03 10:28:23 -07:00
02c188497a Fixes for Apple Silicon
- Only include cpuid.h for x86
- Only set -maes flag for x86
2021-01-06 20:32:21 -05:00
d53d677038 Merge commit '5b1b6f6' 2020-12-31 13:59:27 -08:00
5b1b6f6f80 Update logvisor 2020-12-31 13:58:57 -08:00
393a11ffb5 Update CMakeLists for individual build 2020-05-31 17:23:01 -04:00
Jack Andersen
2783337c36 Update fmtlib 2020-04-11 12:45:06 -10:00
Jack Andersen
dffcac50c5 Exclude CMake CXX standard from MSVC 2020-04-10 19:00:19 -10:00
Jack Andersen
f147e12356 Windows build fixes 2019-11-24 15:17:57 -10:00
Jack Andersen
48a2981a93 Shared NFS data class not necessary 2019-11-24 13:47:48 -10:00
Jack Andersen
19604b2a3b Use correct slash finding function 2019-11-23 20:25:29 -10:00
Jack Andersen
c1a1d1abc8 Use logical block for iv 2019-11-23 20:00:48 -10:00
Jack Andersen
6bf4f07129 Consistent variable names for blocks 2019-11-23 17:29:57 -10:00
Jack Andersen
75fc574f81 Support for Wii U VC NFS images 2019-11-23 17:24:33 -10:00
Jack Andersen
11a0351d1c Merge branch 'master' of ssh://git.axiodl.com:6431/AxioDL/nod 2019-09-30 21:30:57 -10:00
Jack Andersen
4ec6c6697b Update logvisor 2019-09-30 21:30:42 -10:00
ba0c2b7843 Merge pull request #14 from lioncash/athena
General: Be explicit about athena's SeekOrigin type
2019-09-08 17:51:25 -07:00
Lioncash
221bc7c7f2 General: Be explicit about athena's SeekOrigin type
Allows this code to function if the enum is turned into an enum class.
2019-09-08 17:29:38 -04:00
acdadaf963 Merge pull request #5 from henriquegemignani/optional
Use std::optional for CalculateTotalSizeBuild
2019-09-07 01:18:43 -07:00
d658909948 Update athena 2019-09-06 23:31:44 -07:00
Henrique Gemignani Passos Lima
091262ace1 Use std::optional for CalculateTotalSizeBuild 2019-09-06 22:17:06 +02:00
97cfcea14e Merge pull request #13 from lioncash/overrun
DiscBase: Prevent potential off-by-one case within getPartitonNodeCount()
2019-09-05 19:35:01 -07:00
Lioncash
f5c3cbdcd7 DiscBase: Amend typo within getPartitonNodeCount() name
Adds an extra 'i' to correct a typo.
2019-09-05 21:57:44 -04:00
Lioncash
7ddff919c1 DiscBase: Prevent potential off-by-one case within getPartitonNodeCount()
We should be comparing with >= instead of >.
2019-09-05 21:56:49 -04:00
Lioncash
4bba7af2c2 DiscBase: std::move std::function instance
std::function isn't a trivial type (it's allowed to heap allocate to
store any necessary captures), so we can move it in the constructor to
avoid any unnecessary allocations
2019-09-05 21:52:43 -04:00
Lioncash
f443b60bde DiscBase: Mark member functions as const where applicable
These don't modify instance state, so they can be marked as const.
2019-09-05 21:51:20 -04:00
Lioncash
998d6a77c3 General: Remove redundant inline keyword
Functions defined within the class declaration are already inline by
default, so we don't need to specify the keyword here.
2019-09-05 21:48:07 -04:00
34 changed files with 4760 additions and 1041 deletions

4
.gitignore vendored
View File

@@ -4,3 +4,7 @@ version.h
.DS_Store .DS_Store
*.autosave *.autosave
docs/* docs/*
.idea/
cmake-build-*
build/
out/

1
.gitmodules vendored
View File

@@ -1,3 +1,4 @@
[submodule "logvisor"] [submodule "logvisor"]
path = logvisor path = logvisor
url = ../logvisor.git url = ../logvisor.git
branch = master

View File

@@ -1,25 +1,84 @@
if(CMAKE_SOURCE_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR) if(CMAKE_SOURCE_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR)
cmake_minimum_required(VERSION 3.10 FATAL_ERROR) # because of c++17 cmake_minimum_required(VERSION 3.10 FATAL_ERROR) # because of c++17
project(nod VERSION 0.1) project(nod VERSION 0.1)
if (MSVC)
# Shaddup MSVC
add_compile_definitions(UNICODE=1 _UNICODE=1 __SSE__=1
_CRT_SECURE_NO_WARNINGS=1 D_SCL_SECURE_NO_WARNINGS=1
_SCL_SECURE_NO_DEPRECATE=1 _CRT_NONSTDC_NO_WARNINGS=1
_ENABLE_EXTENDED_ALIGNED_STORAGE=1 NOMINMAX=1)
add_compile_options(/IGNORE:4221 /wd4018 /wd4800 /wd4005 /wd4311 /wd4068
/wd4267 /wd4244 /wd4200 /wd4305 /wd4067 /wd4146 /wd4309 /wd4805 /utf-8 ${VS_OPTIONS})
add_compile_options(
# Disable exceptions
$<$<COMPILE_LANGUAGE:CXX>:/EHsc->
# Disable RTTI
$<$<COMPILE_LANGUAGE:CXX>:/GR->
# Enforce various standards compliant behavior.
$<$<COMPILE_LANGUAGE:CXX>:/permissive->
# Enable standard volatile semantics.
$<$<COMPILE_LANGUAGE:CXX>:/volatile:iso>
# Reports the proper value for the __cplusplus preprocessor macro.
$<$<COMPILE_LANGUAGE:CXX>:/Zc:__cplusplus>
# Use latest C++ standard.
$<$<COMPILE_LANGUAGE:CXX>:/std:c++latest>
)
add_compile_definitions(FMT_EXCEPTIONS=0 _HAS_EXCEPTIONS=0)
if ("${CMAKE_CXX_COMPILER_ID}" STREQUAL "MSVC")
# Flags for MSVC (not clang-cl)
add_compile_options(
# Allow constexpr variables to have explicit external linkage.
$<$<COMPILE_LANGUAGE:CXX>:/Zc:externConstexpr>
# Assume that new throws exceptions, allowing better code generation.
$<$<COMPILE_LANGUAGE:CXX>:/Zc:throwingNew>
# Link-time Code Generation for Release builds
$<$<OR:$<CONFIG:Release>,$<CONFIG:RelWithDebInfo>>:/GL>
)
# Link-time Code Generation for Release builds
set(CMAKE_STATIC_LINKER_FLAGS_RELEASE "/LTCG")
set(CMAKE_STATIC_LINKER_FLAGS_RELWITHDEBINFO "/LTCG")
set(CMAKE_EXE_LINKER_FLAGS_RELEASE "/RELEASE /LTCG /OPT:REF /OPT:ICF /INCREMENTAL:NO")
set(CMAKE_EXE_LINKER_FLAGS_RELWITHDEBINFO "/DEBUG /RELEASE /LTCG /OPT:REF /OPT:ICF /INCREMENTAL:NO /DEBUGTYPE:cv,fixup")
endif()
else()
set(CMAKE_CXX_STANDARD 17) set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON) set(CMAKE_CXX_STANDARD_REQUIRED ON)
endif()
endif() endif()
include (CMakePackageConfigHelpers) include (CMakePackageConfigHelpers)
if (MSVC) if (WIN32 AND NOT TARGET nowide)
# Shaddup MSVC include(FetchContent)
add_definitions(-DUNICODE=1 -D_UNICODE=1 -D__SSE__=1 -D_CRT_SECURE_NO_WARNINGS=1 -DD_SCL_SECURE_NO_WARNINGS=1 FetchContent_Declare(
/IGNORE:4221 /wd4018 /wd4800 /wd4005 /wd4311 /wd4267 /wd4244 /wd4200 /wd4305 /wd4067 /wd4146 ${VS_DEFINES}) nowide
endif() URL https://github.com/boostorg/nowide/releases/download/v11.3.0/nowide_standalone_v11.3.0.tar.gz
URL_HASH SHA256=153ac93173c8de9c08e7701e471fa750f84c27e51fe329570c5aa06016591f8c
DOWNLOAD_EXTRACT_TIMESTAMP TRUE
EXCLUDE_FROM_ALL
)
FetchContent_MakeAvailable(nowide)
endif ()
if (NOT TARGET logvisor) if(NOT TARGET spdlog)
add_subdirectory(logvisor) find_package(spdlog REQUIRED)
endif() endif()
add_subdirectory(lib) add_subdirectory(lib)
add_subdirectory(driver) add_subdirectory(driver)
if (CMAKE_SOURCE_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR)
set(version_config_file "${PROJECT_BINARY_DIR}/nodConfigVersion.cmake") set(version_config_file "${PROJECT_BINARY_DIR}/nodConfigVersion.cmake")
set(config_file "${PROJECT_BINARY_DIR}/nodConfig.cmake") set(config_file "${PROJECT_BINARY_DIR}/nodConfig.cmake")
set(config_install_dir "lib/cmake/nod") set(config_install_dir "lib/cmake/nod")
@@ -49,3 +108,4 @@ install(
FILES "${config_file}" "${version_config_file}" FILES "${config_file}" "${version_config_file}"
DESTINATION ${config_install_dir} DESTINATION ${config_install_dir}
) )
endif()

View File

@@ -34,7 +34,7 @@ a content pipeline using the `nod::DiscBuilderBase` interface.
```cpp ```cpp
/* Sample logging lambda for progress feedback */ /* Sample logging lambda for progress feedback */
size_t lastIdx = -1; size_t lastIdx = -1;
auto progFunc = [&](size_t idx, const nod::SystemString& name, size_t bytes) auto progFunc = [&](size_t idx, const std::string& name, size_t bytes)
{ {
if (idx != lastIdx) if (idx != lastIdx)
{ {

View File

@@ -1,11 +1,13 @@
add_executable(nodtool main.cpp) add_executable(nodtool main.cpp)
target_link_libraries(nodtool nod logvisor) target_link_libraries(nodtool nod spdlog::spdlog)
if (NOT WIN32) if (NOT WIN32)
target_link_libraries(nodtool pthread) target_link_libraries(nodtool pthread)
if(${CMAKE_SYSTEM_NAME} MATCHES "FreeBSD") if (${CMAKE_SYSTEM_NAME} MATCHES "FreeBSD")
target_link_libraries(nodtool execinfo) target_link_libraries(nodtool execinfo)
else() else ()
target_link_libraries(nodtool dl) target_link_libraries(nodtool dl)
endif() endif ()
endif() else ()
target_sources(nodtool PRIVATE app.manifest)
endif ()

9
driver/app.manifest Normal file
View File

@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<assembly manifestVersion="1.0" xmlns="urn:schemas-microsoft-com:asm.v1">
<assemblyIdentity type="win32" name="..." version="6.0.0.0"/>
<application>
<windowsSettings>
<activeCodePage xmlns="http://schemas.microsoft.com/SMI/2019/WindowsSettings">UTF-8</activeCodePage>
</windowsSettings>
</application>
</assembly>

View File

@@ -1,227 +1,309 @@
#include <array>
#include <cstddef> #include <cstddef>
#include <cstdint> #include <cstdint>
#include <cstdio> #include <cstdio>
#include <cstring> #include <cstring>
#if _WIN32
#include <fcntl.h>
#include <io.h>
#endif
#include <logvisor/logvisor.hpp> #include <spdlog/spdlog.h>
#include <nod/DiscBase.hpp> #include <nod/DiscBase.hpp>
#include <nod/DiscGCN.hpp> #include <nod/DiscGCN.hpp>
#include <nod/DiscWii.hpp> #include <nod/DiscWii.hpp>
#include <nod/nod.hpp> #include <nod/nod.hpp>
#include "../lib/Util.hpp"
static void printHelp() { static void printHelp() {
fmt::print(stderr, fmt( fmt::print(stderr,
"Usage:\n" "Usage:\n"
" nodtool extract [-f] <image-in> [<dir-out>]\n" " nodtool extract [options] <image-in> [<dir-out>]\n"
" nodtool makegcn <fsroot-in> [<image-out>]\n" " nodtool makegcn [options] <fsroot-in> [<image-out>]\n"
" nodtool makewii <fsroot-in> [<image-out>]\n" " nodtool makewii [options] <fsroot-in> [<image-out>]\n"
" nodtool mergegcn <fsroot-in> <image-in> [<image-out>]\n" " nodtool mergegcn [options] <fsroot-in> <image-in> [<image-out>]\n"
" nodtool mergewii <fsroot-in> <image-in> [<image-out>]\n")); " nodtool mergewii [options] <fsroot-in> <image-in> [<image-out>]\n"
"Options:\n"
" -f Force (extract only)\n"
" -v Verbose details (extract only).\n");
} }
#if NOD_UCS2 #if _MSC_VER
#ifdef strcasecmp #include <nowide/args.hpp>
#undef strcasecmp
#endif
#define strcasecmp _wcsicmp
#define PRISize "Iu" #define PRISize "Iu"
int wmain(int argc, wchar_t* argv[]) int main(int argc, char* argv[]) {
nowide::args _(argc, argv);
#else #else
#define PRISize "zu" #define PRISize "zu"
int main(int argc, char* argv[]) int main(int argc, char* argv[]) {
#endif #endif
{ int argidx = 1;
if (argc < 3 || (!strcasecmp(argv[1], _SYS_STR("makegcn")) && argc < 3) || std::string errand;
(!strcasecmp(argv[1], _SYS_STR("makewii")) && argc < 3) || bool verbose = false;
(!strcasecmp(argv[1], _SYS_STR("mergegcn")) && argc < 4) || nod::ExtractionContext ctx = {true, [&](std::string_view str, float c) {
(!strcasecmp(argv[1], _SYS_STR("mergewii")) && argc < 4)) { if (verbose)
fmt::print(stderr, "Current node: {}, Extraction {:g}% Complete\n", str,
c * 100.f);
}};
while (argidx < argc) {
if (!nod::StrCaseCmp(argv[argidx], "-f")) {
ctx.force = true;
++argidx;
continue;
} else if (!nod::StrCaseCmp(argv[argidx], "-v")) {
verbose = true;
++argidx;
continue;
} else if (errand.empty()) {
errand = argv[argidx];
++argidx;
continue;
} else {
break;
}
}
if (errand.empty()) {
printHelp(); printHelp();
return 1; return 1;
} }
/* Enable logging to console */ auto progFunc = [&](float prog, std::string_view name, size_t bytes) {
logvisor::RegisterStandardExceptions(); fmt::print("\r ");
logvisor::RegisterConsoleLogger();
bool verbose = false;
nod::ExtractionContext ctx = {true, [&](std::string_view str, float c) {
if (verbose)
fmt::print(stderr, fmt("Current node: {}, Extraction {:g}% Complete\n"), str,
c * 100.f);
}};
const nod::SystemChar* inDir = nullptr;
const nod::SystemChar* outDir = _SYS_STR(".");
for (int a = 2; a < argc; ++a) {
if (argv[a][0] == '-' && argv[a][1] == 'f')
ctx.force = true;
else if (argv[a][0] == '-' && argv[a][1] == 'v')
verbose = true;
else if (!inDir)
inDir = argv[a];
else
outDir = argv[a];
}
auto progFunc = [&](float prog, nod::SystemStringView name, size_t bytes) {
fmt::print(fmt(_SYS_STR("\r ")));
if (bytes != SIZE_MAX) if (bytes != SIZE_MAX)
fmt::print(fmt(_SYS_STR("\r{:g}% {} {} B")), prog * 100.f, name, bytes); fmt::print("\r{:g}% {} {} B", prog * 100.f, name, bytes);
else else
fmt::print(fmt(_SYS_STR("\r{:g}% {}")), prog * 100.f, name); fmt::print("\r{:g}% {}", prog * 100.f, name);
fflush(stdout); fflush(stdout);
}; };
if (!strcasecmp(argv[1], _SYS_STR("extract"))) { if (errand == "extract") {
std::string imageIn;
std::string dirOut;
while (argidx < argc) {
if (imageIn.empty()) {
imageIn = argv[argidx];
++argidx;
continue;
} else if (dirOut.empty()) {
dirOut = argv[argidx];
++argidx;
continue;
} else {
printHelp();
return 1;
}
}
if (dirOut.empty())
dirOut = ".";
bool isWii; bool isWii;
std::unique_ptr<nod::DiscBase> disc = nod::OpenDiscFromImage(inDir, isWii); std::unique_ptr<nod::DiscBase> disc = nod::OpenDiscFromImage(imageIn, isWii);
if (!disc) if (!disc)
return 1; return 1;
nod::Mkdir(outDir, 0755); nod::Mkdir(dirOut.c_str(), 0755);
nod::IPartition* dataPart = disc->getDataPartition(); nod::IPartition* dataPart = disc->getDataPartition();
if (!dataPart) if (!dataPart)
return 1; return 1;
if (!dataPart->extractToDirectory(outDir, ctx)) if (!dataPart->extractToDirectory(dirOut, ctx))
return 1; return 1;
} else if (!strcasecmp(argv[1], _SYS_STR("makegcn"))) { } else if (errand == "makegcn") {
std::string fsrootIn;
std::string imageOut;
while (argidx < argc) {
if (fsrootIn.empty()) {
fsrootIn = argv[argidx];
++argidx;
continue;
} else if (imageOut.empty()) {
imageOut = argv[argidx];
++argidx;
continue;
} else {
printHelp();
return 1;
}
}
if (imageOut.empty())
imageOut = fsrootIn + ".gcm";
/* Pre-validate path */ /* Pre-validate path */
nod::Sstat theStat; nod::Sstat theStat;
if (nod::Stat(argv[2], &theStat) || !S_ISDIR(theStat.st_mode)) { if (nod::Stat(fsrootIn.c_str(), &theStat) || !S_ISDIR(theStat.st_mode)) {
nod::LogModule.report(logvisor::Error, fmt(_SYS_STR("unable to stat {} as directory")), argv[2]); spdlog::error("unable to stat {} as directory");
return 1; return 1;
} }
if (nod::DiscBuilderGCN::CalculateTotalSizeRequired(argv[2]) == UINT64_MAX) if (!nod::DiscBuilderGCN::CalculateTotalSizeRequired(fsrootIn))
return 1; return 1;
nod::EBuildResult ret; nod::EBuildResult ret;
if (argc < 4) { nod::DiscBuilderGCN b(imageOut, progFunc);
nod::SystemString outPath(argv[2]); ret = b.buildFromDirectory(fsrootIn);
outPath.append(_SYS_STR(".iso"));
nod::DiscBuilderGCN b(outPath, progFunc);
ret = b.buildFromDirectory(argv[2]);
} else {
nod::DiscBuilderGCN b(argv[3], progFunc);
ret = b.buildFromDirectory(argv[2]);
}
fmt::print(fmt("\n")); fmt::print("\n");
if (ret != nod::EBuildResult::Success) if (ret != nod::EBuildResult::Success)
return 1; return 1;
} else if (!strcasecmp(argv[1], _SYS_STR("makewii"))) { } else if (errand == "makewii") {
std::string fsrootIn;
std::string imageOut;
while (argidx < argc) {
if (fsrootIn.empty()) {
fsrootIn = argv[argidx];
++argidx;
continue;
} else if (imageOut.empty()) {
imageOut = argv[argidx];
++argidx;
continue;
} else {
printHelp();
return 1;
}
}
if (imageOut.empty())
imageOut = fsrootIn + ".iso";
/* Pre-validate path */ /* Pre-validate path */
nod::Sstat theStat; nod::Sstat theStat;
if (nod::Stat(argv[2], &theStat) || !S_ISDIR(theStat.st_mode)) { if (nod::Stat(fsrootIn.c_str(), &theStat) || !S_ISDIR(theStat.st_mode)) {
nod::LogModule.report(logvisor::Error, fmt(_SYS_STR("unable to stat {} as directory")), argv[4]); spdlog::error("unable to stat {} as directory");
return 1; return 1;
} }
bool dual = false; bool dual = false;
if (nod::DiscBuilderWii::CalculateTotalSizeRequired(argv[2], dual) == UINT64_MAX) if (!nod::DiscBuilderWii::CalculateTotalSizeRequired(fsrootIn, dual))
return 1; return 1;
nod::EBuildResult ret; nod::EBuildResult ret;
if (argc < 4) { nod::DiscBuilderWii b(imageOut, dual, progFunc);
nod::SystemString outPath(argv[2]); ret = b.buildFromDirectory(fsrootIn);
outPath.append(_SYS_STR(".iso"));
nod::DiscBuilderWii b(outPath.c_str(), dual, progFunc);
ret = b.buildFromDirectory(argv[2]);
} else {
nod::DiscBuilderWii b(argv[3], dual, progFunc);
ret = b.buildFromDirectory(argv[2]);
}
fmt::print(fmt("\n")); fmt::print("\n");
if (ret != nod::EBuildResult::Success) if (ret != nod::EBuildResult::Success)
return 1; return 1;
} else if (!strcasecmp(argv[1], _SYS_STR("mergegcn"))) { } else if (errand == "mergegcn") {
/* Pre-validate paths */ std::string fsrootIn;
nod::Sstat theStat; std::string imageIn;
if (nod::Stat(argv[2], &theStat) || !S_ISDIR(theStat.st_mode)) { std::string imageOut;
nod::LogModule.report(logvisor::Error, fmt(_SYS_STR("unable to stat {} as directory")), argv[2]); while (argidx < argc) {
if (fsrootIn.empty()) {
fsrootIn = argv[argidx];
++argidx;
continue;
} else if (imageIn.empty()) {
imageIn = argv[argidx];
++argidx;
continue;
} else if (imageOut.empty()) {
imageOut = argv[argidx];
++argidx;
continue;
} else {
printHelp();
return 1; return 1;
} }
if (nod::Stat(argv[3], &theStat) || !S_ISREG(theStat.st_mode)) { }
nod::LogModule.report(logvisor::Error, fmt(_SYS_STR("unable to stat {} as file")), argv[3]); if (imageOut.empty())
imageOut = fsrootIn + ".gcm";
/* Pre-validate paths */
nod::Sstat theStat;
if (nod::Stat(fsrootIn.c_str(), &theStat) || !S_ISDIR(theStat.st_mode)) {
spdlog::error("unable to stat {} as directory");
return 1;
}
if (nod::Stat(imageIn.c_str(), &theStat) || !S_ISREG(theStat.st_mode)) {
spdlog::error("unable to stat {} as file");
return 1; return 1;
} }
bool isWii; bool isWii;
std::unique_ptr<nod::DiscBase> disc = nod::OpenDiscFromImage(argv[3], isWii); std::unique_ptr<nod::DiscBase> disc = nod::OpenDiscFromImage(imageIn, isWii);
if (!disc) { if (!disc) {
nod::LogModule.report(logvisor::Error, fmt(_SYS_STR("unable to open image {}")), argv[3]); spdlog::error("unable to open image {}");
return 1; return 1;
} }
if (isWii) { if (isWii) {
nod::LogModule.report(logvisor::Error, fmt(_SYS_STR("Wii images should be merged with 'mergewii'"))); spdlog::error("Wii images should be merged with 'mergewii'");
return 1; return 1;
} }
if (nod::DiscMergerGCN::CalculateTotalSizeRequired(static_cast<nod::DiscGCN&>(*disc), argv[2]) == UINT64_MAX) if (!nod::DiscMergerGCN::CalculateTotalSizeRequired(static_cast<nod::DiscGCN&>(*disc), fsrootIn))
return 1; return 1;
nod::EBuildResult ret; nod::EBuildResult ret;
if (argc < 5) { nod::DiscMergerGCN b(imageOut, static_cast<nod::DiscGCN&>(*disc), progFunc);
nod::SystemString outPath(argv[2]); ret = b.mergeFromDirectory(fsrootIn);
outPath.append(_SYS_STR(".iso"));
nod::DiscMergerGCN b(outPath.c_str(), static_cast<nod::DiscGCN&>(*disc), progFunc);
ret = b.mergeFromDirectory(argv[2]);
} else {
nod::DiscMergerGCN b(argv[4], static_cast<nod::DiscGCN&>(*disc), progFunc);
ret = b.mergeFromDirectory(argv[2]);
}
fmt::print(fmt("\n")); fmt::print("\n");
if (ret != nod::EBuildResult::Success) if (ret != nod::EBuildResult::Success)
return 1; return 1;
} else if (!strcasecmp(argv[1], _SYS_STR("mergewii"))) { } else if (errand == "mergewii") {
/* Pre-validate paths */ std::string fsrootIn;
nod::Sstat theStat; std::string imageIn;
if (nod::Stat(argv[2], &theStat) || !S_ISDIR(theStat.st_mode)) { std::string imageOut;
nod::LogModule.report(logvisor::Error, fmt(_SYS_STR("unable to stat {} as directory")), argv[2]); while (argidx < argc) {
if (fsrootIn.empty()) {
fsrootIn = argv[argidx];
++argidx;
continue;
} else if (imageIn.empty()) {
imageIn = argv[argidx];
++argidx;
continue;
} else if (imageOut.empty()) {
imageOut = argv[argidx];
++argidx;
continue;
} else {
printHelp();
return 1; return 1;
} }
if (nod::Stat(argv[3], &theStat) || !S_ISREG(theStat.st_mode)) { }
nod::LogModule.report(logvisor::Error, fmt(_SYS_STR("unable to stat {} as file")), argv[3]); if (imageOut.empty())
imageOut = fsrootIn + ".iso";
/* Pre-validate paths */
nod::Sstat theStat;
if (nod::Stat(fsrootIn.c_str(), &theStat) || !S_ISDIR(theStat.st_mode)) {
spdlog::error("unable to stat {} as directory");
return 1;
}
if (nod::Stat(imageIn.c_str(), &theStat) || !S_ISREG(theStat.st_mode)) {
spdlog::error("unable to stat {} as file");
return 1; return 1;
} }
bool isWii; bool isWii;
std::unique_ptr<nod::DiscBase> disc = nod::OpenDiscFromImage(argv[3], isWii); std::unique_ptr<nod::DiscBase> disc = nod::OpenDiscFromImage(imageIn, isWii);
if (!disc) { if (!disc) {
nod::LogModule.report(logvisor::Error, fmt(_SYS_STR("unable to open image {}")), argv[3]); spdlog::error("unable to open image {}");
return 1; return 1;
} }
if (!isWii) { if (!isWii) {
nod::LogModule.report(logvisor::Error, fmt(_SYS_STR("GameCube images should be merged with 'mergegcn'"))); spdlog::error("GameCube images should be merged with 'mergegcn'");
return 1; return 1;
} }
bool dual = false; bool dual = false;
if (nod::DiscMergerWii::CalculateTotalSizeRequired(static_cast<nod::DiscWii&>(*disc), argv[2], dual) == UINT64_MAX) if (!nod::DiscMergerWii::CalculateTotalSizeRequired(static_cast<nod::DiscWii&>(*disc), fsrootIn, dual))
return 1; return 1;
nod::EBuildResult ret; nod::EBuildResult ret;
if (argc < 5) { nod::DiscMergerWii b(imageOut, static_cast<nod::DiscWii&>(*disc), dual, progFunc);
nod::SystemString outPath(argv[2]); ret = b.mergeFromDirectory(fsrootIn);
outPath.append(_SYS_STR(".iso"));
nod::DiscMergerWii b(outPath.c_str(), static_cast<nod::DiscWii&>(*disc), dual, progFunc);
ret = b.mergeFromDirectory(argv[2]);
} else {
nod::DiscMergerWii b(argv[4], static_cast<nod::DiscWii&>(*disc), dual, progFunc);
ret = b.mergeFromDirectory(argv[2]);
}
fmt::print(fmt("\n")); fmt::print("\n");
if (ret != nod::EBuildResult::Success) if (ret != nod::EBuildResult::Success)
return 1; return 1;
} else { } else {
@@ -229,6 +311,6 @@ int main(int argc, char* argv[])
return 1; return 1;
} }
nod::LogModule.report(logvisor::Info, fmt(_SYS_STR("Success!"))); spdlog::info("Success!");
return 0; return 0;
} }

View File

@@ -4,51 +4,49 @@
#include <cctype> #include <cctype>
#include <cstddef> #include <cstddef>
#include <vector> #include <vector>
#include <string>
#include "nod/Util.hpp" #include <string_view>
namespace nod { namespace nod {
/**
* @brief Case-insensitive comparator for std::map sorting
*/
struct CaseInsensitiveCompare { struct CaseInsensitiveCompare {
// Allow heterogeneous lookup with maps that use this comparator.
using is_transparent = void;
bool operator()(std::string_view lhs, std::string_view rhs) const { bool operator()(std::string_view lhs, std::string_view rhs) const {
return std::lexicographical_compare(lhs.begin(), lhs.end(), rhs.begin(), rhs.end(), [](char lhs, char rhs) { return std::lexicographical_compare(lhs.begin(), lhs.end(), rhs.begin(), rhs.end(), [](char lhs, char rhs) {
return std::tolower(static_cast<unsigned char>(lhs)) < std::tolower(static_cast<unsigned char>(rhs)); return std::tolower(static_cast<unsigned char>(lhs)) < std::tolower(static_cast<unsigned char>(rhs));
}); });
} }
#if _WIN32
bool operator()(std::wstring_view lhs, std::wstring_view rhs) const {
return std::lexicographical_compare(lhs.begin(), lhs.end(), rhs.begin(), rhs.end(), [](wchar_t lhs, wchar_t rhs) {
return std::towlower(lhs) < std::towlower(rhs);
});
}
#endif
}; };
class DirectoryEnumerator { class DirectoryEnumerator {
public: public:
enum class Mode { Native, DirsSorted, FilesSorted, DirsThenFilesSorted }; enum class Mode { Native, DirsSorted, FilesSorted, DirsThenFilesSorted };
struct Entry { struct Entry {
SystemString m_path; std::string m_path;
SystemString m_name; std::string m_name;
size_t m_fileSz; size_t m_fileSz;
bool m_isDir; bool m_isDir;
Entry(const SystemString& path, const SystemChar* name, size_t sz, bool isDir) Entry(std::string path, std::string name, size_t sz, bool isDir)
: m_path(path), m_name(name), m_fileSz(sz), m_isDir(isDir) {} : m_path(std::move(path)), m_name(std::move(name)), m_fileSz(sz), m_isDir(isDir) {}
}; };
private: private:
std::vector<Entry> m_entries; std::vector<Entry> m_entries;
public: public:
DirectoryEnumerator(SystemStringView path, Mode mode = Mode::DirsThenFilesSorted, bool sizeSort = false, DirectoryEnumerator(std::string_view path, Mode mode = Mode::DirsThenFilesSorted, bool sizeSort = false,
bool reverse = false, bool noHidden = false); bool reverse = false, bool noHidden = false);
operator bool() const { return m_entries.size() != 0; } explicit operator bool() const { return !m_entries.empty(); }
size_t size() const { return m_entries.size(); } [[nodiscard]] size_t size() const { return m_entries.size(); }
std::vector<Entry>::const_iterator begin() const { return m_entries.cbegin(); } [[nodiscard]] std::vector<Entry>::const_iterator begin() const { return m_entries.cbegin(); }
std::vector<Entry>::const_iterator end() const { return m_entries.cend(); } [[nodiscard]] std::vector<Entry>::const_iterator end() const { return m_entries.cend(); }
}; };
} // namespace nod } // namespace nod

View File

@@ -4,23 +4,23 @@
#include <cstdint> #include <cstdint>
#include <cstring> #include <cstring>
#include <functional> #include <functional>
#include <memory> #include <optional>
#include <string> #include <string>
#include <unordered_map> #include <string_view>
#include <vector>
#include "nod/IDiscIO.hpp" #include "nod/IDiscIO.hpp"
#include "nod/IFileIO.hpp" #include "nod/IFileIO.hpp"
#include "nod/Util.hpp" #include "nod/OSUTF.h"
#include "nod/Endian.hpp"
namespace nod { namespace nod {
using FProgress = std::function<void(float totalProg, SystemStringView fileName, size_t fileBytesXfered)>; using FProgress = std::function<void(float totalProg, std::string_view fileName, size_t fileBytesXfered)>;
enum class EBuildResult { Success, Failed, DiskFull }; enum class EBuildResult { Success, Failed, DiskFull };
enum class PartitionKind : uint32_t { Data, Update, Channel }; enum class PartitionKind : uint32_t { Data, Update, Channel };
const SystemChar* getKindString(PartitionKind kind); const char* getKindString(PartitionKind kind);
class FSTNode { class FSTNode {
uint32_t typeAndNameOffset; uint32_t typeAndNameOffset;
@@ -35,10 +35,10 @@ public:
offset = SBig(off); offset = SBig(off);
length = SBig(len); length = SBig(len);
} }
inline bool isDir() const { return ((SBig(typeAndNameOffset) >> 24) != 0); } bool isDir() const { return ((SBig(typeAndNameOffset) >> 24) != 0); }
inline uint32_t getNameOffset() const { return SBig(typeAndNameOffset) & 0xffffff; } uint32_t getNameOffset() const { return SBig(typeAndNameOffset) & 0xffffff; }
inline uint32_t getOffset() const { return SBig(offset); } uint32_t getOffset() const { return SBig(offset); }
inline uint32_t getLength() const { return SBig(length); } uint32_t getLength() const { return SBig(length); }
void incrementLength() { void incrementLength() {
uint32_t orig = SBig(length); uint32_t orig = SBig(length);
++orig; ++orig;
@@ -187,13 +187,13 @@ private:
public: public:
Node(const IPartition& parent, const FSTNode& node, std::string_view name); Node(const IPartition& parent, const FSTNode& node, std::string_view name);
inline Kind getKind() const { return m_kind; } Kind getKind() const { return m_kind; }
inline std::string_view getName() const { return m_name; } std::string_view getName() const { return m_name; }
inline uint64_t size() const { return m_discLength; } uint64_t size() const { return m_discLength; }
std::unique_ptr<IPartReadStream> beginReadStream(uint64_t offset = 0) const; std::unique_ptr<IPartReadStream> beginReadStream(uint64_t offset = 0) const;
std::unique_ptr<uint8_t[]> getBuf() const; std::unique_ptr<uint8_t[]> getBuf() const;
inline std::vector<Node>::iterator rawBegin() const { return m_childrenBegin; } std::vector<Node>::iterator rawBegin() const { return m_childrenBegin; }
inline std::vector<Node>::iterator rawEnd() const { return m_childrenEnd; } std::vector<Node>::iterator rawEnd() const { return m_childrenEnd; }
class DirectoryIterator { class DirectoryIterator {
friend class Node; friend class Node;
@@ -207,25 +207,27 @@ public:
using pointer = Node*; using pointer = Node*;
using reference = Node&; using reference = Node&;
inline bool operator!=(const DirectoryIterator& other) { return m_it != other.m_it; } bool operator==(const DirectoryIterator& other) const { return m_it == other.m_it; }
inline bool operator==(const DirectoryIterator& other) { return m_it == other.m_it; } bool operator!=(const DirectoryIterator& other) const { return !operator==(other); }
inline DirectoryIterator& operator++() { DirectoryIterator& operator++() {
if (m_it->m_kind == Kind::Directory) if (m_it->m_kind == Kind::Directory)
m_it = m_it->rawEnd(); m_it = m_it->rawEnd();
else else
++m_it; ++m_it;
return *this; return *this;
} }
inline Node& operator*() { return *m_it; } Node& operator*() { return *m_it; }
inline Node* operator->() { return &*m_it; } const Node& operator*() const { return *m_it; }
Node* operator->() { return &*m_it; }
const Node* operator->() const { return &*m_it; }
}; };
inline DirectoryIterator begin() const { return DirectoryIterator(m_childrenBegin); } DirectoryIterator begin() const { return DirectoryIterator(m_childrenBegin); }
inline DirectoryIterator end() const { return DirectoryIterator(m_childrenEnd); } DirectoryIterator end() const { return DirectoryIterator(m_childrenEnd); }
inline DirectoryIterator find(std::string_view name) const { DirectoryIterator find(std::string_view name) const {
if (m_kind == Kind::Directory) { if (m_kind == Kind::Directory) {
DirectoryIterator it = begin(); DirectoryIterator it = begin();
for (; it != end(); ++it) { for (; it != end(); ++it) {
if (!it->getName().compare(name)) if (it->getName() == name)
return it; return it;
} }
return it; return it;
@@ -233,7 +235,7 @@ public:
return end(); return end();
} }
bool extractToDirectory(SystemStringView basePath, const ExtractionContext& ctx) const; bool extractToDirectory(std::string_view basePath, const ExtractionContext& ctx) const;
}; };
class IPartition { class IPartition {
@@ -289,49 +291,49 @@ public:
IPartition(const DiscBase& parent, PartitionKind kind, bool isWii, uint64_t offset) IPartition(const DiscBase& parent, PartitionKind kind, bool isWii, uint64_t offset)
: m_parent(parent), m_kind(kind), m_offset(offset), m_isWii(isWii) {} : m_parent(parent), m_kind(kind), m_offset(offset), m_isWii(isWii) {}
virtual uint64_t normalizeOffset(uint64_t anOffset) const { return anOffset; } virtual uint64_t normalizeOffset(uint64_t anOffset) const { return anOffset; }
inline PartitionKind getKind() const { return m_kind; } PartitionKind getKind() const { return m_kind; }
inline bool isWii() const { return m_isWii; } bool isWii() const { return m_isWii; }
inline uint64_t getDiscOffset() const { return m_offset; } uint64_t getDiscOffset() const { return m_offset; }
virtual std::unique_ptr<IPartReadStream> beginReadStream(uint64_t offset = 0) const = 0; virtual std::unique_ptr<IPartReadStream> beginReadStream(uint64_t offset = 0) const = 0;
inline std::unique_ptr<IPartReadStream> beginDOLReadStream(uint64_t offset = 0) const { std::unique_ptr<IPartReadStream> beginDOLReadStream(uint64_t offset = 0) const {
return beginReadStream(m_dolOff + offset); return beginReadStream(m_dolOff + offset);
} }
inline std::unique_ptr<IPartReadStream> beginFSTReadStream(uint64_t offset = 0) const { std::unique_ptr<IPartReadStream> beginFSTReadStream(uint64_t offset = 0) const {
return beginReadStream(m_fstOff + offset); return beginReadStream(m_fstOff + offset);
} }
inline std::unique_ptr<IPartReadStream> beginApploaderReadStream(uint64_t offset = 0) const { std::unique_ptr<IPartReadStream> beginApploaderReadStream(uint64_t offset = 0) const {
return beginReadStream(0x2440 + offset); return beginReadStream(0x2440 + offset);
} }
inline const Node& getFSTRoot() const { return m_nodes[0]; } const Node& getFSTRoot() const { return m_nodes[0]; }
inline Node& getFSTRoot() { return m_nodes[0]; } Node& getFSTRoot() { return m_nodes[0]; }
bool extractToDirectory(SystemStringView path, const ExtractionContext& ctx); bool extractToDirectory(std::string_view path, const ExtractionContext& ctx);
inline uint64_t getDOLSize() const { return m_dolSz; } uint64_t getDOLSize() const { return m_dolSz; }
inline std::unique_ptr<uint8_t[]> getDOLBuf() const { std::unique_ptr<uint8_t[]> getDOLBuf() const {
std::unique_ptr<uint8_t[]> buf(new uint8_t[m_dolSz]); std::unique_ptr<uint8_t[]> buf(new uint8_t[m_dolSz]);
beginDOLReadStream()->read(buf.get(), m_dolSz); beginDOLReadStream()->read(buf.get(), m_dolSz);
return buf; return buf;
} }
inline uint64_t getFSTSize() const { return m_fstSz; } uint64_t getFSTSize() const { return m_fstSz; }
inline std::unique_ptr<uint8_t[]> getFSTBuf() const { std::unique_ptr<uint8_t[]> getFSTBuf() const {
std::unique_ptr<uint8_t[]> buf(new uint8_t[m_fstSz]); std::unique_ptr<uint8_t[]> buf(new uint8_t[m_fstSz]);
beginFSTReadStream()->read(buf.get(), m_fstSz); beginFSTReadStream()->read(buf.get(), m_fstSz);
return buf; return buf;
} }
inline uint64_t getApploaderSize() const { return m_apploaderSz; } uint64_t getApploaderSize() const { return m_apploaderSz; }
inline std::unique_ptr<uint8_t[]> getApploaderBuf() const { std::unique_ptr<uint8_t[]> getApploaderBuf() const {
std::unique_ptr<uint8_t[]> buf(new uint8_t[m_apploaderSz]); std::unique_ptr<uint8_t[]> buf(new uint8_t[m_apploaderSz]);
beginApploaderReadStream()->read(buf.get(), m_apploaderSz); beginApploaderReadStream()->read(buf.get(), m_apploaderSz);
return buf; return buf;
} }
inline size_t getNodeCount() const { return m_nodes.size(); } size_t getNodeCount() const { return m_nodes.size(); }
inline const Header& getHeader() const { return m_header; } const Header& getHeader() const { return m_header; }
inline const BI2Header& getBI2() const { return m_bi2Header; } const BI2Header& getBI2() const { return m_bi2Header; }
virtual bool extractCryptoFiles(SystemStringView path, const ExtractionContext& ctx) const { return true; } virtual bool extractCryptoFiles(std::string_view path, const ExtractionContext& ctx) const { return true; }
bool extractSysFiles(SystemStringView path, const ExtractionContext& ctx) const; bool extractSysFiles(std::string_view path, const ExtractionContext& ctx) const;
}; };
class DiscBase { class DiscBase {
@@ -346,34 +348,35 @@ protected:
public: public:
DiscBase(std::unique_ptr<IDiscIO>&& dio, bool& err) : m_discIO(std::move(dio)), m_header(*m_discIO, err) {} DiscBase(std::unique_ptr<IDiscIO>&& dio, bool& err) : m_discIO(std::move(dio)), m_header(*m_discIO, err) {}
inline const Header& getHeader() const { return m_header; } const Header& getHeader() const { return m_header; }
inline const IDiscIO& getDiscIO() const { return *m_discIO; } const IDiscIO& getDiscIO() const { return *m_discIO; }
inline size_t getPartitonNodeCount(size_t partition = 0) const { size_t getPartitionNodeCount(size_t partition = 0) const {
if (partition > m_partitions.size()) if (partition >= m_partitions.size()) {
return -1; return -1;
}
return m_partitions[partition]->getNodeCount(); return m_partitions[partition]->getNodeCount();
} }
inline IPartition* getDataPartition() { IPartition* getDataPartition() {
for (const std::unique_ptr<IPartition>& part : m_partitions) for (const std::unique_ptr<IPartition>& part : m_partitions)
if (part->getKind() == PartitionKind::Data) if (part->getKind() == PartitionKind::Data)
return part.get(); return part.get();
return nullptr; return nullptr;
} }
inline IPartition* getUpdatePartition() { IPartition* getUpdatePartition() {
for (const std::unique_ptr<IPartition>& part : m_partitions) for (const std::unique_ptr<IPartition>& part : m_partitions)
if (part->getKind() == PartitionKind::Update) if (part->getKind() == PartitionKind::Update)
return part.get(); return part.get();
return nullptr; return nullptr;
} }
inline void extractToDirectory(SystemStringView path, const ExtractionContext& ctx) { void extractToDirectory(std::string_view path, const ExtractionContext& ctx) {
for (std::unique_ptr<IPartition>& part : m_partitions) for (std::unique_ptr<IPartition>& part : m_partitions)
part->extractToDirectory(path, ctx); part->extractToDirectory(path, ctx);
} }
virtual bool extractDiscHeaderFiles(SystemStringView path, const ExtractionContext& ctx) const = 0; virtual bool extractDiscHeaderFiles(std::string_view path, const ExtractionContext& ctx) const = 0;
}; };
class DiscBuilderBase { class DiscBuilderBase {
@@ -385,30 +388,30 @@ public:
virtual ~PartitionBuilderBase() = default; virtual ~PartitionBuilderBase() = default;
protected: protected:
std::unordered_map<SystemString, std::pair<uint64_t, uint64_t>> m_fileOffsetsSizes; std::unordered_map<std::string, std::pair<uint64_t, uint64_t>> m_fileOffsetsSizes;
std::vector<FSTNode> m_buildNodes; std::vector<FSTNode> m_buildNodes;
std::vector<std::string> m_buildNames; std::vector<std::string> m_buildNames;
size_t m_buildNameOff = 0; size_t m_buildNameOff = 0;
virtual uint64_t userAllocate(uint64_t reqSz, IPartWriteStream& ws) = 0; virtual uint64_t userAllocate(uint64_t reqSz, IPartWriteStream& ws) = 0;
virtual uint32_t packOffset(uint64_t offset) const = 0; virtual uint32_t packOffset(uint64_t offset) const = 0;
void recursiveBuildNodesPre(SystemStringView dirIn); void recursiveBuildNodesPre(std::string_view dirIn);
bool recursiveBuildNodes(IPartWriteStream& ws, bool system, SystemStringView dirIn); bool recursiveBuildNodes(IPartWriteStream& ws, bool system, std::string_view dirIn);
bool recursiveBuildFST(SystemStringView dirIn, std::function<void(void)> incParents, size_t parentDirIdx); bool recursiveBuildFST(std::string_view dirIn, std::function<void(void)> incParents, size_t parentDirIdx);
void recursiveMergeNodesPre(const Node* nodeIn, SystemStringView dirIn); void recursiveMergeNodesPre(const Node* nodeIn, std::string_view dirIn);
bool recursiveMergeNodes(IPartWriteStream& ws, bool system, const Node* nodeIn, SystemStringView dirIn, bool recursiveMergeNodes(IPartWriteStream& ws, bool system, const Node* nodeIn, std::string_view dirIn,
SystemStringView keyPath); std::string_view keyPath);
bool recursiveMergeFST(const Node* nodeIn, SystemStringView dirIn, std::function<void(void)> incParents, bool recursiveMergeFST(const Node* nodeIn, std::string_view dirIn, std::function<void(void)> incParents,
SystemStringView keyPath); size_t parentDirIdx, std::string_view keyPath);
static bool RecursiveCalculateTotalSize(uint64_t& totalSz, const Node* nodeIn, SystemStringView dirIn); static bool RecursiveCalculateTotalSize(uint64_t& totalSz, const Node* nodeIn, std::string_view dirIn);
void addBuildName(SystemStringView str) { void addBuildName(std::string_view str) {
SystemUTF8Conv utf8View(str); UTF8ToSJIS nameView(str);
m_buildNames.emplace_back(utf8View.utf8_str()); m_buildNames.emplace_back(nameView.str());
m_buildNameOff += str.size() + 1; m_buildNameOff += nameView.str().size() + 1;
} }
DiscBuilderBase& m_parent; DiscBuilderBase& m_parent;
@@ -421,14 +424,14 @@ public:
PartitionBuilderBase(DiscBuilderBase& parent, PartitionKind kind, bool isWii) PartitionBuilderBase(DiscBuilderBase& parent, PartitionKind kind, bool isWii)
: m_parent(parent), m_kind(kind), m_isWii(isWii) {} : m_parent(parent), m_kind(kind), m_isWii(isWii) {}
virtual std::unique_ptr<IPartWriteStream> beginWriteStream(uint64_t offset) = 0; virtual std::unique_ptr<IPartWriteStream> beginWriteStream(uint64_t offset) = 0;
bool buildFromDirectory(IPartWriteStream& ws, SystemStringView dirIn); bool buildFromDirectory(IPartWriteStream& ws, std::string_view dirIn);
static uint64_t CalculateTotalSizeBuild(SystemStringView dirIn, PartitionKind kind, bool isWii); static std::optional<uint64_t> CalculateTotalSizeBuild(std::string_view dirIn, PartitionKind kind, bool isWii);
bool mergeFromDirectory(IPartWriteStream& ws, const IPartition* partIn, SystemStringView dirIn); bool mergeFromDirectory(IPartWriteStream& ws, const IPartition* partIn, std::string_view dirIn);
static uint64_t CalculateTotalSizeMerge(const IPartition* partIn, SystemStringView dirIn); static std::optional<uint64_t> CalculateTotalSizeMerge(const IPartition* partIn, std::string_view dirIn);
}; };
protected: protected:
SystemString m_outPath; std::string m_outPath;
std::unique_ptr<IFileIO> m_fileIO; std::unique_ptr<IFileIO> m_fileIO;
std::vector<std::unique_ptr<PartitionBuilderBase>> m_partitions; std::vector<std::unique_ptr<PartitionBuilderBase>> m_partitions;
int64_t m_discCapacity; int64_t m_discCapacity;
@@ -451,11 +454,11 @@ public:
} }
virtual ~DiscBuilderBase() = default; virtual ~DiscBuilderBase() = default;
DiscBuilderBase(SystemStringView outPath, int64_t discCapacity, FProgress progressCB) DiscBuilderBase(std::string_view outPath, int64_t discCapacity, FProgress progressCB)
: m_outPath(outPath) : m_outPath(outPath)
, m_fileIO(NewFileIO(outPath, discCapacity)) , m_fileIO(NewFileIO(outPath, discCapacity))
, m_discCapacity(discCapacity) , m_discCapacity(discCapacity)
, m_progressCB(progressCB) {} , m_progressCB(std::move(progressCB)) {}
DiscBuilderBase(DiscBuilderBase&&) = default; DiscBuilderBase(DiscBuilderBase&&) = default;
DiscBuilderBase& operator=(DiscBuilderBase&&) = default; DiscBuilderBase& operator=(DiscBuilderBase&&) = default;

View File

@@ -7,20 +7,20 @@ class DiscBuilderGCN;
class DiscGCN : public DiscBase { class DiscGCN : public DiscBase {
friend class DiscMergerGCN; friend class DiscMergerGCN;
DiscBuilderGCN makeMergeBuilder(SystemStringView outPath, FProgress progressCB); DiscBuilderGCN makeMergeBuilder(std::string_view outPath, FProgress progressCB);
public: public:
DiscGCN(std::unique_ptr<IDiscIO>&& dio, bool& err); DiscGCN(std::unique_ptr<IDiscIO>&& dio, bool& err);
bool extractDiscHeaderFiles(SystemStringView path, const ExtractionContext& ctx) const override; bool extractDiscHeaderFiles(std::string_view path, const ExtractionContext& ctx) const override;
}; };
class DiscBuilderGCN : public DiscBuilderBase { class DiscBuilderGCN : public DiscBuilderBase {
friend class DiscMergerGCN; friend class DiscMergerGCN;
public: public:
DiscBuilderGCN(SystemStringView outPath, FProgress progressCB); DiscBuilderGCN(std::string_view outPath, FProgress progressCB);
EBuildResult buildFromDirectory(SystemStringView dirIn); EBuildResult buildFromDirectory(std::string_view dirIn);
static uint64_t CalculateTotalSizeRequired(SystemStringView dirIn); static std::optional<uint64_t> CalculateTotalSizeRequired(std::string_view dirIn);
}; };
class DiscMergerGCN { class DiscMergerGCN {
@@ -28,9 +28,9 @@ class DiscMergerGCN {
DiscBuilderGCN m_builder; DiscBuilderGCN m_builder;
public: public:
DiscMergerGCN(SystemStringView outPath, DiscGCN& sourceDisc, FProgress progressCB); DiscMergerGCN(std::string_view outPath, DiscGCN& sourceDisc, FProgress progressCB);
EBuildResult mergeFromDirectory(SystemStringView dirIn); EBuildResult mergeFromDirectory(std::string_view dirIn);
static uint64_t CalculateTotalSizeRequired(DiscGCN& sourceDisc, SystemStringView dirIn); static std::optional<uint64_t> CalculateTotalSizeRequired(DiscGCN& sourceDisc, std::string_view dirIn);
}; };
} // namespace nod } // namespace nod

View File

@@ -8,15 +8,15 @@ class DiscBuilderWii;
class DiscWii : public DiscBase { class DiscWii : public DiscBase {
public: public:
DiscWii(std::unique_ptr<IDiscIO>&& dio, bool& err); DiscWii(std::unique_ptr<IDiscIO>&& dio, bool& err);
DiscBuilderWii makeMergeBuilder(SystemStringView outPath, bool dualLayer, FProgress progressCB); DiscBuilderWii makeMergeBuilder(std::string_view outPath, bool dualLayer, FProgress progressCB);
bool extractDiscHeaderFiles(SystemStringView path, const ExtractionContext& ctx) const override; bool extractDiscHeaderFiles(std::string_view path, const ExtractionContext& ctx) const override;
}; };
class DiscBuilderWii : public DiscBuilderBase { class DiscBuilderWii : public DiscBuilderBase {
public: public:
DiscBuilderWii(SystemStringView outPath, bool dualLayer, FProgress progressCB); DiscBuilderWii(std::string_view outPath, bool dualLayer, FProgress progressCB);
EBuildResult buildFromDirectory(SystemStringView dirIn); EBuildResult buildFromDirectory(std::string_view dirIn);
static uint64_t CalculateTotalSizeRequired(SystemStringView dirIn, bool& dualLayer); static std::optional<uint64_t> CalculateTotalSizeRequired(std::string_view dirIn, bool& dualLayer);
}; };
class DiscMergerWii { class DiscMergerWii {
@@ -24,9 +24,9 @@ class DiscMergerWii {
DiscBuilderWii m_builder; DiscBuilderWii m_builder;
public: public:
DiscMergerWii(SystemStringView outPath, DiscWii& sourceDisc, bool dualLayer, FProgress progressCB); DiscMergerWii(std::string_view outPath, DiscWii& sourceDisc, bool dualLayer, FProgress progressCB);
EBuildResult mergeFromDirectory(SystemStringView dirIn); EBuildResult mergeFromDirectory(std::string_view dirIn);
static uint64_t CalculateTotalSizeRequired(DiscWii& sourceDisc, SystemStringView dirIn, bool& dualLayer); static std::optional<uint64_t> CalculateTotalSizeRequired(DiscWii& sourceDisc, std::string_view dirIn, bool& dualLayer);
}; };
} // namespace nod } // namespace nod

78
include/nod/Endian.hpp Normal file
View File

@@ -0,0 +1,78 @@
#pragma once
#include <cstdint>
#undef bswap16
#undef bswap32
#undef bswap64
namespace nod {
/* Type-sensitive byte swappers */
template <typename T>
static inline T bswap16(T val) {
#if __GNUC__
return __builtin_bswap16(val);
#elif _WIN32
return _byteswap_ushort(val);
#else
return (val = (val << 8) | ((val >> 8) & 0xFF));
#endif
}
template <typename T>
static inline T bswap32(T val) {
#if __GNUC__
return __builtin_bswap32(val);
#elif _WIN32
return _byteswap_ulong(val);
#else
val = (val & 0x0000FFFF) << 16 | (val & 0xFFFF0000) >> 16;
val = (val & 0x00FF00FF) << 8 | (val & 0xFF00FF00) >> 8;
return val;
#endif
}
template <typename T>
static inline T bswap64(T val) {
#if __GNUC__
return __builtin_bswap64(val);
#elif _WIN32
return _byteswap_uint64(val);
#else
return ((val & 0xFF00000000000000ULL) >> 56) | ((val & 0x00FF000000000000ULL) >> 40) |
((val & 0x0000FF0000000000ULL) >> 24) | ((val & 0x000000FF00000000ULL) >> 8) |
((val & 0x00000000FF000000ULL) << 8) | ((val & 0x0000000000FF0000ULL) << 24) |
((val & 0x000000000000FF00ULL) << 40) | ((val & 0x00000000000000FFULL) << 56);
#endif
}
#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
static inline int16_t SBig(int16_t val) { return bswap16(val); }
static inline uint16_t SBig(uint16_t val) { return bswap16(val); }
static inline int32_t SBig(int32_t val) { return bswap32(val); }
static inline uint32_t SBig(uint32_t val) { return bswap32(val); }
static inline int64_t SBig(int64_t val) { return bswap64(val); }
static inline uint64_t SBig(uint64_t val) { return bswap64(val); }
static inline int16_t SLittle(int16_t val) { return val; }
static inline uint16_t SLittle(uint16_t val) { return val; }
static inline int32_t SLittle(int32_t val) { return val; }
static inline uint32_t SLittle(uint32_t val) { return val; }
static inline int64_t SLittle(int64_t val) { return val; }
static inline uint64_t SLittle(uint64_t val) { return val; }
#else
static inline int16_t SLittle(int16_t val) { return bswap16(val); }
static inline uint16_t SLittle(uint16_t val) { return bswap16(val); }
static inline int32_t SLittle(int32_t val) { return bswap32(val); }
static inline uint32_t SLittle(uint32_t val) { return bswap32(val); }
static inline int64_t SLittle(int64_t val) { return bswap64(val); }
static inline uint64_t SLittle(uint64_t val) { return bswap64(val); }
static inline int16_t SBig(int16_t val) { return val; }
static inline uint16_t SBig(uint16_t val) { return val; }
static inline int32_t SBig(int32_t val) { return val; }
static inline uint32_t SBig(uint32_t val) { return val; }
static inline int64_t SBig(int64_t val) { return val; }
static inline uint64_t SBig(uint64_t val) { return val; }
#endif
} // namespace nod

View File

@@ -28,6 +28,7 @@ public:
virtual ~IDiscIO() = default; virtual ~IDiscIO() = default;
virtual std::unique_ptr<IReadStream> beginReadStream(uint64_t offset = 0) const = 0; virtual std::unique_ptr<IReadStream> beginReadStream(uint64_t offset = 0) const = 0;
virtual std::unique_ptr<IWriteStream> beginWriteStream(uint64_t offset = 0) const = 0; virtual std::unique_ptr<IWriteStream> beginWriteStream(uint64_t offset = 0) const = 0;
virtual bool hasWiiCrypto() const { return true; } /* NFS overrides this to false */
}; };
struct IPartReadStream : IReadStream { struct IPartReadStream : IReadStream {
@@ -49,11 +50,12 @@ public:
AthenaPartReadStream(std::unique_ptr<IPartReadStream>&& rs) : m_rs(std::move(rs)) {} AthenaPartReadStream(std::unique_ptr<IPartReadStream>&& rs) : m_rs(std::move(rs)) {}
void seek(atInt64 off, athena::SeekOrigin origin) override { void seek(atInt64 off, athena::SeekOrigin origin) override {
if (origin == athena::Begin) if (origin == athena::SeekOrigin::Begin) {
m_rs->seek(off, SEEK_SET); m_rs->seek(off, SEEK_SET);
else if (origin == athena::Current) } else if (origin == athena::SeekOrigin::Current) {
m_rs->seek(off, SEEK_CUR); m_rs->seek(off, SEEK_CUR);
} }
}
atUint64 position() const override { return m_rs->position(); } atUint64 position() const override { return m_rs->position(); }
atUint64 length() const override { return 0; } atUint64 length() const override { return 0; }
atUint64 readUBytesToBuf(void* buf, atUint64 sz) override { atUint64 readUBytesToBuf(void* buf, atUint64 sz) override {

View File

@@ -3,11 +3,9 @@
#include <cstdint> #include <cstdint>
#include <functional> #include <functional>
#include <memory> #include <memory>
#include <string_view>
#include "nod/IDiscIO.hpp" #include "nod/IDiscIO.hpp"
#include "nod/Util.hpp"
#include <logvisor/logvisor.hpp>
namespace nod { namespace nod {
@@ -18,46 +16,8 @@ public:
virtual uint64_t size() = 0; virtual uint64_t size() = 0;
struct IWriteStream : nod::IWriteStream { struct IWriteStream : nod::IWriteStream {
uint64_t copyFromDisc(IPartReadStream& discio, uint64_t length) { uint64_t copyFromDisc(IPartReadStream& discio, uint64_t length);
uint64_t read = 0; uint64_t copyFromDisc(IPartReadStream& discio, uint64_t length, const std::function<void(float)>& prog);
uint8_t buf[0x7c00];
while (length) {
uint64_t thisSz = nod::min(uint64_t(0x7c00), length);
uint64_t readSz = discio.read(buf, thisSz);
if (thisSz != readSz) {
LogModule.report(logvisor::Error, fmt("unable to read enough from disc"));
return read;
}
if (write(buf, readSz) != readSz) {
LogModule.report(logvisor::Error, fmt("unable to write in file"));
return read;
}
length -= thisSz;
read += thisSz;
}
return read;
}
uint64_t copyFromDisc(IPartReadStream& discio, uint64_t length, const std::function<void(float)>& prog) {
uint64_t read = 0;
uint8_t buf[0x7c00];
uint64_t total = length;
while (length) {
uint64_t thisSz = nod::min(uint64_t(0x7c00), length);
uint64_t readSz = discio.read(buf, thisSz);
if (thisSz != readSz) {
LogModule.report(logvisor::Error, fmt("unable to read enough from disc"));
return read;
}
if (write(buf, readSz) != readSz) {
LogModule.report(logvisor::Error, fmt("unable to write in file"));
return read;
}
length -= thisSz;
read += thisSz;
prog(read / float(total));
}
return read;
}
}; };
virtual std::unique_ptr<IWriteStream> beginWriteStream() const = 0; virtual std::unique_ptr<IWriteStream> beginWriteStream() const = 0;
virtual std::unique_ptr<IWriteStream> beginWriteStream(uint64_t offset) const = 0; virtual std::unique_ptr<IWriteStream> beginWriteStream(uint64_t offset) const = 0;
@@ -69,6 +29,6 @@ public:
virtual std::unique_ptr<IReadStream> beginReadStream(uint64_t offset) const = 0; virtual std::unique_ptr<IReadStream> beginReadStream(uint64_t offset) const = 0;
}; };
std::unique_ptr<IFileIO> NewFileIO(SystemStringView path, int64_t maxWriteSize = -1); std::unique_ptr<IFileIO> NewFileIO(std::string_view path, int64_t maxWriteSize = -1);
} // namespace nod } // namespace nod

113
include/nod/OSUTF.h Normal file
View File

@@ -0,0 +1,113 @@
#ifdef __cplusplus
#include <array>
#include <cstdint>
#if !__cpp_char8_t
using char8_t = uint8_t;
#endif
#define OS_CONSTEXPR constexpr
extern "C" {
#else
#include <stdint.h>
#include <stdbool.h>
#define OS_CONSTEXPR
typedef uint8_t char8_t;
typedef uint16_t char16_t;
typedef uint32_t char32_t;
#endif
OS_CONSTEXPR inline bool IsSjisLeadByte(char8_t c) { return (c > 0x80 && c < 0xa0) || (c > 0xdf && c < 0xfd); }
OS_CONSTEXPR inline bool IsSjisTrailByte(char8_t c) { return c > 0x3f && c < 0xfd && c != 0x7f; }
char32_t OSSJISToUTF32(char16_t sjis);
char16_t OSUTF32ToSJIS(char32_t utf32);
char8_t OSUTF32ToAnsi(char32_t utf32);
char16_t* OSUTF32To16(char32_t utf32, char16_t* utf16);
char8_t* OSUTF32To8(char32_t utf32, char8_t* utf8);
const char16_t* OSUTF16To32(const char16_t* utf16, char32_t* utf32);
const char8_t* OSUTF8To32(const char8_t* utf8, char32_t* utf32);
#ifdef __cplusplus
}
class SJISToUTF8 {
private:
std::string out;
public:
SJISToUTF8(std::string_view sv) {
const auto* in = reinterpret_cast<const char8_t*>(sv.data());
const auto* end = in + sv.size();
std::array<char8_t, 4> u8arr{};
while (in < end) {
if (IsSjisLeadByte(*in)) {
char16_t sjis = static_cast<char16_t>(*in) << 8 | *(in + 1);
in += 2;
char32_t utf32 = OSSJISToUTF32(sjis);
if (utf32 == 0) {
continue;
}
char8_t* u8out = u8arr.data();
char8_t* u8end = OSUTF32To8(utf32, u8out);
if (u8end == nullptr) {
continue;
}
auto length = static_cast<size_t>(u8end - u8out);
out.append(std::string_view{reinterpret_cast<char*>(u8out), length});
} else {
out.push_back(static_cast<char>(*in++));
}
}
}
[[nodiscard]] const std::string& str() const { return out; }
[[nodiscard]] std::string& str() { return out; }
[[nodiscard]] const char* c_str() const { return out.c_str(); }
};
class UTF8ToSJIS {
private:
std::string out;
public:
UTF8ToSJIS(std::string_view sv) {
const auto* in = reinterpret_cast<const char8_t*>(sv.data());
const auto* end = in + sv.size();
while (in < end) {
char32_t utf32 = 0;
const char8_t* next = OSUTF8To32(in, &utf32);
if (next == nullptr) {
utf32 = *in;
in++;
} else {
in = next;
}
char16_t sjis = OSUTF32ToSJIS(utf32);
char8_t lead = (sjis >> 8) & 0xFF;
if (IsSjisLeadByte(lead)) {
out.push_back(static_cast<char>(lead));
}
out.push_back(static_cast<char>(sjis & 0xFF));
}
}
[[nodiscard]] const std::string& str() const { return out; }
[[nodiscard]] std::string& str() { return out; }
[[nodiscard]] const char* c_str() const { return out.c_str(); }
};
#endif

View File

@@ -1,317 +0,0 @@
#pragma once
#if _WIN32 && UNICODE
#include <cwctype>
#include <direct.h>
#ifndef WIN32_LEAN_AND_MEAN
#define WIN32_LEAN_AND_MEAN 1
#endif
#ifndef NOMINMAX
#define NOMINMAX
#endif
#include <windows.h>
#if defined(WINAPI_FAMILY) && WINAPI_FAMILY != WINAPI_FAMILY_DESKTOP_APP
#define WINDOWS_STORE 1
#else
#define WINDOWS_STORE 0
#endif
#else
#include <cctype>
#include <cerrno>
#include <sys/file.h>
#include <sys/param.h>
#include <sys/statvfs.h>
#include <unistd.h>
#endif
#include <sys/stat.h>
#include <algorithm>
#include <cstring>
#include <string>
#include <string_view>
#include <logvisor/logvisor.hpp>
#ifdef _MSC_VER
#pragma warning(disable : 4996)
#include <sys/stat.h>
#if !defined(S_ISREG) && defined(S_IFMT) && defined(S_IFREG)
#define S_ISREG(m) (((m)&S_IFMT) == S_IFREG)
#endif
#if !defined(S_ISDIR) && defined(S_IFMT) && defined(S_IFDIR)
#define S_ISDIR(m) (((m)&S_IFMT) == S_IFDIR)
#endif
#if !defined(S_ISLNK)
#define S_ISLNK(m) 0
#endif
#endif
#undef min
#undef max
namespace nod {
/* define our own min/max to avoid MSVC BS */
template <typename T>
inline T min(T a, T b) {
return a < b ? a : b;
}
template <typename T>
inline T max(T a, T b) {
return a > b ? a : b;
}
/* Log Module */
extern logvisor::Module LogModule;
/* filesystem char type */
#if _WIN32 && UNICODE
#define NOD_UCS2 1
typedef struct _stat Sstat;
static inline int Mkdir(const wchar_t* path, int) { return _wmkdir(path); }
static inline int Stat(const wchar_t* path, Sstat* statout) { return _wstat(path, statout); }
#else
typedef struct stat Sstat;
static inline int Mkdir(const char* path, mode_t mode) { return mkdir(path, mode); }
static inline int Stat(const char* path, Sstat* statout) { return stat(path, statout); }
#endif
/* String-converting views */
#if NOD_UCS2
typedef wchar_t SystemChar;
typedef std::wstring SystemString;
typedef std::wstring_view SystemStringView;
static inline void ToLower(SystemString& str) { std::transform(str.begin(), str.end(), str.begin(), towlower); }
static inline void ToUpper(SystemString& str) { std::transform(str.begin(), str.end(), str.begin(), towupper); }
static inline size_t StrLen(const SystemChar* str) { return wcslen(str); }
class SystemUTF8Conv {
std::string m_utf8;
public:
explicit SystemUTF8Conv(SystemStringView str) {
int len = WideCharToMultiByte(CP_UTF8, 0, str.data(), str.size(), nullptr, 0, nullptr, nullptr);
m_utf8.assign(len, '\0');
WideCharToMultiByte(CP_UTF8, 0, str.data(), str.size(), &m_utf8[0], len, nullptr, nullptr);
}
inline std::string_view utf8_str() const { return m_utf8; }
inline const char* c_str() const { return m_utf8.c_str(); }
};
class SystemStringConv {
std::wstring m_sys;
public:
explicit SystemStringConv(std::string_view str) {
int len = MultiByteToWideChar(CP_UTF8, 0, str.data(), str.size(), nullptr, 0);
m_sys.assign(len, L'\0');
MultiByteToWideChar(CP_UTF8, 0, str.data(), str.size(), &m_sys[0], len);
}
inline SystemStringView sys_str() const { return m_sys; }
inline const SystemChar* c_str() const { return m_sys.c_str(); }
};
#ifndef _SYS_STR
#define _SYS_STR(val) L##val
#endif
#else
typedef char SystemChar;
typedef std::string SystemString;
typedef std::string_view SystemStringView;
static inline void ToLower(SystemString& str) { std::transform(str.begin(), str.end(), str.begin(), tolower); }
static inline void ToUpper(SystemString& str) { std::transform(str.begin(), str.end(), str.begin(), toupper); }
static inline size_t StrLen(const SystemChar* str) { return strlen(str); }
class SystemUTF8Conv {
std::string_view m_utf8;
public:
explicit SystemUTF8Conv(SystemStringView str) : m_utf8(str) {}
inline std::string_view utf8_str() const { return m_utf8; }
inline const char* c_str() const { return m_utf8.data(); }
};
class SystemStringConv {
SystemStringView m_sys;
public:
explicit SystemStringConv(std::string_view str) : m_sys(str) {}
inline SystemStringView sys_str() const { return m_sys; }
inline const SystemChar* c_str() const { return m_sys.data(); }
};
#ifndef _SYS_STR
#define _SYS_STR(val) val
#endif
#endif
static inline void Unlink(const SystemChar* file) {
#if NOD_UCS2
_wunlink(file);
#else
unlink(file);
#endif
}
static inline int StrCmp(const SystemChar* str1, const SystemChar* str2) {
#if NOD_UCS2
return wcscmp(str1, str2);
#else
return strcmp(str1, str2);
#endif
}
static inline int StrCaseCmp(const SystemChar* str1, const SystemChar* str2) {
#if NOD_UCS2
return _wcsicmp(str1, str2);
#else
return strcasecmp(str1, str2);
#endif
}
#undef bswap16
#undef bswap32
#undef bswap64
/* Type-sensitive byte swappers */
template <typename T>
static inline T bswap16(T val) {
#if __GNUC__
return __builtin_bswap16(val);
#elif _WIN32
return _byteswap_ushort(val);
#else
return (val = (val << 8) | ((val >> 8) & 0xFF));
#endif
}
template <typename T>
static inline T bswap32(T val) {
#if __GNUC__
return __builtin_bswap32(val);
#elif _WIN32
return _byteswap_ulong(val);
#else
val = (val & 0x0000FFFF) << 16 | (val & 0xFFFF0000) >> 16;
val = (val & 0x00FF00FF) << 8 | (val & 0xFF00FF00) >> 8;
return val;
#endif
}
template <typename T>
static inline T bswap64(T val) {
#if __GNUC__
return __builtin_bswap64(val);
#elif _WIN32
return _byteswap_uint64(val);
#else
return ((val & 0xFF00000000000000ULL) >> 56) | ((val & 0x00FF000000000000ULL) >> 40) |
((val & 0x0000FF0000000000ULL) >> 24) | ((val & 0x000000FF00000000ULL) >> 8) |
((val & 0x00000000FF000000ULL) << 8) | ((val & 0x0000000000FF0000ULL) << 24) |
((val & 0x000000000000FF00ULL) << 40) | ((val & 0x00000000000000FFULL) << 56);
#endif
}
#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
static inline int16_t SBig(int16_t val) { return bswap16(val); }
static inline uint16_t SBig(uint16_t val) { return bswap16(val); }
static inline int32_t SBig(int32_t val) { return bswap32(val); }
static inline uint32_t SBig(uint32_t val) { return bswap32(val); }
static inline int64_t SBig(int64_t val) { return bswap64(val); }
static inline uint64_t SBig(uint64_t val) { return bswap64(val); }
static inline int16_t SLittle(int16_t val) { return val; }
static inline uint16_t SLittle(uint16_t val) { return val; }
static inline int32_t SLittle(int32_t val) { return val; }
static inline uint32_t SLittle(uint32_t val) { return val; }
static inline int64_t SLittle(int64_t val) { return val; }
static inline uint64_t SLittle(uint64_t val) { return val; }
#else
static inline int16_t SLittle(int16_t val) { return bswap16(val); }
static inline uint16_t SLittle(uint16_t val) { return bswap16(val); }
static inline int32_t SLittle(int32_t val) { return bswap32(val); }
static inline uint32_t SLittle(uint32_t val) { return bswap32(val); }
static inline int64_t SLittle(int64_t val) { return bswap64(val); }
static inline uint64_t SLittle(uint64_t val) { return bswap64(val); }
static inline int16_t SBig(int16_t val) { return val; }
static inline uint16_t SBig(uint16_t val) { return val; }
static inline int32_t SBig(int32_t val) { return val; }
static inline uint32_t SBig(uint32_t val) { return val; }
static inline int64_t SBig(int64_t val) { return val; }
static inline uint64_t SBig(uint64_t val) { return val; }
#endif
#ifndef ROUND_UP_32
#define ROUND_UP_32(val) (((val) + 31) & ~31)
#define ROUND_UP_16(val) (((val) + 15) & ~15)
#endif
enum class FileLockType { None = 0, Read, Write };
static inline FILE* Fopen(const SystemChar* path, const SystemChar* mode, FileLockType lock = FileLockType::None) {
#if NOD_UCS2
FILE* fp = _wfopen(path, mode);
if (!fp)
return nullptr;
#else
FILE* fp = fopen(path, mode);
if (!fp)
return nullptr;
#endif
if (lock != FileLockType::None) {
#if _WIN32
OVERLAPPED ov = {};
LockFileEx((HANDLE)(uintptr_t)_fileno(fp), (lock == FileLockType::Write) ? LOCKFILE_EXCLUSIVE_LOCK : 0, 0, 0, 1,
&ov);
#else
if (flock(fileno(fp), ((lock == FileLockType::Write) ? LOCK_EX : LOCK_SH) | LOCK_NB))
LogModule.report(logvisor::Error, fmt("flock {}: {}"), path, strerror(errno));
#endif
}
return fp;
}
static inline int FSeek(FILE* fp, int64_t offset, int whence) {
#if _WIN32
return _fseeki64(fp, offset, whence);
#elif __APPLE__ || __FreeBSD__
return fseeko(fp, offset, whence);
#else
return fseeko64(fp, offset, whence);
#endif
}
static inline int64_t FTell(FILE* fp) {
#if _WIN32
return _ftelli64(fp);
#elif __APPLE__ || __FreeBSD__
return ftello(fp);
#else
return ftello64(fp);
#endif
}
static inline bool CheckFreeSpace(const SystemChar* path, size_t reqSz) {
#if _WIN32
ULARGE_INTEGER freeBytes;
wchar_t buf[1024];
wchar_t* end;
DWORD ret = GetFullPathNameW(path, 1024, buf, &end);
if (!ret || ret > 1024) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("GetFullPathNameW {}")), path);
return false;
}
if (end)
end[0] = L'\0';
if (!GetDiskFreeSpaceExW(buf, &freeBytes, nullptr, nullptr)) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("GetDiskFreeSpaceExW {}: {}")), path, GetLastError());
return false;
}
return reqSz < freeBytes.QuadPart;
#else
struct statvfs svfs;
if (statvfs(path, &svfs)) {
LogModule.report(logvisor::Error, fmt("statvfs {}: {}"), path, strerror(errno));
return false;
}
return reqSz < svfs.f_frsize * svfs.f_bavail;
#endif
}
} // namespace nod

View File

@@ -2,9 +2,7 @@
#include <functional> #include <functional>
#include <memory> #include <memory>
#include <string> #include <string_view>
#include "nod/Util.hpp"
namespace nod { namespace nod {
@@ -15,7 +13,7 @@ struct ExtractionContext final {
std::function<void(std::string_view, float)> progressCB; std::function<void(std::string_view, float)> progressCB;
}; };
std::unique_ptr<DiscBase> OpenDiscFromImage(SystemStringView path); std::unique_ptr<DiscBase> OpenDiscFromImage(std::string_view path);
std::unique_ptr<DiscBase> OpenDiscFromImage(SystemStringView path, bool& isWii); std::unique_ptr<DiscBase> OpenDiscFromImage(std::string_view path, bool& isWii);
} // namespace nod } // namespace nod

View File

@@ -6,36 +6,43 @@ add_library(nod
DiscBase.cpp DiscBase.cpp
DiscGCN.cpp DiscGCN.cpp
DiscIOISO.cpp DiscIOISO.cpp
DiscIONFS.cpp
DiscIOWBFS.cpp DiscIOWBFS.cpp
DiscWii.cpp DiscWii.cpp
IFileIO.cpp
nod.cpp nod.cpp
OSUTF.c
Util.cpp
Util.hpp
../include/nod/aes.hpp ../include/nod/aes.hpp
../include/nod/DirectoryEnumerator.hpp ../include/nod/DirectoryEnumerator.hpp
../include/nod/DiscBase.hpp ../include/nod/DiscBase.hpp
../include/nod/DiscGCN.hpp ../include/nod/DiscGCN.hpp
../include/nod/DiscWii.hpp ../include/nod/DiscWii.hpp
../include/nod/Endian.hpp
../include/nod/IDiscIO.hpp ../include/nod/IDiscIO.hpp
../include/nod/IFileIO.hpp ../include/nod/IFileIO.hpp
../include/nod/nod.hpp ../include/nod/nod.hpp
../include/nod/OSUTF.h
../include/nod/sha1.h ../include/nod/sha1.h
../include/nod/Util.hpp
) )
target_include_directories(nod PUBLIC target_include_directories(nod PUBLIC
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/../include> $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/../include>
) )
target_link_libraries(nod PUBLIC logvisor) target_link_libraries(nod PUBLIC $<BUILD_INTERFACE:spdlog::spdlog>)
if(WIN32) if(WIN32)
target_sources(nod PRIVATE FileIOWin32.cpp) target_sources(nod PRIVATE FileIOWin32.cpp)
target_link_libraries(nod PRIVATE nowide::nowide)
else() else()
target_compile_options(nod PRIVATE -Wno-multichar) target_compile_options(nod PRIVATE -Wno-multichar)
target_sources(nod PRIVATE FileIOFILE.cpp) target_sources(nod PRIVATE FileIOFILE.cpp)
endif() endif()
if(NOT MSVC AND NOT NX) if(NOT MSVC AND ${CMAKE_SYSTEM_PROCESSOR} STREQUAL x86_64)
set_source_files_properties(aes.cpp PROPERTIES COMPILE_FLAGS -maes) set_source_files_properties(aes.cpp PROPERTIES COMPILE_FLAGS -maes)
endif() endif()

View File

@@ -1,10 +1,14 @@
#include "nod/DirectoryEnumerator.hpp" #include "nod/DirectoryEnumerator.hpp"
#include "Util.hpp"
#ifdef _WIN32 #ifdef _WIN32
#ifndef WIN32_LEAN_AND_MEAN
#define WIN32_LEAN_AND_MEAN
#endif
#include <windows.h> #include <windows.h>
#else #else
#include <sys/stat.h>
#include <dirent.h> #include <dirent.h>
#include <sys/stat.h>
#endif #endif
#include <cstring> #include <cstring>
@@ -12,124 +16,153 @@
namespace nod { namespace nod {
DirectoryEnumerator::DirectoryEnumerator(SystemStringView path, Mode mode, bool sizeSort, bool reverse, bool noHidden) { DirectoryEnumerator::DirectoryEnumerator(std::string_view path, Mode mode, bool sizeSort, bool reverse, bool noHidden) {
Sstat theStat; Sstat theStat;
if (Stat(path.data(), &theStat) || !S_ISDIR(theStat.st_mode)) if (Stat(path.data(), &theStat) || !S_ISDIR(theStat.st_mode)) {
return; return;
}
#if _WIN32 #if _WIN32
SystemString wc(path); std::wstring wc = nowide::widen(path);
wc += _SYS_STR("/*"); wc += L"/*";
WIN32_FIND_DATAW d; WIN32_FIND_DATAW d;
HANDLE dir = FindFirstFileW(wc.c_str(), &d); HANDLE dir = FindFirstFileW(wc.c_str(), &d);
if (dir == INVALID_HANDLE_VALUE) if (dir == INVALID_HANDLE_VALUE) {
return; return;
}
switch (mode) { switch (mode) {
case Mode::Native: case Mode::Native:
do { do {
if (!wcscmp(d.cFileName, _SYS_STR(".")) || !wcscmp(d.cFileName, _SYS_STR(".."))) if (!wcscmp(d.cFileName, L".") || !wcscmp(d.cFileName, L"..")) {
continue; continue;
if (noHidden && (d.cFileName[0] == L'.' || (d.dwFileAttributes & FILE_ATTRIBUTE_HIDDEN) != 0)) }
if (noHidden && (d.cFileName[0] == L'.' || (d.dwFileAttributes & FILE_ATTRIBUTE_HIDDEN) != 0)) {
continue; continue;
SystemString fp(path); }
fp += _SYS_STR('/'); std::string fileName = nowide::narrow(d.cFileName);
fp += d.cFileName; std::string fp(path);
fp += '/';
fp += fileName;
Sstat st; Sstat st;
if (Stat(fp.c_str(), &st)) if (Stat(fp.c_str(), &st))
continue; continue;
size_t sz = 0; size_t sz = 0;
bool isDir = false; bool isDir = false;
if (S_ISDIR(st.st_mode)) if (S_ISDIR(st.st_mode)) {
isDir = true; isDir = true;
else if (S_ISREG(st.st_mode)) } else if (S_ISREG(st.st_mode)) {
sz = st.st_size; sz = st.st_size;
else } else {
continue; continue;
}
m_entries.push_back(Entry(fp, d.cFileName, sz, isDir)); m_entries.emplace_back(fp, fileName, sz, isDir);
} while (FindNextFileW(dir, &d)); } while (FindNextFileW(dir, &d));
break; break;
case Mode::DirsThenFilesSorted: case Mode::DirsThenFilesSorted:
case Mode::DirsSorted: { case Mode::DirsSorted: {
std::map<SystemString, Entry, CaseInsensitiveCompare> sort; std::map<std::string, Entry, CaseInsensitiveCompare> sort;
do { do {
if (!wcscmp(d.cFileName, _SYS_STR(".")) || !wcscmp(d.cFileName, _SYS_STR(".."))) if (!wcscmp(d.cFileName, L".") || !wcscmp(d.cFileName, L"..")) {
continue; continue;
if (noHidden && (d.cFileName[0] == L'.' || (d.dwFileAttributes & FILE_ATTRIBUTE_HIDDEN) != 0)) }
if (noHidden && (d.cFileName[0] == L'.' || (d.dwFileAttributes & FILE_ATTRIBUTE_HIDDEN) != 0)) {
continue; continue;
SystemString fp(path); }
fp += _SYS_STR('/'); std::string fileName = nowide::narrow(d.cFileName);
fp += d.cFileName; std::string fp(path);
fp += '/';
fp += fileName;
Sstat st; Sstat st;
if (Stat(fp.c_str(), &st) || !S_ISDIR(st.st_mode)) if (Stat(fp.c_str(), &st) || !S_ISDIR(st.st_mode)) {
continue; continue;
sort.emplace(std::make_pair(d.cFileName, Entry(fp, d.cFileName, 0, true))); }
sort.emplace(fileName, Entry{fp, fileName, 0, true});
} while (FindNextFileW(dir, &d)); } while (FindNextFileW(dir, &d));
m_entries.reserve(sort.size()); m_entries.reserve(sort.size());
if (reverse) if (reverse) {
for (auto it = sort.crbegin(); it != sort.crend(); ++it) for (auto it = sort.crbegin(); it != sort.crend(); ++it) {
m_entries.push_back(std::move(it->second)); m_entries.emplace_back(std::move(it->second));
else }
for (auto& e : sort) } else {
m_entries.push_back(std::move(e.second)); for (auto& e : sort) {
m_entries.emplace_back(std::move(e.second));
}
}
if (mode == Mode::DirsSorted) if (mode == Mode::DirsSorted) {
break; break;
}
FindClose(dir); FindClose(dir);
dir = FindFirstFileW(wc.c_str(), &d); dir = FindFirstFileW(wc.c_str(), &d);
} }
case Mode::FilesSorted: { case Mode::FilesSorted: {
if (mode == Mode::FilesSorted) if (mode == Mode::FilesSorted) {
m_entries.clear(); m_entries.clear();
}
if (sizeSort) { if (sizeSort) {
std::multimap<size_t, Entry> sort; std::multimap<size_t, Entry> sort;
do { do {
if (!wcscmp(d.cFileName, _SYS_STR(".")) || !wcscmp(d.cFileName, _SYS_STR(".."))) if (!wcscmp(d.cFileName, L".") || !wcscmp(d.cFileName, L"..")) {
continue; continue;
if (noHidden && (d.cFileName[0] == L'.' || (d.dwFileAttributes & FILE_ATTRIBUTE_HIDDEN) != 0)) }
if (noHidden && (d.cFileName[0] == L'.' || (d.dwFileAttributes & FILE_ATTRIBUTE_HIDDEN) != 0)) {
continue; continue;
SystemString fp(path); }
fp += _SYS_STR('/'); std::string fileName = nowide::narrow(d.cFileName);
fp += d.cFileName; std::string fp(path);
fp += '/';
fp += fileName;
Sstat st; Sstat st;
if (Stat(fp.c_str(), &st) || !S_ISREG(st.st_mode)) if (Stat(fp.c_str(), &st) || !S_ISREG(st.st_mode)) {
continue; continue;
sort.emplace(std::make_pair(st.st_size, Entry(fp, d.cFileName, st.st_size, false))); }
sort.emplace(st.st_size, Entry{fp, fileName, static_cast<size_t>(st.st_size), false});
} while (FindNextFileW(dir, &d)); } while (FindNextFileW(dir, &d));
m_entries.reserve(sort.size()); m_entries.reserve(m_entries.size() + sort.size());
if (reverse) if (reverse) {
for (auto it = sort.crbegin(); it != sort.crend(); ++it) for (auto it = sort.crbegin(); it != sort.crend(); ++it) {
m_entries.push_back(std::move(it->second)); m_entries.emplace_back(std::move(it->second));
else }
for (auto& e : sort)
m_entries.push_back(std::move(e.second));
} else { } else {
std::map<SystemString, Entry, CaseInsensitiveCompare> sort; for (auto& e : sort) {
m_entries.emplace_back(std::move(e.second));
}
}
} else {
std::map<std::string, Entry, CaseInsensitiveCompare> sort;
do { do {
if (!wcscmp(d.cFileName, _SYS_STR(".")) || !wcscmp(d.cFileName, _SYS_STR(".."))) if (!wcscmp(d.cFileName, L".") || !wcscmp(d.cFileName, L"..")) {
continue; continue;
if (noHidden && (d.cFileName[0] == L'.' || (d.dwFileAttributes & FILE_ATTRIBUTE_HIDDEN) != 0)) }
if (noHidden && (d.cFileName[0] == L'.' || (d.dwFileAttributes & FILE_ATTRIBUTE_HIDDEN) != 0)) {
continue; continue;
SystemString fp(path); }
fp += _SYS_STR('/'); std::string fileName = nowide::narrow(d.cFileName);
fp += d.cFileName; std::string fp(path);
fp += '/';
fp += fileName;
Sstat st; Sstat st;
if (Stat(fp.c_str(), &st) || !S_ISREG(st.st_mode)) if (Stat(fp.c_str(), &st) || !S_ISREG(st.st_mode)) {
continue; continue;
sort.emplace(std::make_pair(d.cFileName, Entry(fp, d.cFileName, st.st_size, false))); }
sort.emplace(fileName, Entry{fp, fileName, static_cast<size_t>(st.st_size), false});
} while (FindNextFileW(dir, &d)); } while (FindNextFileW(dir, &d));
m_entries.reserve(sort.size()); m_entries.reserve(m_entries.size() + sort.size());
if (reverse) if (reverse) {
for (auto it = sort.crbegin(); it != sort.crend(); ++it) for (auto it = sort.crbegin(); it != sort.crend(); ++it) {
m_entries.push_back(std::move(it->second)); m_entries.emplace_back(std::move(it->second));
else }
for (auto& e : sort) } else {
m_entries.push_back(std::move(e.second)); for (auto& e : sort) {
m_entries.emplace_back(std::move(e.second));
}
}
} }
break; break;
@@ -150,7 +183,7 @@ DirectoryEnumerator::DirectoryEnumerator(SystemStringView path, Mode mode, bool
continue; continue;
if (noHidden && d->d_name[0] == '.') if (noHidden && d->d_name[0] == '.')
continue; continue;
SystemString fp(path); std::string fp(path);
fp += '/'; fp += '/';
fp += d->d_name; fp += d->d_name;
Sstat st; Sstat st;
@@ -171,13 +204,13 @@ DirectoryEnumerator::DirectoryEnumerator(SystemStringView path, Mode mode, bool
break; break;
case Mode::DirsThenFilesSorted: case Mode::DirsThenFilesSorted:
case Mode::DirsSorted: { case Mode::DirsSorted: {
std::map<SystemString, Entry, CaseInsensitiveCompare> sort; std::map<std::string, Entry, CaseInsensitiveCompare> sort;
while ((d = readdir(dir))) { while ((d = readdir(dir))) {
if (!strcmp(d->d_name, ".") || !strcmp(d->d_name, "..")) if (!strcmp(d->d_name, ".") || !strcmp(d->d_name, ".."))
continue; continue;
if (noHidden && d->d_name[0] == '.') if (noHidden && d->d_name[0] == '.')
continue; continue;
SystemString fp(path); std::string fp(path);
fp += '/'; fp += '/';
fp += d->d_name; fp += d->d_name;
Sstat st; Sstat st;
@@ -210,7 +243,7 @@ DirectoryEnumerator::DirectoryEnumerator(SystemStringView path, Mode mode, bool
continue; continue;
if (noHidden && d->d_name[0] == '.') if (noHidden && d->d_name[0] == '.')
continue; continue;
SystemString fp(path); std::string fp(path);
fp += '/'; fp += '/';
fp += d->d_name; fp += d->d_name;
Sstat st; Sstat st;
@@ -227,13 +260,13 @@ DirectoryEnumerator::DirectoryEnumerator(SystemStringView path, Mode mode, bool
for (auto& e : sort) for (auto& e : sort)
m_entries.push_back(std::move(e.second)); m_entries.push_back(std::move(e.second));
} else { } else {
std::map<SystemString, Entry, CaseInsensitiveCompare> sort; std::map<std::string, Entry, CaseInsensitiveCompare> sort;
while ((d = readdir(dir))) { while ((d = readdir(dir))) {
if (!strcmp(d->d_name, ".") || !strcmp(d->d_name, "..")) if (!strcmp(d->d_name, ".") || !strcmp(d->d_name, ".."))
continue; continue;
if (noHidden && d->d_name[0] == '.') if (noHidden && d->d_name[0] == '.')
continue; continue;
SystemString fp(path); std::string fp(path);
fp += '/'; fp += '/';
fp += d->d_name; fp += d->d_name;
Sstat st; Sstat st;

View File

@@ -9,13 +9,13 @@
#include "nod/DirectoryEnumerator.hpp" #include "nod/DirectoryEnumerator.hpp"
#include "nod/IFileIO.hpp" #include "nod/IFileIO.hpp"
#include "nod/nod.hpp" #include "nod/nod.hpp"
#include "nod/Util.hpp" #include "Util.hpp"
#ifndef _WIN32 #ifndef _WIN32
#include <unistd.h> #include <unistd.h>
#else #else
#include <logvisor/logvisor.hpp> #include <spdlog/spdlog.h>
static void* memmem(const void* haystack, size_t hlen, const void* needle, size_t nlen) { static void* memmem(const void* haystack, size_t hlen, const void* needle, size_t nlen) {
int needle_first; int needle_first;
@@ -40,17 +40,18 @@ static void* memmem(const void* haystack, size_t hlen, const void* needle, size_
#endif #endif
#include <algorithm> #include <algorithm>
#include <spdlog/spdlog.h>
namespace nod { namespace nod {
const SystemChar* getKindString(PartitionKind kind) { const char* getKindString(PartitionKind kind) {
switch (kind) { switch (kind) {
case PartitionKind::Data: case PartitionKind::Data:
return _SYS_STR("DATA"); return "DATA";
case PartitionKind::Update: case PartitionKind::Update:
return _SYS_STR("UPDATE"); return "UPDATE";
case PartitionKind::Channel: case PartitionKind::Channel:
return _SYS_STR("CHANNEL"); return "CHANNEL";
default: default:
return nullptr; return nullptr;
} }
@@ -109,7 +110,7 @@ Node::Node(const IPartition& parent, const FSTNode& node, std::string_view name)
std::unique_ptr<IPartReadStream> Node::beginReadStream(uint64_t offset) const { std::unique_ptr<IPartReadStream> Node::beginReadStream(uint64_t offset) const {
if (m_kind != Kind::File) { if (m_kind != Kind::File) {
LogModule.report(logvisor::Error, fmt("unable to stream a non-file {}"), m_name); spdlog::error("unable to stream a non-file {}", m_name);
return std::unique_ptr<IPartReadStream>(); return std::unique_ptr<IPartReadStream>();
} }
return m_parent.beginReadStream(m_discOffset + offset); return m_parent.beginReadStream(m_discOffset + offset);
@@ -117,7 +118,7 @@ std::unique_ptr<IPartReadStream> Node::beginReadStream(uint64_t offset) const {
std::unique_ptr<uint8_t[]> Node::getBuf() const { std::unique_ptr<uint8_t[]> Node::getBuf() const {
if (m_kind != Kind::File) { if (m_kind != Kind::File) {
LogModule.report(logvisor::Error, fmt("unable to buffer a non-file {}"), m_name); spdlog::error("unable to buffer a non-file {}", m_name);
return nullptr; return nullptr;
} }
@@ -126,16 +127,16 @@ std::unique_ptr<uint8_t[]> Node::getBuf() const {
return buf; return buf;
} }
bool Node::extractToDirectory(SystemStringView basePath, const ExtractionContext& ctx) const { bool Node::extractToDirectory(std::string_view basePath, const ExtractionContext& ctx) const {
SystemStringConv nameView(getName()); SJISToUTF8 nameView(getName());
SystemString path = SystemString(basePath) + _SYS_STR('/') + nameView.sys_str().data(); std::string path = std::string(basePath) + '/' + nameView.str().data();
if (m_kind == Kind::Directory) { if (m_kind == Kind::Directory) {
++m_parent.m_curNodeIdx; ++m_parent.m_curNodeIdx;
if (ctx.progressCB && !getName().empty()) if (ctx.progressCB && !getName().empty())
ctx.progressCB(getName(), m_parent.m_curNodeIdx / float(m_parent.getNodeCount())); ctx.progressCB(nameView.str(), m_parent.m_curNodeIdx / float(m_parent.getNodeCount()));
if (Mkdir(path.c_str(), 0755) && errno != EEXIST) { if (Mkdir(path.c_str(), 0755) && errno != EEXIST) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("unable to mkdir '{}'")), path); spdlog::error("unable to mkdir '{}'", path);
return false; return false;
} }
for (Node& subnode : *this) for (Node& subnode : *this)
@@ -144,7 +145,7 @@ bool Node::extractToDirectory(SystemStringView basePath, const ExtractionContext
} else if (m_kind == Kind::File) { } else if (m_kind == Kind::File) {
Sstat theStat; Sstat theStat;
if (ctx.progressCB) if (ctx.progressCB)
ctx.progressCB(getName(), m_parent.m_curNodeIdx / float(m_parent.getNodeCount())); ctx.progressCB(nameView.str(), m_parent.m_curNodeIdx / float(m_parent.getNodeCount()));
if (ctx.force || Stat(path.c_str(), &theStat)) { if (ctx.force || Stat(path.c_str(), &theStat)) {
std::unique_ptr<IPartReadStream> rs = beginReadStream(); std::unique_ptr<IPartReadStream> rs = beginReadStream();
@@ -153,7 +154,7 @@ bool Node::extractToDirectory(SystemStringView basePath, const ExtractionContext
return false; return false;
ws->copyFromDisc(*rs, m_discLength, [&](float prog) { ws->copyFromDisc(*rs, m_discLength, [&](float prog) {
if (ctx.progressCB) if (ctx.progressCB)
ctx.progressCB(getName(), (m_parent.m_curNodeIdx + prog) / float(m_parent.getNodeCount())); ctx.progressCB(nameView.str(), (m_parent.m_curNodeIdx + prog) / float(m_parent.getNodeCount()));
}); });
} }
++m_parent.m_curNodeIdx; ++m_parent.m_curNodeIdx;
@@ -161,17 +162,17 @@ bool Node::extractToDirectory(SystemStringView basePath, const ExtractionContext
return true; return true;
} }
bool IPartition::extractToDirectory(SystemStringView path, const ExtractionContext& ctx) { bool IPartition::extractToDirectory(std::string_view path, const ExtractionContext& ctx) {
m_curNodeIdx = 0; m_curNodeIdx = 0;
if (Mkdir(path.data(), 0755) && errno != EEXIST) { if (Mkdir(path.data(), 0755) && errno != EEXIST) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("unable to mkdir '{}'")), path); spdlog::error("unable to mkdir '{}'", path);
return false; return false;
} }
SystemString basePath = m_isWii ? SystemString(path) + _SYS_STR("/") + getKindString(m_kind) : SystemString(path); std::string basePath = m_isWii ? std::string(path) + "/" + getKindString(m_kind) : std::string(path);
if (m_isWii) { if (m_isWii) {
if (Mkdir(basePath.c_str(), 0755) && errno != EEXIST) { if (Mkdir(basePath.c_str(), 0755) && errno != EEXIST) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("unable to mkdir '{}'")), basePath); spdlog::error("unable to mkdir '{}'", basePath);
return false; return false;
} }
} }
@@ -188,25 +189,25 @@ bool IPartition::extractToDirectory(SystemStringView path, const ExtractionConte
return false; return false;
/* Extract Filesystem */ /* Extract Filesystem */
SystemString fsPath = basePath + _SYS_STR("/files"); std::string fsPath = basePath + "/files";
if (Mkdir(fsPath.c_str(), 0755) && errno != EEXIST) { if (Mkdir(fsPath.c_str(), 0755) && errno != EEXIST) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("unable to mkdir '{}'")), fsPath); spdlog::error("unable to mkdir '{}'", fsPath);
return false; return false;
} }
return m_nodes[0].extractToDirectory(fsPath, ctx); return m_nodes[0].extractToDirectory(fsPath, ctx);
} }
bool IPartition::extractSysFiles(SystemStringView basePath, const ExtractionContext& ctx) const { bool IPartition::extractSysFiles(std::string_view basePath, const ExtractionContext& ctx) const {
SystemString basePathStr(basePath); std::string basePathStr(basePath);
if (Mkdir((basePathStr + _SYS_STR("/sys")).c_str(), 0755) && errno != EEXIST) { if (Mkdir((basePathStr + "/sys").c_str(), 0755) && errno != EEXIST) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("unable to mkdir '{}/sys'")), basePath); spdlog::error("unable to mkdir '{}/sys'", basePath);
return false; return false;
} }
Sstat theStat; Sstat theStat;
/* Extract Apploader */ /* Extract Apploader */
SystemString apploaderPath = basePathStr + _SYS_STR("/sys/apploader.img"); std::string apploaderPath = basePathStr + "/sys/apploader.img";
if (ctx.force || Stat(apploaderPath.c_str(), &theStat)) { if (ctx.force || Stat(apploaderPath.c_str(), &theStat)) {
if (ctx.progressCB) if (ctx.progressCB)
ctx.progressCB("apploader.bin", 0.f); ctx.progressCB("apploader.bin", 0.f);
@@ -218,7 +219,7 @@ bool IPartition::extractSysFiles(SystemStringView basePath, const ExtractionCont
} }
/* Extract Dol */ /* Extract Dol */
SystemString dolPath = basePathStr + _SYS_STR("/sys/main.dol"); std::string dolPath = basePathStr + "/sys/main.dol";
if (ctx.force || Stat(dolPath.c_str(), &theStat)) { if (ctx.force || Stat(dolPath.c_str(), &theStat)) {
if (ctx.progressCB) if (ctx.progressCB)
ctx.progressCB("main.dol", 0.f); ctx.progressCB("main.dol", 0.f);
@@ -230,7 +231,7 @@ bool IPartition::extractSysFiles(SystemStringView basePath, const ExtractionCont
} }
/* Extract Boot info */ /* Extract Boot info */
SystemString bootPath = basePathStr + _SYS_STR("/sys/boot.bin"); std::string bootPath = basePathStr + "/sys/boot.bin";
if (ctx.force || Stat(bootPath.c_str(), &theStat)) { if (ctx.force || Stat(bootPath.c_str(), &theStat)) {
if (ctx.progressCB) if (ctx.progressCB)
ctx.progressCB("boot.bin", 0.f); ctx.progressCB("boot.bin", 0.f);
@@ -241,7 +242,7 @@ bool IPartition::extractSysFiles(SystemStringView basePath, const ExtractionCont
} }
/* Extract BI2 info */ /* Extract BI2 info */
SystemString bi2Path = basePathStr + _SYS_STR("/sys/bi2.bin"); std::string bi2Path = basePathStr + "/sys/bi2.bin";
if (ctx.force || Stat(bi2Path.c_str(), &theStat)) { if (ctx.force || Stat(bi2Path.c_str(), &theStat)) {
if (ctx.progressCB) if (ctx.progressCB)
ctx.progressCB("bi2.bin", 0.f); ctx.progressCB("bi2.bin", 0.f);
@@ -255,26 +256,26 @@ bool IPartition::extractSysFiles(SystemStringView basePath, const ExtractionCont
return true; return true;
} }
static bool IsSystemFile(SystemStringView name, bool& isDol) { static bool IsSystemFile(std::string_view name, bool& isDol) {
isDol = false; isDol = false;
if (name.size() < 4) if (name.size() < 4)
return false; return false;
if (!StrCaseCmp((&*(name.cend() - 4)), _SYS_STR(".dol"))) { if (!StrCaseCmp((&*(name.cend() - 4)), ".dol")) {
isDol = true; isDol = true;
return true; return true;
} }
if (!StrCaseCmp((&*(name.cend() - 4)), _SYS_STR(".rel"))) if (!StrCaseCmp((&*(name.cend() - 4)), ".rel"))
return true; return true;
if (!StrCaseCmp((&*(name.cend() - 4)), _SYS_STR(".rso"))) if (!StrCaseCmp((&*(name.cend() - 4)), ".rso"))
return true; return true;
if (!StrCaseCmp((&*(name.cend() - 4)), _SYS_STR(".sel"))) if (!StrCaseCmp((&*(name.cend() - 4)), ".sel"))
return true; return true;
if (!StrCaseCmp((&*(name.cend() - 4)), _SYS_STR(".bnr"))) if (!StrCaseCmp((&*(name.cend() - 4)), ".bnr"))
return true; return true;
if (!StrCaseCmp((&*(name.cend() - 4)), _SYS_STR(".elf"))) if (!StrCaseCmp((&*(name.cend() - 4)), ".elf"))
return true; return true;
if (!StrCaseCmp((&*(name.cend() - 4)), _SYS_STR(".wad"))) if (!StrCaseCmp((&*(name.cend() - 4)), ".wad"))
return true; return true;
return false; return false;
@@ -301,7 +302,7 @@ static size_t PatchDOL(IFileIO::IReadStream& in, IPartWriteStream& out, size_t s
return out.write(buf.get(), sz); return out.write(buf.get(), sz);
} }
void DiscBuilderBase::PartitionBuilderBase::recursiveBuildNodesPre(SystemStringView filesIn) { void DiscBuilderBase::PartitionBuilderBase::recursiveBuildNodesPre(std::string_view filesIn) {
DirectoryEnumerator dEnum(filesIn, DirectoryEnumerator::Mode::DirsThenFilesSorted, false, false, true); DirectoryEnumerator dEnum(filesIn, DirectoryEnumerator::Mode::DirsThenFilesSorted, false, false, true);
for (const DirectoryEnumerator::Entry& e : dEnum) { for (const DirectoryEnumerator::Entry& e : dEnum) {
if (e.m_isDir) if (e.m_isDir)
@@ -312,7 +313,7 @@ void DiscBuilderBase::PartitionBuilderBase::recursiveBuildNodesPre(SystemStringV
} }
bool DiscBuilderBase::PartitionBuilderBase::recursiveBuildNodes(IPartWriteStream& ws, bool system, bool DiscBuilderBase::PartitionBuilderBase::recursiveBuildNodes(IPartWriteStream& ws, bool system,
SystemStringView filesIn) { std::string_view filesIn) {
DirectoryEnumerator dEnum(filesIn, DirectoryEnumerator::Mode::DirsThenFilesSorted, false, false, true); DirectoryEnumerator dEnum(filesIn, DirectoryEnumerator::Mode::DirsThenFilesSorted, false, false, true);
for (const DirectoryEnumerator::Entry& e : dEnum) { for (const DirectoryEnumerator::Entry& e : dEnum) {
if (e.m_isDir) { if (e.m_isDir) {
@@ -337,7 +338,7 @@ bool DiscBuilderBase::PartitionBuilderBase::recursiveBuildNodes(IPartWriteStream
bool patched; bool patched;
xferSz = PatchDOL(*rs, ws, e.m_fileSz, patched); xferSz = PatchDOL(*rs, ws, e.m_fileSz, patched);
m_parent.m_progressCB(m_parent.getProgressFactor(), m_parent.m_progressCB(m_parent.getProgressFactor(),
e.m_name + (patched ? _SYS_STR(" [PATCHED]") : _SYS_STR("")), xferSz); e.m_name + (patched ? " [PATCHED]" : ""), xferSz);
++m_parent.m_progressIdx; ++m_parent.m_progressIdx;
} else { } else {
char buf[0x8000]; char buf[0x8000];
@@ -359,7 +360,7 @@ bool DiscBuilderBase::PartitionBuilderBase::recursiveBuildNodes(IPartWriteStream
return true; return true;
} }
bool DiscBuilderBase::PartitionBuilderBase::recursiveBuildFST(SystemStringView filesIn, bool DiscBuilderBase::PartitionBuilderBase::recursiveBuildFST(std::string_view filesIn,
std::function<void(void)> incParents, std::function<void(void)> incParents,
size_t parentDirIdx) { size_t parentDirIdx) {
DirectoryEnumerator dEnum(filesIn, DirectoryEnumerator::Mode::DirsThenFilesSorted, false, false, true); DirectoryEnumerator dEnum(filesIn, DirectoryEnumerator::Mode::DirsThenFilesSorted, false, false, true);
@@ -387,7 +388,7 @@ bool DiscBuilderBase::PartitionBuilderBase::recursiveBuildFST(SystemStringView f
return true; return true;
} }
void DiscBuilderBase::PartitionBuilderBase::recursiveMergeNodesPre(const Node* nodeIn, SystemStringView dirIn) { void DiscBuilderBase::PartitionBuilderBase::recursiveMergeNodesPre(const Node* nodeIn, std::string_view dirIn) {
/* Build map of existing nodes to write-through later */ /* Build map of existing nodes to write-through later */
std::unordered_map<std::string, const Node*> fileNodes; std::unordered_map<std::string, const Node*> fileNodes;
std::unordered_map<std::string, const Node*> dirNodes; std::unordered_map<std::string, const Node*> dirNodes;
@@ -406,10 +407,9 @@ void DiscBuilderBase::PartitionBuilderBase::recursiveMergeNodesPre(const Node* n
if (!dirIn.empty()) { if (!dirIn.empty()) {
DirectoryEnumerator dEnum(dirIn, DirectoryEnumerator::Mode::DirsThenFilesSorted, false, false, true); DirectoryEnumerator dEnum(dirIn, DirectoryEnumerator::Mode::DirsThenFilesSorted, false, false, true);
for (const DirectoryEnumerator::Entry& e : dEnum) { for (const DirectoryEnumerator::Entry& e : dEnum) {
SystemUTF8Conv nameView(e.m_name); UTF8ToSJIS nameView(e.m_name);
if (e.m_isDir) { if (e.m_isDir) {
auto search = dirNodes.find(nameView.utf8_str().data()); auto search = dirNodes.find(nameView.str());
if (search != dirNodes.cend()) { if (search != dirNodes.cend()) {
recursiveMergeNodesPre(search->second, e.m_path.c_str()); recursiveMergeNodesPre(search->second, e.m_path.c_str());
dirNodes.erase(search); dirNodes.erase(search);
@@ -417,7 +417,7 @@ void DiscBuilderBase::PartitionBuilderBase::recursiveMergeNodesPre(const Node* n
recursiveMergeNodesPre(nullptr, e.m_path.c_str()); recursiveMergeNodesPre(nullptr, e.m_path.c_str());
} }
} else { } else {
fileNodes.erase(nameView.utf8_str().data()); fileNodes.erase(nameView.str());
++m_parent.m_progressTotal; ++m_parent.m_progressTotal;
} }
} }
@@ -433,7 +433,7 @@ void DiscBuilderBase::PartitionBuilderBase::recursiveMergeNodesPre(const Node* n
} }
bool DiscBuilderBase::PartitionBuilderBase::recursiveMergeNodes(IPartWriteStream& ws, bool system, const Node* nodeIn, bool DiscBuilderBase::PartitionBuilderBase::recursiveMergeNodes(IPartWriteStream& ws, bool system, const Node* nodeIn,
SystemStringView dirIn, SystemStringView keyPath) { std::string_view dirIn, std::string_view keyPath) {
/* Build map of existing nodes to write-through later */ /* Build map of existing nodes to write-through later */
std::unordered_map<std::string, const Node*> fileNodes; std::unordered_map<std::string, const Node*> fileNodes;
std::unordered_map<std::string, const Node*> dirNodes; std::unordered_map<std::string, const Node*> dirNodes;
@@ -452,11 +452,11 @@ bool DiscBuilderBase::PartitionBuilderBase::recursiveMergeNodes(IPartWriteStream
if (!dirIn.empty()) { if (!dirIn.empty()) {
DirectoryEnumerator dEnum(dirIn, DirectoryEnumerator::Mode::DirsThenFilesSorted, false, false, true); DirectoryEnumerator dEnum(dirIn, DirectoryEnumerator::Mode::DirsThenFilesSorted, false, false, true);
for (const DirectoryEnumerator::Entry& e : dEnum) { for (const DirectoryEnumerator::Entry& e : dEnum) {
SystemUTF8Conv nameView(e.m_name); UTF8ToSJIS nameView(e.m_name);
SystemString chKeyPath = SystemString(keyPath) + _SYS_STR('/') + e.m_name; std::string chKeyPath = std::string(keyPath) + '/' + e.m_name;
if (e.m_isDir) { if (e.m_isDir) {
auto search = dirNodes.find(nameView.utf8_str().data()); auto search = dirNodes.find(nameView.str());
if (search != dirNodes.cend()) { if (search != dirNodes.cend()) {
if (!recursiveMergeNodes(ws, system, search->second, e.m_path.c_str(), chKeyPath)) if (!recursiveMergeNodes(ws, system, search->second, e.m_path.c_str(), chKeyPath))
return false; return false;
@@ -471,7 +471,7 @@ bool DiscBuilderBase::PartitionBuilderBase::recursiveMergeNodes(IPartWriteStream
if (system ^ isSys) if (system ^ isSys)
continue; continue;
fileNodes.erase(nameView.utf8_str().data()); fileNodes.erase(nameView.str());
size_t fileSz = ROUND_UP_32(e.m_fileSz); size_t fileSz = ROUND_UP_32(e.m_fileSz);
uint64_t fileOff = userAllocate(fileSz, ws); uint64_t fileOff = userAllocate(fileSz, ws);
@@ -486,7 +486,7 @@ bool DiscBuilderBase::PartitionBuilderBase::recursiveMergeNodes(IPartWriteStream
bool patched; bool patched;
xferSz = PatchDOL(*rs, ws, e.m_fileSz, patched); xferSz = PatchDOL(*rs, ws, e.m_fileSz, patched);
m_parent.m_progressCB(m_parent.getProgressFactor(), m_parent.m_progressCB(m_parent.getProgressFactor(),
e.m_name + (patched ? _SYS_STR(" [PATCHED]") : _SYS_STR("")), xferSz); e.m_name + (patched ? " [PATCHED]" : ""), xferSz);
++m_parent.m_progressIdx; ++m_parent.m_progressIdx;
} else { } else {
char buf[0x8000]; char buf[0x8000];
@@ -508,8 +508,8 @@ bool DiscBuilderBase::PartitionBuilderBase::recursiveMergeNodes(IPartWriteStream
/* Write-through remaining dir nodes */ /* Write-through remaining dir nodes */
for (const auto& p : dirNodes) { for (const auto& p : dirNodes) {
SystemStringConv sysName(p.second->getName()); SJISToUTF8 sysName(p.second->getName());
SystemString chKeyPath = SystemString(keyPath) + _SYS_STR('/') + sysName.c_str(); std::string chKeyPath = std::string(keyPath) + '/' + sysName.str();
if (!recursiveMergeNodes(ws, system, p.second, {}, chKeyPath)) if (!recursiveMergeNodes(ws, system, p.second, {}, chKeyPath))
return false; return false;
} }
@@ -517,11 +517,11 @@ bool DiscBuilderBase::PartitionBuilderBase::recursiveMergeNodes(IPartWriteStream
/* Write-through remaining file nodes */ /* Write-through remaining file nodes */
for (const auto& p : fileNodes) { for (const auto& p : fileNodes) {
const Node& ch = *p.second; const Node& ch = *p.second;
SystemStringConv sysName(ch.getName()); SJISToUTF8 sysName(ch.getName());
SystemString chKeyPath = SystemString(keyPath) + _SYS_STR('/') + sysName.c_str(); std::string chKeyPath = std::string(keyPath) + '/' + sysName.str();
bool isDol; bool isDol;
bool isSys = IsSystemFile(sysName.sys_str(), isDol); bool isSys = IsSystemFile(ch.getName(), isDol);
if (system ^ isSys) if (system ^ isSys)
continue; continue;
@@ -541,7 +541,7 @@ bool DiscBuilderBase::PartitionBuilderBase::recursiveMergeNodes(IPartWriteStream
PatchDOL(dolBuf, xferSz, patched); PatchDOL(dolBuf, xferSz, patched);
ws.write(dolBuf.get(), xferSz); ws.write(dolBuf.get(), xferSz);
m_parent.m_progressCB(m_parent.getProgressFactor(), m_parent.m_progressCB(m_parent.getProgressFactor(),
SystemString(sysName.sys_str()) + (patched ? _SYS_STR(" [PATCHED]") : _SYS_STR("")), std::string(ch.getName()) + (patched ? " [PATCHED]" : ""),
xferSz); xferSz);
++m_parent.m_progressIdx; ++m_parent.m_progressIdx;
} else { } else {
@@ -552,7 +552,7 @@ bool DiscBuilderBase::PartitionBuilderBase::recursiveMergeNodes(IPartWriteStream
break; break;
ws.write(buf, rdSz); ws.write(buf, rdSz);
xferSz += rdSz; xferSz += rdSz;
m_parent.m_progressCB(m_parent.getProgressFactorMidFile(xferSz, ch.size()), sysName.sys_str(), xferSz); m_parent.m_progressCB(m_parent.getProgressFactorMidFile(xferSz, ch.size()), ch.getName(), xferSz);
} }
++m_parent.m_progressIdx; ++m_parent.m_progressIdx;
} }
@@ -563,9 +563,9 @@ bool DiscBuilderBase::PartitionBuilderBase::recursiveMergeNodes(IPartWriteStream
return true; return true;
} }
bool DiscBuilderBase::PartitionBuilderBase::recursiveMergeFST(const Node* nodeIn, SystemStringView dirIn, bool DiscBuilderBase::PartitionBuilderBase::recursiveMergeFST(const Node* nodeIn, std::string_view dirIn,
std::function<void(void)> incParents, std::function<void(void)> incParents,
SystemStringView keyPath) { size_t parentDirIdx, std::string_view keyPath) {
/* Build map of existing nodes to write-through later */ /* Build map of existing nodes to write-through later */
std::unordered_map<std::string, const Node*> fileNodes; std::unordered_map<std::string, const Node*> fileNodes;
std::unordered_map<std::string, const Node*> dirNodes; std::unordered_map<std::string, const Node*> dirNodes;
@@ -584,23 +584,23 @@ bool DiscBuilderBase::PartitionBuilderBase::recursiveMergeFST(const Node* nodeIn
if (!dirIn.empty()) { if (!dirIn.empty()) {
DirectoryEnumerator dEnum(dirIn, DirectoryEnumerator::Mode::DirsThenFilesSorted, false, false, true); DirectoryEnumerator dEnum(dirIn, DirectoryEnumerator::Mode::DirsThenFilesSorted, false, false, true);
for (const DirectoryEnumerator::Entry& e : dEnum) { for (const DirectoryEnumerator::Entry& e : dEnum) {
SystemUTF8Conv nameView(e.m_name); UTF8ToSJIS nameView(e.m_name);
SystemString chKeyPath = SystemString(keyPath) + _SYS_STR('/') + e.m_name; std::string chKeyPath = std::string(keyPath) + '/' + e.m_name;
if (e.m_isDir) { if (e.m_isDir) {
size_t dirNodeIdx = m_buildNodes.size(); size_t dirNodeIdx = m_buildNodes.size();
m_buildNodes.emplace_back(true, m_buildNameOff, 0, dirNodeIdx + 1); m_buildNodes.emplace_back(true, m_buildNameOff, parentDirIdx, dirNodeIdx + 1);
addBuildName(e.m_name); addBuildName(e.m_name);
incParents(); incParents();
auto search = dirNodes.find(nameView.utf8_str().data()); auto search = dirNodes.find(nameView.str());
if (search != dirNodes.cend()) { if (search != dirNodes.cend()) {
if (!recursiveMergeFST(search->second, e.m_path.c_str(), if (!recursiveMergeFST(search->second, e.m_path.c_str(),
[&]() { [&]() {
m_buildNodes[dirNodeIdx].incrementLength(); m_buildNodes[dirNodeIdx].incrementLength();
incParents(); incParents();
}, },
chKeyPath)) dirNodeIdx, chKeyPath))
return false; return false;
dirNodes.erase(search); dirNodes.erase(search);
} else { } else {
@@ -609,11 +609,11 @@ bool DiscBuilderBase::PartitionBuilderBase::recursiveMergeFST(const Node* nodeIn
m_buildNodes[dirNodeIdx].incrementLength(); m_buildNodes[dirNodeIdx].incrementLength();
incParents(); incParents();
}, },
chKeyPath)) dirNodeIdx, chKeyPath))
return false; return false;
} }
} else { } else {
fileNodes.erase(nameView.utf8_str().data()); fileNodes.erase(nameView.str());
std::pair<uint64_t, uint64_t> fileOffSz = m_fileOffsetsSizes.at(chKeyPath); std::pair<uint64_t, uint64_t> fileOffSz = m_fileOffsetsSizes.at(chKeyPath);
m_buildNodes.emplace_back(false, m_buildNameOff, packOffset(fileOffSz.first), fileOffSz.second); m_buildNodes.emplace_back(false, m_buildNameOff, packOffset(fileOffSz.first), fileOffSz.second);
addBuildName(e.m_name); addBuildName(e.m_name);
@@ -624,12 +624,12 @@ bool DiscBuilderBase::PartitionBuilderBase::recursiveMergeFST(const Node* nodeIn
/* Write-through remaining dir nodes */ /* Write-through remaining dir nodes */
for (const auto& p : dirNodes) { for (const auto& p : dirNodes) {
SystemStringConv sysName(p.second->getName()); SJISToUTF8 sysName(p.second->getName());
SystemString chKeyPath = SystemString(keyPath) + _SYS_STR('/') + sysName.sys_str().data(); std::string chKeyPath = std::string(keyPath) + '/' + sysName.str();
size_t dirNodeIdx = m_buildNodes.size(); size_t dirNodeIdx = m_buildNodes.size();
m_buildNodes.emplace_back(true, m_buildNameOff, 0, dirNodeIdx + 1); m_buildNodes.emplace_back(true, m_buildNameOff, parentDirIdx, dirNodeIdx + 1);
addBuildName(sysName.sys_str()); addBuildName(sysName.str());
incParents(); incParents();
if (!recursiveMergeFST(p.second, {}, if (!recursiveMergeFST(p.second, {},
@@ -637,19 +637,19 @@ bool DiscBuilderBase::PartitionBuilderBase::recursiveMergeFST(const Node* nodeIn
m_buildNodes[dirNodeIdx].incrementLength(); m_buildNodes[dirNodeIdx].incrementLength();
incParents(); incParents();
}, },
chKeyPath)) dirNodeIdx, chKeyPath))
return false; return false;
} }
/* Write-through remaining file nodes */ /* Write-through remaining file nodes */
for (const auto& p : fileNodes) { for (const auto& p : fileNodes) {
const Node& ch = *p.second; const Node& ch = *p.second;
SystemStringConv sysName(ch.getName()); SJISToUTF8 sysName(ch.getName());
SystemString chKeyPath = SystemString(keyPath) + _SYS_STR('/') + sysName.sys_str().data(); std::string chKeyPath = std::string(keyPath) + '/' + sysName.str();
std::pair<uint64_t, uint64_t> fileOffSz = m_fileOffsetsSizes.at(chKeyPath); std::pair<uint64_t, uint64_t> fileOffSz = m_fileOffsetsSizes.at(chKeyPath);
m_buildNodes.emplace_back(false, m_buildNameOff, packOffset(fileOffSz.first), fileOffSz.second); m_buildNodes.emplace_back(false, m_buildNameOff, packOffset(fileOffSz.first), fileOffSz.second);
addBuildName(sysName.sys_str()); addBuildName(ch.getName());
incParents(); incParents();
} }
@@ -657,7 +657,7 @@ bool DiscBuilderBase::PartitionBuilderBase::recursiveMergeFST(const Node* nodeIn
} }
bool DiscBuilderBase::PartitionBuilderBase::RecursiveCalculateTotalSize(uint64_t& totalSz, const Node* nodeIn, bool DiscBuilderBase::PartitionBuilderBase::RecursiveCalculateTotalSize(uint64_t& totalSz, const Node* nodeIn,
SystemStringView dirIn) { std::string_view dirIn) {
/* Build map of existing nodes to write-through later */ /* Build map of existing nodes to write-through later */
std::unordered_map<std::string, const Node*> fileNodes; std::unordered_map<std::string, const Node*> fileNodes;
std::unordered_map<std::string, const Node*> dirNodes; std::unordered_map<std::string, const Node*> dirNodes;
@@ -676,20 +676,20 @@ bool DiscBuilderBase::PartitionBuilderBase::RecursiveCalculateTotalSize(uint64_t
if (!dirIn.empty()) { if (!dirIn.empty()) {
DirectoryEnumerator dEnum(dirIn, DirectoryEnumerator::Mode::DirsThenFilesSorted, false, false, true); DirectoryEnumerator dEnum(dirIn, DirectoryEnumerator::Mode::DirsThenFilesSorted, false, false, true);
for (const DirectoryEnumerator::Entry& e : dEnum) { for (const DirectoryEnumerator::Entry& e : dEnum) {
SystemUTF8Conv nameView(e.m_name); UTF8ToSJIS nameView(e.m_name);
if (e.m_isDir) { if (e.m_isDir) {
auto search = dirNodes.find(nameView.utf8_str().data()); auto search = dirNodes.find(nameView.str());
if (search != dirNodes.cend()) { if (search != dirNodes.cend()) {
if (!RecursiveCalculateTotalSize(totalSz, search->second, e.m_path.c_str())) if (!RecursiveCalculateTotalSize(totalSz, search->second, e.m_path))
return false; return false;
dirNodes.erase(search); dirNodes.erase(search);
} else { } else {
if (!RecursiveCalculateTotalSize(totalSz, nullptr, e.m_path.c_str())) if (!RecursiveCalculateTotalSize(totalSz, nullptr, e.m_path))
return false; return false;
} }
} else { } else {
fileNodes.erase(nameView.utf8_str().data()); fileNodes.erase(nameView.str());
totalSz += ROUND_UP_32(e.m_fileSz); totalSz += ROUND_UP_32(e.m_fileSz);
} }
} }
@@ -710,34 +710,34 @@ bool DiscBuilderBase::PartitionBuilderBase::RecursiveCalculateTotalSize(uint64_t
return true; return true;
} }
bool DiscBuilderBase::PartitionBuilderBase::buildFromDirectory(IPartWriteStream& ws, SystemStringView dirIn) { bool DiscBuilderBase::PartitionBuilderBase::buildFromDirectory(IPartWriteStream& ws, std::string_view dirIn) {
if (dirIn.empty()) { if (dirIn.empty()) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("all arguments must be supplied to buildFromDirectory()"))); spdlog::error("all arguments must be supplied to buildFromDirectory()");
return false; return false;
} }
SystemString dirStr(dirIn); std::string dirStr(dirIn);
SystemString basePath = m_isWii ? dirStr + _SYS_STR("/") + getKindString(m_kind) : dirStr; std::string basePath = m_isWii ? dirStr + "/" + getKindString(m_kind) : dirStr;
SystemString dolIn = basePath + _SYS_STR("/sys/main.dol"); std::string dolIn = basePath + "/sys/main.dol";
SystemString filesIn = basePath + _SYS_STR("/files"); std::string filesIn = basePath + "/files";
/* 1st pass - Tally up total progress steps */ /* 1st pass - Tally up total progress steps */
m_parent.m_progressTotal += 2; /* Prep and DOL */ m_parent.m_progressTotal += 2; /* Prep and DOL */
recursiveBuildNodesPre(filesIn.c_str()); recursiveBuildNodesPre(filesIn.c_str());
/* Clear file */ /* Clear file */
m_parent.m_progressCB(m_parent.getProgressFactor(), _SYS_STR("Preparing output image"), -1); m_parent.m_progressCB(m_parent.getProgressFactor(), "Preparing output image", -1);
++m_parent.m_progressIdx; ++m_parent.m_progressIdx;
/* Add root node */ /* Add root node */
m_buildNodes.emplace_back(true, m_buildNameOff, 0, 1); m_buildNodes.emplace_back(true, m_buildNameOff, 0, 1);
addBuildName(_SYS_STR("<root>")); addBuildName("<root>");
/* Write Boot DOL first (first thing seeked to after Apploader) */ /* Write Boot DOL first (first thing seeked to after Apploader) */
{ {
Sstat dolStat; Sstat dolStat;
if (Stat(dolIn.c_str(), &dolStat)) { if (Stat(dolIn.c_str(), &dolStat)) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("unable to stat {}")), dolIn); spdlog::error("unable to stat {}", dolIn);
return false; return false;
} }
size_t fileSz = ROUND_UP_32(dolStat.st_size); size_t fileSz = ROUND_UP_32(dolStat.st_size);
@@ -751,7 +751,7 @@ bool DiscBuilderBase::PartitionBuilderBase::buildFromDirectory(IPartWriteStream&
return false; return false;
bool patched; bool patched;
size_t xferSz = PatchDOL(*rs, ws, dolStat.st_size, patched); size_t xferSz = PatchDOL(*rs, ws, dolStat.st_size, patched);
m_parent.m_progressCB(m_parent.getProgressFactor(), dolIn + (patched ? _SYS_STR(" [PATCHED]") : _SYS_STR("")), m_parent.m_progressCB(m_parent.getProgressFactor(), dolIn + (patched ? " [PATCHED]" : ""),
xferSz); xferSz);
++m_parent.m_progressIdx; ++m_parent.m_progressIdx;
for (size_t i = 0; i < fileSz - xferSz; ++i) for (size_t i = 0; i < fileSz - xferSz; ++i)
@@ -769,46 +769,46 @@ bool DiscBuilderBase::PartitionBuilderBase::buildFromDirectory(IPartWriteStream&
return true; return true;
} }
uint64_t DiscBuilderBase::PartitionBuilderBase::CalculateTotalSizeBuild(SystemStringView dirIn, PartitionKind kind, std::optional<uint64_t> DiscBuilderBase::PartitionBuilderBase::CalculateTotalSizeBuild(std::string_view dirIn,
bool isWii) { PartitionKind kind, bool isWii) {
SystemString dirStr(dirIn); std::string dirStr(dirIn);
SystemString basePath = isWii ? dirStr + _SYS_STR("/") + getKindString(kind) : dirStr; std::string basePath = isWii ? dirStr + "/" + getKindString(kind) : dirStr;
SystemString dolIn = basePath + _SYS_STR("/sys/main.dol"); std::string dolIn = basePath + "/sys/main.dol";
SystemString filesIn = basePath + _SYS_STR("/files"); std::string filesIn = basePath + "/files";
Sstat dolStat; Sstat dolStat;
if (Stat(dolIn.c_str(), &dolStat)) { if (Stat(dolIn.c_str(), &dolStat)) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("unable to stat {}")), dolIn); spdlog::error("unable to stat {}", dolIn);
return UINT64_MAX; return std::nullopt;
} }
uint64_t totalSz = ROUND_UP_32(dolStat.st_size); uint64_t totalSz = ROUND_UP_32(dolStat.st_size);
if (!RecursiveCalculateTotalSize(totalSz, nullptr, filesIn.c_str())) if (!RecursiveCalculateTotalSize(totalSz, nullptr, filesIn))
return UINT64_MAX; return std::nullopt;
return totalSz; return totalSz;
} }
bool DiscBuilderBase::PartitionBuilderBase::mergeFromDirectory(IPartWriteStream& ws, const IPartition* partIn, bool DiscBuilderBase::PartitionBuilderBase::mergeFromDirectory(IPartWriteStream& ws, const IPartition* partIn,
SystemStringView dirIn) { std::string_view dirIn) {
if (dirIn.empty()) { if (dirIn.empty()) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("all arguments must be supplied to mergeFromDirectory()"))); spdlog::error("all arguments must be supplied to mergeFromDirectory()");
return false; return false;
} }
SystemString dirStr(dirIn); std::string dirStr(dirIn);
SystemString basePath = m_isWii ? dirStr + _SYS_STR("/") + getKindString(m_kind) : dirStr; std::string basePath = m_isWii ? dirStr + "/" + getKindString(m_kind) : dirStr;
SystemString filesIn = basePath + _SYS_STR("/files"); std::string filesIn = basePath + "/files";
/* 1st pass - Tally up total progress steps */ /* 1st pass - Tally up total progress steps */
m_parent.m_progressTotal += 2; /* Prep and DOL */ m_parent.m_progressTotal += 2; /* Prep and DOL */
recursiveMergeNodesPre(&partIn->getFSTRoot(), filesIn.c_str()); recursiveMergeNodesPre(&partIn->getFSTRoot(), filesIn.c_str());
/* Clear file */ /* Clear file */
m_parent.m_progressCB(m_parent.getProgressFactor(), _SYS_STR("Preparing output image"), -1); m_parent.m_progressCB(m_parent.getProgressFactor(), "Preparing output image", -1);
++m_parent.m_progressIdx; ++m_parent.m_progressIdx;
/* Add root node */ /* Add root node */
m_buildNodes.emplace_back(true, m_buildNameOff, 0, 1); m_buildNodes.emplace_back(true, m_buildNameOff, 0, 1);
addBuildName(_SYS_STR("<root>")); addBuildName("<root>");
/* Write Boot DOL first (first thing seeked to after Apploader) */ /* Write Boot DOL first (first thing seeked to after Apploader) */
{ {
@@ -824,7 +824,7 @@ bool DiscBuilderBase::PartitionBuilderBase::mergeFromDirectory(IPartWriteStream&
PatchDOL(dolBuf, xferSz, patched); PatchDOL(dolBuf, xferSz, patched);
ws.write(dolBuf.get(), xferSz); ws.write(dolBuf.get(), xferSz);
m_parent.m_progressCB(m_parent.getProgressFactor(), m_parent.m_progressCB(m_parent.getProgressFactor(),
SystemString(_SYS_STR("<boot-dol>")) + (patched ? _SYS_STR(" [PATCHED]") : _SYS_STR("")), std::string("<boot-dol>") + (patched ? " [PATCHED]" : ""),
xferSz); xferSz);
++m_parent.m_progressIdx; ++m_parent.m_progressIdx;
for (size_t i = 0; i < fileSz - xferSz; ++i) for (size_t i = 0; i < fileSz - xferSz; ++i)
@@ -832,26 +832,26 @@ bool DiscBuilderBase::PartitionBuilderBase::mergeFromDirectory(IPartWriteStream&
} }
/* Gather files in root directory */ /* Gather files in root directory */
SystemString keyPath; std::string keyPath;
if (!recursiveMergeNodes(ws, true, &partIn->getFSTRoot(), filesIn.c_str(), keyPath)) if (!recursiveMergeNodes(ws, true, &partIn->getFSTRoot(), filesIn.c_str(), keyPath))
return false; return false;
if (!recursiveMergeNodes(ws, false, &partIn->getFSTRoot(), filesIn.c_str(), keyPath)) if (!recursiveMergeNodes(ws, false, &partIn->getFSTRoot(), filesIn.c_str(), keyPath))
return false; return false;
if (!recursiveMergeFST(&partIn->getFSTRoot(), filesIn.c_str(), [&]() { m_buildNodes[0].incrementLength(); }, keyPath)) if (!recursiveMergeFST(&partIn->getFSTRoot(), filesIn.c_str(), [&]() { m_buildNodes[0].incrementLength(); }, 0, keyPath))
return false; return false;
return true; return true;
} }
uint64_t DiscBuilderBase::PartitionBuilderBase::CalculateTotalSizeMerge(const IPartition* partIn, std::optional<uint64_t> DiscBuilderBase::PartitionBuilderBase::CalculateTotalSizeMerge(const IPartition* partIn,
SystemStringView dirIn) { std::string_view dirIn) {
SystemString dirStr(dirIn); std::string dirStr(dirIn);
SystemString basePath = partIn->isWii() ? dirStr + _SYS_STR("/") + getKindString(partIn->getKind()) : dirStr; std::string basePath = partIn->isWii() ? dirStr + "/" + getKindString(partIn->getKind()) : dirStr;
SystemString filesIn = basePath + _SYS_STR("/files"); std::string filesIn = basePath + "/files";
uint64_t totalSz = ROUND_UP_32(partIn->getDOLSize()); uint64_t totalSz = ROUND_UP_32(partIn->getDOLSize());
if (!RecursiveCalculateTotalSize(totalSz, &partIn->getFSTRoot(), filesIn.c_str())) if (!RecursiveCalculateTotalSize(totalSz, &partIn->getFSTRoot(), filesIn))
return UINT64_MAX; return std::nullopt;
return totalSz; return totalSz;
} }

View File

@@ -7,9 +7,9 @@
#include <string> #include <string>
#include "nod/nod.hpp" #include "nod/nod.hpp"
#include "nod/Util.hpp" #include "Util.hpp"
#include <logvisor/logvisor.hpp> #include <spdlog/spdlog.h>
#define BUFFER_SZ 0x8000 #define BUFFER_SZ 0x8000
@@ -126,11 +126,11 @@ DiscGCN::DiscGCN(std::unique_ptr<IDiscIO>&& dio, bool& err) : DiscBase(std::move
m_partitions.emplace_back(std::make_unique<PartitionGCN>(*this, 0, err)); m_partitions.emplace_back(std::make_unique<PartitionGCN>(*this, 0, err));
} }
DiscBuilderGCN DiscGCN::makeMergeBuilder(SystemStringView outPath, FProgress progressCB) { DiscBuilderGCN DiscGCN::makeMergeBuilder(std::string_view outPath, FProgress progressCB) {
return DiscBuilderGCN(outPath, progressCB); return DiscBuilderGCN(outPath, progressCB);
} }
bool DiscGCN::extractDiscHeaderFiles(SystemStringView path, const ExtractionContext& ctx) const { return true; } bool DiscGCN::extractDiscHeaderFiles(std::string_view path, const ExtractionContext& ctx) const { return true; }
class PartitionBuilderGCN : public DiscBuilderBase::PartitionBuilderBase { class PartitionBuilderGCN : public DiscBuilderBase::PartitionBuilderBase {
uint64_t m_curUser = 0x57058000; uint64_t m_curUser = 0x57058000;
@@ -168,7 +168,7 @@ public:
m_curUser -= reqSz; m_curUser -= reqSz;
m_curUser &= 0xfffffffffffffff0; m_curUser &= 0xfffffffffffffff0;
if (m_curUser < 0x30000) { if (m_curUser < 0x30000) {
LogModule.report(logvisor::Error, fmt("user area low mark reached")); spdlog::error("user area low mark reached");
return -1; return -1;
} }
static_cast<PartWriteStream&>(ws).seek(m_curUser); static_cast<PartWriteStream&>(ws).seek(m_curUser);
@@ -211,7 +211,7 @@ public:
fstSz = ROUND_UP_32(fstSz); fstSz = ROUND_UP_32(fstSz);
if (fstOff + fstSz >= m_curUser) { if (fstOff + fstSz >= m_curUser) {
LogModule.report(logvisor::Error, fmt("FST flows into user area (one or the other is too big)")); spdlog::error("FST flows into user area (one or the other is too big)");
return false; return false;
} }
@@ -226,7 +226,7 @@ public:
return true; return true;
} }
bool buildFromDirectory(SystemStringView dirIn) { bool buildFromDirectory(std::string_view dirIn) {
std::unique_ptr<IPartWriteStream> ws = beginWriteStream(0); std::unique_ptr<IPartWriteStream> ws = beginWriteStream(0);
if (!ws) if (!ws)
return false; return false;
@@ -234,29 +234,29 @@ public:
if (!result) if (!result)
return false; return false;
SystemString dirStr(dirIn); std::string dirStr(dirIn);
/* Check Apploader */ /* Check Apploader */
SystemString apploaderIn = dirStr + _SYS_STR("/sys/apploader.img"); std::string apploaderIn = dirStr + "/sys/apploader.img";
Sstat apploaderStat; Sstat apploaderStat;
if (Stat(apploaderIn.c_str(), &apploaderStat)) { if (Stat(apploaderIn.c_str(), &apploaderStat)) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("unable to stat {}")), apploaderIn); spdlog::error("unable to stat {}", apploaderIn);
return false; return false;
} }
/* Check Boot */ /* Check Boot */
SystemString bootIn = dirStr + _SYS_STR("/sys/boot.bin"); std::string bootIn = dirStr + "/sys/boot.bin";
Sstat bootStat; Sstat bootStat;
if (Stat(bootIn.c_str(), &bootStat)) { if (Stat(bootIn.c_str(), &bootStat)) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("unable to stat {}")), bootIn); spdlog::error("unable to stat {}", bootIn);
return false; return false;
} }
/* Check BI2 */ /* Check BI2 */
SystemString bi2In = dirStr + _SYS_STR("/sys/bi2.bin"); std::string bi2In = dirStr + "/sys/bi2.bin";
Sstat bi2Stat; Sstat bi2Stat;
if (Stat(bi2In.c_str(), &bi2Stat)) { if (Stat(bi2In.c_str(), &bi2Stat)) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("unable to stat {}")), bi2In); spdlog::error("unable to stat {}", bi2In);
return false; return false;
} }
@@ -298,7 +298,7 @@ public:
ws.write(buf, rdSz); ws.write(buf, rdSz);
xferSz += rdSz; xferSz += rdSz;
if (0x2440 + xferSz >= m_curUser) { if (0x2440 + xferSz >= m_curUser) {
LogModule.report(logvisor::Error, fmt("apploader flows into user area (one or the other is too big)")); spdlog::error("apploader flows into user area (one or the other is too big)");
return false; return false;
} }
m_parent.m_progressCB(m_parent.getProgressFactor(), apploaderIn, xferSz); m_parent.m_progressCB(m_parent.getProgressFactor(), apploaderIn, xferSz);
@@ -308,7 +308,7 @@ public:
}); });
} }
bool mergeFromDirectory(const PartitionGCN* partIn, SystemStringView dirIn) { bool mergeFromDirectory(const PartitionGCN* partIn, std::string_view dirIn) {
std::unique_ptr<IPartWriteStream> ws = beginWriteStream(0); std::unique_ptr<IPartWriteStream> ws = beginWriteStream(0);
if (!ws) if (!ws)
return false; return false;
@@ -336,11 +336,11 @@ public:
[this, partIn](IPartWriteStream& ws, size_t& xferSz) -> bool { [this, partIn](IPartWriteStream& ws, size_t& xferSz) -> bool {
std::unique_ptr<uint8_t[]> apploaderBuf = partIn->getApploaderBuf(); std::unique_ptr<uint8_t[]> apploaderBuf = partIn->getApploaderBuf();
size_t apploaderSz = partIn->getApploaderSize(); size_t apploaderSz = partIn->getApploaderSize();
SystemString apploaderName(_SYS_STR("<apploader>")); std::string apploaderName("<apploader>");
ws.write(apploaderBuf.get(), apploaderSz); ws.write(apploaderBuf.get(), apploaderSz);
xferSz += apploaderSz; xferSz += apploaderSz;
if (0x2440 + xferSz >= m_curUser) { if (0x2440 + xferSz >= m_curUser) {
LogModule.report(logvisor::Error, fmt("apploader flows into user area (one or the other is too big)")); spdlog::error("apploader flows into user area (one or the other is too big)");
return false; return false;
} }
m_parent.m_progressCB(m_parent.getProgressFactor(), apploaderName, xferSz); m_parent.m_progressCB(m_parent.getProgressFactor(), apploaderName, xferSz);
@@ -350,14 +350,14 @@ public:
} }
}; };
EBuildResult DiscBuilderGCN::buildFromDirectory(SystemStringView dirIn) { EBuildResult DiscBuilderGCN::buildFromDirectory(std::string_view dirIn) {
if (!m_fileIO->beginWriteStream()) if (!m_fileIO->beginWriteStream())
return EBuildResult::Failed; return EBuildResult::Failed;
if (!CheckFreeSpace(m_outPath.c_str(), 0x57058000)) { if (!CheckFreeSpace(m_outPath.c_str(), 0x57058000)) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("not enough free disk space for {}")), m_outPath); spdlog::error("not enough free disk space for {}", m_outPath);
return EBuildResult::DiskFull; return EBuildResult::DiskFull;
} }
m_progressCB(getProgressFactor(), _SYS_STR("Preallocating image"), -1); m_progressCB(getProgressFactor(), "Preallocating image", -1);
++m_progressIdx; ++m_progressIdx;
{ {
auto ws = m_fileIO->beginWriteStream(0); auto ws = m_fileIO->beginWriteStream(0);
@@ -372,34 +372,35 @@ EBuildResult DiscBuilderGCN::buildFromDirectory(SystemStringView dirIn) {
return pb.buildFromDirectory(dirIn) ? EBuildResult::Success : EBuildResult::Failed; return pb.buildFromDirectory(dirIn) ? EBuildResult::Success : EBuildResult::Failed;
} }
uint64_t DiscBuilderGCN::CalculateTotalSizeRequired(SystemStringView dirIn) { std::optional<uint64_t> DiscBuilderGCN::CalculateTotalSizeRequired(std::string_view dirIn) {
uint64_t sz = DiscBuilderBase::PartitionBuilderBase::CalculateTotalSizeBuild(dirIn, PartitionKind::Data, false); std::optional<uint64_t> sz =
if (sz == UINT64_MAX) DiscBuilderBase::PartitionBuilderBase::CalculateTotalSizeBuild(dirIn, PartitionKind::Data, false);
return UINT64_MAX; if (!sz)
sz += 0x30000; return sz;
*sz += 0x30000;
if (sz > 0x57058000) { if (sz > 0x57058000) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("disc capacity exceeded [{} / {}]")), sz, 0x57058000); spdlog::error("disc capacity exceeded [{} / {}]", *sz, 0x57058000);
return UINT64_MAX; return std::nullopt;
} }
return sz; return sz;
} }
DiscBuilderGCN::DiscBuilderGCN(SystemStringView outPath, FProgress progressCB) DiscBuilderGCN::DiscBuilderGCN(std::string_view outPath, FProgress progressCB)
: DiscBuilderBase(outPath, 0x57058000, progressCB) { : DiscBuilderBase(outPath, 0x57058000, progressCB) {
m_partitions.emplace_back(std::make_unique<PartitionBuilderGCN>(*this)); m_partitions.emplace_back(std::make_unique<PartitionBuilderGCN>(*this));
} }
DiscMergerGCN::DiscMergerGCN(SystemStringView outPath, DiscGCN& sourceDisc, FProgress progressCB) DiscMergerGCN::DiscMergerGCN(std::string_view outPath, DiscGCN& sourceDisc, FProgress progressCB)
: m_sourceDisc(sourceDisc), m_builder(sourceDisc.makeMergeBuilder(outPath, progressCB)) {} : m_sourceDisc(sourceDisc), m_builder(sourceDisc.makeMergeBuilder(outPath, progressCB)) {}
EBuildResult DiscMergerGCN::mergeFromDirectory(SystemStringView dirIn) { EBuildResult DiscMergerGCN::mergeFromDirectory(std::string_view dirIn) {
if (!m_builder.getFileIO().beginWriteStream()) if (!m_builder.getFileIO().beginWriteStream())
return EBuildResult::Failed; return EBuildResult::Failed;
if (!CheckFreeSpace(m_builder.m_outPath.c_str(), 0x57058000)) { if (!CheckFreeSpace(m_builder.m_outPath.c_str(), 0x57058000)) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("not enough free disk space for {}")), m_builder.m_outPath); spdlog::error("not enough free disk space for {}", m_builder.m_outPath);
return EBuildResult::DiskFull; return EBuildResult::DiskFull;
} }
m_builder.m_progressCB(m_builder.getProgressFactor(), _SYS_STR("Preallocating image"), -1); m_builder.m_progressCB(m_builder.getProgressFactor(), "Preallocating image", -1);
++m_builder.m_progressIdx; ++m_builder.m_progressIdx;
{ {
auto ws = m_builder.m_fileIO->beginWriteStream(0); auto ws = m_builder.m_fileIO->beginWriteStream(0);
@@ -416,14 +417,15 @@ EBuildResult DiscMergerGCN::mergeFromDirectory(SystemStringView dirIn) {
: EBuildResult::Failed; : EBuildResult::Failed;
} }
uint64_t DiscMergerGCN::CalculateTotalSizeRequired(DiscGCN& sourceDisc, SystemStringView dirIn) { std::optional<uint64_t> DiscMergerGCN::CalculateTotalSizeRequired(DiscGCN& sourceDisc, std::string_view dirIn) {
uint64_t sz = DiscBuilderBase::PartitionBuilderBase::CalculateTotalSizeMerge(sourceDisc.getDataPartition(), dirIn); std::optional<uint64_t> sz =
if (sz == UINT64_MAX) DiscBuilderBase::PartitionBuilderBase::CalculateTotalSizeMerge(sourceDisc.getDataPartition(), dirIn);
return UINT64_MAX; if (!sz)
sz += 0x30000; return std::nullopt;
*sz += 0x30000;
if (sz > 0x57058000) { if (sz > 0x57058000) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("disc capacity exceeded [{} / {}]")), sz, 0x57058000); spdlog::error("disc capacity exceeded [{} / {}]", *sz, 0x57058000);
return -1; return std::nullopt;
} }
return sz; return sz;
} }

View File

@@ -1,6 +1,7 @@
#include "nod/IDiscIO.hpp" #include "nod/IDiscIO.hpp"
#include "nod/IFileIO.hpp" #include "nod/IFileIO.hpp"
#include "nod/Util.hpp"
#include "Util.hpp"
namespace nod { namespace nod {
@@ -8,7 +9,7 @@ class DiscIOISO : public IDiscIO {
std::unique_ptr<IFileIO> m_fio; std::unique_ptr<IFileIO> m_fio;
public: public:
DiscIOISO(SystemStringView fpin) : m_fio(NewFileIO(fpin)) {} DiscIOISO(std::string_view fpin) : m_fio(NewFileIO(fpin)) {}
class ReadStream : public IReadStream { class ReadStream : public IReadStream {
friend class DiscIOISO; friend class DiscIOISO;
@@ -28,9 +29,8 @@ public:
bool err = false; bool err = false;
auto ret = std::unique_ptr<IReadStream>(new ReadStream(m_fio->beginReadStream(offset), err)); auto ret = std::unique_ptr<IReadStream>(new ReadStream(m_fio->beginReadStream(offset), err));
if (err) { if (err)
return nullptr; return {};
}
return ret; return ret;
} }
@@ -51,14 +51,13 @@ public:
bool err = false; bool err = false;
auto ret = std::unique_ptr<IWriteStream>(new WriteStream(m_fio->beginWriteStream(offset), err)); auto ret = std::unique_ptr<IWriteStream>(new WriteStream(m_fio->beginWriteStream(offset), err));
if (err) { if (err)
return nullptr; return {};
}
return ret; return ret;
} }
}; };
std::unique_ptr<IDiscIO> NewDiscIOISO(SystemStringView path) { return std::make_unique<DiscIOISO>(path); } std::unique_ptr<IDiscIO> NewDiscIOISO(std::string_view path) { return std::make_unique<DiscIOISO>(path); }
} // namespace nod } // namespace nod

272
lib/DiscIONFS.cpp Normal file
View File

@@ -0,0 +1,272 @@
#include "nod/IDiscIO.hpp"
#include "nod/IFileIO.hpp"
#include "nod/aes.hpp"
#include "nod/Endian.hpp"
#include "Util.hpp"
#include <spdlog/spdlog.h>
namespace nod {
/*
* NFS is the image format used to distribute Wii VC games for the Wii U.
* It is an LBA format similar to WBFS but adds its own encryption layer.
* It logically stores a standard Wii disc image with partitions.
*/
class DiscIONFS : public IDiscIO {
std::vector<std::unique_ptr<IFileIO>> files;
struct NFSHead {
uint32_t magic; // EGGS
uint32_t version;
uint32_t unknown[2]; // Signature, UUID?
uint32_t lbaRangeCount;
struct {
uint32_t startBlock;
uint32_t numBlocks;
} lbaRanges[61];
uint32_t endMagic; // SGGE
} nfsHead;
uint8_t key[16];
uint32_t calculateNumFiles() const {
uint32_t totalBlockCount = 0;
for (uint32_t i = 0; i < nfsHead.lbaRangeCount; ++i)
totalBlockCount += nfsHead.lbaRanges[i].numBlocks;
return (uint64_t(totalBlockCount) * uint64_t(0x8000) + (uint64_t(0x200) + uint64_t(0xF9FFFFF))) /
uint64_t(0xFA00000);
}
struct FBO {
uint32_t file, block, lblock, offset;
};
FBO logicalToFBO(uint64_t offset) const {
auto blockAndRemBytes = nod::div(offset, uint64_t(0x8000)); /* 32768 bytes per block */
uint32_t block = UINT32_MAX;
for (uint32_t i = 0, physicalBlock = 0; i < nfsHead.lbaRangeCount; ++i) {
const auto& range = nfsHead.lbaRanges[i];
if (blockAndRemBytes.quot >= range.startBlock && blockAndRemBytes.quot - range.startBlock < range.numBlocks) {
block = physicalBlock + (blockAndRemBytes.quot - range.startBlock);
break;
}
physicalBlock += range.numBlocks;
}
/* This offset has no physical mapping, read zeroes */
if (block == UINT32_MAX)
return {UINT32_MAX, UINT32_MAX, UINT32_MAX, UINT32_MAX};
auto fileAndRemBlocks = nod::div(block, uint32_t(8000)); /* 8000 blocks per file */
return {uint32_t(fileAndRemBlocks.quot), uint32_t(fileAndRemBlocks.rem), uint32_t(blockAndRemBytes.quot),
uint32_t(blockAndRemBytes.rem)};
}
public:
DiscIONFS(std::string_view fpin, bool& err) {
/* Validate file path format */
using SignedSize = std::make_signed<std::string::size_type>::type;
const auto dotPos = SignedSize(fpin.rfind('.'));
const auto slashPos = SignedSize(fpin.find_last_of("/\\"));
if (fpin.size() <= 4 || dotPos == -1 || dotPos <= slashPos || fpin.compare(slashPos + 1, 4, "hif_") ||
fpin.compare(dotPos, fpin.size() - dotPos, ".nfs")) {
spdlog::error("'{}' must begin with 'hif_' and end with '.nfs' to be accepted as an NFS image", fpin);
err = true;
return;
}
/* Load key file */
const std::string dir(fpin.begin(), fpin.begin() + slashPos + 1);
auto keyFile = NewFileIO(dir + "../code/htk.bin")->beginReadStream();
if (!keyFile)
keyFile = NewFileIO(dir + "htk.bin")->beginReadStream();
if (!keyFile) {
spdlog::error("Unable to open '{}../code/htk.bin' or '{}htk.bin'", dir, dir);
err = true;
return;
}
if (keyFile->read(key, 16) != 16) {
spdlog::error("Unable to read from '{}../code/htk.bin' or '{}htk.bin'", dir, dir);
err = true;
return;
}
/* Load header from first file */
const std::string firstPath = fmt::format("{}hif_{:06}.nfs", dir, 0);
files.push_back(NewFileIO(firstPath));
auto rs = files.back()->beginReadStream();
if (!rs) {
spdlog::error("'{}' does not exist", firstPath);
err = true;
return;
}
if (rs->read(&nfsHead, 0x200) != 0x200) {
spdlog::error("Unable to read header from '{}'", firstPath);
err = true;
return;
}
if (std::memcmp(&nfsHead.magic, "EGGS", 4)) {
spdlog::error("Invalid magic in '{}'", firstPath);
err = true;
return;
}
nfsHead.lbaRangeCount = SBig(nfsHead.lbaRangeCount);
for (uint32_t i = 0; i < nfsHead.lbaRangeCount; ++i) {
auto& range = nfsHead.lbaRanges[i];
range.startBlock = SBig(range.startBlock);
range.numBlocks = SBig(range.numBlocks);
}
/* Ensure remaining files exist */
const uint32_t numFiles = calculateNumFiles();
files.reserve(numFiles);
for (uint32_t i = 1; i < numFiles; ++i) {
std::string path = fmt::format("{}hif_{:06}.nfs", dir, i);
files.push_back(NewFileIO(path));
if (!files.back()->exists()) {
spdlog::error("'{}' does not exist", path);
err = true;
return;
}
}
}
class ReadStream : public IReadStream {
friend class DiscIONFS;
const DiscIONFS& m_parent;
std::unique_ptr<IReadStream> m_rs;
std::unique_ptr<IAES> m_aes;
/* Physical address - all UINT32_MAX indicates logical zero block */
DiscIONFS::FBO m_physAddr;
/* Logical address */
uint64_t m_offset;
/* Active file stream and its offset as set in the system.
* Block is typically one ahead of the presently decrypted block. */
uint32_t m_curFile = UINT32_MAX;
uint32_t m_curBlock = UINT32_MAX;
ReadStream(const DiscIONFS& parent, uint64_t offset, bool& err)
: m_parent(parent)
, m_aes(NewAES())
, m_physAddr({UINT32_MAX, UINT32_MAX, UINT32_MAX, UINT32_MAX})
, m_offset(offset) {
m_aes->setKey(m_parent.key);
setLogicalAddr(offset);
}
uint8_t m_encBuf[0x8000] = {};
uint8_t m_decBuf[0x8000] = {};
void setCurFile(uint32_t curFile) {
if (curFile >= m_parent.files.size()) {
spdlog::error("Out of bounds NFS file access");
return;
}
m_curFile = curFile;
m_curBlock = UINT32_MAX;
m_rs = m_parent.files[m_curFile]->beginReadStream();
}
void setCurBlock(uint32_t curBlock) {
m_curBlock = curBlock;
m_rs->seek(m_curBlock * 0x8000 + 0x200);
}
void setPhysicalAddr(DiscIONFS::FBO physAddr) {
/* If we're just changing the offset, nothing else needs to be done */
if (m_physAddr.file == physAddr.file && m_physAddr.block == physAddr.block) {
m_physAddr.offset = physAddr.offset;
return;
}
m_physAddr = physAddr;
/* Set logical zero block */
if (m_physAddr.file == UINT32_MAX) {
memset(m_decBuf, 0, 0x8000);
return;
}
/* Make necessary file and block current with system */
if (m_physAddr.file != m_curFile)
setCurFile(m_physAddr.file);
if (m_physAddr.block != m_curBlock)
setCurBlock(m_physAddr.block);
/* Read block, handling 0x200 overlap case */
if (m_physAddr.block == 7999) {
m_rs->read(m_encBuf, 0x7E00);
setCurFile(m_curFile + 1);
m_rs->read(m_encBuf + 0x7E00, 0x200);
m_curBlock = 0;
} else {
m_rs->read(m_encBuf, 0x8000);
++m_curBlock;
}
/* Decrypt */
const uint32_t ivBuf[] = {0, 0, 0, SBig(m_physAddr.lblock)};
m_aes->decrypt((const uint8_t*)ivBuf, m_encBuf, m_decBuf, 0x8000);
}
void setLogicalAddr(uint64_t addr) { setPhysicalAddr(m_parent.logicalToFBO(m_offset)); }
public:
uint64_t read(void* buf, uint64_t length) override {
uint64_t rem = length;
uint8_t* dst = (uint8_t*)buf;
/* Perform reads on block boundaries */
while (rem) {
uint64_t readSize = rem;
uint32_t blockOffset = (m_physAddr.offset == UINT32_MAX) ? 0 : m_physAddr.offset;
if (readSize + blockOffset > 0x8000)
readSize = 0x8000 - blockOffset;
memmove(dst, m_decBuf + blockOffset, readSize);
dst += readSize;
rem -= readSize;
m_offset += readSize;
setLogicalAddr(m_offset);
}
return dst - (uint8_t*)buf;
}
uint64_t position() const override { return m_offset; }
void seek(int64_t offset, int whence) override {
if (whence == SEEK_SET)
m_offset = offset;
else if (whence == SEEK_CUR)
m_offset += offset;
else
return;
setLogicalAddr(m_offset);
}
};
std::unique_ptr<IReadStream> beginReadStream(uint64_t offset) const override {
bool err = false;
auto ret = std::unique_ptr<IReadStream>(new ReadStream(*this, offset, err));
if (err)
return {};
return ret;
}
std::unique_ptr<IWriteStream> beginWriteStream(uint64_t offset) const override { return {}; }
bool hasWiiCrypto() const override { return false; }
};
std::unique_ptr<IDiscIO> NewDiscIONFS(std::string_view path) {
bool err = false;
auto ret = std::make_unique<DiscIONFS>(path, err);
if (err)
return {};
return ret;
}
} // namespace nod

View File

@@ -5,9 +5,10 @@
#include "nod/IDiscIO.hpp" #include "nod/IDiscIO.hpp"
#include "nod/IFileIO.hpp" #include "nod/IFileIO.hpp"
#include "nod/Util.hpp" #include "nod/Endian.hpp"
#include "Util.hpp"
#include <logvisor/logvisor.hpp> #include <spdlog/spdlog.h>
namespace nod { namespace nod {
@@ -23,7 +24,7 @@ static uint8_t size_to_shift(uint32_t size) {
} }
class DiscIOWBFS : public IDiscIO { class DiscIOWBFS : public IDiscIO {
SystemString filepath; std::unique_ptr<IFileIO> m_fio;
struct WBFSHead { struct WBFSHead {
uint32_t magic; uint32_t magic;
@@ -76,24 +77,23 @@ class DiscIOWBFS : public IDiscIO {
off *= 512ULL; off *= 512ULL;
rs.seek(off, SEEK_SET); rs.seek(off, SEEK_SET);
if (rs.read(buf, count * 512ULL) != count * 512ULL) { if (rs.read(buf, count * 512ULL) != count * 512ULL) {
LogModule.report(logvisor::Error, fmt("error reading disc")); spdlog::error("error reading disc");
return 1; return 1;
} }
return 0; return 0;
} }
public: public:
DiscIOWBFS(SystemStringView fpin) : filepath(fpin) { DiscIOWBFS(std::string_view fpin) : m_fio(NewFileIO(fpin)) {
/* Temporary file handle to read LBA table */ /* Temporary file handle to read LBA table */
std::unique_ptr<IFileIO> fio = NewFileIO(filepath); std::unique_ptr<IFileIO::IReadStream> rs = m_fio->beginReadStream();
std::unique_ptr<IFileIO::IReadStream> rs = fio->beginReadStream();
if (!rs) if (!rs)
return; return;
WBFS* p = &wbfs; WBFS* p = &wbfs;
WBFSHead tmpHead; WBFSHead tmpHead;
if (rs->read(&tmpHead, sizeof(tmpHead)) != sizeof(tmpHead)) { if (rs->read(&tmpHead, sizeof(tmpHead)) != sizeof(tmpHead)) {
LogModule.report(logvisor::Error, fmt("unable to read WBFS head")); spdlog::error("unable to read WBFS head");
return; return;
} }
unsigned hd_sector_size = 1 << tmpHead.hd_sec_sz_s; unsigned hd_sector_size = 1 << tmpHead.hd_sec_sz_s;
@@ -103,7 +103,7 @@ public:
WBFSHead* head = (WBFSHead*)wbfsHead.get(); WBFSHead* head = (WBFSHead*)wbfsHead.get();
rs->seek(0, SEEK_SET); rs->seek(0, SEEK_SET);
if (rs->read(head, hd_sector_size) != hd_sector_size) { if (rs->read(head, hd_sector_size) != hd_sector_size) {
LogModule.report(logvisor::Error, fmt("unable to read WBFS head")); spdlog::error("unable to read WBFS head");
return; return;
} }
@@ -116,11 +116,11 @@ public:
if (_wbfsReadSector(*rs, p->part_lba, 1, head)) if (_wbfsReadSector(*rs, p->part_lba, 1, head))
return; return;
if (hd_sector_size && head->hd_sec_sz_s != size_to_shift(hd_sector_size)) { if (hd_sector_size && head->hd_sec_sz_s != size_to_shift(hd_sector_size)) {
LogModule.report(logvisor::Error, fmt("hd sector size doesn't match")); spdlog::error("hd sector size doesn't match");
return; return;
} }
if (num_hd_sector && head->n_hd_sec != SBig(num_hd_sector)) { if (num_hd_sector && head->n_hd_sec != SBig(num_hd_sector)) {
LogModule.report(logvisor::Error, fmt("hd num sector doesn't match")); spdlog::error("hd num sector doesn't match");
return; return;
} }
p->hd_sec_sz = 1 << head->hd_sec_sz_s; p->hd_sec_sz = 1 << head->hd_sec_sz_s;
@@ -148,7 +148,7 @@ public:
if (head->disc_table[0]) { if (head->disc_table[0]) {
wbfsDiscInfo.reset(new uint8_t[p->disc_info_sz]); wbfsDiscInfo.reset(new uint8_t[p->disc_info_sz]);
if (!wbfsDiscInfo) { if (!wbfsDiscInfo) {
LogModule.report(logvisor::Error, fmt("allocating memory")); spdlog::error("allocating memory");
return; return;
} }
if (_wbfsReadSector(*rs, p->part_lba + 1, disc_info_sz_lba, wbfsDiscInfo.get())) if (_wbfsReadSector(*rs, p->part_lba + 1, disc_info_sz_lba, wbfsDiscInfo.get()))
@@ -265,18 +265,17 @@ public:
std::unique_ptr<IReadStream> beginReadStream(uint64_t offset) const override { std::unique_ptr<IReadStream> beginReadStream(uint64_t offset) const override {
bool err = false; bool err = false;
auto ret = std::unique_ptr<IReadStream>(new ReadStream(*this, NewFileIO(filepath)->beginReadStream(), offset, err)); auto ret = std::unique_ptr<IReadStream>(new ReadStream(*this, m_fio->beginReadStream(), offset, err));
if (err) { if (err)
return nullptr; return {};
}
return ret; return ret;
} }
std::unique_ptr<IWriteStream> beginWriteStream(uint64_t offset) const override { return nullptr; } std::unique_ptr<IWriteStream> beginWriteStream(uint64_t offset) const override { return {}; }
}; };
std::unique_ptr<IDiscIO> NewDiscIOWBFS(SystemStringView path) { return std::make_unique<DiscIOWBFS>(path); } std::unique_ptr<IDiscIO> NewDiscIOWBFS(std::string_view path) { return std::make_unique<DiscIOWBFS>(path); }
} // namespace nod } // namespace nod

View File

@@ -9,7 +9,9 @@
#include "nod/aes.hpp" #include "nod/aes.hpp"
#include "nod/nod.hpp" #include "nod/nod.hpp"
#include "nod/sha1.h" #include "nod/sha1.h"
#include "nod/Util.hpp" #include "Util.hpp"
#include <spdlog/spdlog.h>
namespace nod { namespace nod {
@@ -332,14 +334,22 @@ public:
uint8_t m_decBuf[0x7c00]; uint8_t m_decBuf[0x7c00];
void decryptBlock() { void decryptBlock() {
if (m_aes) {
m_dio->read(m_encBuf, 0x8000); m_dio->read(m_encBuf, 0x8000);
m_aes->decrypt(&m_encBuf[0x3d0], &m_encBuf[0x400], m_decBuf, 0x7c00); m_aes->decrypt(&m_encBuf[0x3d0], &m_encBuf[0x400], m_decBuf, 0x7c00);
} else {
m_dio->seek(0x400, SEEK_CUR);
m_dio->read(m_decBuf, 0x7c00);
}
} }
public: public:
PartReadStream(const PartitionWii& parent, uint64_t baseOffset, uint64_t offset, bool& err) PartReadStream(const PartitionWii& parent, uint64_t baseOffset, uint64_t offset, bool& err)
: m_aes(NewAES()), m_parent(parent), m_baseOffset(baseOffset), m_offset(offset) { : m_parent(parent), m_baseOffset(baseOffset), m_offset(offset) {
if (m_parent.m_parent.getDiscIO().hasWiiCrypto()) {
m_aes = NewAES();
m_aes->setKey(parent.m_decKey); m_aes->setKey(parent.m_decKey);
}
size_t block = m_offset / 0x7c00; size_t block = m_offset / 0x7c00;
m_dio = m_parent.m_parent.getDiscIO().beginReadStream(m_baseOffset + block * 0x8000); m_dio = m_parent.m_parent.getDiscIO().beginReadStream(m_baseOffset + block * 0x8000);
if (!m_dio) { if (!m_dio) {
@@ -365,27 +375,25 @@ public:
} }
uint64_t position() const override { return m_offset; } uint64_t position() const override { return m_offset; }
uint64_t read(void* buf, uint64_t length) override { uint64_t read(void* buf, uint64_t length) override {
size_t block = m_offset / 0x7c00; auto blockAndRemOff = nod::div(m_offset, uint64_t(0x7c00));
size_t cacheOffset = m_offset % 0x7c00;
uint64_t cacheSize;
uint64_t rem = length; uint64_t rem = length;
uint8_t* dst = (uint8_t*)buf; uint8_t* dst = (uint8_t*)buf;
while (rem) { while (rem) {
if (block != m_curBlock) { if (blockAndRemOff.quot != m_curBlock) {
decryptBlock(); decryptBlock();
m_curBlock = block; m_curBlock = blockAndRemOff.quot;
} }
cacheSize = rem; uint64_t cacheSize = rem;
if (cacheSize + cacheOffset > 0x7c00) if (cacheSize + blockAndRemOff.rem > 0x7c00)
cacheSize = 0x7c00 - cacheOffset; cacheSize = 0x7c00 - blockAndRemOff.rem;
memmove(dst, m_decBuf + cacheOffset, cacheSize); memmove(dst, m_decBuf + blockAndRemOff.rem, cacheSize);
dst += cacheSize; dst += cacheSize;
rem -= cacheSize; rem -= cacheSize;
cacheOffset = 0; blockAndRemOff.rem = 0;
++block; ++blockAndRemOff.quot;
} }
m_offset += length; m_offset += length;
@@ -415,7 +423,7 @@ public:
uint32_t h3; uint32_t h3;
if (rs->read(&h3, 4) != 4) { if (rs->read(&h3, 4) != 4) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("unable to read H3 offset apploader"))); spdlog::error("unable to read H3 offset apploader");
return nullptr; return nullptr;
} }
h3 = SBig(h3); h3 = SBig(h3);
@@ -433,12 +441,12 @@ public:
return buf; return buf;
} }
bool extractCryptoFiles(SystemStringView basePath, const ExtractionContext& ctx) const override { bool extractCryptoFiles(std::string_view basePath, const ExtractionContext& ctx) const override {
Sstat theStat; Sstat theStat;
SystemString basePathStr(basePath); std::string basePathStr(basePath);
/* Extract Ticket */ /* Extract Ticket */
SystemString ticketPath = basePathStr + _SYS_STR("/ticket.bin"); std::string ticketPath = basePathStr + "/ticket.bin";
if (ctx.force || Stat(ticketPath.c_str(), &theStat)) { if (ctx.force || Stat(ticketPath.c_str(), &theStat)) {
if (ctx.progressCB) if (ctx.progressCB)
ctx.progressCB("ticket.bin", 0.f); ctx.progressCB("ticket.bin", 0.f);
@@ -449,7 +457,7 @@ public:
} }
/* Extract TMD */ /* Extract TMD */
SystemString tmdPath = basePathStr + _SYS_STR("/tmd.bin"); std::string tmdPath = basePathStr + "/tmd.bin";
if (ctx.force || Stat(tmdPath.c_str(), &theStat)) { if (ctx.force || Stat(tmdPath.c_str(), &theStat)) {
if (ctx.progressCB) if (ctx.progressCB)
ctx.progressCB("tmd.bin", 0.f); ctx.progressCB("tmd.bin", 0.f);
@@ -460,7 +468,7 @@ public:
} }
/* Extract Certs */ /* Extract Certs */
SystemString certPath = basePathStr + _SYS_STR("/cert.bin"); std::string certPath = basePathStr + "/cert.bin";
if (ctx.force || Stat(certPath.c_str(), &theStat)) { if (ctx.force || Stat(certPath.c_str(), &theStat)) {
if (ctx.progressCB) if (ctx.progressCB)
ctx.progressCB("cert.bin", 0.f); ctx.progressCB("cert.bin", 0.f);
@@ -473,7 +481,7 @@ public:
} }
/* Extract H3 */ /* Extract H3 */
SystemString h3Path = basePathStr + _SYS_STR("/h3.bin"); std::string h3Path = basePathStr + "/h3.bin";
if (ctx.force || Stat(h3Path.c_str(), &theStat)) { if (ctx.force || Stat(h3Path.c_str(), &theStat)) {
if (ctx.progressCB) if (ctx.progressCB)
ctx.progressCB("h3.bin", 0.f); ctx.progressCB("h3.bin", 0.f);
@@ -533,7 +541,7 @@ DiscWii::DiscWii(std::unique_ptr<IDiscIO>&& dio, bool& err) : DiscBase(std::move
kind = part.partType; kind = part.partType;
break; break;
default: default:
LogModule.report(logvisor::Error, fmt("invalid partition type {}"), part.partType); spdlog::error("invalid partition type {}", static_cast<uint32_t>(part.partType));
err = true; err = true;
return; return;
} }
@@ -543,22 +551,22 @@ DiscWii::DiscWii(std::unique_ptr<IDiscIO>&& dio, bool& err) : DiscBase(std::move
} }
} }
DiscBuilderWii DiscWii::makeMergeBuilder(SystemStringView outPath, bool dualLayer, FProgress progressCB) { DiscBuilderWii DiscWii::makeMergeBuilder(std::string_view outPath, bool dualLayer, FProgress progressCB) {
return DiscBuilderWii(outPath, dualLayer, progressCB); return DiscBuilderWii(outPath, dualLayer, progressCB);
} }
bool DiscWii::extractDiscHeaderFiles(SystemStringView basePath, const ExtractionContext& ctx) const { bool DiscWii::extractDiscHeaderFiles(std::string_view basePath, const ExtractionContext& ctx) const {
SystemString basePathStr(basePath); std::string basePathStr(basePath);
if (Mkdir((basePathStr + _SYS_STR("/disc")).c_str(), 0755) && errno != EEXIST) { if (Mkdir((basePathStr + "/disc").c_str(), 0755) && errno != EEXIST) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("unable to mkdir '{}/disc'")), basePathStr); spdlog::error("unable to mkdir '{}/disc'", basePathStr);
return false; return false;
} }
Sstat theStat; Sstat theStat;
/* Extract Header */ /* Extract Header */
SystemString headerPath = basePathStr + _SYS_STR("/disc/header.bin"); std::string headerPath = basePathStr + "/disc/header.bin";
if (ctx.force || Stat(headerPath.c_str(), &theStat)) { if (ctx.force || Stat(headerPath.c_str(), &theStat)) {
if (ctx.progressCB) if (ctx.progressCB)
ctx.progressCB("header.bin", 0.f); ctx.progressCB("header.bin", 0.f);
@@ -574,7 +582,7 @@ bool DiscWii::extractDiscHeaderFiles(SystemStringView basePath, const Extraction
} }
/* Extract Region info */ /* Extract Region info */
SystemString regionPath = basePathStr + _SYS_STR("/disc/region.bin"); std::string regionPath = basePathStr + "/disc/region.bin";
if (ctx.force || Stat(regionPath.c_str(), &theStat)) { if (ctx.force || Stat(regionPath.c_str(), &theStat)) {
if (ctx.progressCB) if (ctx.progressCB)
ctx.progressCB("header.bin", 0.f); ctx.progressCB("header.bin", 0.f);
@@ -669,7 +677,7 @@ public:
} }
if (m_fio->write(m_buf, 0x200000) != 0x200000) { if (m_fio->write(m_buf, 0x200000) != 0x200000) {
LogModule.report(logvisor::Error, fmt("unable to write full disc group")); spdlog::error("unable to write full disc group");
return; return;
} }
} }
@@ -678,7 +686,7 @@ public:
PartWriteStream(PartitionBuilderWii& parent, uint64_t baseOffset, uint64_t offset, bool& err) PartWriteStream(PartitionBuilderWii& parent, uint64_t baseOffset, uint64_t offset, bool& err)
: m_parent(parent), m_baseOffset(baseOffset), m_offset(offset) { : m_parent(parent), m_baseOffset(baseOffset), m_offset(offset) {
if (offset % 0x1F0000) { if (offset % 0x1F0000) {
LogModule.report(logvisor::Error, fmt("partition write stream MUST begin on 0x1F0000-aligned boundary")); spdlog::error("partition write stream MUST begin on 0x1F0000-aligned boundary");
err = true; err = true;
return; return;
} }
@@ -748,13 +756,13 @@ public:
uint64_t userAllocate(uint64_t reqSz, IPartWriteStream& ws) override { uint64_t userAllocate(uint64_t reqSz, IPartWriteStream& ws) override {
reqSz = ROUND_UP_32(reqSz); reqSz = ROUND_UP_32(reqSz);
if (m_curUser + reqSz >= 0x1FB450000) { if (m_curUser + reqSz >= 0x1FB450000) {
LogModule.report(logvisor::Error, fmt("partition exceeds maximum single-partition capacity")); spdlog::error("partition exceeds maximum single-partition capacity");
return -1; return -1;
} }
uint64_t ret = m_curUser; uint64_t ret = m_curUser;
PartWriteStream& cws = static_cast<PartWriteStream&>(ws); PartWriteStream& cws = static_cast<PartWriteStream&>(ws);
if (cws.m_offset > ret) { if (cws.m_offset > ret) {
LogModule.report(logvisor::Error, fmt("partition overwrite error")); spdlog::error("partition overwrite error");
return -1; return -1;
} }
while (cws.m_offset < ret) while (cws.m_offset < ret)
@@ -832,7 +840,7 @@ public:
fstSz = ROUND_UP_32(fstSz); fstSz = ROUND_UP_32(fstSz);
if (fstOff + fstSz >= 0x1F0000) { if (fstOff + fstSz >= 0x1F0000) {
LogModule.report(logvisor::Error, fmt("FST flows into user area (one or the other is too big)")); spdlog::error("FST flows into user area (one or the other is too big)");
return -1; return -1;
} }
@@ -848,7 +856,7 @@ public:
size_t fstOffRel = fstOff - 0x2440; size_t fstOffRel = fstOff - 0x2440;
if (xferSz > fstOffRel) { if (xferSz > fstOffRel) {
LogModule.report(logvisor::Error, fmt("apploader unexpectedly flows into FST")); spdlog::error("apploader unexpectedly flows into FST");
return -1; return -1;
} }
for (size_t i = 0; i < fstOffRel - xferSz; ++i) for (size_t i = 0; i < fstOffRel - xferSz; ++i)
@@ -896,7 +904,7 @@ public:
}* bfWindow = (BFWindow*)(tmdData.get() + 0x19A); }* bfWindow = (BFWindow*)(tmdData.get() + 0x19A);
bool good = false; bool good = false;
uint64_t attempts = 0; uint64_t attempts = 0;
SystemString bfName(_SYS_STR("Brute force attempts")); std::string bfName("Brute force attempts");
for (int w = 0; w < 7; ++w) { for (int w = 0; w < 7; ++w) {
for (uint64_t i = 0; i < UINT64_MAX; ++i) { for (uint64_t i = 0; i < UINT64_MAX; ++i) {
bfWindow->word[w] = i; bfWindow->word[w] = i;
@@ -924,55 +932,55 @@ public:
return m_baseOffset + dataOff + groupCount * 0x200000; return m_baseOffset + dataOff + groupCount * 0x200000;
} }
uint64_t buildFromDirectory(SystemStringView dirIn) { uint64_t buildFromDirectory(std::string_view dirIn) {
SystemString dirStr(dirIn); std::string dirStr(dirIn);
SystemString basePath = dirStr + _SYS_STR("/") + getKindString(m_kind); std::string basePath = dirStr + "/" + getKindString(m_kind);
/* Check Ticket */ /* Check Ticket */
SystemString ticketIn = basePath + _SYS_STR("/ticket.bin"); std::string ticketIn = basePath + "/ticket.bin";
Sstat theStat; Sstat theStat;
if (Stat(ticketIn.c_str(), &theStat)) { if (Stat(ticketIn.c_str(), &theStat)) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("unable to stat {}")), ticketIn); spdlog::error("unable to stat {}", ticketIn);
return -1; return -1;
} }
/* Check TMD */ /* Check TMD */
SystemString tmdIn = basePath + _SYS_STR("/tmd.bin"); std::string tmdIn = basePath + "/tmd.bin";
Sstat tmdStat; Sstat tmdStat;
if (Stat(tmdIn.c_str(), &tmdStat)) { if (Stat(tmdIn.c_str(), &tmdStat)) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("unable to stat {}")), tmdIn); spdlog::error("unable to stat {}", tmdIn);
return -1; return -1;
} }
/* Check Cert */ /* Check Cert */
SystemString certIn = basePath + _SYS_STR("/cert.bin"); std::string certIn = basePath + "/cert.bin";
Sstat certStat; Sstat certStat;
if (Stat(certIn.c_str(), &certStat)) { if (Stat(certIn.c_str(), &certStat)) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("unable to stat {}")), certIn); spdlog::error("unable to stat {}", certIn);
return -1; return -1;
} }
/* Check Apploader */ /* Check Apploader */
SystemString apploaderIn = basePath + _SYS_STR("/sys/apploader.img"); std::string apploaderIn = basePath + "/sys/apploader.img";
Sstat apploaderStat; Sstat apploaderStat;
if (Stat(apploaderIn.c_str(), &apploaderStat)) { if (Stat(apploaderIn.c_str(), &apploaderStat)) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("unable to stat {}")), apploaderIn); spdlog::error("unable to stat {}", apploaderIn);
return -1; return -1;
} }
/* Check Boot */ /* Check Boot */
SystemString bootIn = basePath + _SYS_STR("/sys/boot.bin"); std::string bootIn = basePath + "/sys/boot.bin";
Sstat bootStat; Sstat bootStat;
if (Stat(bootIn.c_str(), &bootStat)) { if (Stat(bootIn.c_str(), &bootStat)) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("unable to stat {}")), bootIn); spdlog::error("unable to stat {}", bootIn);
return -1; return -1;
} }
/* Check BI2 */ /* Check BI2 */
SystemString bi2In = basePath + _SYS_STR("/sys/bi2.bin"); std::string bi2In = basePath + "/sys/bi2.bin";
Sstat bi2Stat; Sstat bi2Stat;
if (Stat(bi2In.c_str(), &bi2Stat)) { if (Stat(bi2In.c_str(), &bi2Stat)) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("unable to stat {}")), bi2In); spdlog::error("unable to stat {}", bi2In);
return -1; return -1;
} }
@@ -1065,7 +1073,7 @@ public:
cws.write(buf, rdSz); cws.write(buf, rdSz);
xferSz += rdSz; xferSz += rdSz;
if (0x2440 + xferSz >= 0x1F0000) { if (0x2440 + xferSz >= 0x1F0000) {
LogModule.report(logvisor::Error, fmt("apploader flows into user area (one or the other is too big)")); spdlog::error("apploader flows into user area (one or the other is too big)");
return false; return false;
} }
m_parent.m_progressCB(m_parent.getProgressFactor(), apploaderIn, xferSz); m_parent.m_progressCB(m_parent.getProgressFactor(), apploaderIn, xferSz);
@@ -1079,7 +1087,7 @@ public:
apploaderStat.st_size); apploaderStat.st_size);
} }
uint64_t mergeFromDirectory(const PartitionWii* partIn, SystemStringView dirIn) { uint64_t mergeFromDirectory(const PartitionWii* partIn, std::string_view dirIn) {
size_t phSz; size_t phSz;
std::unique_ptr<uint8_t[]> phBuf = partIn->readPartitionHeaderBuf(phSz); std::unique_ptr<uint8_t[]> phBuf = partIn->readPartitionHeaderBuf(phSz);
@@ -1119,11 +1127,11 @@ public:
[this, partIn](IPartWriteStream& cws, size_t& xferSz) -> bool { [this, partIn](IPartWriteStream& cws, size_t& xferSz) -> bool {
std::unique_ptr<uint8_t[]> apploaderBuf = partIn->getApploaderBuf(); std::unique_ptr<uint8_t[]> apploaderBuf = partIn->getApploaderBuf();
size_t apploaderSz = partIn->getApploaderSize(); size_t apploaderSz = partIn->getApploaderSize();
SystemString apploaderName(_SYS_STR("<apploader>")); std::string apploaderName("<apploader>");
cws.write(apploaderBuf.get(), apploaderSz); cws.write(apploaderBuf.get(), apploaderSz);
xferSz += apploaderSz; xferSz += apploaderSz;
if (0x2440 + xferSz >= 0x1F0000) { if (0x2440 + xferSz >= 0x1F0000) {
LogModule.report(logvisor::Error, fmt("apploader flows into user area (one or the other is too big)")); spdlog::error("apploader flows into user area (one or the other is too big)");
return false; return false;
} }
m_parent.m_progressCB(m_parent.getProgressFactor(), apploaderName, xferSz); m_parent.m_progressCB(m_parent.getProgressFactor(), apploaderName, xferSz);
@@ -1137,9 +1145,9 @@ public:
} }
}; };
EBuildResult DiscBuilderWii::buildFromDirectory(SystemStringView dirIn) { EBuildResult DiscBuilderWii::buildFromDirectory(std::string_view dirIn) {
SystemString dirStr(dirIn); std::string dirStr(dirIn);
SystemString basePath = SystemString(dirStr) + _SYS_STR("/") + getKindString(PartitionKind::Data); std::string basePath = std::string(dirStr) + "/" + getKindString(PartitionKind::Data);
PartitionBuilderWii& pb = static_cast<PartitionBuilderWii&>(*m_partitions[0]); PartitionBuilderWii& pb = static_cast<PartitionBuilderWii&>(*m_partitions[0]);
uint64_t filledSz = pb.m_baseOffset; uint64_t filledSz = pb.m_baseOffset;
@@ -1147,10 +1155,10 @@ EBuildResult DiscBuilderWii::buildFromDirectory(SystemStringView dirIn) {
return EBuildResult::Failed; return EBuildResult::Failed;
if (!CheckFreeSpace(m_outPath.c_str(), m_discCapacity)) { if (!CheckFreeSpace(m_outPath.c_str(), m_discCapacity)) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("not enough free disk space for {}")), m_outPath); spdlog::error("not enough free disk space for {}", m_outPath);
return EBuildResult::DiskFull; return EBuildResult::DiskFull;
} }
m_progressCB(getProgressFactor(), _SYS_STR("Preallocating image"), -1); m_progressCB(getProgressFactor(), "Preallocating image", -1);
++m_progressIdx; ++m_progressIdx;
{ {
std::unique_ptr<IFileIO::IWriteStream> ws = m_fileIO->beginWriteStream(0); std::unique_ptr<IFileIO::IWriteStream> ws = m_fileIO->beginWriteStream(0);
@@ -1166,18 +1174,18 @@ EBuildResult DiscBuilderWii::buildFromDirectory(SystemStringView dirIn) {
if (filledSz == UINT64_MAX) if (filledSz == UINT64_MAX)
return EBuildResult::Failed; return EBuildResult::Failed;
else if (filledSz >= uint64_t(m_discCapacity)) { else if (filledSz >= uint64_t(m_discCapacity)) {
LogModule.report(logvisor::Error, fmt("data partition exceeds disc capacity")); spdlog::error("data partition exceeds disc capacity");
return EBuildResult::Failed; return EBuildResult::Failed;
} }
m_progressCB(getProgressFactor(), _SYS_STR("Finishing Disc"), -1); m_progressCB(getProgressFactor(), "Finishing Disc", -1);
++m_progressIdx; ++m_progressIdx;
/* Populate disc header */ /* Populate disc header */
std::unique_ptr<IFileIO::IWriteStream> ws = m_fileIO->beginWriteStream(0); std::unique_ptr<IFileIO::IWriteStream> ws = m_fileIO->beginWriteStream(0);
if (!ws) if (!ws)
return EBuildResult::Failed; return EBuildResult::Failed;
SystemString headerPath = basePath + _SYS_STR("/disc/header.bin"); std::string headerPath = basePath + "/disc/header.bin";
std::unique_ptr<IFileIO::IReadStream> rs = NewFileIO(headerPath.c_str())->beginReadStream(); std::unique_ptr<IFileIO::IReadStream> rs = NewFileIO(headerPath.c_str())->beginReadStream();
if (!rs) if (!rs)
return EBuildResult::Failed; return EBuildResult::Failed;
@@ -1199,7 +1207,7 @@ EBuildResult DiscBuilderWii::buildFromDirectory(SystemStringView dirIn) {
ws->write(vals, 4); ws->write(vals, 4);
/* Populate region info */ /* Populate region info */
SystemString regionPath = basePath + _SYS_STR("/disc/region.bin"); std::string regionPath = basePath + "/disc/region.bin";
rs = NewFileIO(regionPath.c_str())->beginReadStream(); rs = NewFileIO(regionPath.c_str())->beginReadStream();
if (!rs) if (!rs)
return EBuildResult::Failed; return EBuildResult::Failed;
@@ -1229,42 +1237,42 @@ EBuildResult DiscBuilderWii::buildFromDirectory(SystemStringView dirIn) {
return EBuildResult::Success; return EBuildResult::Success;
} }
uint64_t DiscBuilderWii::CalculateTotalSizeRequired(SystemStringView dirIn, bool& dualLayer) { std::optional<uint64_t> DiscBuilderWii::CalculateTotalSizeRequired(std::string_view dirIn, bool& dualLayer) {
uint64_t sz = DiscBuilderBase::PartitionBuilderBase::CalculateTotalSizeBuild(dirIn, PartitionKind::Data, true); std::optional<uint64_t> sz = DiscBuilderBase::PartitionBuilderBase::CalculateTotalSizeBuild(dirIn, PartitionKind::Data, true);
if (sz == UINT64_MAX) if (!sz)
return UINT64_MAX; return sz;
auto szDiv = std::lldiv(sz, 0x1F0000); auto szDiv = nod::div(*sz, uint64_t(0x1F0000));
if (szDiv.rem) if (szDiv.rem)
++szDiv.quot; ++szDiv.quot;
sz = szDiv.quot * 0x200000; sz = szDiv.quot * 0x200000;
sz += 0x200000; *sz += 0x200000;
dualLayer = (sz > 0x118240000); dualLayer = (sz > 0x118240000);
if (sz > 0x1FB4E0000) { if (sz > 0x1FB4E0000) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("disc capacity exceeded [{} / {}]")), sz, 0x1FB4E0000); spdlog::error("disc capacity exceeded [{} / {}]", *sz, 0x1FB4E0000);
return UINT64_MAX; return std::nullopt;
} }
return sz; return sz;
} }
DiscBuilderWii::DiscBuilderWii(SystemStringView outPath, bool dualLayer, FProgress progressCB) DiscBuilderWii::DiscBuilderWii(std::string_view outPath, bool dualLayer, FProgress progressCB)
: DiscBuilderBase(outPath, dualLayer ? 0x1FB4E0000 : 0x118240000, progressCB) { : DiscBuilderBase(outPath, dualLayer ? 0x1FB4E0000 : 0x118240000, progressCB) {
m_partitions.emplace_back(std::make_unique<PartitionBuilderWii>(*this, PartitionKind::Data, 0x200000)); m_partitions.emplace_back(std::make_unique<PartitionBuilderWii>(*this, PartitionKind::Data, 0x200000));
} }
DiscMergerWii::DiscMergerWii(SystemStringView outPath, DiscWii& sourceDisc, bool dualLayer, FProgress progressCB) DiscMergerWii::DiscMergerWii(std::string_view outPath, DiscWii& sourceDisc, bool dualLayer, FProgress progressCB)
: m_sourceDisc(sourceDisc), m_builder(sourceDisc.makeMergeBuilder(outPath, dualLayer, progressCB)) {} : m_sourceDisc(sourceDisc), m_builder(sourceDisc.makeMergeBuilder(outPath, dualLayer, progressCB)) {}
EBuildResult DiscMergerWii::mergeFromDirectory(SystemStringView dirIn) { EBuildResult DiscMergerWii::mergeFromDirectory(std::string_view dirIn) {
PartitionBuilderWii& pb = static_cast<PartitionBuilderWii&>(*m_builder.m_partitions[0]); PartitionBuilderWii& pb = static_cast<PartitionBuilderWii&>(*m_builder.m_partitions[0]);
uint64_t filledSz = pb.m_baseOffset; uint64_t filledSz = pb.m_baseOffset;
if (!m_builder.m_fileIO->beginWriteStream()) if (!m_builder.m_fileIO->beginWriteStream())
return EBuildResult::Failed; return EBuildResult::Failed;
if (!CheckFreeSpace(m_builder.m_outPath.c_str(), m_builder.m_discCapacity)) { if (!CheckFreeSpace(m_builder.m_outPath.c_str(), m_builder.m_discCapacity)) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("not enough free disk space for {}")), m_builder.m_outPath); spdlog::error("not enough free disk space for {}", m_builder.m_outPath);
return EBuildResult::DiskFull; return EBuildResult::DiskFull;
} }
m_builder.m_progressCB(m_builder.getProgressFactor(), _SYS_STR("Preallocating image"), -1); m_builder.m_progressCB(m_builder.getProgressFactor(), "Preallocating image", -1);
++m_builder.m_progressIdx; ++m_builder.m_progressIdx;
{ {
std::unique_ptr<IFileIO::IWriteStream> ws = m_builder.m_fileIO->beginWriteStream(0); std::unique_ptr<IFileIO::IWriteStream> ws = m_builder.m_fileIO->beginWriteStream(0);
@@ -1280,11 +1288,11 @@ EBuildResult DiscMergerWii::mergeFromDirectory(SystemStringView dirIn) {
if (filledSz == UINT64_MAX) if (filledSz == UINT64_MAX)
return EBuildResult::Failed; return EBuildResult::Failed;
else if (filledSz >= uint64_t(m_builder.m_discCapacity)) { else if (filledSz >= uint64_t(m_builder.m_discCapacity)) {
LogModule.report(logvisor::Error, fmt("data partition exceeds disc capacity")); spdlog::error("data partition exceeds disc capacity");
return EBuildResult::Failed; return EBuildResult::Failed;
} }
m_builder.m_progressCB(m_builder.getProgressFactor(), _SYS_STR("Finishing Disc"), -1); m_builder.m_progressCB(m_builder.getProgressFactor(), "Finishing Disc", -1);
++m_builder.m_progressIdx; ++m_builder.m_progressIdx;
/* Populate disc header */ /* Populate disc header */
@@ -1336,19 +1344,19 @@ EBuildResult DiscMergerWii::mergeFromDirectory(SystemStringView dirIn) {
return EBuildResult::Success; return EBuildResult::Success;
} }
uint64_t DiscMergerWii::CalculateTotalSizeRequired(DiscWii& sourceDisc, SystemStringView dirIn, bool& dualLayer) { std::optional<uint64_t> DiscMergerWii::CalculateTotalSizeRequired(DiscWii& sourceDisc, std::string_view dirIn, bool& dualLayer) {
uint64_t sz = DiscBuilderBase::PartitionBuilderBase::CalculateTotalSizeMerge(sourceDisc.getDataPartition(), dirIn); std::optional<uint64_t> sz = DiscBuilderBase::PartitionBuilderBase::CalculateTotalSizeMerge(sourceDisc.getDataPartition(), dirIn);
if (sz == UINT64_MAX) if (!sz)
return UINT64_MAX; return std::nullopt;
auto szDiv = std::lldiv(sz, 0x1F0000); auto szDiv = nod::div(*sz, uint64_t(0x1F0000));
if (szDiv.rem) if (szDiv.rem)
++szDiv.quot; ++szDiv.quot;
sz = szDiv.quot * 0x200000; sz = szDiv.quot * 0x200000;
sz += 0x200000; *sz += 0x200000;
dualLayer = (sz > 0x118240000); dualLayer = (sz > 0x118240000);
if (sz > 0x1FB4E0000) { if (sz > 0x1FB4E0000) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("disc capacity exceeded [{} / {}]")), sz, 0x1FB4E0000); spdlog::error("disc capacity exceeded [{} / {}]", *sz, 0x1FB4E0000);
return UINT64_MAX; return std::nullopt;
} }
return sz; return sz;
} }

View File

@@ -4,21 +4,21 @@
#include <cstdlib> #include <cstdlib>
#include "nod/IFileIO.hpp" #include "nod/IFileIO.hpp"
#include "nod/Util.hpp" #include "Util.hpp"
#include <logvisor/logvisor.hpp> #include <spdlog/spdlog.h>
namespace nod { namespace nod {
class FileIOFILE : public IFileIO { class FileIOFILE : public IFileIO {
SystemString m_path; std::string m_path;
int64_t m_maxWriteSize; int64_t m_maxWriteSize;
public: public:
FileIOFILE(SystemStringView path, int64_t maxWriteSize) : m_path(path), m_maxWriteSize(maxWriteSize) {} FileIOFILE(std::string_view path, int64_t maxWriteSize) : m_path(path), m_maxWriteSize(maxWriteSize) {}
bool exists() override { bool exists() override {
FILE* fp = Fopen(m_path.c_str(), _SYS_STR("rb")); FILE* fp = Fopen(m_path.c_str(), "rb");
if (!fp) if (!fp)
return false; return false;
fclose(fp); fclose(fp);
@@ -26,7 +26,7 @@ public:
} }
uint64_t size() override { uint64_t size() override {
FILE* fp = Fopen(m_path.c_str(), _SYS_STR("rb")); FILE* fp = Fopen(m_path.c_str(), "rb");
if (!fp) if (!fp)
return 0; return 0;
FSeek(fp, 0, SEEK_END); FSeek(fp, 0, SEEK_END);
@@ -38,34 +38,33 @@ public:
struct WriteStream : public IFileIO::IWriteStream { struct WriteStream : public IFileIO::IWriteStream {
FILE* fp; FILE* fp;
int64_t m_maxWriteSize; int64_t m_maxWriteSize;
WriteStream(SystemStringView path, int64_t maxWriteSize, bool& err) : m_maxWriteSize(maxWriteSize) { WriteStream(std::string_view path, int64_t maxWriteSize, bool& err) : m_maxWriteSize(maxWriteSize) {
fp = Fopen(path.data(), _SYS_STR("wb")); fp = Fopen(path.data(), "wb");
if (!fp) { if (!fp) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("unable to open '{}' for writing")), path); spdlog::error("unable to open '{}' for writing", path);
err = true; err = true;
} }
} }
WriteStream(SystemStringView path, uint64_t offset, int64_t maxWriteSize, bool& err) WriteStream(std::string_view path, uint64_t offset, int64_t maxWriteSize, bool& err)
: m_maxWriteSize(maxWriteSize) { : m_maxWriteSize(maxWriteSize) {
fp = Fopen(path.data(), _SYS_STR("ab")); fp = Fopen(path.data(), "ab");
if (!fp) if (!fp)
goto FailLoc; goto FailLoc;
fclose(fp); fclose(fp);
fp = Fopen(path.data(), _SYS_STR("r+b")); fp = Fopen(path.data(), "r+b");
if (!fp) if (!fp)
goto FailLoc; goto FailLoc;
FSeek(fp, offset, SEEK_SET); FSeek(fp, offset, SEEK_SET);
return; return;
FailLoc: FailLoc:
LogModule.report(logvisor::Error, fmt(_SYS_STR("unable to open '{}' for writing")), path); spdlog::error("unable to open '{}' for writing", path);
err = true; err = true;
} }
~WriteStream() override { fclose(fp); } ~WriteStream() override { fclose(fp); }
uint64_t write(const void* buf, uint64_t length) override { uint64_t write(const void* buf, uint64_t length) override {
if (m_maxWriteSize >= 0) { if (m_maxWriteSize >= 0) {
if (FTell(fp) + length > m_maxWriteSize) { if (FTell(fp) + length > m_maxWriteSize) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("write operation exceeds file's {}-byte limit")), spdlog::error("write operation exceeds file's {}-byte limit", m_maxWriteSize);
m_maxWriteSize);
return 0; return 0;
} }
} }
@@ -77,9 +76,8 @@ public:
bool err = false; bool err = false;
auto ret = std::unique_ptr<IWriteStream>(new WriteStream(m_path, m_maxWriteSize, err)); auto ret = std::unique_ptr<IWriteStream>(new WriteStream(m_path, m_maxWriteSize, err));
if (err) { if (err)
return nullptr; return {};
}
return ret; return ret;
} }
@@ -88,23 +86,22 @@ public:
bool err = false; bool err = false;
auto ret = std::unique_ptr<IWriteStream>(new WriteStream(m_path, offset, m_maxWriteSize, err)); auto ret = std::unique_ptr<IWriteStream>(new WriteStream(m_path, offset, m_maxWriteSize, err));
if (err) { if (err)
return nullptr; return {};
}
return ret; return ret;
} }
struct ReadStream : public IFileIO::IReadStream { struct ReadStream : public IFileIO::IReadStream {
FILE* fp; FILE* fp;
ReadStream(SystemStringView path, bool& err) { ReadStream(std::string_view path, bool& err) {
fp = Fopen(path.data(), _SYS_STR("rb")); fp = Fopen(path.data(), "rb");
if (!fp) { if (!fp) {
err = true; err = true;
LogModule.report(logvisor::Error, fmt(_SYS_STR("unable to open '{}' for reading")), path); spdlog::error("unable to open '{}' for reading", path);
} }
} }
ReadStream(SystemStringView path, uint64_t offset, bool& err) : ReadStream(path, err) { ReadStream(std::string_view path, uint64_t offset, bool& err) : ReadStream(path, err) {
if (err) if (err)
return; return;
FSeek(fp, offset, SEEK_SET); FSeek(fp, offset, SEEK_SET);
@@ -119,11 +116,11 @@ public:
while (length) { while (length) {
uint64_t thisSz = nod::min(uint64_t(0x7c00), length); uint64_t thisSz = nod::min(uint64_t(0x7c00), length);
if (read(buf, thisSz) != thisSz) { if (read(buf, thisSz) != thisSz) {
LogModule.report(logvisor::Error, fmt("unable to read enough from file")); spdlog::error("unable to read enough from file");
return written; return written;
} }
if (discio.write(buf, thisSz) != thisSz) { if (discio.write(buf, thisSz) != thisSz) {
LogModule.report(logvisor::Error, fmt("unable to write enough to disc")); spdlog::error("unable to write enough to disc");
return written; return written;
} }
length -= thisSz; length -= thisSz;
@@ -137,9 +134,8 @@ public:
bool err = false; bool err = false;
auto ret = std::unique_ptr<IReadStream>(new ReadStream(m_path, err)); auto ret = std::unique_ptr<IReadStream>(new ReadStream(m_path, err));
if (err) { if (err)
return nullptr; return {};
}
return ret; return ret;
} }
@@ -148,15 +144,14 @@ public:
bool err = false; bool err = false;
auto ret = std::unique_ptr<IReadStream>(new ReadStream(m_path, offset, err)); auto ret = std::unique_ptr<IReadStream>(new ReadStream(m_path, offset, err));
if (err) { if (err)
return nullptr; return {};
}
return ret; return ret;
} }
}; };
std::unique_ptr<IFileIO> NewFileIO(SystemStringView path, int64_t maxWriteSize) { std::unique_ptr<IFileIO> NewFileIO(std::string_view path, int64_t maxWriteSize) {
return std::make_unique<FileIOFILE>(path, maxWriteSize); return std::make_unique<FileIOFILE>(path, maxWriteSize);
} }

View File

@@ -1,28 +1,38 @@
#include <cinttypes>
#include <cstdint> #include <cstdint>
#include <cstdio> #if _WIN32
#include <cstdlib> #ifndef WIN32_LEAN_AND_MEAN
#define WIN32_LEAN_AND_MEAN 1
#endif
#ifndef NOMINMAX
#define NOMINMAX
#endif
#include <Windows.h>
#endif
#include "nod/IFileIO.hpp" #include "nod/IFileIO.hpp"
#include "nod/Util.hpp" #include "Util.hpp"
#include <logvisor/logvisor.hpp> #include <spdlog/spdlog.h>
#include <nowide/convert.hpp>
#include <nowide/stackstring.hpp>
namespace nod { namespace nod {
class FileIOWin32 : public IFileIO { class FileIOWin32 : public IFileIO {
SystemString m_path; std::wstring m_wpath;
int64_t m_maxWriteSize; int64_t m_maxWriteSize;
public: public:
FileIOWin32(SystemStringView path, int64_t maxWriteSize) : m_path(path), m_maxWriteSize(maxWriteSize) {} FileIOWin32(std::string_view path, int64_t maxWriteSize)
: m_wpath(nowide::widen(path)), m_maxWriteSize(maxWriteSize) {}
bool exists() override { bool exists() override {
#if !WINDOWS_STORE #if !WINDOWS_STORE
HANDLE fp = CreateFileW(m_path.c_str(), GENERIC_READ, FILE_SHARE_READ, nullptr, OPEN_EXISTING, HANDLE fp = CreateFileW(m_wpath.data(), GENERIC_READ, FILE_SHARE_READ, nullptr, OPEN_EXISTING,
FILE_ATTRIBUTE_NORMAL, nullptr); FILE_ATTRIBUTE_NORMAL, nullptr);
#else #else
HANDLE fp = CreateFile2(m_path.c_str(), GENERIC_READ, FILE_SHARE_READ, OPEN_EXISTING, nullptr); HANDLE fp = CreateFile2(m_path.get(), GENERIC_READ, FILE_SHARE_READ, OPEN_EXISTING, nullptr);
#endif #endif
if (fp == INVALID_HANDLE_VALUE) if (fp == INVALID_HANDLE_VALUE)
return false; return false;
@@ -32,10 +42,10 @@ public:
uint64_t size() override { uint64_t size() override {
#if !WINDOWS_STORE #if !WINDOWS_STORE
HANDLE fp = CreateFileW(m_path.c_str(), GENERIC_READ, FILE_SHARE_READ, nullptr, OPEN_EXISTING, HANDLE fp = CreateFileW(m_wpath.data(), GENERIC_READ, FILE_SHARE_READ, nullptr, OPEN_EXISTING,
FILE_ATTRIBUTE_NORMAL, nullptr); FILE_ATTRIBUTE_NORMAL, nullptr);
#else #else
HANDLE fp = CreateFile2(m_path.c_str(), GENERIC_READ, FILE_SHARE_READ, OPEN_EXISTING, nullptr); HANDLE fp = CreateFile2(m_path.get(), GENERIC_READ, FILE_SHARE_READ, OPEN_EXISTING, nullptr);
#endif #endif
if (fp == INVALID_HANDLE_VALUE) if (fp == INVALID_HANDLE_VALUE)
return 0; return 0;
@@ -51,28 +61,30 @@ public:
struct WriteStream : public IFileIO::IWriteStream { struct WriteStream : public IFileIO::IWriteStream {
HANDLE fp; HANDLE fp;
int64_t m_maxWriteSize; int64_t m_maxWriteSize;
WriteStream(SystemStringView path, int64_t maxWriteSize, bool& err) : m_maxWriteSize(maxWriteSize) { WriteStream(std::wstring_view wpath, int64_t maxWriteSize, bool& err) : m_maxWriteSize(maxWriteSize) {
#if !WINDOWS_STORE #if !WINDOWS_STORE
fp = CreateFileW(path.data(), GENERIC_WRITE, FILE_SHARE_WRITE, nullptr, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, fp = CreateFileW(wpath.data(), GENERIC_WRITE, FILE_SHARE_WRITE, nullptr, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL,
nullptr); nullptr);
#else #else
fp = CreateFile2(path.data(), GENERIC_WRITE, FILE_SHARE_WRITE, CREATE_ALWAYS, nullptr); fp = CreateFile2(wpath.data(), GENERIC_WRITE, FILE_SHARE_WRITE, CREATE_ALWAYS, nullptr);
#endif #endif
if (fp == INVALID_HANDLE_VALUE) { if (fp == INVALID_HANDLE_VALUE) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("unable to open '{}' for writing")), path); const nowide::stackstring path(wpath.data(), wpath.data() + wpath.size());
spdlog::error("unable to open '{}' for writing", path.get());
err = true; err = true;
} }
} }
WriteStream(SystemStringView path, uint64_t offset, int64_t maxWriteSize, bool& err) WriteStream(std::wstring_view wpath, uint64_t offset, int64_t maxWriteSize, bool& err)
: m_maxWriteSize(maxWriteSize) { : m_maxWriteSize(maxWriteSize) {
#if !WINDOWS_STORE #if !WINDOWS_STORE
fp = CreateFileW(path.data(), GENERIC_WRITE, FILE_SHARE_WRITE, nullptr, OPEN_ALWAYS, FILE_ATTRIBUTE_NORMAL, fp = CreateFileW(wpath.data(), GENERIC_WRITE, FILE_SHARE_WRITE, nullptr, OPEN_ALWAYS, FILE_ATTRIBUTE_NORMAL,
nullptr); nullptr);
#else #else
fp = CreateFile2(path.data(), GENERIC_WRITE, FILE_SHARE_WRITE, OPEN_ALWAYS, nullptr); fp = CreateFile2(wpath.data(), GENERIC_WRITE, FILE_SHARE_WRITE, OPEN_ALWAYS, nullptr);
#endif #endif
if (fp == INVALID_HANDLE_VALUE) { if (fp == INVALID_HANDLE_VALUE) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("unable to open '{}' for writing")), path); const nowide::stackstring path(wpath.data(), wpath.data() + wpath.size());
spdlog::error("unable to open '{}' for writing", path.get());
err = true; err = true;
return; return;
} }
@@ -87,8 +99,7 @@ public:
LARGE_INTEGER res; LARGE_INTEGER res;
SetFilePointerEx(fp, li, &res, FILE_CURRENT); SetFilePointerEx(fp, li, &res, FILE_CURRENT);
if (res.QuadPart + int64_t(length) > m_maxWriteSize) { if (res.QuadPart + int64_t(length) > m_maxWriteSize) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("write operation exceeds file's {}-byte limit")), spdlog::error("write operation exceeds file's {}-byte limit", m_maxWriteSize);
m_maxWriteSize);
return 0; return 0;
} }
} }
@@ -100,7 +111,7 @@ public:
}; };
std::unique_ptr<IWriteStream> beginWriteStream() const override { std::unique_ptr<IWriteStream> beginWriteStream() const override {
bool err = false; bool err = false;
auto ret = std::make_unique<WriteStream>(m_path, m_maxWriteSize, err); auto ret = std::make_unique<WriteStream>(m_wpath, m_maxWriteSize, err);
if (err) { if (err) {
return nullptr; return nullptr;
@@ -110,7 +121,7 @@ public:
} }
std::unique_ptr<IWriteStream> beginWriteStream(uint64_t offset) const override { std::unique_ptr<IWriteStream> beginWriteStream(uint64_t offset) const override {
bool err = false; bool err = false;
auto ret = std::make_unique<WriteStream>(m_path, offset, m_maxWriteSize, err); auto ret = std::make_unique<WriteStream>(m_wpath, offset, m_maxWriteSize, err);
if (err) { if (err) {
return nullptr; return nullptr;
@@ -121,19 +132,20 @@ public:
struct ReadStream : public IFileIO::IReadStream { struct ReadStream : public IFileIO::IReadStream {
HANDLE fp; HANDLE fp;
ReadStream(SystemStringView path, bool& err) { ReadStream(std::wstring_view wpath, bool& err) {
#if !WINDOWS_STORE #if !WINDOWS_STORE
fp = CreateFileW(path.data(), GENERIC_READ, FILE_SHARE_READ, nullptr, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, fp = CreateFileW(wpath.data(), GENERIC_READ, FILE_SHARE_READ, nullptr, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL,
nullptr); nullptr);
#else #else
fp = CreateFile2(path.data(), GENERIC_READ, FILE_SHARE_READ, OPEN_EXISTING, nullptr); fp = CreateFile2(wpath.data(), GENERIC_READ, FILE_SHARE_READ, OPEN_EXISTING, nullptr);
#endif #endif
if (fp == INVALID_HANDLE_VALUE) { if (fp == INVALID_HANDLE_VALUE) {
err = true; err = true;
LogModule.report(logvisor::Error, fmt(_SYS_STR("unable to open '{}' for reading")), path); const nowide::stackstring path(wpath.data(), wpath.data() + wpath.size());
spdlog::error("unable to open '{}' for reading", path.get());
} }
} }
ReadStream(SystemStringView path, uint64_t offset, bool& err) : ReadStream(path, err) { ReadStream(std::wstring_view wpath, uint64_t offset, bool& err) : ReadStream(wpath, err) {
if (err) if (err)
return; return;
LARGE_INTEGER lioffset; LARGE_INTEGER lioffset;
@@ -163,11 +175,11 @@ public:
while (length) { while (length) {
uint64_t thisSz = nod::min(uint64_t(0x7c00), length); uint64_t thisSz = nod::min(uint64_t(0x7c00), length);
if (read(buf, thisSz) != thisSz) { if (read(buf, thisSz) != thisSz) {
LogModule.report(logvisor::Error, fmt("unable to read enough from file")); spdlog::error("unable to read enough from file");
return written; return written;
} }
if (discio.write(buf, thisSz) != thisSz) { if (discio.write(buf, thisSz) != thisSz) {
LogModule.report(logvisor::Error, fmt("unable to write enough to disc")); spdlog::error("unable to write enough to disc");
return written; return written;
} }
length -= thisSz; length -= thisSz;
@@ -179,7 +191,7 @@ public:
std::unique_ptr<IReadStream> beginReadStream() const override { std::unique_ptr<IReadStream> beginReadStream() const override {
bool err = false; bool err = false;
auto ret = std::make_unique<ReadStream>(m_path, err); auto ret = std::make_unique<ReadStream>(m_wpath, err);
if (err) { if (err) {
return nullptr; return nullptr;
@@ -190,7 +202,7 @@ public:
std::unique_ptr<IReadStream> beginReadStream(uint64_t offset) const override { std::unique_ptr<IReadStream> beginReadStream(uint64_t offset) const override {
bool err = false; bool err = false;
auto ret = std::make_unique<ReadStream>(m_path, offset, err); auto ret = std::make_unique<ReadStream>(m_wpath, offset, err);
if (err) { if (err) {
return nullptr; return nullptr;
@@ -200,7 +212,7 @@ public:
} }
}; };
std::unique_ptr<IFileIO> NewFileIO(SystemStringView path, int64_t maxWriteSize) { std::unique_ptr<IFileIO> NewFileIO(std::string_view path, int64_t maxWriteSize) {
return std::make_unique<FileIOWin32>(path, maxWriteSize); return std::make_unique<FileIOWin32>(path, maxWriteSize);
} }

48
lib/IFileIO.cpp Normal file
View File

@@ -0,0 +1,48 @@
#include "nod/IFileIO.hpp"
#include "Util.hpp"
#include <spdlog/spdlog.h>
namespace nod {
uint64_t IFileIO::IWriteStream::copyFromDisc(IPartReadStream& discio, uint64_t length) {
uint64_t read = 0;
uint8_t buf[0x7c00];
while (length) {
uint64_t thisSz = nod::min(uint64_t(0x7c00), length);
uint64_t readSz = discio.read(buf, thisSz);
if (thisSz != readSz) {
spdlog::error("unable to read enough from disc");
return read;
}
if (write(buf, readSz) != readSz) {
spdlog::error("unable to write in file");
return read;
}
length -= thisSz;
read += thisSz;
}
return read;
}
uint64_t IFileIO::IWriteStream::copyFromDisc(IPartReadStream& discio, uint64_t length,
const std::function<void(float)>& prog) {
uint64_t read = 0;
uint8_t buf[0x7c00];
uint64_t total = length;
while (length) {
uint64_t thisSz = nod::min(uint64_t(0x7c00), length);
uint64_t readSz = discio.read(buf, thisSz);
if (thisSz != readSz) {
spdlog::error("unable to read enough from disc");
return read;
}
if (write(buf, readSz) != readSz) {
spdlog::error("unable to write in file");
return read;
}
length -= thisSz;
read += thisSz;
prog(read / float(total));
}
return read;
}
} // namespace nod

3149
lib/OSUTF.c Normal file

File diff suppressed because it is too large Load Diff

71
lib/Util.cpp Normal file
View File

@@ -0,0 +1,71 @@
#include "Util.hpp"
#include <spdlog/spdlog.h>
#if _WIN32
#ifndef WIN32_LEAN_AND_MEAN
#define WIN32_LEAN_AND_MEAN 1
#endif
#ifndef NOMINMAX
#define NOMINMAX
#endif
#include <Windows.h>
#endif
namespace nod {
FILE* Fopen(const char* path, const char* mode, FileLockType lock) {
#if _WIN32
const nowide::wstackstring wpath(path);
const nowide::wshort_stackstring wmode(mode);
FILE* fp = _wfopen(wpath.get(), wmode.get());
if (!fp)
return nullptr;
#else
FILE* fp = fopen(path, mode);
if (!fp)
return nullptr;
#endif
if (lock != FileLockType::None) {
#if _WIN32
OVERLAPPED ov = {};
LockFileEx((HANDLE)(uintptr_t)_fileno(fp), (lock == FileLockType::Write) ? LOCKFILE_EXCLUSIVE_LOCK : 0, 0, 0, 1,
&ov);
#else
if (flock(fileno(fp), ((lock == FileLockType::Write) ? LOCK_EX : LOCK_SH) | LOCK_NB))
spdlog::error("flock {}: {}", path, strerror(errno));
#endif
}
return fp;
}
bool CheckFreeSpace(const char* path, size_t reqSz) {
#if _WIN32
ULARGE_INTEGER freeBytes;
const nowide::wstackstring wpath(path);
std::array<wchar_t, 1024> buf{};
wchar_t* end = nullptr;
DWORD ret = GetFullPathNameW(wpath.get(), 1024, buf.data(), &end);
if (ret == 0 || ret > 1024) {
spdlog::error("GetFullPathNameW {}", path);
return false;
}
if (end != nullptr) {
end[0] = L'\0';
}
if (!GetDiskFreeSpaceExW(buf.data(), &freeBytes, nullptr, nullptr)) {
spdlog::error("GetDiskFreeSpaceExW {}: {}", path, GetLastError());
return false;
}
return reqSz < freeBytes.QuadPart;
#else
struct statvfs svfs;
if (statvfs(path, &svfs)) {
spdlog::error("statvfs {}: {}", path, strerror(errno));
return false;
}
return reqSz < svfs.f_frsize * svfs.f_bavail;
#endif
}
} // namespace nod

126
lib/Util.hpp Normal file
View File

@@ -0,0 +1,126 @@
#pragma once
#if _WIN32
#include <array>
#include <cwctype>
#include <direct.h>
#include <nowide/stackstring.hpp>
#include <winapifamily.h>
#if defined(WINAPI_FAMILY) && WINAPI_FAMILY != WINAPI_FAMILY_DESKTOP_APP
#define WINDOWS_STORE 1
#else
#define WINDOWS_STORE 0
#endif
#else
#include <cctype>
#include <cerrno>
#include <sys/file.h>
#include <sys/param.h>
#include <sys/statvfs.h>
#include <unistd.h>
#endif
#include <sys/stat.h>
#include <algorithm>
#include <cstring>
#include <string>
#include <string_view>
#ifdef _MSC_VER
#pragma warning(disable : 4996)
#include <sys/stat.h>
#if !defined(S_ISREG) && defined(S_IFMT) && defined(S_IFREG)
#define S_ISREG(m) (((m)&S_IFMT) == S_IFREG)
#endif
#if !defined(S_ISDIR) && defined(S_IFMT) && defined(S_IFDIR)
#define S_ISDIR(m) (((m)&S_IFMT) == S_IFDIR)
#endif
#if !defined(S_ISLNK)
#define S_ISLNK(m) 0
#endif
#endif
#undef min
#undef max
namespace nod {
/* define our own min/max to avoid MSVC BS */
template <typename T>
constexpr T min(T a, T b) {
return a < b ? a : b;
}
template <typename T>
constexpr T max(T a, T b) {
return a > b ? a : b;
}
/* template-based div for flexible typing and avoiding a library call */
template <typename T>
constexpr auto div(T a, T b) {
struct DivTp {
T quot, rem;
};
return DivTp{a / b, a % b};
}
/* filesystem char type */
#if _WIN32
static inline int Mkdir(const char* path, int) {
const nowide::wstackstring str(path);
return _wmkdir(str.get());
}
using Sstat = struct ::_stat64;
static inline int Stat(const char* path, Sstat* statout) {
const nowide::wstackstring wpath(path);
return _wstat64(wpath.get(), statout);
}
#else
static inline int Mkdir(const char* path, mode_t mode) { return mkdir(path, mode); }
typedef struct stat Sstat;
static inline int Stat(const char* path, Sstat* statout) { return stat(path, statout); }
#endif
static inline int StrCaseCmp(const char* str1, const char* str2) {
#ifdef _MSC_VER
return _stricmp(str1, str2);
#else
return strcasecmp(str1, str2);
#endif
}
#ifndef ROUND_UP_32
#define ROUND_UP_32(val) (((val) + 31) & ~31)
#define ROUND_UP_16(val) (((val) + 15) & ~15)
#endif
enum class FileLockType { None = 0, Read, Write };
FILE* Fopen(const char* path, const char* mode, FileLockType lock = FileLockType::None);
static inline int FSeek(FILE* fp, int64_t offset, int whence) {
#if _WIN32
return _fseeki64(fp, offset, whence);
#elif __APPLE__ || __FreeBSD__
return fseeko(fp, offset, whence);
#else
return fseeko64(fp, offset, whence);
#endif
}
static inline int64_t FTell(FILE* fp) {
#if _WIN32
return _ftelli64(fp);
#elif __APPLE__ || __FreeBSD__
return ftello(fp);
#else
return ftello64(fp);
#endif
}
bool CheckFreeSpace(const char* path, size_t reqSz);
} // namespace nod

View File

@@ -4,7 +4,7 @@
#if _WIN32 #if _WIN32
#include <intrin.h> #include <intrin.h>
#else #elif __x86_64__
#include <cpuid.h> #include <cpuid.h>
#endif #endif

View File

@@ -6,18 +6,18 @@
#include "nod/DiscGCN.hpp" #include "nod/DiscGCN.hpp"
#include "nod/DiscWii.hpp" #include "nod/DiscWii.hpp"
#include <spdlog/spdlog.h>
namespace nod { namespace nod {
std::unique_ptr<IDiscIO> NewDiscIOISO(std::string_view path);
std::unique_ptr<IDiscIO> NewDiscIOWBFS(std::string_view path);
std::unique_ptr<IDiscIO> NewDiscIONFS(std::string_view path);
logvisor::Module LogModule("nod"); std::unique_ptr<DiscBase> OpenDiscFromImage(std::string_view path, bool& isWii) {
std::unique_ptr<IDiscIO> NewDiscIOISO(SystemStringView path);
std::unique_ptr<IDiscIO> NewDiscIOWBFS(SystemStringView path);
std::unique_ptr<DiscBase> OpenDiscFromImage(SystemStringView path, bool& isWii) {
/* Temporary file handle to determine image type */ /* Temporary file handle to determine image type */
std::unique_ptr<IFileIO> fio = NewFileIO(path); std::unique_ptr<IFileIO> fio = NewFileIO(path);
if (!fio->exists()) { if (!fio->exists()) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("Unable to open '{}'")), path); spdlog::error("Unable to open '{}'", path);
return {}; return {};
} }
std::unique_ptr<IFileIO::IReadStream> rs = fio->beginReadStream(); std::unique_ptr<IFileIO::IReadStream> rs = fio->beginReadStream();
@@ -28,13 +28,21 @@ std::unique_ptr<DiscBase> OpenDiscFromImage(SystemStringView path, bool& isWii)
std::unique_ptr<IDiscIO> discIO; std::unique_ptr<IDiscIO> discIO;
uint32_t magic = 0; uint32_t magic = 0;
if (rs->read(&magic, 4) != 4) { if (rs->read(&magic, 4) != 4) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("Unable to read magic from '{}'")), path); spdlog::error("Unable to read magic from '{}'", path);
return {}; return {};
} }
using SignedSize = std::make_signed<std::string::size_type>::type;
const auto dotPos = SignedSize(path.rfind('.'));
const auto slashPos = SignedSize(path.find_last_of("/\\"));
if (magic == nod::SBig((uint32_t)'WBFS')) { if (magic == nod::SBig((uint32_t)'WBFS')) {
discIO = NewDiscIOWBFS(path); discIO = NewDiscIOWBFS(path);
isWii = true; isWii = true;
} else if (path.size() > 4 && dotPos != -1 && dotPos > slashPos &&
!path.compare(slashPos + 1, 4, "hif_") &&
!path.compare(dotPos, path.size() - dotPos, ".nfs")) {
discIO = NewDiscIONFS(path);
isWii = true;
} else { } else {
rs->seek(0x18, SEEK_SET); rs->seek(0x18, SEEK_SET);
rs->read(&magic, 4); rs->read(&magic, 4);
@@ -51,7 +59,7 @@ std::unique_ptr<DiscBase> OpenDiscFromImage(SystemStringView path, bool& isWii)
} }
if (!discIO) { if (!discIO) {
LogModule.report(logvisor::Error, fmt(_SYS_STR("'{}' is not a valid image")), path); spdlog::error("'{}' is not a valid image", path);
return {}; return {};
} }
@@ -59,20 +67,18 @@ std::unique_ptr<DiscBase> OpenDiscFromImage(SystemStringView path, bool& isWii)
std::unique_ptr<DiscBase> ret; std::unique_ptr<DiscBase> ret;
if (isWii) { if (isWii) {
ret = std::make_unique<DiscWii>(std::move(discIO), err); ret = std::make_unique<DiscWii>(std::move(discIO), err);
if (err) { if (err)
return nullptr; return {};
}
return ret; return ret;
} }
ret = std::make_unique<DiscGCN>(std::move(discIO), err); ret = std::make_unique<DiscGCN>(std::move(discIO), err);
if (err) { if (err)
return nullptr; return {};
}
return ret; return ret;
} }
std::unique_ptr<DiscBase> OpenDiscFromImage(SystemStringView path) { std::unique_ptr<DiscBase> OpenDiscFromImage(std::string_view path) {
bool isWii; bool isWii;
return OpenDiscFromImage(path, isWii); return OpenDiscFromImage(path, isWii);
} }

Submodule logvisor deleted from f623ace3b4