Browse Source
[cmake, tools, docs] Update CPMUtil (#2790)
[cmake, tools, docs] Update CPMUtil (#2790)
updates CPMUtil to
[`8f9b39fc98`](8f9b39fc98)
Makes the tooling/docs better and enforces usage of tar.gz everywhere as
opposed to zips
Signed-off-by: crueter <crueter@eden-emu.dev>
Reviewed-on: https://git.eden-emu.dev/eden-emu/eden/pulls/2790
Reviewed-by: CamilleLaVey <camillelavey99@gmail.com>
Reviewed-by: MaranBr <maranbr@eden-emu.dev>
pull/2794/head
crueter
2 months ago
No known key found for this signature in database
GPG Key ID: 425ACD2D4830EBC6
30 changed files with 569 additions and 117 deletions
-
4.ci/license-header.sh
-
5CMakeModules/CPM.cmake
-
4CMakeModules/CPMUtil.cmake
-
8cpmfile.json
-
17docs/CPMUtil.md
-
17docs/CPMUtil/AddCIPackage
-
17docs/CPMUtil/AddCIPackage.md
-
104docs/CPMUtil/AddJsonPackage.md
-
116docs/CPMUtil/AddPackage.md
-
46docs/CPMUtil/README.md
-
6docs/README.md
-
14externals/cpmfile.json
-
2externals/ffmpeg/cpmfile.json
-
6src/dynarmic/externals/cpmfile.json
-
4src/qt_common/externals/cpmfile.json
-
24tools/cpm/README.md
-
4tools/cpm/check-hash-all.sh
-
53tools/cpm/check-hash.sh
-
4tools/cpm/check-updates-all.sh
-
45tools/cpm/check-updates.sh
-
15tools/cpm/common.sh
-
3tools/cpm/download.sh
-
2tools/cpm/fetch-all.sh
-
30tools/cpm/fetch.sh
-
5tools/cpm/format.sh
-
94tools/cpm/hash.sh
-
9tools/cpm/package.sh
-
4tools/cpm/which.sh
-
10tools/shellcheck.sh
-
14tools/update-cpm.sh
@ -1,17 +0,0 @@ |
|||||
# CPMUtil |
|
||||
|
|
||||
CPMUtil is a wrapper around CPM that aims to reduce boilerplate and add useful utility functions to make dependency management a piece of cake. |
|
||||
|
|
||||
See more in [its repository](https://git.crueter.xyz/CMake/CPMUtil) |
|
||||
|
|
||||
Eden-specific options: |
|
||||
|
|
||||
- `YUZU_USE_CPM` is set by default on MSVC and Android. Other platforms should use this if certain "required" system dependencies (e.g. OpenSSL) are broken or missing |
|
||||
* If this is `OFF`, required system dependencies will be searched via `find_package`, although most externals use CPM regardless. |
|
||||
- Force system libraries via CMake arguments: |
|
||||
* SDL2: `YUZU_USE_BUNDLED_SDL2` and `YUZU_USE_EXTERNAL_SDL2` |
|
||||
* FFmpeg: `YUZU_USE_EXTERNAL_FFMPEG` |
|
||||
|
|
||||
## Tooling |
|
||||
|
|
||||
See the [tooling docs](../tools/cpm) |
|
||||
@ -0,0 +1,17 @@ |
|||||
|
# AddPackage |
||||
|
|
||||
|
- `VERSION` (required): The version to get (the tag will be `v${VERSION}`) |
||||
|
- `NAME` (required): Name used within the artifacts |
||||
|
- `REPO` (required): CI repository, e.g. `crueter-ci/OpenSSL` |
||||
|
- `PACKAGE` (required): `find_package` package name |
||||
|
- `EXTENSION`: Artifact extension (default `tar.zst`) |
||||
|
- `MIN_VERSION`: Minimum version for `find_package`. Only used if platform does not support this package as a bundled artifact |
||||
|
- `DISABLED_PLATFORMS`: List of platforms that lack artifacts for this package. Options: |
||||
|
* `windows-amd64` |
||||
|
* `windows-arm64` |
||||
|
* `android` |
||||
|
* `solaris-amd64` |
||||
|
* `freebsd-amd64` |
||||
|
* `linux-amd64` |
||||
|
* `linux-aarch64` |
||||
|
* `macos-universal` |
||||
@ -0,0 +1,17 @@ |
|||||
|
# AddPackage |
||||
|
|
||||
|
- `VERSION` (required): The version to get (the tag will be `v${VERSION}`) |
||||
|
- `NAME` (required): Name used within the artifacts |
||||
|
- `REPO` (required): CI repository, e.g. `crueter-ci/OpenSSL` |
||||
|
- `PACKAGE` (required): `find_package` package name |
||||
|
- `EXTENSION`: Artifact extension (default `tar.zst`) |
||||
|
- `MIN_VERSION`: Minimum version for `find_package`. Only used if platform does not support this package as a bundled artifact |
||||
|
- `DISABLED_PLATFORMS`: List of platforms that lack artifacts for this package. Options: |
||||
|
* `windows-amd64` |
||||
|
* `windows-arm64` |
||||
|
* `android` |
||||
|
* `solaris-amd64` |
||||
|
* `freebsd-amd64` |
||||
|
* `linux-amd64` |
||||
|
* `linux-aarch64` |
||||
|
* `macos-universal` |
||||
@ -0,0 +1,104 @@ |
|||||
|
# AddJsonPackage |
||||
|
|
||||
|
In each directory that utilizes `CPMUtil`, there must be a `cpmfile.json` that defines dependencies in a similar manner to the individual calls. |
||||
|
|
||||
|
The cpmfile is an object of objects, with each sub-object being named according to the package's identifier, e.g. `openssl`, which can then be fetched with `AddJsonPackage(<identifier>)`. Options are designed to map closely to the argument names, and are always strings unless otherwise specified. |
||||
|
<!-- TOC --> |
||||
|
- [Options](#options) |
||||
|
- [Examples](#examples) |
||||
|
<!-- /TOC --> |
||||
|
|
||||
|
## Options |
||||
|
|
||||
|
- `package` -> `NAME` (`PACKAGE` for CI), defaults to the object key |
||||
|
- `repo` -> `REPO` |
||||
|
- `version` -> `VERSION` |
||||
|
- `ci` (bool) |
||||
|
|
||||
|
If `ci` is `false`: |
||||
|
|
||||
|
- `hash` -> `HASH` |
||||
|
- `hash_suffix` -> `HASH_SUFFIX` |
||||
|
- `sha` -> `SHA` |
||||
|
- `key` -> `KEY` |
||||
|
- `tag` -> `TAG` |
||||
|
* If the tag contains `%VERSION%`, that part will be replaced by the `git_version`, OR `version` if `git_version` is not specified |
||||
|
- `url` -> `URL` |
||||
|
- `artifact` -> `ARTIFACT` |
||||
|
* If the artifact contains `%VERSION%`, that part will be replaced by the `git_version`, OR `version` if `git_version` is not specified |
||||
|
* If the artifact contains `%TAG%`, that part will be replaced by the `tag` (with its replacement already done) |
||||
|
- `git_version` -> `GIT_VERSION` |
||||
|
- `git_host` -> `GIT_HOST` |
||||
|
- `source_subdir` -> `SOURCE_SUBDIR` |
||||
|
- `bundled` -> `BUNDLED_PACKAGE` |
||||
|
- `find_args` -> `FIND_PACKAGE_ARGUMENTS` |
||||
|
- `download_only` -> `DOWNLOAD_ONLY` |
||||
|
- `patches` -> `PATCHES` (array) |
||||
|
- `options` -> `OPTIONS` (array) |
||||
|
- `skip_updates`: Tells `check-updates.sh` to not check for new updates on this package. |
||||
|
|
||||
|
Other arguments aren't currently supported. If you wish to add them, see the `AddJsonPackage` function in `CMakeModules/CPMUtil.cmake`. |
||||
|
|
||||
|
If `ci` is `true`: |
||||
|
|
||||
|
- `name` -> `NAME`, defaults to the object key |
||||
|
- `extension` -> `EXTENSION`, defaults to `tar.zst` |
||||
|
- `min_version` -> `MIN_VERSION` |
||||
|
- `extension` -> `EXTENSION` |
||||
|
- `disabled_platforms` -> `DISABLED_PLATFORMS` (array) |
||||
|
|
||||
|
## Examples |
||||
|
|
||||
|
In order: OpenSSL CI, Boost (tag + artifact), Opus (options + find_args), discord-rpc (sha + options + patches). |
||||
|
|
||||
|
```json |
||||
|
{ |
||||
|
"openssl": { |
||||
|
"ci": true, |
||||
|
"package": "OpenSSL", |
||||
|
"name": "openssl", |
||||
|
"repo": "crueter-ci/OpenSSL", |
||||
|
"version": "3.6.0", |
||||
|
"min_version": "1.1.1", |
||||
|
"disabled_platforms": [ |
||||
|
"macos-universal" |
||||
|
] |
||||
|
}, |
||||
|
"boost": { |
||||
|
"package": "Boost", |
||||
|
"repo": "boostorg/boost", |
||||
|
"tag": "boost-%VERSION%", |
||||
|
"artifact": "%TAG%-cmake.7z", |
||||
|
"hash": "e5b049e5b61964480ca816395f63f95621e66cb9bcf616a8b10e441e0e69f129e22443acb11e77bc1e8170f8e4171b9b7719891efc43699782bfcd4b3a365f01", |
||||
|
"git_version": "1.88.0", |
||||
|
"version": "1.57" |
||||
|
}, |
||||
|
"opus": { |
||||
|
"package": "Opus", |
||||
|
"repo": "xiph/opus", |
||||
|
"sha": "5ded705cf4", |
||||
|
"hash": "0dc89e58ddda1f3bc6a7037963994770c5806c10e66f5cc55c59286fc76d0544fe4eca7626772b888fd719f434bc8a92f792bdb350c807968b2ac14cfc04b203", |
||||
|
"version": "1.3", |
||||
|
"find_args": "MODULE", |
||||
|
"options": [ |
||||
|
"OPUS_BUILD_TESTING OFF", |
||||
|
"OPUS_BUILD_PROGRAMS OFF", |
||||
|
"OPUS_INSTALL_PKG_CONFIG_MODULE OFF", |
||||
|
"OPUS_INSTALL_CMAKE_CONFIG_MODULE OFF" |
||||
|
] |
||||
|
}, |
||||
|
"discord-rpc": { |
||||
|
"repo": "discord/discord-rpc", |
||||
|
"sha": "963aa9f3e5", |
||||
|
"hash": "386e1344e9a666d730f2d335ee3aef1fd05b1039febefd51aa751b705009cc764411397f3ca08dffd46205c72f75b235c870c737b2091a4ed0c3b061f5919bde", |
||||
|
"options": [ |
||||
|
"BUILD_EXAMPLES OFF" |
||||
|
], |
||||
|
"patches": [ |
||||
|
"0001-cmake-version.patch", |
||||
|
"0002-no-clang-format.patch", |
||||
|
"0003-fix-cpp17.patch" |
||||
|
] |
||||
|
} |
||||
|
} |
||||
|
``` |
||||
@ -0,0 +1,116 @@ |
|||||
|
# `AddPackage` |
||||
|
|
||||
|
<!-- TOC --> |
||||
|
- [Identification/Fetching](#identificationfetching) |
||||
|
- [Hashing](#hashing) |
||||
|
- [Other Options](#other-options) |
||||
|
- [Extra Variables](#extra-variables) |
||||
|
- [System/Bundled Packages](#systembundled-packages) |
||||
|
- [Identification](#identification) |
||||
|
<!-- /TOC --> |
||||
|
|
||||
|
## Identification/Fetching |
||||
|
|
||||
|
- `NAME` (required): The package name (must be the same as the `find_package` name if applicable) |
||||
|
- `VERSION`: The minimum version of this package that can be used on the system |
||||
|
- `GIT_VERSION`: The "version" found within git |
||||
|
- `URL`: The URL to fetch. |
||||
|
- `REPO`: The repo to use (`owner/repo`). |
||||
|
- `GIT_HOST`: The Git host to use |
||||
|
* Defaults to `github.com`. Do not include the protocol, as HTTPS is enforced. |
||||
|
- `TAG`: The tag to fetch, if applicable. |
||||
|
- `ARTIFACT`: The name of the artifact, if applicable. |
||||
|
- `SHA`: Commit sha to fetch, if applicable. |
||||
|
- `BRANCH`: Branch to fetch, if applicable. |
||||
|
|
||||
|
The following configurations are supported, in descending order of precedence: |
||||
|
|
||||
|
- `URL`: Bare URL download, useful for custom artifacts |
||||
|
* If this is set, `GIT_URL` or `REPO` should be set to allow the dependency viewer to link to the project's Git repository. |
||||
|
* If this is NOT set, `REPO` must be defined. |
||||
|
- `REPO + TAG + ARTIFACT`: GitHub release artifact |
||||
|
* The final download URL will be `https://github.com/${REPO}/releases/download/${TAG}/${ARTIFACT}` |
||||
|
* Useful for prebuilt libraries and prefetched archives |
||||
|
- `REPO + TAG`: GitHub tag archive |
||||
|
* The final download URL will be `https://github.com/${REPO}/archive/refs/tags/${TAG}.tar.gz` |
||||
|
* Useful for pinning to a specific tag, better for build identification |
||||
|
- `REPO + SHA`: GitHub commit archive |
||||
|
* The final download URL will be `https://github.com/${REPO}/archive/${SHA}.zip` |
||||
|
* Useful for pinning to a specific commit |
||||
|
- `REPO + BRANCH`: GitHub branch archive |
||||
|
* The final download URL will be `https://github.com/${REPO}/archive/refs/heads/${BRANCH}.zip` |
||||
|
* Generally not recommended unless the branch is frozen |
||||
|
- `REPO`: GitHub master archive |
||||
|
* The final download URL will be `https://github.com/${REPO}/archive/refs/heads/master.zip` |
||||
|
* Generally not recommended unless the project is dead |
||||
|
|
||||
|
## Hashing |
||||
|
|
||||
|
Hashing is used for verifying downloads. It's highly recommended to use these. |
||||
|
|
||||
|
- `HASH_ALGO` (default `SHA512`): Hash algorithm to use |
||||
|
|
||||
|
Hashing strategies, descending order of precedence: |
||||
|
|
||||
|
- `HASH`: Bare hash verification, useful for static downloads e.g. commit archives |
||||
|
- `HASH_SUFFIX`: Download the hash as `${DOWNLOAD_URL}.${HASH_SUFFIX}` |
||||
|
* The downloaded hash *must* match the hash algorithm and contain nothing but the hash; no filenames or extra content. |
||||
|
- `HASH_URL`: Download the hash from a separate URL |
||||
|
|
||||
|
## Other Options |
||||
|
|
||||
|
- `KEY`: Custom cache key to use (stored as `.cache/cpm/${packagename_lower}/${key}`) |
||||
|
* Default is based on, in descending order of precedence: |
||||
|
- First 4 characters of the sha |
||||
|
- `GIT_VERSION` |
||||
|
- Tag |
||||
|
- `VERSION` |
||||
|
- Otherwise, CPM defaults will be used. This is not recommended as it doesn't produce reproducible caches |
||||
|
- `DOWNLOAD_ONLY`: Whether or not to configure the downloaded package via CMake |
||||
|
* Useful to turn `OFF` if the project doesn't use CMake |
||||
|
- `SOURCE_SUBDIR`: Subdirectory of the project containing a CMakeLists.txt file |
||||
|
- `FIND_PACKAGE_ARGUMENTS`: Arguments to pass to the `find_package` call |
||||
|
- `BUNDLED_PACKAGE`: Set to `ON` to default to the bundled package |
||||
|
- `FORCE_BUNDLED_PACKAGE`: Set to `ON` to force the usage of the bundled package, regardless of CPMUTIL_FORCE_SYSTEM or `<package>_FORCE_SYSTEM` |
||||
|
- `OPTIONS`: Options to pass to the configuration of the package |
||||
|
- `PATCHES`: Patches to apply to the package, stored in `.patch/${packagename_lower}/0001-patch-name.patch` and so on |
||||
|
- Other arguments can be passed to CPM as well |
||||
|
|
||||
|
## Extra Variables |
||||
|
|
||||
|
For each added package, users may additionally force usage of the system/bundled package. |
||||
|
|
||||
|
- `${package}_FORCE_SYSTEM`: Require the package to be installed on the system |
||||
|
- `${package}_FORCE_BUNDLED`: Force the package to be fetched and use the bundled version |
||||
|
|
||||
|
## System/Bundled Packages |
||||
|
|
||||
|
Descending order of precedence: |
||||
|
- If `${package}_FORCE_SYSTEM` is true, requires the package to be on the system |
||||
|
- If `${package}_FORCE_BUNDLED` is true, forcefully uses the bundled package |
||||
|
- If `CPMUTIL_FORCE_SYSTEM` is true, requires the package to be on the system |
||||
|
- If `CPMUTIL_FORCE_BUNDLED` is true, forcefully uses the bundled package |
||||
|
- If the `BUNDLED_PACKAGE` argument is true, forcefully uses the bundled package |
||||
|
- Otherwise, CPM will search for the package first, and if not found, will use the bundled package |
||||
|
|
||||
|
## Identification |
||||
|
|
||||
|
All dependencies must be identifiable in some way for usage in the dependency viewer. Lists are provided in descending order of precedence. |
||||
|
|
||||
|
URLs: |
||||
|
|
||||
|
- `GIT_URL` |
||||
|
- `REPO` as a Git repository |
||||
|
* You may optionally specify `GIT_HOST` to use a custom host, e.g. `GIT_HOST git.crueter.xyz`. Note that the git host MUST be GitHub-like in its artifact/archive downloads, e.g. Forgejo |
||||
|
* If `GIT_HOST` is unspecified, defaults to `github.com` |
||||
|
- `URL` |
||||
|
|
||||
|
Versions (bundled): |
||||
|
|
||||
|
- `SHA` |
||||
|
- `GIT_VERSION` |
||||
|
- `VERSION` |
||||
|
- `TAG` |
||||
|
- "unknown" |
||||
|
|
||||
|
If the package is a system package, AddPackage will attempt to determine the package version and append ` (system)` to the identifier. Otherwise, it will be marked as `unknown (system)` |
||||
@ -0,0 +1,46 @@ |
|||||
|
# CPMUtil |
||||
|
|
||||
|
CPMUtil is a wrapper around CPM that aims to reduce boilerplate and add useful utility functions to make dependency management a piece of cake. |
||||
|
|
||||
|
Global Options: |
||||
|
|
||||
|
- `CPMUTIL_FORCE_SYSTEM` (default `OFF`): Require all CPM dependencies to use system packages. NOT RECOMMENDED! |
||||
|
* You may optionally override this (section) |
||||
|
- `CPMUTIL_FORCE_BUNDLED` (default `ON` on MSVC and Android, `OFF` elsewhere): Require all CPM dependencies to use bundled packages. |
||||
|
|
||||
|
You are highly encouraged to read AddPackage first, even if you plan to only interact with CPMUtil via `AddJsonPackage`. |
||||
|
|
||||
|
<!-- TOC --> |
||||
|
- [AddPackage](#addpackage) |
||||
|
- [AddCIPackage](#addcipackage) |
||||
|
- [AddJsonPackage](#addjsonpackage) |
||||
|
- [Lists](#lists) |
||||
|
<!-- /TOC --> |
||||
|
|
||||
|
## AddPackage |
||||
|
|
||||
|
The core of CPMUtil is the [`AddPackage`](./AddPackage.md) function. [`AddPackage`](./AddPackage.md) itself is fully CMake-based, and largely serves as an interface between CPM and the rest of CPMUtil. |
||||
|
|
||||
|
## AddCIPackage |
||||
|
|
||||
|
[`AddCIPackage`](./AddCIPackage.md) adds a package that follows [crueter's CI repository spec](https://github.com/crueter-ci). |
||||
|
|
||||
|
## AddJsonPackage |
||||
|
|
||||
|
[`AddJsonPackage`](./AddJsonPackage.md) is the recommended method of usage for CPMUtil. |
||||
|
|
||||
|
## Lists |
||||
|
|
||||
|
CPMUtil will create three lists of dependencies where `AddPackage` or similar was used. Each is in order of addition. |
||||
|
|
||||
|
- `CPM_PACKAGE_NAMES`: The names of packages included by CPMUtil |
||||
|
- `CPM_PACKAGE_URLS`: The URLs to project/repo pages of packages |
||||
|
- `CPM_PACKAGE_SHAS`: Short version identifiers for each package |
||||
|
* If the package was included as a system package, ` (system)` is appended thereafter |
||||
|
* Packages whose versions can't be deduced will be left as `unknown`. |
||||
|
|
||||
|
For an example of how this might be implemented in an application, see Eden's implementation: |
||||
|
|
||||
|
- [`dep_hashes.h.in`](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/src/dep_hashes.h.in) |
||||
|
- [`GenerateDepHashes.cmake`](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/CMakeModules/GenerateDepHashes.cmake) |
||||
|
- [`deps_dialog.cpp`](https://git.eden-emu.dev/eden-emu/eden/src/branch/master/src/yuzu/deps_dialog.cpp) |
||||
@ -1,10 +1,10 @@ |
|||||
# Eden Build Documentation |
# Eden Build Documentation |
||||
|
|
||||
This contains documentation created by developers. This contains build instructions, guidelines, instructions/layouts for [cool stuff we made](CPM.md), and more. |
|
||||
|
This contains documentation created by developers. This contains build instructions, guidelines, instructions/layouts for [cool stuff we made](./CPMUtil), and more. |
||||
|
|
||||
- **[General Build Instructions](Build.md)** |
- **[General Build Instructions](Build.md)** |
||||
- **[Development Guidelines](Development.md)** |
- **[Development Guidelines](Development.md)** |
||||
- **[Dependencies](Deps.md)** |
- **[Dependencies](Deps.md)** |
||||
- **[CPM - CMake Package Manager](CPMUtil.md)** |
|
||||
|
- **[CPM - CMake Package Manager](./CPMUtil)** |
||||
- **[Platform-Specific Caveats](Caveats.md)** |
- **[Platform-Specific Caveats](Caveats.md)** |
||||
- **[User Handbook](User.md)** |
|
||||
|
- **[User Handbook](User.md)** |
||||
@ -1,11 +1,7 @@ |
|||||
#!/bin/sh -e |
#!/bin/sh -e |
||||
|
|
||||
# SPDX-FileCopyrightText: Copyright 2025 Eden Emulator Project |
|
||||
|
# SPDX-FileCopyrightText: Copyright 2025 crueter |
||||
# SPDX-License-Identifier: GPL-3.0-or-later |
# SPDX-License-Identifier: GPL-3.0-or-later |
||||
|
|
||||
# fd is slightly faster on NVMe (the syntax sux though) |
|
||||
if command -v fd > /dev/null; then |
|
||||
fd . tools -esh -x shellcheck |
|
||||
else |
|
||||
find tools -name "*.sh" -exec shellcheck -s sh {} \; |
|
||||
fi |
|
||||
|
# do NOT use fd in scripts, PLEASE |
||||
|
find tools -name "*.sh" -exec shellcheck -s sh {} \; |
||||
@ -1,6 +1,16 @@ |
|||||
#!/bin/sh -e |
#!/bin/sh -e |
||||
|
|
||||
# SPDX-FileCopyrightText: Copyright 2025 Eden Emulator Project |
|
||||
|
# SPDX-FileCopyrightText: Copyright 2025 crueter |
||||
# SPDX-License-Identifier: GPL-3.0-or-later |
# SPDX-License-Identifier: GPL-3.0-or-later |
||||
|
|
||||
wget -O CMakeModules/CPM.cmake https://github.com/cpm-cmake/CPM.cmake/releases/latest/download/CPM.cmake |
|
||||
|
# updates CPMUtil, its docs, and related tools from the latest release |
||||
|
|
||||
|
if command -v zstd > /dev/null; then |
||||
|
EXT=tar.zst |
||||
|
else |
||||
|
EXT=tar.gz |
||||
|
fi |
||||
|
|
||||
|
wget "https://git.crueter.xyz/CMake/CPMUtil/releases/download/continuous/CPMUtil.$EXT" |
||||
|
tar xf "CPMUtil.$EXT" |
||||
|
rm "CPMUtil.$EXT" |
||||
Write
Preview
Loading…
Cancel
Save
Reference in new issue