Users can add additional VS project configurations with their own
custom settings, but to support this workflow, we can't rely directly
on $(Platform) and $(Configuration), because VS needs those to be
both unique Configuration|Platform combos, and we need to allow for
different combos of Configuration|Platform to point to the same
scons build configuration.
GodotPlatform and GodotConfiguration properties lets us decouple from
the magic VS properties that we don't control, so users can add
however many Platform|Configuration combos they want and still
point to a specific GodotPlatform|GodotConfiguration build config.
Make sure we include any user-specified project settings in our
project definitions, this way users can customize their VS
environment to more closely match what they're building for, and
they better can control debugging and deployment from VS.
Add support for setting VS-only compiler defines, include paths, and
additional linker options, as a hint to VS to use when loading projects
and parsing code. VS would usually know these on non nmake projects,
but for nmake projects we have to tell it about any implicit information
that the compiler has, so it can resolve symbols in the editor.
Custom Visual Studio project generation logic that supports any platform that has a msvs.py
script, so Visual Studio can be used to run scons for any platform, with the right defines per target.
Invoked with `scons vsproj=yes`
To generate build configuration files for all platforms+targets+arch combinations, users should call
```
scons vsproj=yes platform=XXX target=YYY [other build flags]
```
for each combination of platform+target[+arch]. This will generate the relevant vs project files but
skip the build process, so that project files can be quickly generated without waiting for a command line
build. This lets project files be quickly generated even if there are build errors.
All possible combinations of platform+target are created in the solution file by default, but they
won't do anything until each one is set up with a scons vsproj=yes command for the respective platform
in the appropriate command line. This lets users only generate the combinations they need, and VS
won't have to parse settings for other combos.
Only platforms that opt in to vs proj generation by having a msvs.py file in the platform folder are included.
Platforms with a msvs.py file will be added to the solution, but only the current active platform+target+arch
will have a build configuration generated, because we only know what the right defines/includes/flags/etc are
on the active build target currently being processed by scons.
Platforms that don't support an editor target will have a dummy editor target that won't do anything on build,
but will have the files and configuration for the windows editor target.
To generate AND build from the command line, run
```
scons vsproj=yes vsproj_gen_only=no
```
Verbose output is meant for debugging the SCU mode itself and can be
triggered by changing the `_verbose` bool manually.
Prefix all prints with "SCU:" for context, and print the processed
folders all at once instead of when adding the sources.
This fixes multiple issues/inconsistencies around `get_compiler_version()`:
* With no shell allocated, launching the compiler could fail even
with proper paths being set.
* The return value was described as "an array of version numbers as ints",
but the function actually returned a `Dictionary` (or `None`).
* Not all calls were properly handling a `None` return value in case of errors.
On Windows this broke compiling for me since #81869 with default settings.
* Some calls defined inconsistent defaults/fallbacks (`0` or `-1`).
Even if you specify the subsystem to be the console one, the vsproj doesn't carry over the setting, which makes working with this mode in the IDE a bit annoying since it'll regenerate the vsproj right afterwards. Since there's only two options and 'gui' is the default, we only carry over the 'console' setting.
Follow-up to #75932.
Since these icons are only used by the export plugin, it makes sense to
move them and generate the headers there.
The whole `detect.is_active()` logic seems to be a leftover from before
times, as far back as 1.0-stable it already wasn't used for anything.
So I'm removing it and moving the export icon generation to
`platform_methods`, where it makes more sense.
Ensures that the versions always match the Godot version, albeit following
SemVer 2.0 so inserting a dot between "beta" and the build number.
For "stable" status, we omit the suffix as this would be interpreted as a
pre-release build too.
So we have:
| Godot version | Nupkg version |
| -------------- | -------------- |
| 4.0.0-beta | 4.0.0-beta |
| 4.0.0-beta2 | 4.0.0-beta.2 |
| 4.0.0-rc1 | 4.0.0-rc.1 |
| 4.0.0-stable | 4.0.0 |
This adds support for building solutions with dev_mode and/or float=64 enabled.
Additionally, it adds solution generation to the Windows CI to catch future regressions.
Implements https://github.com/godotengine/godot-proposals/issues/3371.
New `target` presets
====================
The `tools` option is removed and `target` changes to use three new presets,
which match the builds users are familiar with. These targets control the
default optimization level and enable editor-specific and debugging code:
- `editor`: Replaces `tools=yes target=release_debug`.
* Defines: `TOOLS_ENABLED`, `DEBUG_ENABLED`, `-O2`/`/O2`
- `template_debug`: Replaces `tools=no target=release_debug`.
* Defines: `DEBUG_ENABLED`, `-O2`/`/O2`
- `template_release`: Replaces `tools=no target=release`.
* Defines: `-O3`/`/O2`
New `dev_build` option
======================
The previous `target=debug` is now replaced by a separate `dev_build=yes`
option, which can be used in combination with either of the three targets,
and changes the following:
- `dev_build`: Defines `DEV_ENABLED`, disables optimization (`-O0`/`/0d`),
enables generating debug symbols, does not define `NDEBUG` so `assert()`
works in thirdparty libraries, adds a `.dev` suffix to the binary name.
Note: Unlike previously, `dev_build` defaults to off so that users who
compile Godot from source get an optimized and small build by default.
Engine contributors should now set `dev_build=yes` in their build scripts or
IDE configuration manually.
Changed binary names
====================
The name of generated binaries and object files are changed too, to follow
this format:
`godot.<platform>.<target>[.dev][.double].<arch>[.<extra_suffix>][.<ext>]`
For example:
- `godot.linuxbsd.editor.dev.arm64`
- `godot.windows.template_release.double.x86_64.mono.exe`
Be sure to update your links/scripts/IDE config accordingly.
More flexible `optimize` and `debug_symbols` options
====================================================
The optimization level and whether to generate debug symbols can be further
specified with the `optimize` and `debug_symbols` options. So the default
values listed above for the various `target` and `dev_build` combinations
are indicative and can be replaced when compiling, e.g.:
`scons p=linuxbsd target=template_debug dev_build=yes optimize=debug`
will make a "debug" export template with dev-only code enabled, `-Og`
optimization level for GCC/Clang, and debug symbols. Perfect for debugging
complex crashes at runtime in an exported project.
This makes it possible to retrieve all relevant versioning info used to
generate `core/version_generated.gen.h` in the buildsystem.
Notably it makes the custom logic parsing the `GODOT_VERSION_STATUS`
environment variable to override status easy to reuse.
We want to replace libnethost as it gives us issues with some compilers.
Our implementation tries to mimic libnethost's hostfxr_resolver search
logic. We try to use the same function names for easier comparing in
case we need to update this in the future.
We're targeting .NET 5 for now to make development easier while
.NET 6 is not yet released.
TEMPORARY REGRESSIONS
---------------------
Assembly unloading is not implemented yet. As such, many Godot
resources are leaked at exit. This will be re-implemented later
together with assembly hot-reloading.
Modules can now call:
env.module_add_dependencies(name: str, deps: list, optional: bool)
To add required or optional dependencies during the "can_build" step.
Required dependencies will be checked and the module will be not be
enabled when they are missing, printing a warning to notify the user.
* Changed to use the same stages as extensions.
* Makes the initialization more coherent, helping solve problems due to lack of stages.
* Makes it easier to port between module and extension.
* removed the DRIVER initialization level (no longer needed).
All compilation messages are now written in blue to ensure that
compiler errors and warnings stand out more. Messages were also
slightly shortened to make them easier to fit on a single line on
narrow terminals.
This makes it possible to change the branch of the documentation that
URLs are pointing to without having to modify all class reference
files.
In the XML class reference, the `$DOCS_URL` placeholder should be used,
and will be replaced automatically in the editor and when generating
the RST class reference.
The documentation branch string is set in `version.py`.
Co-authored-by: Hugo Locurcio <hugo.locurcio@hugo.pro>
Mapping and other "abstract base classes" were moved after python 3.3 from collections to collections.abc
Python 3.3 is long gone and a newer version of python won't support this code.
Whenever we change the name (or remove) generated cpp files with the `.gen.cpp`
extension, users run into build issues when switching between branches (i.e.
switching before and after the name change/removal). This is because we glob
`*.cpp` so if a now-obsolete file from a previous build is present, we'll
include it too, potentially leading to bugs or compilation failure (due to
missing headers or invalid code).
So globbing patterns in `add_source_files` will now skip files ending with
`.gen.cpp`, which should instead be passed explicitly where they're used.
Add support for specifying defines for each configuration
Add support for specifying extra cli args for each configuration
Add support for specifying extra includes for each configuration
This could cause spurious errors on CI when trying to prune the cache,
as for some reason it tries to remove files/paths which do not exist.
That points at a bug in the `cache_progress` logic but at least this
workaround should prevent CI failures.
Emscripten is LLVM-based so we want to follow the same logic. But we can't just
put it as a match in `methods.using_clang()` as that would mess with the
compiler version detection logic used to restrict old GCC and Clang releases.
`VERSION_STATUS` is part of what constitutes the reference version for a given
Godot build, and is part of the version check for compatible export templates.
For dev snapshots (alpha, beta, RCs), we usually set the `VERSION_STATUS` to
a specific build number (e.g. `beta2`), but this change doesn't end up
committed to the Git repository as we don't want to keep changing `version.py`
for testing builds.
So this new environment override will be what can be used in official builds
and by users making custom builds for specific snapshots.
* This PR adds the ability to disable classes when building.
* For now it's only possible to do this via command like:
`scons disable_classes=RayCast2D,Area3D`
* Eventually, a proper UI will be implemented to create a build config file to do this at large scale, as well as detect what is used in the project.
Test sources and build parameter were not supplied to the visual studio project. This resulted in a build that was not able to be test using the --test command. Adding build parameter ensures we can test, and supplying the sources ensures we have all files to write new tests and edit existing ones.
The `dev=yes` and `production=yes` options work as aliases to set a number of
options, while still aiming to allow overriding specific options if the user
wishes so. (E.g. `production=yes use_lto=no` should work to enable production
defaults *but* disable LTO.)
That wasn't working as `ARGUMENTS.get()` returns a string and not a boolean as
expected by `BoolVariable`, and this wasn't flagged as a bug... So added a
helper method using SCons' `BoolVariable._text2bool` to do the conversion
manually.
This adds `custom_modules_recursive` which allows to detect and collect
all nested C++ modules which may reside in any directory specified by
`custom_modules` option.
The detection logic is made to be more strict because `SCSub` may be
used for organizing hierarchical builds within a module itself, so the
existence of `register_types.h` and `config.py` is checked as well
(these are all required for a C++ module to be compiled by Godot).
For performance reasons, built-in modules are not checked recursively,
and there's no benefit of doing so in the first place.
It's now possible to specify a directory path pointing to a *single*
module, as it may contain nested modules which are detected recursively.
First, compile the engine normally with:
```
scons custom_modules="path/to/your/modules" vsproj=yes
```
Then run the Visual Studio project. You can now rebuild the engine if
you need to make changes to custom modules directly within IDE.
A new `env.Run` method is added which allows to control the verbosity
of builders output automatically depending on whether the "verbose"
option is set. It also allows to optionally run any SCons commands in a
subprocess using the existing `run_in_subprocess` method, unifying
the interface. `Action` objects wrap all builder functions to include a
short build message associated with any action.
Notably, this removes quite verbose output generated by `make_doc_header`
and `make_editor_icons_action` builders.
Until https://github.com/psf/black/pull/1328 makes it in a stable release,
we have to use the latest from Git.
Apply new style fixes done by latest black.
A new `methods.dump(env)` is added to dump the construction environment
used by SCons to build Godot to a `.scons_env.json`. The file can be used
for debugging purposes and any external tool.
This is still a bit hacky and eventually we should rework the way we handle
optional dependencies (especially with regard to builtin/system libs), but
it's a simple first step.
Fixes#39219.
The insertion order for dictionaries is only a language feature for
Python 3.6/3.7+ implementations, and not prior to that.
This ensures that the engine won't be rebuilt if the order of detected
modules changes in any way, as the `OrderedDict` should guarantee
inerstion order.
The existence of `SCsub` is checked instead. This file is required for
all modules, and prevents the build system to leave modules without
`config.py` undetected, leading to silently ignoring the module during
compilation.
This patch adds ability to include external, user-defined C++ modules
to be compiled as part of Godot via `custom_modules` build option
which can be passed to `scons`.
```
scons platform=x11 tools=yes custom_modules="../project/modules"
```
Features:
- detects all available modules under `custom_modules` directory the
same way as it does for built-in modules (not recursive);
- works with both relative and absolute paths on the filesystem;
- multiple search paths can be specified as a comma-separated list.
Module custom documentation and editor icons collection and generation
process is adapted to work with absolute paths needed by such modules.
Also fixed doctool bug mixing absolute and relative paths respectively.
Implementation details:
- `env.module_list` is a dictionary now, which holds both module name as
key and either a relative or absolute path to a module as a value.
- `methods.detect_modules` is run twice: once for built-in modules, and
second for external modules, all combined later.
- `methods.detect_modules` was not doing what it says on the tin. It is
split into `detect_modules` which collects a list of available modules
and `write_modules` which generates `register_types` sources for each.
- whether a module is built-in or external is distinguished by relative
or absolute paths respectively. `custom_modules` scons converter
ensures that the path is absolute even if relative path is supplied,
including expanding user paths and symbolic links.
- treats the parent directory as if it was Godot's base directory, so
that there's no need to change include paths in cases where custom
modules are included as dependencies in other modules.
Some required changes are made:
- locally imported SCons-specific packages within the method;
- `global` variables converted to `nonlocal` (used in nested functions).
Configured for a max line length of 120 characters.
psf/black is very opinionated and purposely doesn't leave much room for
configuration. The output is mostly OK so that should be fine for us,
but some things worth noting:
- Manually wrapped strings will be reflowed, so by using a line length
of 120 for the sake of preserving readability for our long command
calls, it also means that some manually wrapped strings are back on
the same line and should be manually merged again.
- Code generators using string concatenation extensively look awful,
since black puts each operand on a single line. We need to refactor
these generators to use more pythonic string formatting, for which
many options are available (`%`, `format` or f-strings).
- CI checks and a pre-commit hook will be added to ensure that future
buildsystem changes are well-formatted.
Scons' `Environment.subst()` does that, and was already used in the
other place where we query an env variable (`env["LINK"]` in x11 code).
Fixes `3.2` iOS build after cherry-pick of #36559 (previously it only
ran for GCC code, not iOS's Clang), and the same issue would likely
affect `master` if iOS builds were enabled right now.
