build.zig: add a 'compile' step to compile the self-hosted compiler
without installing it.
Compilation: set cache mode to whole when using the LLVM backend and
--enable-cache is passed.
This makes `zig build` act the same as it does with stage1. Upside is
that a second invocation of `zig build` on an unmodified source tree
will avoid redoing the compilation again. Downside is that it will
proliferate more garbage in the project-local cache (same as stage1).
This can eventually be fixed when Zig's incremental compilation is more
robust; we can go back to having LLVM use CacheMode.incremental and rely
on it detecting no changes and avoiding doing the flush() step.
Previously, you might obtain `-lLLVM-15` from the CMake configuration,
but we might not be able to locate the library if it's not in your
system library path.
Previously, you might obtain `-lLLVM-15` from the CMake configuration,
but we might not be able to locate the library if it's not in your
system library path.
No longer introduce build options for tests. Instead, ZIG_EXE
environment variable is added to any invocation of `zig run` or `zig
test`.
The end result of this branch is the same: there is no longer a
mandatory positional command line argument when invoking zig test
binaries directly.
We now warn the user if config.h could not be located.
This also updates the search to stop early upon encountering a
`.git` directory, so that we avoid recursing outside of the zig
source if possible.
* test/link: initial wasm support
This adds basic parsing and dumping of wasm section so they
can be tested using the new linker-test infrastructure.
* test/link: all wasm sections parsing and dumping
We now parse and dump all sections for the wasm binary format.
Currently, this only dumps the name of a custom section.
Later this should also dump symbol table, name, linking metadata and relocations.
All of those live within the custom sections.
* Add wasm linker test
This also fixes a parser mistake in reading the flags.
* test/link: implement linker tests wasm & fixes
Adds several test cases to test the wasm self-hosted linker.
This also introduces fixes that were caught during the implementation
of those tests.
* test-runner: obey omit_stage2 for standalone
When a standalone test requires stage2, but stage2 is omit
from the compiler, such test case will not be included as part
of the test suite that is being ran. This is to support CI's
where we omit stage2 to lower the memory usage.
Instead of always using std.testing.allocator, the test harness now follows
the same logic as self-hosted for choosing an allocator - that is - it
uses C allocator when linking libc, std.testing.allocator otherwise, and
respects `-Dforce-gpa` to override the decision. I did this because
I found GeneralPurposeAllocator to be prohibitively slow when doing
multi-threading, even in the context of a debug build.
There is now a second thread pool which is used to spawn each
test case. The stage2 tests are passed the first thread pool. If it were
only multi-threading the stage1 tests then we could use the same thread
pool for everything. However, the problem with this strategy with stage2
is that stage2 wants to spawn tasks and then call wait() on the main
thread. If we use the same thread pool for everything, we get a deadlock
because all the threads end up all hanging at wait() and nothing is
getting done. So we use our second thread pool to simulate a "process pool"
of sorts.
I spent most of the time working on this commit scratching my head trying
to figure out why I was getting ETXTBSY when spawning the test cases.
Turns out it's a fundamental Unix design flaw, already a known, unsolved
issue by Go and Java maintainers:
https://github.com/golang/go/issues/22315https://bugs.openjdk.org/browse/JDK-8068370
With this change, the following command, executed on my laptop, went from
6m24s to 1m44s:
```
stage1/bin/zig build test-cases -fqemu -fwasmtime -Denable-llvm
```
closes#11818
* migrate runtime safety tests to the new test harness
- this required adding compare output / execution support for stage1
to the test harness.
* rename `zig build test-stage2` to `zig build test-cases` since it now
does quite a bit of stage1 testing actually. I named it this way
since the main directory in the source tree associated with these
tests is "test/cases/".
* add some documentation for the test manifest format.
stage2: change logic for detecting whether the main package is inside
the std package. Previously it relied on realpath() which is not portable.
This uses resolve() which is how imports already work.
* stage2: fix cleanup bug when creating Module
* flatten lib/std/special/* to lib/*
- this was motivated by making main_pkg_is_inside_std false for
compiler_rt & friends.
* rename "mini libc" to "universal libc"
We can't yet run the behavior tests with stage3, but at least we can run
them with stage2, and we can use the proper test matrix.
This commit also adds use_llvm and ofmt to the zig build system.
* `-Dskip-compile-errors` is removed; `-Dskip-stage1` is added.
* Use `std.testing.allocator` instead of a new instance of GPA.
- Fix the memory leaks this revealed.
* Show the file name when it is not parsed correctly such as when the
manifest is missing.
- Better error messages when test files are not parsed correctly.
* Ignore unknown files such as swap files.
* Move logic from declarative file to the test harness implementation.
* Move stage1 tests to stage2 tests where appropriate.
using the provided -L directories before checking if we need to integrate
with system library paths. This prevents false positive of invoking
system cc to find native paths when in fact all dependencies are
satisfied by -L (or --search-prefix to zig build).
* build.zig: remove detour through zig0. I'll add it back as an option
later. This means you can build stage1 with a freshly built zig
without detouring through zig0 again.
* remove unused file windows_script.bat
* drone.yml: remove unnecessary steps
* ninja doesn't need a jobs parameter
* build the release version of zig with zig build instead of cmake.
* build stage2 with -Dstatic-llvm -Duse-zig-libcxx
Instead of assuming that the zig version used with `zig build` is
appropriate for building the self-hosted compiler component, we follow
the same path as the cmake build script and build zig0, using that to
produce zig1.o, re-linking to produce stage1.
This allows an arbitrarily old Zig version and corresponding build.zig
script to build all future versions of Zig from source via the bootstrap
compiler. In other words, it allows us to use `zig build` as an
alternative to cmake when bootstrapping.
In accordance with the requesting issue (#10750):
- `zig test` skips any tests that it cannot spawn, returning success
- `zig run` and `zig build` exit with failure, reporting the command the cannot be run
- `zig clang`, `zig ar`, etc. already punt directly to the appropriate clang/lld main(), even before this change
- Native `libc` Detection is not supported
Additionally, `exec()` and related Builder functions error at run-time, reporting the command that cannot be run
Instead of defaulting to false, just keep the option as optional to
communicate default to the build system.
Fixes one problem with building the compiler for single-threaded
targets.
