This rather large commit adds/fixes missing WASI functionality
in `libstd` needed to pass the `libstd` tests. As such, now by
default tests targeting `wasm32-wasi` target are enabled in
`test/tests.zig` module. However, they can be disabled by passing
the `-Dskip-wasi=true` flag when invoking the `zig build test`
command. When the flag is set to `false`, i.e., when WASI tests are
included, `wasmtime` with `--dir=.` is used as the default testing
command.
Since the majority of `libstd` tests were relying on `fs.cwd()`
call to get current working directory handle wrapped in `Dir`
struct, in order to make the tests WASI-friendly, `fs.cwd()`
call was replaced with `testing.getTestDir()` function which
resolved to either `fs.cwd()` for non-WASI targets, or tries to
fetch the preopen list from the WASI runtime and extract a
preopen for '.' path.
The summary of changes introduced by this commit:
* implement `Dir.makeDir` and `Dir.openDir` targeting WASI
* implement `Dir.deleteFile` and `Dir.deleteDir` targeting WASI
* fix `os.close` and map errors in `unlinkat`
* move WASI-specific `mkdirat` and `unlinkat` from `std.fs.wasi`
to `std.os` module
* implement `lseek_{SET, CUR, END}` targeting WASI
* implement `futimens` targeting WASI
* implement `ftruncate` targeting WASI
* implement `readv`, `writev`, `pread{v}`, `pwrite{v}` targeting WASI
* make sure ANSI escape codes are _not_ used in stderr or stdout
in WASI, as WASI always sanitizes stderr, and sanitizes stdout if
fd is a TTY
* fix specifying WASI rights when opening/creating files/dirs
* tweak `AtomicFile` to be WASI-compatible
* implement `os.renameatWasi` for WASI-compliant `os.renameat` function
* implement sleep() targeting WASI
* fix `process.getEnvMap` targeting WASI
After this commit there are no more bit rotted files.
The testing program that was in ir.zig has been moved to main.zig
Unsupported command line options have been deleted, or error messages
added.
The compiler repl is available from the build-exe, build-lib,
build-obj commands with the --watch option.
The main zig build script now builds the self-hosted compiler
unconditionally. Linking against LLVM is behind a -Denable-llvm
flag that defaults to off.
Rather than stuffing configuration information into the Zig binary, the
build script reads it from config.h. This solves a problem for package
maintainers and improves the use case of deterministic builds.
closes#3758
closes#4682
The self-hosted compiler is still bit rotted and still not compiling
successfully yet. I have a more serious rework of the code in a
different branch.
Zig now supports a more fine-grained sense of what is native and what is
not. Some examples:
This is now allowed:
-target native
Different OS but native CPU, default Windows C ABI:
-target native-windows
This could be useful for example when running in Wine.
Different CPU but native OS, native C ABI.
-target x86_64-native -mcpu=skylake
Different C ABI but otherwise native target:
-target native-native-musl
-target native-native-gnu
Lots of breaking changes to related std lib APIs.
Calls to getOs() will need to be changed to getOsTag().
Calls to getArch() will need to be changed to getCpuArch().
Usage of Target.Cross and Target.Native need to be updated to use
CrossTarget API.
`std.build.Builder.standardTargetOptions` is changed to accept its
parameters as a struct with default values. It now has the ability to
specify a whitelist of targets allowed, as well as the default target.
Rather than two different ways of collecting the target, it's now always
a string that is validated, and prints helpful diagnostics for invalid
targets. This feature should now be actually useful, and contributions
welcome to further improve the user experience.
`std.build.LibExeObjStep.setTheTarget` is removed.
`std.build.LibExeObjStep.setTarget` is updated to take a CrossTarget
parameter.
`std.build.LibExeObjStep.setTargetGLibC` is removed. glibc versions are
handled in the CrossTarget API and can be specified with the `-target`
triple.
`std.builtin.Version` gains a `format` method.
Rather than `zig0 build ...` the build now does
`zig0 build-lib ...`, avoiding the requirement of linking the build
script, and thus avoiding the requirement of finding native libc,
for systems where libc is the system ABI.
* libc_installation.cpp is deleted.
src-self-hosted/libc_installation.zig is now used for both stage1 and
stage2 compilers.
* (breaking) move `std.fs.File.access` to `std.fs.Dir.access`. The API
now encourages use with an open directory handle.
* Add `std.os.faccessat` and related functions.
* Deprecate the "C" suffix naming convention for null-terminated
parameters. "C" should be used when it is related to libc. However
null-terminated parameters often have to do with the native system
ABI rather than libc. "Z" suffix is the new convention. For example,
`std.os.openC` is deprecated in favor of `std.os.openZ`.
* Add `std.mem.dupeZ` for using an allocator to copy memory and add a
null terminator.
* Remove dead struct field `std.ChildProcess.llnode`.
* Introduce `std.event.Batch`. This API allows expressing concurrency
without forcing code to be async. It requires no Allocator and does
not introduce any failure conditions. However it is not thread-safe.
* There is now an ongoing experiment to transition away from
`std.event.Group` in favor of `std.event.Batch`.
* `std.os.execvpeC` calls `getenvZ` rather than `getenv`. This is
slightly more efficient on most systems, and works around a
limitation of `getenv` lack of integration with libc.
* (breaking) `std.os.AccessError` gains `FileBusy`, `SymLinkLoop`, and
`ReadOnlyFileSystem`. Previously these error codes were all reported
as `PermissionDenied`.
* Add `std.Target.isDragonFlyBSD`.
* stage2: access to the windows_sdk functions is done with a manually
maintained .zig binding file instead of `@cImport`.
* Update src-self-hosted/libc_installation.zig with all the
improvements that stage1 has seen to src/libc_installation.cpp until
now. In addition, it now takes advantage of Batch so that evented I/O
mode takes advantage of concurrency, but it still works in blocking
I/O mode, which is how it is used in stage1.
1. behavior tests with --test-evented-io
2. std lib tests with --test-evented-io
3. fuzz test evented I/O a bit, make it robust
4. make sure it works on all platforms (kqueue, Windows IOCP,
epoll/other)
5. restart efforts on self-hosted
* Implements #3768. This is a sweeping breaking change that requires
many (trivial) edits to Zig source code. Array values no longer
coerced to slices; however one may use `&` to obtain a reference to
an array value, which may then be coerced to a slice.
* Adds `IrInstruction::dump`, for debugging purposes. It's useful to
call to inspect the instruction when debugging Zig IR.
* Fixes bugs with result location semantics. See the new behavior test
cases, and compile error test cases.
* Fixes bugs with `@typeInfo` not properly resolving const values.
* Behavior tests are passing but std lib tests are not yet. There
is more work to do before merging this branch.
we don't really have a way to determine whether the stage1
zig compiler requires PIC so to be safe we always enable it
when building libuserland.
fixes build on some configurations of alpine linux.
* add zig build option `-Dskip-libc` to skip tests that build libc
(e.g. if you don't want to wait for musl to build)
* add `-Denable-wine` option which uses wine to run cross compiled
windows tests on non-windows hosts
* add `-Denable-qemu` option which uses qemu to run cross compiled
foreign architecture tests
* add `-Denable-foreign-glibc=path` option which combined with
`-Denable-qemu` enables running cross compiled tests that link
against glibc. See
https://github.com/ziglang/zig/wiki/Updating-libc#glibc for how to
produce this directory.
* the test matrix is done manually. release test builds are only
enabled by default for the native target. this should save us some CI
time, while still providing decent coverage of release builds.
- add test coverage for `x86_64-linux-musl -lc` (building musl libc)
- add test coverage for `x86_64-linux-gnu -lc` (building glibc)
- add test coverage for `aarch64v8_5a-linux-none`
- add test coverage for `aarch64v8_5a-linux-musl -lc` (building musl libc)
- add test coverage for `aarch64v8_5a-linux-gnu -lc` (building glibc)
- add test coverage for `arm-linux-none`
- test coverage for `arm-linux-musleabihf -lc` (building musl libc) is
disabled due to #3286
- test coverage for `arm-linux-gnueabihf -lc` (building glibc) is disabled
due to #3287
- test coverage for `x86_64-windows-gnu -lc` (building mingw-w64) is
disabled due to #3285
* enable qemu testing on the Linux CI job. There's not really a good
reason to enable wine, since we have a Windows CI job as well.
* remove the no longer needed `--build-file ../build.zig` from CI
scripts
* fix bug in glibc compilation where it wasn't properly reading the abi
list txt files, resulting in "key not found" error.
* std.build.Target gains:
- isNetBSD
- isLinux
- osRequiresLibC
- getArchPtrBitWidth
- getExternalExecutor
* zig build system gains support for enabling wine and enabling qemu.
`artifact.enable_wine = true;`, `artifact.enable_qemu = true;`. This
communicates that the system has these tools installed and the build
system will use them to run tests.
* zig build system gains support for overriding the dynamic linker of
an executable artifact.
* fix std.c.lseek prototype. makes behavior tests for
arm-linux-musleabihf pass.
* disable std lib tests that are failing on ARM. See #3288, #3289
* provide `std.os.off_t`.
* disable some of the compiler_rt symbols for arm 32 bit. Fixes
compiler_rt tests for arm 32 bit
* add __stack_chk_guard when linking against glibc. Fixes std lib tests
for aarch64-linux-gnu
* workaround for "unable to inline function" using `@inlineCall`. Fixes
compiler_rt tests for arm 32 bit.
`zig build test-compare-panic`
Create basic tests to compare panic output. The address field
is replaced by a symbolic constant and each expected output is
specific to os. Tests will only run for explicitly defined
platforms.
see also #2485
also start prefering NtDll API. so far:
* NtQueryInformationFile
* NtClose
adds a performance workaround for windows unicode conversion. but that
should probably be removed before merging
This moves the installation of shipped source files from large
CMakeLists.txt lists to zig build recursive directory installation.
On my computer a cmake `make install` takes 2.4 seconds even when it has
to do nothing, and prints a lot of unnecessary lines to stdout that say
"up-to-date: [some file it is installing]".
After this commit, the default output of `make` is down to 1
second, and it does not print any junk to stdout. Further, a `make
install` is no longer required and `make` is sufficient.
This closes#2874.
It also closes#2585. `make` now always invokes `zig build` for
installing files and libuserland.a, and zig's own caching system makes
that go fast.
* fix crash when doing field access of slice types. closes#2486
* remove the deprecated Child property from slice types
* add -Dskip-non-native build option to build script
Sadly due to a workaround for LLD linker limitations on macOS
we cannot put libuserland into an .a file; instead we have to use object
files. Again due to linker limitations, bundling compiler_rt.o into
another relocatable object also doesn't work. So we're left with
disabling stack probing on macOS for the stage1 self-hosted code.
These workarounds could all be removed if the macos support in the LLD
linker improved, or if Zig project had its own linker that did not have
these issues.
and use it when building libuserland.a
The self-hosted part of stage1 relies on zig's compiler-rt, and so we
include it in libuserland.a.
This should potentially be the default, but for now it's behind a linker
option.
self-hosted translate-c: small progress on translating functions.
Previously libuserland was being built for the native target,
which could depend on native CPU features such as AVX. The
CI infrastructure is intending to create binaries that are
widely compatible and do not make use of specific CPU features.
There could be something to gain from enabling native CPU features
sometimes, by introducing a new cmake configuration option, but
since it's planned to eventually ship self-hosted rather than stage1,
I don't think it really matters.
closes#2348
This modifies the build process of Zig to put all of the source files
into libcompiler.a, except main.cpp and userland.cpp.
Next, the build process links main.cpp, userland.cpp, and libcompiler.a
into zig1. userland.cpp is a shim for functions that will later be
replaced with self-hosted implementations.
Next, the build process uses zig1 to build src-self-hosted/stage1.zig
into libuserland.a, which does not depend on any of the things that
are shimmed in userland.cpp, such as translate-c.
Finally, the build process re-links main.cpp and libcompiler.a, except
with libuserland.a instead of userland.cpp. Now the shims are replaced
with .zig code. This provides all of the Zig standard library to the
stage1 C++ compiler, and enables us to move certain things to userland,
such as translate-c.
As a proof of concept I have made the `zig zen` command use text defined
in userland. I added `zig translate-c-2` which is a work-in-progress
reimplementation of translate-c in userland, which currently calls
`std.debug.panic("unimplemented")` and you can see the stack trace makes
it all the way back into the C++ main() function (Thanks LemonBoy for
improving that!).
This could potentially let us move other things into userland, such as
hashing algorithms, the entire cache system, .d file parsing, pretty
much anything that libuserland.a itself doesn't need to depend on.
This can also let us have `zig fmt` in stage1 without the overhead
of child process execution, and without the initial compilation delay
before it gets cached.
See #1964
* in Zig build scripts, getOutputPath() is no longer a valid function
to call, unless setOutputDir() was used, or within a custom make()
function. Instead there is more convenient API to use which takes
advantage of the caching system. Search this commit diff for
`exe.run()` for an example.
* Zig build by default enables caching. All build artifacts will go
into zig-cache. If you want to access build artifacts in a convenient
location, it is recommended to add an `install` step. Otherwise
you can use the `run()` API mentioned above to execute programs
directly from their location in the cache. Closes#330.
`addSystemCommand` is available for programs not built with Zig
build.
* Please note that Zig does no cache evicting yet. You may have to
manually delete zig-cache directories periodically to keep disk
usage down. It's planned for this to be a simple Least Recently
Used eviction system eventually.
* `--output`, `--output-lib`, and `--output-h` are removed. Instead,
use `--output-dir` which defaults to the current working directory.
Or take advantage of `--cache on`, which will print the main output
path to stdout, and the other artifacts will be in the same directory
with predictable file names. `--disable-gen-h` is available when
one wants to prevent .h file generation.
* `@cImport` is always independently cached now. Closes#2015.
It always writes the generated Zig code to disk which makes debug
info and compile errors better. No more "TODO: remember C source
location to display here"
* Fix .d file parsing. (Fixes the MacOS CI failure)
* Zig no longer creates "temporary files" other than inside a
zig-cache directory.
This breaks the CLI API that Godbolt uses. The suggested new invocation
can be found in this commit diff, in the changes to `test/cli.zig`.
* zig fmt
* std.mem.join takes a slice of slices instead of var args
* std.mem.join takes a separator slice rather than byte,
and always inserts it. Previously it would not insert the separator
if there already was one, violating the documented behavior.
* std.mem.join calculates exactly the correct amount to allocate
and has no call to allocator.shrink()
* bring back joinWindows and joinPosix and the corresponding tests.
it is intended to be able to call these functions from any OS.
* rename std.os.path.resolveSlice to resolve (now resolve takes
a slice of slices instead of var args)
* rename std.mem.split to std.mem.tokenize
* add future deprecation notice to docs
* (unrelated) add note to std.os.path.resolve docs
* std.mem.separate - assert delimiter.len not zero
* fix implementation of std.mem.separate to respect the delimiter
* separate the two iterators to different structs
This commit contains everything from the copy-elision-2
branch that does not have to do with copy elision directly,
but is generally useful for master branch.
* All const values know their parents, when applicable, not
just structs and unions.
* Null pointers in const values are represented explicitly,
rather than as a HardCodedAddr value of 0.
* Rename "maybe" to "optional" in various code locations.
* Separate DeclVarSrc and DeclVarGen
* Separate PtrCastSrc and PtrCastGen
* Separate CmpxchgSrc and CmpxchgGen
* Represent optional error set as an integer, using the 0 value.
In a const value, it uses nullptr.
* Introduce type_has_one_possible_value and use it where applicable.
* Fix debug builds not setting memory to 0xaa when storing
undefined.
* Separate the type of a variable from the const value of a variable.
* Use copy_const_val where appropriate.
* Rearrange structs to pack data more efficiently.
* Move test/cases/* to test/behavior/*
* Use `std.debug.assertOrPanic` in behavior tests instead of
`std.debug.assert`.
* Fix outdated slice syntax in docs.
* add a --system-linker-hack command line parameter to work around
poor LLD macho code. See #1535
* build.zig correctly handles static as well as dynamic dependencies
when building the self hosted compiler.
- no more unnecessary libxml2 dependency
- a static build on macos produces a completely static self-hosted
compiler for macos (except for libSystem as intended).
init-lib creates a working static library with tests, and
init-exe creates a working hello world with a `run` target.
both now have test coverage with the new "cli tests" file.
closes#1035
* fix race condition in std.event.Channel deinit
* add support to zig build for --no-rosegment
* add passing self-hosted compare-output test for calling a function
* put a global lock on LLD linking because it's not thread safe
* introduce std.atomic.Int
* add src-self-hosted/test.zig which is tested by the main test suite
- it fully utilizes the multithreaded async/await event loop so the
tests should Go Fast
* `stage2/bin/zig build-obj test.zig` is able to spit out an error if 2 exported
functions collide
* ability for `zig test` to accept `--object` and `--assembly`
arguments
* std.build: TestStep supports addLibPath and addObjectFile
Add wasm32 support to the build-obj, build-exe and build-lib commands
of the stage 1 compiler. Wasm64 should work transparently once it's
supported in upstream LLVM.
To export a function:
// lib.zig - for exposition, not necessary for this example
pub use @import("add.zig");
// add.zig
export fn add(a: i32, b: i32) i32 {
return a + b;
}
To import a function:
// cube.zig
extern fn square(x: i32) i32;
export fn cube(x: i32) i32 {
return x * square(x);
}
See #770
To help automatically translate code, see the
zig-fmt-pointer-reform-2 branch.
This will convert all & into *. Due to the syntax
ambiguity (which is why we are making this change),
even address-of & will turn into *, so you'll have
to manually fix thes instances. You will be guaranteed
to get compile errors for them - expected 'type', found 'foo'
Before we accepted a nullable allocator for some stuff like
opening files. Now we require an allocator.
Use the mem.FixedBufferAllocator pattern if a bound on the amount
to allocate is known.
This also establishes the pattern that usually an allocator is the
first argument to a function (possibly after "self").
fix docs for std.cstr.addNullByte
self hosted compiler:
* only build docs when explicitly asked to
* clean up main
* stub out zig fmt
The purpose of this is:
* Only one way to do things
* Changing a function with void return type to return a possible
error becomes a 1 character change, subtly encouraging
people to use errors.
See #632
Here are some imperfect sed commands for performing this update:
remove arrow:
```
sed -i 's/\(\bfn\b.*\)-> /\1/g' $(find . -name "*.zig")
```
add void:
```
sed -i 's/\(\bfn\b.*\))\s*{/\1) void {/g' $(find ../ -name "*.zig")
```
Some cleanup may be necessary, but this should do the bulk of the work.
* docgen supports obj_err code kind for demonstrating
errors without explicit test cases
* add documentation for `extern enum`. See #367
* remove coldcc keyword and add @setIsCold. See #661
* add compile errors for non-extern struct, enum, unions
in function signatures
* add .h file generation for extern struct, enum, unions
I started working on #465 and made some corresponding std.io
API changes.
New structs:
* std.io.FileInStream
* std.io.FileOutStream
* std.io.BufferedOutStream
* std.io.BufferedInStream
Removed:
* std.io.File.in_stream
* std.io.File.out_stream
Now instead of &file.out_stream or &file.in_stream to get access to
the stream API for a file, you get it like this:
var file_in_stream = io.FileInStream.init(&file);
const in_stream = &file_in_stream.stream;
var file_out_stream = io.FileOutStream.init(&file);
const out_stream = &file_out_stream.stream;
This is evidence that we might not need any OOP features -
See #130.
* add std.os.deleteTree
* add std.os.deleteDir
* add std.os.page_size
* add std.os API for iterating over directories
* refactor duplication in build.zig
* update documentation on how to run tests
