* std.zig.parse is moved to std.zig.Ast.parse
* the new function has an additional parameter that requires passing
Mode.zig or Mode.zon
* moved parser.zig code to Parse.zig
* added parseZon function next to parseRoot function
- Revert the addition of CLANG_SYSTEM_LIBARIES and LLVM_SYSTEM_LIBRARIES
- Change addCMakeLibraryList to parse non-absolute path .lib dependencies as system libraries
- Remove ZIG_ENABLE_ZSTD in favour of allowing ZIG_STATIC_ZSTD to be toggled off explicitly when ZIG_STATIC is on
- Remove ZIG_ENABLE_LIBCPP (now unused)
Using zig cc with CMake on Windows was failing during compiler
detection. -nostdinc was causing the crt not to be linked, and Coff/lld.zig
assumed that wWinMainCRTStartup would be present in this case.
-nostdlib did not prevent the default behaviour of linking libc++ when
zig c++ was used. This caused libc++ to be built when CMake ran
ABI detection using zig c++, which fails as libcxxabi cannot compile
under MSVC.
- Change the behaviour of COFF -nostdinc to set /entry to the function that the
default CRT method for the specified subsystem would have called.
- Fix -ENTRY being passed twice if it was specified explicitly and -nostdlib was present.
- Add support for /pdb, /version, /implib, and /subsystem as linker args (passed by CMake)
- Remove -Ddisable-zstd, no longer needed
- Add -Ddisable-libcpp for use when bootstrapping on msvc
- first convert to lower-case then perform mappings
- effect: freebsd, netbsd and openbsd convert arch `amd64` to `x86_64`
- effect: convert `arm64` to `aarch64`
By pulling out the parallel hashing setup from `CodeSignature.zig`,
we can now reuse it different places across MachO linker (for now;
I can totally see its usefulness beyond MachO, eg. in COFF or ELF too).
The parallel hasher is generic over actual hasher such as Sha256 or MD5.
The implementation is kept as it was.
For UUID calculation, depending on the linking mode:
* incremental - since it only supports debug mode, we don't bother with MD5
hashing of the contents, and populate it with random data but only once
per a sequence of in-place binary patches
* traditional - in debug, we use random string (for speed); in release,
we calculate the hash, however we use LLVM/LLD's trick in that we
calculate a series of MD5 hashes in parallel and then one an MD5 of MD5
final hash to generate digest.
This allows using `zig ar` for `CMAKE_AR`. Unfortunately, it requires a
patch to CMakeLists.txt and cannot be done merely with flags to the
cmake line.
On windows we get:
lld-link: error: undefined symbol: __stack_chk_fail
>>> referenced by CMakeFiles/zig2.dir/zig2.c.obj:(main)
>>> referenced by CMakeFiles/zig2.dir/zig2.c.obj:(main_main)
>>> referenced by CMakeFiles/zig2.dir/zig2.c.obj:(log_scoped_28_default_29_err__anon_2764)
>>> referenced 36192 more times
lld-link: error: undefined symbol: __stack_chk_guard
>>> referenced by CMakeFiles/zig2.dir/zig2.c.obj:(.refptr.__stack_chk_guard)
>>> referenced by CMakeFiles/zig2.dir/compiler_rt.c.obj
Before it was checking if the compiler was AppleClang, however, this did
not handle the case when using a compiled-from-source Clang on a macOS
computer, in which case the linker is still Apple ld64, and it is in
fact the linker that needs to have the different flag to communicate
stack size.
This flag makes CI servers print garbage to the terminal. I started
implementing detection in wasi.c, but it would have destroyed the
beautiful code that only touches the C stdlib, does not do any
conditional compilation based on the operating system, or rely on any
POSIX functions.
So instead, let's just do without this flag to retain simplicity in this
step of the build process.
This takes a bit longer since the interpreted part has to do more work
but it saves a round trip through the compiler by allowing `zig2 build`
to be the final step. 1-2-3, done.
For me this is currently failing due to compilation errors generated by
GCC when compiling zig2.c but in theory if those are fixed, it should
work!
* synchronize zig1.c from zig-wasi external project
* change the way argv works to avoid absolute paths
* autodetect isatty
* compiler_rt: disable some functions when object format is C
* add missing flag from config.zig.in
The next problem is that compiling compiler_rt.c with gcc gives
"conflicting types" errors for `__eqhf2` and friends.
This ensures that the Zig version will be re-computed when jumping
through the source tree, which is especially important if bisecting
across AstGen- or other changes that must not use the old cache.
* Old cmake option: `-DZIG_SKIP_INSTALL_LIB_FILES=ON`
* New cmake option: `-DZIG_NO_LIB=ON`
* Old build.zig option: `-Dskip-install-lib-files`
* New build.zig option: `-Dno-lib`
Motivation is making build commands easier to type.
CMake recognizes the CMAKE_PREFIX_PATH environment variable for some
things, and also the CMAKE_PREFIX_PATH cache variable for other things.
However, it does not relate these two things, i.e. if the environment
variable is set, CMake does not populate the cache variable in a
corresponding manner. Some package systems, such as Homebrew, set the
environment variable but not the cache variable. Furthermore, the
environment variable follows the system path separator, such as ':' on
POSIX and ';' on Windows, but the cache variable follows CMake's array
behavior, i.e. always ';' for a separator.
Closes#13242
build.zig: add a 'compile' step to compile the self-hosted compiler
without installing it.
Compilation: set cache mode to whole when using the LLVM backend and
--enable-cache is passed.
This makes `zig build` act the same as it does with stage1. Upside is
that a second invocation of `zig build` on an unmodified source tree
will avoid redoing the compilation again. Downside is that it will
proliferate more garbage in the project-local cache (same as stage1).
This can eventually be fixed when Zig's incremental compilation is more
robust; we can go back to having LLVM use CacheMode.incremental and rely
on it detecting no changes and avoiding doing the flush() step.
* CMakeLists: pass `-Dstrip` for release zig builds
* pass -target and -mcpu to zig1. works around llvm on freebsd
incorrectly detecting "freestanding" instead of "freebsd" for the
native OS.
* ci.ziglang.org is now responsible for creating aarch64-macos tarballs
rather than Azure.
This is a simplification of the cmake build script which introduces a
new "stage3" target that is built by default, which builds and installs
a stage3 zig.
It greatly simplifies the build instructions for Zig, making it conform
to the regular cmake routine, while still producing a stage3 artifact.
Now instead of zig.h being baked into the compiler binary, it is a
header file distributed along with all the other header files
distributed with Zig.
Closes#11643
This commit reworks the LLVM/Clang/LLD discovery process for CMake. The
biggest changes are that:
1. We search for LLVM from most preferred directory to least, skipping
any `llvm-config` that is the wrong version, or that doesn't
support the requested link mode ("static" or "shared").
2. `ZIG_PREFER_CLANG_CPP_DYLIB` has been renamed to `ZIG_SHARED_LLVM`,
to better align with `ZIG_STATIC_LLVM`.
3. We only search for LLVM in the same directory alongside LLVM.
4. LLVM's link mode is forwarded to Clang, so that we can look for the
appropriate shared/static libraries.
5. We use `--link-static` when querying `--system-libs` from llvm-config,
so that this will include libz and other dependencies for
statically linking LLD
CMake has a surprising default behavior where looking up a library by
multiple names gives the name order higher priority than the directory
search order.
For example, if your system provides "llvm-config-14" and
CMAKE_PREFIX_PATH includes "llvm-config", CMake will always end up
choosing the system-provided llvm-config-14.
This change add NAMES_PER_DIR to request the more sensible behavior:
directory search order has higher priority than name order, so
CMAKE_PREFIX_PATH always wins over system-provided tools/libraries.
LLVM 14 deprecated a bunch of C API functions in preparation for opaque
pointer changes. However, they did not actually implement opaque pointer
semantics, so the deprecations are largely masturbatory. I have nothing
against masturbation, I am just busy trying to get the self hosted
compiler done for 0.10.0, so we will come back to this later.
Rather than lowering float negation as `0.0 - x`.
* Add AIR instruction for float negation.
* Add compiler-rt functions for f128, f80 negation
closes#11853
After doing performance testing, it seems that multi-compilation-unit
compiler-rt did not bring the performance improvements that we expected
it to. The idea is that it makes linking faster, however, it incurred a
cost in the frontend that was not offset by any gains in linking.
Furthermore, the single-object compiler-rt (with -ffunction-sections and
--gc-sections) ends up being fewer bytes on disk and so it's actually
the same or faster linking speed than the multi-compilation-unit
version.
So we are planning to keep using single-compilation-unit compiler-rt for
the foreseeable future, but may experiment with this again in the
future, in which case this commit can be reverted.
The purpose of this branch is to switch to using an object file for each
independent function, in order to make linking simpler - instead of
relying on `-ffunction-sections` and `--gc-sections`, which involves the
linker doing the work of linking everything and then undoing work via
garbage collection, this will allow the linker to only include the
compilation units that are depended on in the first place.
This commit makes progress towards that goal.
Rename all references of sparcv9 to sparc64, to make Zig align more with
other projects. Also, added new function to convert glibc arch name to Zig
arch name, since it refers to the architecture as sparcv9.
This is based on the suggestion by @kubkon in PR 11847.
(https://github.com/ziglang/zig/pull/11487#pullrequestreview-963761757)
stage2: change logic for detecting whether the main package is inside
the std package. Previously it relied on realpath() which is not portable.
This uses resolve() which is how imports already work.
* stage2: fix cleanup bug when creating Module
* flatten lib/std/special/* to lib/*
- this was motivated by making main_pkg_is_inside_std false for
compiler_rt & friends.
* rename "mini libc" to "universal libc"
* unify the logic for exporting math functions from compiler-rt,
with the appropriate suffixes and prefixes.
- add all missing f128 and f80 exports. Functions with missing
implementations call other functions and have TODO comments.
- also add f16 functions
* move math functions from freestanding libc to compiler-rt (#7265)
* enable all the f128 and f80 code in the stage2 compiler and behavior
tests (#11161).
* update std lib to use builtins rather than `std.math`.
Add placeholder files for Codegen, Emit, and Mir stages, complete with
a placeholder implementation of generate() to make it able to be plugged in
to the frontend. At the moment the implementation just panics, it'll be
worked on incrementally later.
Also, this registers the sparcv9 backend files into CMakeLists.txt.
Currently, the new API will only be available on macOS with
the intention of adding more POSIX systems to it incrementally
(such as Linux, etc.).
Changes:
* add `posix_spawn` wrappers in a separate container in
`os/posix_spawn.zig`
* rewrite `ChildProcess.spawnPosix` using `posix_spawn` targeting macOS
as `ChildProcess.spawnMacos`
* introduce a `posix_spawn` specific `std.c.waitpid` wrapper which
does return an error in case the child process failed to exec - this
is required for any process that was spawned using `posix_spawn`
mechanism as, by definition, the errors returned by `posix_spawn`
routine cover only the `fork`-equivalent; `pre-exec()` and `exec()`
steps are covered by a catch-all error `ECHILD` returned by `waitpid`
on unsuccessful execution, e.g., no such file error, etc.
- neg can only overflow, if a == MIN
- case `-0` is properly handled by hardware, so overflow check by comparing
`a == MIN` is sufficient
- tests: MIN, MIN+1, MIN+4, -42, -7, -1, 0, 1, 7..
See #1290
- abs can only overflow, if a == MIN
- comparing the sign change from wrapping addition is branchless
- tests: MIN, MIN+1,..MIN+4, -42, -7, -1, 0, 1, 7..
See #1290
- adds __cmpsi2, __cmpdi2, __cmpti2
- adds __ucmpsi2, __ucmpdi2, __ucmpti2
- use 2 if statements with 2 temporaries and a constant
- tests: MIN, MIN+1, MIN/2, -1, 0, 1, MAX/2, MAX-1, MAX if applicable
See #1290
- use negXi2.zig to prevent confusion with negXf2.zig
- used for size optimized builds and machines without carry instruction
- tests: special cases 0, -INT_MIN
* use divTrunc range and shift with constant offsets
See #1290
- each byte gets masked, shifted and combined
- use boring masks instead of comptime for readability
- tests: bit patterns with reverse operation, if applicable
See #1290
`getExternalExecutor` is moved from `std.zig.CrossTarget` to
`std.zig.system.NativeTargetInfo.getExternalExecutor`.
The function also now communicates a bit more information about *why*
the host is unable to execute a binary. The CLI is updated to report
this information in a useful manner.
`getExternalExecutor` is also improved to detect such patterns as:
* x86_64 is able to execute x86 binaries
* aarch64 is able to execute arm binaries
* etc.
Added qemu-hexagon support to `getExternalExecutor`.
`std.Target.canExecBinaries` of is removed; callers should use the more
powerful `getExternalExecutor` instead.
Now that `zig test` tries to run the resulting binary no matter what,
this commit has a follow-up change to the build system and docgen to
utilize the `getExternalExecutor` function and pass `--test-no-exec`
in some cases to avoid getting the error.
Additionally:
* refactor: extract NativePaths and NativeTargetInfo into their own
files named after the structs.
* small improvement to langref to reduce the complexity of the `callconv`
expression in a couple examples.
- use Bit Twiddling Hacks: Compute parity in parallel
- test cases derived from popcount.zig
- tests: compare naive approach 10_000 times with random numbers created
from naive seed 42
- compiler_rt.zig: sort by LLVM builtin order and add comments to improve structure
See #1290
- apply simpler approach than LLVM for __popcountdi2
taken from The Art of Computer Programming and generalized
- rename popcountdi2.zig to popcount.zig
- test cases derived from popcountdi2_test.zig
- tests: compare naive approach 10_000 times with
random numbers created from naive seed 42
See #1290
Add additional search paths pointing at homebrew prefixes as Apple
doesn't ship a static libncurses for linking - only a stub for dynamic
linking `libncurses.tbd`.
* wasm: Move wasm's codegen to arch/wasm/CodeGen.zig
* wasm: Define Wasm's Mir
This declares the initial most-used instructions for wasm as
well as the data that represents them.
TODO: Add binary operand opcodes.
By re-using the wasm opcode values, we can emit each opcode very easily
by simply using `@enumToInt()`. However, this poses a possible problem:
If we use all of wasm's opcodes, it leaves us no room to use synthetic opcodes such as debugging instructions.
We could use reserved opcodes, but the wasm spec may use them at some point.
TODO: Check if we should perhaps use a 16bit tag where the highest bits are used for synthetic opcodes.
* wasm: Define basic Emit structure
* wasm: Implement corresponding Emit functions for MIR
* wasm: Initial lowering to MIR
- This implements lowering to MIR from AIR for storing and loading of locals
as well as emitting immediates.
- Relocating function indexes has been simplified a lot as well as we no
longer need to patch offsets and we write a relocatable value instead.
- Locals are now emitted at the beginning of the function section entry
meaning all offsets we generate are stable.
* wasm: Lower all AIR instructions to MIR
* wasm: Implement remaining MIR instructions
* wasm: Fix function relocations
* wasm: Get all tests working
* wasm: Make `Data` 4 bytes instead of 8.
- 64bit immediates are now stored in 2 seperate u32's.
- 64bit floats are now stored in 2 seperate u32's.
- `mem_arg` is now stored as a seperate payload in extra.
Systems with multiple LLVM toolchains installed (e.g. one globally and one
in $HOME/local) would get confused and fail to compile. Being explicit
about the version required will force CMake to find the right version of LLVM.
The main purpose of this branch is to explore avoiding the
`usingnamespace` feature of the zig language, specifically with regards
to `std.os` and related functionality.
If this experiment is successful, it will provide a data point on
whether or not it would be practical to entirely remove `usingnamespace`
from the language.
In this commit, `usingnamespace` has been completely eliminated from
the Linux x86_64 compilation path, aside from io_uring.
The behavior tests pass, however that's as far as this branch goes. It is
very breaking, and a lot more work is needed before it could be
considered mergeable. I wanted to put a pull requset up early so that
zig programmers have time to provide feedback.
This is progress towards closing #6600 since it clarifies where the
actual "owner" of each declaration is, and reduces the number of
different ways to import the same declarations.
One of the main organizational strategies used here is to do namespacing
with real namespaces (e.g. structs) rather than by having declarations
share a common prefix (the C strategy). It's no coincidence that
`usingnamespace` has similar semantics to `#include` and becomes much
less necessary when using proper namespaces.
The primary purpose of this change is to eliminate one usage of
`usingnamespace` in the standard library - specifically the usage for
errno values in `std.os.linux`.
This is accomplished by truncating the `E` prefix from error values, and
making errno a proper enum.
A similar strategy can be used to eliminate some other `usingnamespace`
sites in the std lib.
When using `build-exe` or `build-lib -dynamic`, `-fcompiler-rt` means building
compiler-rt into a static library and then linking it into the executable.
When using `build-lib`, `-fcompiler-rt` means building compiler-rt into an
object file and then adding it into the static archive.
Before this commit, when using `build-obj`, zig would build compiler-rt
into an object file, and then on ELF, use `lld -r` to merge it into the
main object file. Other linker backends of LLD do not support `-r` to
merge objects, so this failed with error messages for those targets.
Now, `-fcompiler-rt` when used with `build-obj` acts as if the user puts
`_ = @import("compiler_rt");` inside their root source file. The symbols
of compiler-rt go into the same compilation unit as the root source file.
This is hooked up for stage1 only for now. Once stage2 is capable of
building compiler-rt, it should be hooked up there as well.
This commit changes the AIR file and the documentation of the memory
layout. The actual work of modifying the surrounding code (in Sema and
codegen) is not yet done.
It makes sense to have them as a dependent type since they only ever
deal with TextBlocks. Simplify Relocations to rely on symbol indices
and symbol resolver rather than pointers.
* rename files to adhere to conventions
* remove unnecessary function / optionality
* fix merge conflict
* better panic message
* remove unnecessary TODO comment
* proper namespacing of declarations
* clean up documentation comments
* no copyright header needed for a brand new zig file that is not
copied from anywhere
After giving it more thought, it doesn't make sense to separate
the two structurally. Instead, there should be two constructors
for a Dylib struct: one from binary file, and the other from a stub
file. This cleans up a lot of code and opens the way for recursive
parsing of re-exports from a dylib which are a hard requirement for
native feel when linking frameworks.
Instead of trying to fit a stub file into the frame of a Dylib struct,
I think it makes more sense to keep them as separate entities with
possibly shared interface (which would be added in the future).
This cleaned up a lot of logic in Dylib as well as Stub. Also, while
here I've made creating actual *Symbols lazy in the sense Dylib and
Stub only store hash maps of symbol names that they expose but we
defer create and referencing given dylib/stub until link time when
a symbol is actually referenced. This should reduce memory usage
and speed things up a bit.
This breaking change disambiguates between overriding the lib dir when
performing an installation with the Zig Build System, and overriding the
lib dir that the Zig installation itself uses.
- deprecates `std.Thread.spinLoopHint` and moves it to `std.atomic.spinLoopHint`
- added an Atomic(T) generic wrapper type which replaces atomic.Bool and atomic.Int
- in Atomic(T), selectively expose member functions depending on T and include bitwise atomic methods when T is an Integer
- added fence() and compilerFence() to std.atomic
The same entrypoint supports the following commands:
* ar
* ranlib
* dlltool
* lib
For now, our strategy is to bundle the (renamed) `main()` function of
llvm-ar, same as our strategy for `zig clang`. However, as Zig matures,
a goal will be to replace the dependency on LLVM with our own
implementation of this tool, so that it is available in builds of zig
that do not have LLVM extensions enabled.
This commit also categorizes the subcommands into categories in the
--help menu.
* Extracts AstGen logic from ir.cpp into astgen.cpp. Reduces the
largest file of stage1 from 33,551 lines to 25,510.
* tokenizer: rework it completely to match the stage2 tokenizer logic.
They can now be maintained together; when one is changed, the other
can be changed in the same way.
- Each token now takes up 13 bytes instead of 64 bytes. The tokenizer
does not parse char literals, string literals, integer literals,
etc into meaningful data. Instead, that happens during parsing or
astgen.
- no longer store line offsets. Error messages scan source
files to find the line/column as needed (same as stage2).
- main loop: instead of checking the loop, handle a null byte
explicitly in the switch statements. This is a nice improvement
that we may want to backport to stage2.
- delete some dead tokens, artifacts of past syntax that no longer
exists.
* Parser: fix a TODO by parsing builtin functions as tokens rather than
`@` as a separate token. This is how stage2 does it.
* Remove some debugging infrastructure. These will need to be redone,
if at all, as the code migrates to match stage2.
- remove the ast_render code.
- remove the IR debugging stuff
- remove teh token printing code
We've settled on the nomenclature for the artifacts the compiler
pipeline produces:
1. Tokens
2. AST (Abstract Syntax Tree)
3. ZIR (Zig Intermediate Representation)
4. AIR (Analyzed Intermediate Representation)
5. Machine Code
Renaming `ir` identifiers to `air` will come with the inevitable
air-memory-layout branch that I plan to start after the 0.8.0 release.
Rename include dir to match the convention:
from `wasm32-wasi` to `wasm-wasi-musl`
Add building stubs which will be used to build and cache WASI
libc sysroot.
The support is minimalistic in the sense that we only support actual
dylib files and not stubs/tbds yet, and we also don't support re-exports
just yet.
Store only globals and undefs at the linker level, while all locals
stay scoped to the actual object file they were defined in. This is
fine since the relocations referencing locals will always be resolved
first using the local symbol table before checking for the reference
within the linker's global symbol table.
This also paves the way for proper symbol resolution from within static
and dynamic libraries.
Now that we ship our own linker for MachO by default in both stage1
and stage2, we need a way to enable logs for verbose debugging.
This commit adds `ZIG_ENABLE_LOGGING` cmake option which is equivalent
to stage2's `-Dlog` flag.
To enable it when building stage1 with cmake, add:
```
cmake .. -DZIG_ENABLE_LOGGING=on
```
new pipeline `BuildMacOS_arm64`
- `vmImage: 'macOS-10.15' `
new `macos_arm64_script`
- switch from using `make` to `ninja`
- select xcode 12.4
- set zig-cache env variables
- build host-zig binary with xcode, link against llvm for x86_64 (target macos 10.15)
- build arm64-zig binary with xcode and host-zig, link against llvm for arm64 (target macos 11.0)
- ad-hoc codesign arm64 binary with linker
- use host-zig for docgen
- use host-zig for experimental std lib docs
- sync final `release/` hierarchy with `linux_script`
- use gnu-tar for good-practices (set owner, set sort)
enhance `CMakeLists.txt`
- do not build `zig0` when cross-compiling
- disable `BYPRODUCTS` directive `zig1.o` to avoid `ninja` error
see #8265
Previously, there was an option ZIG_PREFER_LLVM_CONFIG which would
override the default of not using llvm-config when cross compiling.
That option is now removed in favor of the more powerful
ZIG_USE_LLVM_CONFIG which defaults to OFF for cross compiling and ON for
native compilation. The option overrides the default.
This will be used in zig-bootstrap to improve support for native builds.
Reverts bf642204b3 and uses a different
workaround, suggested by @LemonBoy.
There is either a compiler bug or a design flaw somewhere around here.
It does not have to block this branch, but I need to understand exactly
what's going on here and make it so that nobody ever has to run into
this problem again.
This type is not widely applicable enough to be a public part of the
public interface of the std.
The current implementation in only fully utilized by the zig fmt
implementation, which could benefit by even tighter integration as
will be demonstrated in the next commit. Therefore, move the current
io.AutoIndentingStream to lib/std/zig/render.zig.
The C backend of the self hosted compiler also use this type currently,
but it does not require anywhere near its full complexity. Therefore,
implement a greatly simplified version of this interface in
src/codegen/c.zig.
* move concurrency primitives that always operate on kernel threads to
the std.Thread namespace
* remove std.SpinLock. Nobody should use this in a non-freestanding
environment; the other primitives are always preferable. In
freestanding, it will be necessary to put custom spin logic in there,
so there are no use cases for a std lib version.
* move some std lib files to the top level fields convention
* add std.Thread.spinLoopHint
* add std.Thread.Condition
* add std.Thread.Semaphore
* new implementation of std.Thread.Mutex for Windows and non-pthreads Linux
* add std.Thread.RwLock
Implementations provided by @kprotty
