This is direct result of review comments left by andrewrk and
daurnimator. It makes sense to map `ENOTCAPABLE` into a more generic
`error.AccessDenied`.
This commit adds `error.NotCapable` enum value and makes sure that
every applicable WASI syscall that can return `ENOTCAPABLE` errno
remaps it to `error.NotCapable.
Also, add more informative `@compileError` in a few `std.os` functions
that would otherwise yield a cryptic compile error when targeting
WASI. Finally, enhance docs in a few places and add test case for
`fstatat`.
Adds Windows stub (still needs to be implemented on Windows),
adds WASI implementation, adds unit test testing basic chain of
ops: create file -> symlink -> readlink.
* improve docs
* add TODO comments for things that don't have open issues
* remove redundant namespacing of struct fields
* guard against ioctl returning EINTR
* remove the general std.os.ioctl function in favor of the specific
ioctl_SIOCGIFINDEX function. This allows us to have a more precise
error set, and more type-safe API.
Remove the constants that assume a base unit in favor of explicit
x_per_y constants.
nanosecond calendar timestamps now use i128 for the type. This affects
fs.File.Stat, std.time.nanoTimestamp, and fs.File.updateTimes.
calendar timestamps are now signed, because the value can be less than
the epoch (the user can set their computer time to whatever they wish).
implement std.os.clock_gettime for Windows when clock id is
CLOCK_CALENDAR.
This rather large commit adds/fixes missing WASI functionality
in `libstd` needed to pass the `libstd` tests. As such, now by
default tests targeting `wasm32-wasi` target are enabled in
`test/tests.zig` module. However, they can be disabled by passing
the `-Dskip-wasi=true` flag when invoking the `zig build test`
command. When the flag is set to `false`, i.e., when WASI tests are
included, `wasmtime` with `--dir=.` is used as the default testing
command.
Since the majority of `libstd` tests were relying on `fs.cwd()`
call to get current working directory handle wrapped in `Dir`
struct, in order to make the tests WASI-friendly, `fs.cwd()`
call was replaced with `testing.getTestDir()` function which
resolved to either `fs.cwd()` for non-WASI targets, or tries to
fetch the preopen list from the WASI runtime and extract a
preopen for '.' path.
The summary of changes introduced by this commit:
* implement `Dir.makeDir` and `Dir.openDir` targeting WASI
* implement `Dir.deleteFile` and `Dir.deleteDir` targeting WASI
* fix `os.close` and map errors in `unlinkat`
* move WASI-specific `mkdirat` and `unlinkat` from `std.fs.wasi`
to `std.os` module
* implement `lseek_{SET, CUR, END}` targeting WASI
* implement `futimens` targeting WASI
* implement `ftruncate` targeting WASI
* implement `readv`, `writev`, `pread{v}`, `pwrite{v}` targeting WASI
* make sure ANSI escape codes are _not_ used in stderr or stdout
in WASI, as WASI always sanitizes stderr, and sanitizes stdout if
fd is a TTY
* fix specifying WASI rights when opening/creating files/dirs
* tweak `AtomicFile` to be WASI-compatible
* implement `os.renameatWasi` for WASI-compliant `os.renameat` function
* implement sleep() targeting WASI
* fix `process.getEnvMap` targeting WASI
According to documentation ETIMEDOUT (110) is a valid error code for the read function. I just had my long-running (been running for about 7 weeks) network program crash because it did not handle the ETIMEDOUT error code from "read".
Before it was possible for .intended_io_mode = .blocking,
.capable_io_mode = .evented, and then the implementation would put a
request on the fs thread, which is the wrong behavior. Now it always
calls the appropriate WriteFile/ReadFile function, passing the intended
io mode directly as a parameter.
This makes the behavior tests pass on Windows with --test-evented-io.
`flock` locks based on the file handle, instead of the process id.
This brings the file locking on unix based systems closer to file
locking on Windows.
Now, this function first attempts a case-sensitive lookup.
If no match is found, and `key` is ASCII, then it attempts a
second case-insensitive lookup.
It is not planned to support full Unicode case-insensitivity
on Windows, and in fact relying on non-ASCII case-insensitive
environment variables is fundamentally problematic.
Previously the zig build system incorrectly assumed that the only build
artifact was a binary. Now, when you enable the cache, only the output
dir is printed to stdout, and the zig build system iterates over the
files in that directory, copying them to the output directory.
To support this change:
* Add `std.os.renameat`, `std.os.renameatZ`, and `std.os.renameatW`.
* Fix `std.os.linux.renameat` not compiling due to typos.
* Deprecate `std.fs.updateFile` and `std.fs.updateFileMode`.
* Add `std.fs.Dir.updateFile`, which supports using open directory
handles for both the source and destination paths, as well as an
options parameter which allows overriding the mode.
* Update `std.fs.AtomicFile` to support operating based on an open
directory handle. Instead of `std.fs.AtomicFile.init`, use
`std.fs.Dir.atomicFile`.
* `std.fs.AtomicFile` deinit() better handles the situation when the
rename fails but the temporary file still exists, by still
attempting to remove the temporary file.
* `std.fs.Dir.openFileWindows` is moved to `std.os.windows.OpenFileW`.
* `std.os.RenameError` gains the error codes `NoDevice`,
`SharingViolation`, and `PipeBusy` which have been observed from
Windows.
Closes#4733
The main goal here is to make the function pointers comptime, so that we
don't have to do the crazy stuff with async function frames.
Since InStream, OutStream, and SeekableStream are already generic
across error sets, it's not really worse to make them generic across the
vtable as well.
See #764 for the open issue acknowledging that using generics for these
abstractions is a design flaw.
See #130 for the efforts to make these abstractions non-generic.
This commit also changes the OutStream API so that `write` returns
number of bytes written, and `writeAll` is the one that loops until the
whole buffer is written.
* improve `std.fs.AtomicFile` to use sendfile()
- also fix AtomicFile cleanup not destroying tmp files under some
error conditions
* improve `std.fs.updateFile` to take advantage of the new `makePath`
which no longer needs an Allocator.
* rename std.fs.makeDir to std.fs.makeDirAbsolute
* rename std.fs.Dir.makeDirC to std.fs.Dir.makeDirZ
* add std.fs.Dir.makeDirW and provide Windows implementation of
std.os.mkdirat. std.os.windows.CreateDirectory is now implemented
by calling ntdll, supports an optional root directory handle,
and returns an open directory handle. Its error set has a few more
errors in it.
* rename std.fs.Dir.changeTo to std.fs.Dir.setAsCwd
* fix std.fs.File.writevAll and related functions when len 0 iovecs
supplied.
* introduce `std.fs.File.writeFileAll`, exposing a convenient
cross-platform API on top of sendfile().
* `NoDevice` added to std.os.MakeDirError error set.
* std.os.fchdir gets a smaller error set.
* std.os.windows.CloseHandle is implemented with ntdll call rather than
kernel32.
* rework os.sendfile and add macosx support, and a fallback
implementation for any OS.
* fix sendto compile error
* std.os write functions support partial writes. closes#3443.
* std.os pread / pwrite functions can now return `error.Unseekable`.
* std.fs.File read/write functions now have readAll/writeAll variants
which loop to complete operations even when partial reads/writes
happen.
* Audit std.os read/write functions with respect to Linux returning
EINVAL for lengths greater than 0x7fff0000.
* std.os read/write shim functions do not unnecessarily loop. Since
partial reads/writes are part of the API, the caller will be forced
to loop anyway, and so that would just be code bloat.
* Improve doc comments
* Add a non-trivial test for std.os.sendfile
* Fix std.os.pread on 32 bit Linux
* Add missing SYS_sendfile bit on aarch64
This changset adds a `sendfile(2)` syscall bindings to the linux bits
component. Where available, the `sendfile64(2)` syscall will be
transparently called.
A wrapping function has also been added to the std.os to transform
errno returns to Zig errors.
Change-Id: I86769fc4382c0771e3656e7b21137bafd99a4411
* re-introduce `std.build.Target` which is distinct from `std.Target`.
`std.build.Target` wraps `std.Target` so that it can be annotated as
"the native target" or an explicitly specified target.
* `std.Target.Os` is moved to `std.Target.Os.Tag`. The former is now a
struct which has the tag as well as version range information.
* `std.elf` gains some more ELF header constants.
* `std.Target.parse` gains the ability to parse operating system
version ranges as well as glibc version.
* Added `std.Target.isGnuLibC()`.
* self-hosted dynamic linker detection and glibc version detection.
This also adds the improved logic using `/usr/bin/env` rather than
invoking the system C compiler to find the dynamic linker when zig
is statically linked. Related: #2084
Note: this `/usr/bin/env` code is work-in-progress.
* `-target-glibc` CLI option is removed in favor of the new `-target`
syntax. Example: `-target x86_64-linux-gnu.2.27`
closes#1907
std.os.getenv and std.os.getenvZ have nice compile errors when not linking
libc and using Windows.
std.os.getenvW is provided as a Windows-only API that does not require
an allocator. It uses the Process Environment Block.
std.process.getEnvVarOwned is improved to be a simple wrapper on top of
std.os.getenvW.
std.process.getEnvMap is improved to use the Process Environment Block
rather than calling GetEnvironmentVariableW.
std.zig.system.NativePaths uses process.getEnvVarOwned instead of
std.os.getenvZ, which works on Windows as well as POSIX.
This function expands argv[0] into the absolute path resolved with PATH
environment variable before making the execve syscall. However, in case
the execve fails, e.g. with ENOENT, it did not restore argv to how it
was before it was passed in. This resulted in the caller performing an
invalid free.
This commit also adds verbose debug info when native system C compiler
detection fails. See #4521.
Some C compilers, such as Clang, are known to rely on
argv[0] to find the path to their own executable,
without even bothering to resolve PATH. This results
in the message:
error: unable to execute command: Executable "" doesn't exist!
So we tell ChildProcess to expand argv[0] to the absolute path
to give them a helping hand.
* libc_installation.cpp is deleted.
src-self-hosted/libc_installation.zig is now used for both stage1 and
stage2 compilers.
* (breaking) move `std.fs.File.access` to `std.fs.Dir.access`. The API
now encourages use with an open directory handle.
* Add `std.os.faccessat` and related functions.
* Deprecate the "C" suffix naming convention for null-terminated
parameters. "C" should be used when it is related to libc. However
null-terminated parameters often have to do with the native system
ABI rather than libc. "Z" suffix is the new convention. For example,
`std.os.openC` is deprecated in favor of `std.os.openZ`.
* Add `std.mem.dupeZ` for using an allocator to copy memory and add a
null terminator.
* Remove dead struct field `std.ChildProcess.llnode`.
* Introduce `std.event.Batch`. This API allows expressing concurrency
without forcing code to be async. It requires no Allocator and does
not introduce any failure conditions. However it is not thread-safe.
* There is now an ongoing experiment to transition away from
`std.event.Group` in favor of `std.event.Batch`.
* `std.os.execvpeC` calls `getenvZ` rather than `getenv`. This is
slightly more efficient on most systems, and works around a
limitation of `getenv` lack of integration with libc.
* (breaking) `std.os.AccessError` gains `FileBusy`, `SymLinkLoop`, and
`ReadOnlyFileSystem`. Previously these error codes were all reported
as `PermissionDenied`.
* Add `std.Target.isDragonFlyBSD`.
* stage2: access to the windows_sdk functions is done with a manually
maintained .zig binding file instead of `@cImport`.
* Update src-self-hosted/libc_installation.zig with all the
improvements that stage1 has seen to src/libc_installation.cpp until
now. In addition, it now takes advantage of Batch so that evented I/O
mode takes advantage of concurrency, but it still works in blocking
I/O mode, which is how it is used in stage1.
* `zig test` gainst `--test-evented-io` parameter and gains the ability
to seamlessly run async tests.
* `std.ChildProcess` opens its child process pipe with O_NONBLOCK when
using evented I/O
* `std.io.getStdErr()` gives a File that is blocking even in evented
I/O mode.
* Delete `std.event.fs`. The functionality is now merged into `std.fs`
and async file system access (using a dedicated thread) is
automatically handled.
* `std.fs.File` can be configured to specify whether its handle is
expected to block, and whether that is OK to block even when in
async I/O mode. This makes async I/O work correctly for e.g. the
file system as well as network.
* `std.fs.File` has some deprecated functions removed.
* Missing readv,writev,pread,pwrite,preadv,pwritev functions are added
to `std.os` and `std.fs.File`. They are all integrated with async
I/O.
* `std.fs.Watch` is still bit rotted and needs to be audited in light
of the new async/await syntax.
* `std.io.OutStream` integrates with async I/O
* linked list nodes in the std lib have default `null` values for
`prev` and `next`.
* Windows async I/O integration is enabled for reading/writing file
handles.
* Added `std.os.mode_t`. Integer sizes need to be audited.
* Fixed#4403 which was causing compiler to crash.
This is working towards:
./zig test ../test/stage1/behavior.zig --test-evented-io
Which does not successfully build yet. I'd like to enable behavioral
tests and std lib tests with --test-evented-io in the test matrix in the
future, to prevent regressions.
* move test from std/io/test.zig to std/os/test.zig
* do glibc version check, and make direct system call if
glibc is too old
* disable test when not linking libc, to avoid not working
with outdated qemu version on the CI server. see #4019
In the code review I accidentally encouraged Luna to remove some
handling of errors that are possible according to POSIX, but I think how
Luna had it before was better, so I fixed it, and now the branch should
be good to merge.
This change was mostly made with `zig fmt` and this also modified some whitespace. Note that in some files, `zig fmt` produced incorrect code, so the change was made manually.
The new plan to support hobby operating systems is #3784.
And what kind of name is "Zen" anyway? There's already a
[Zen programming language](http://zenlang.sourceforge.net/)
and that's just confusing.
this also deletes C string literals from the language, and then makes
the std lib changes and compiler changes necessary to get the behavior
tests and std lib tests passing again.
* Delete `std.net.TmpWinAddr`. I don't think that was ever meant to
be a real thing.
* Delete `std.net.OsAddress`. This abstraction was not helpful.
* Rename `std.net.Address` to `std.net.IpAddress`. It is now an extern
union of IPv4 and IPv6 addresses.
* Move `std.net.parseIp4` and `std.net.parseIp6` to the
`std.net.IpAddress` namespace. They now return `IpAddress` instead of
`u32` and `std.net.Ip6Addr`, which is deleted.
* Add `std.net.IpAddress.parse` which accepts a port and parses either
an IPv4 or IPv6 address.
* Add `std.net.IpAddress.parseExpectingFamily` which additionally
accepts a `family` parameter.
* `std.net.IpAddress.initIp4` and `std.net.IpAddress.initIp6` are
improved to directly take the address fields instead of a weird
in-between type.
* `std.net.IpAddress.port` is renamed to `std.net.IpAddress.getPort`.
* Added `std.net.IpAddress.setPort`.
* `os.sockaddr` struct on all targets is improved to match the
corresponding system struct. Previously I had made it a union of
sockaddr_in, sockaddr_in6, and sockaddr_un. The new abstraction for
this is now `std.net.IpAddress`.
* `os.sockaddr` and related bits are added for Windows.
* `os.sockaddr` and related bits now have the `zero` fields default
to zero initialization, and `len` fields default to the correct size.
This is enough to abstract the differences across targets, and so
no more switch on the target OS is needed in `std.net.IpAddress`.
* Add the missing `os.sockaddr_un` on FreeBSD and NetBSD.
* `std.net.IpAddress.initPosix` now takes a pointer to `os.sockaddr`.
* delete the std/event/net directory
* `std.event.Loop.waitUntilFdReadable` and related functions
no longer have possibility of failure. On Linux, they fall
back to poll() and then fall back to sleep().
* add some missing `noasync` decorations in `std.event.Loop`
* redo the `std.net.Server` API. it's quite nice now, but
shutdown does not work cleanly. There is a race condition with
close() that I am actively working on.
* move `std.io.OutStream` to its own file to match `std.io.InStream`.
I started working on making `write` integrated with evented I/O,
but it got tricky so I backed off and filed #3557. However
I did integrate `std.os.writev` and `std.os.pwritev` with evented I/O.
* add `std.Target.stack_align`
* move networking tests to `lib/std/net/test.zig`
* add `std.net.tcpConnectToHost` and `std.net.tcpConnectToAddress`.
* rename `error.UnknownName` to `error.UnknownHostName` within the
context of DNS resolution.
* add `std.os.readv`, which is integrated with evented I/O.
* `std.os.preadv`, is now integrated with evented I/O.
* `std.os.accept4` now asserts that ENOTSOCK and EOPNOTSUPP never
occur (misuse of API), instead of returning errors.
* `std.os.connect` is now integrated with evented I/O.
`std.os.connect_async` is gone. Just use `std.os.connect`.
* fix false positive dependency loop regarding async function frames
* add more compile notes to help when dependency loops occur
in determining whether a function is async.
* ir: change an assert to ir_assert to make it easier to find
workarounds for when such an assert is triggered. In this case
it was trying to parse an IPv4 address at comptime.
It had the downside of running all the comptime blocks and resolving
all the usingnamespaces of each system, when just trying to discover if
the current system is a particular one.
For Darwin, where it's nice to use `std.Target.current.isDarwin()`, this
demonstrates the utility that #425 would provide.
* All the data types from `@import("builtin")` are moved to
`@import("std").builtin`. The target-related types are moved
to `std.Target`. This allows the data types to have methods, such as
`std.Target.current.isDarwin()`.
* `std.os.windows.subsystem` is moved to
`std.Target.current.subsystem`.
* Remove the concept of the panic package from the compiler
implementation. Instead, `std.builtin.panic` is always the panic
function. It checks for `@hasDecl(@import("root"), "panic")`,
or else provides a default implementation.
This is an important step for multibuilds (#3028). Without this change,
the types inside the builtin namespace look like different types, when
trying to merge builds with different target settings. With this change,
Zig can figure out that, e.g., `std.builtin.Os` (the enum type) from one
compilation and `std.builtin.Os` from another compilation are the same
type, even if the target OS value differs.
The sockaddr pointer and size of the accept function points to a data structure that can only be determined at runtime. The only requirement is that it must be large enough to hold 2 bytes for the address family value. Typeical usage of the socket API is for UDP/TCP IPv4 and IPv6 sockets, which use sockaddr_in and sockaddr_in6. And some sockets can actually support both simultaneously in which case the app may want to have access to the size of the returned sockaddr. Operating systems can even support custom protocols where they use custom sockaddr data structures. In this case the standard library would have no knowledge of the actual size of the sockaddr being passed into the accept function. In this case the standard library should defer to the app to pass in the size of their structure.
* Added `std.c.unlinkat` and `std.os.unlinkat`.
* Removed `std.fs.MAX_BUF_BYTES` (this declaration never made it to
master branch)
* Added `std.fs.Dir.deleteTree` to be used on an open directory handle.
* `std.fs.deleteTree` has better behavior for both relative and
absolute paths. For absolute paths, it opens the base directory
and uses that handle for subsequent operations. For relative paths,
it does a similar strategy, using the cwd handle.
* The error set of `std.fs.deleteTree` is improved to no longer have
these possible errors:
- OutOfMemory
- FileTooBig
- IsDir
- DirNotEmpty
- PathAlreadyExists
- NoSpaceLeft
* Added `std.fs.Dir.posix_cwd` which is a statically initialized
directory representing the current working directory.
* The error set of `std.Dir.open` is improved to no longer have these
possible errors:
- FileTooBig
- IsDir
- NoSpaceLeft
- PathAlreadyExists
- OutOfMemory
* Added more alternative functions to `std.fs` for when the path
parameter is a null terminated string. This can sometimes be more
effecient on systems which have an ABI based on null terminated
strings.
* Added `std.fs.Dir.openDir`, `std.fs.Dir.deleteFile`, and
`std.fs.Dir.deleteDir` which all operate on an open directory handle.
* `std.fs.Walker.Entry` now has a `dir` field, which can be used to do
operations directly on `std.fs.Walker.Entry.basename`, avoiding
`error.NameTooLong` for deeply nested paths.
* Added more docs to `std.os.OpenError`
This commit does the POSIX components for these changes. I plan to
follow up shortly with a commit for Windows.
* `std.os.execve` had the wrong name; it should have been
`std.os.execvpe`. This is now corrected.
* introduce `std.os.execveC` which does not look at PATH, and uses
null terminated parameters, matching POSIX ABIs. It does not
require an allocator.
* fix typo nonsense doc comment in `std.fs.MAX_PATH_BYTES`.
* introduce `std.os.execvpeC`, which is like `execvpe` except it
uses null terminated parameters, matching POSIX ABIs, and thus
does not require an allocator.
* `std.os.execvpe` implementation is reworked to only convert
parameters and then delegate to `std.os.execvpeC`.
* `std.os.execvpeC` improved to handle `ENOTDIR`. See #3415
