Commit Graph

123 Commits

Author SHA1 Message Date
Andrew Kelley
3b77f1ed7e rename zig-cache to .zig-cache
closes #20077
2024-05-29 10:20:15 -07:00
Andrew Kelley
f97c2f28fd update the codebase for the new std.Progress API 2024-05-27 20:56:48 -07:00
Christofer Nolander
8f6b1f2c38
zig fetch: resolve branch/tag names to commit SHA (#19941)
* Revert "Revert "Merge pull request #19349 from nolanderc/save-commit""

This reverts commit 6ca4ed5948.

* update to new URI changes, rework `--save` type

* initialize `latest_commit` to null everywhere
2024-05-11 14:19:35 -07:00
Andrew Kelley
6ca4ed5948 Revert "Merge pull request #19349 from nolanderc/save-commit"
This reverts commit 7fa2357d05, reversing
changes made to cb77bd672c.
2024-05-10 16:41:15 -07:00
Andrew Kelley
7fa2357d05
Merge pull request #19349 from nolanderc/save-commit
`zig fetch`: resolve branch/tag names to commit SHA
2024-05-10 16:27:30 -07:00
Andrew Kelley
a72292513e add std.Thread.Pool.spawnWg
This function accepts a WaitGroup parameter and manages the reference
counting therein. It also is infallible.

The existing `spawn` function is still handy when the job wants to
further schedule more tasks.
2024-05-03 20:58:02 -07:00
Jonathan Marler
a96b78c170 add std.zip and support zip files in build.zig.zon
fixes #17408

Helpful reviewers/testers include Joshe Wolfe, Auguste Rame, Andrew
Kelley and Jacob Young.

Co-authored-by: Joel Gustafson <joelg@mit.edu>
2024-05-03 16:58:53 -04:00
Igor Anić
9cfac4718d fetch: combine unpack error validation in a single function
Follow up of: #19500
[discussion](https://github.com/ziglang/zig/pull/19500#discussion_r1558238138)
2024-04-11 15:44:40 -07:00
Jacob Young
c4587dc9f4 Uri: propagate per-component encoding
This allows `std.Uri.resolve_inplace` to properly preserve the fact
that `new` is already escaped but `base` may not be.  I originally tried
just moving `raw_uri` around, but it made uri resolution unmanagably
complicated, so I instead added per-component information to `Uri` which
allows extra allocations to be avoided when constructing uris with
components from different sources, and in some cases, deferring the work
all the way to when the uri is printed, where an allocator may not even
be needed.

Closes #19587
2024-04-10 02:11:54 -07:00
Igor Anić
4151e6c31b fetch: use arena allocator for diagnostic/UnpackResult
Reference:
https://github.com/ziglang/zig/pull/19500#discussion_r1556476973

Arena is now used for Diagnostic (tar and git). `deinit` is not called on Diagnostic
allowing us to reference strings from Diagnostic in UnpackResult without
dupe.

That seamed reasonable to me. Instead of using gpa for Diagnostic, and
then dupe to arena. Or using arena for both and making dupe so we can deinit
Diagnostic.
2024-04-09 15:00:22 +02:00
Igor Anić
f8dd2a1064 fetch: add executable bit test 2024-04-09 15:00:22 +02:00
Igor Anić
8c58b8fe01 fetch: refactor package root in errors
Use stripRoot in less places. Strip it while copying error from
diagnostic to unpack result so other palaces can be free of this logic.
2024-04-09 15:00:22 +02:00
Igor Anić
b422e4a202 fetch: save test cases
Prepare test cases, store them in Fetch/testdata.
They cover changes in this PR as well from previous one #19111.
2024-04-09 15:00:22 +02:00
Igor Anić
3d5a9237f7 fetch: use empty string instead of null for root_dir
Make it consistent with Cache.Path sub_path.
Remove null check in many locations.
2024-04-09 15:00:22 +02:00
Igor Anić
5f0b434f90 fetch: remove root_dir from error messages
To be consistent with paths in manifest.
2024-04-09 15:00:22 +02:00
Igor Anić
22e9c50376 fetch: fix test tarball
Should include folder structure, at least root folder so it can be found
in pipeToFileSystem.
2024-04-09 15:00:21 +02:00
Igor Anić
fc745fb05c fetch: remove test with mount and net dependencies 2024-04-09 15:00:21 +02:00
Igor Anić
84fac2242c fix zig fmt 2024-04-09 15:00:21 +02:00
Igor Anić
dc61c2e904 add comments 2024-04-09 15:00:21 +02:00
Igor Anić
373d48212f fetch: add pathological packages test
Using test cases from:
https://github.com/ianprime0509/pathological-packages repository.
Depends on existence of the FAT32 file system. Folder is in FAT32 file
system because it is case insensitive and and does not support symlinks.

It is complicated test case requires internet connection, depends on
existence of FAT32 in the specific location. But it is so valuable for
development. Running `zig test Package.zig` is so much faster than
building zig binary and running `zig fetch URL`. Committing it here
although it should probably be removed.
2024-04-09 15:00:21 +02:00
Igor Anić
5a38924a7d fetch.git: collect file create diagnostic errors
On case insensitive file systems, don't overwrite files with same name
in different casing. Add diagnostic error so caller could decide what to do.
2024-04-09 15:00:21 +02:00
Igor Anić
ad60b6c1ed fetch: update comments 2024-04-09 15:00:21 +02:00
Igor Anić
dfec4918a3 fetch: remove absolute path from tests 2024-04-09 15:00:21 +02:00
Igor Anić
4d6a7e074b fetch: filter unpack errors
Report only errors which are not filtered by paths in build.zig.zon.
2024-04-09 15:00:21 +02:00
Igor Anić
a0790914b4 fetch: return UnpackResult from unpackResource
Test that we are still outputing same errors.
2024-04-09 15:00:21 +02:00
Igor Anić
b3339f3cd5 tar: store diagnostic errors into UnpackResult
Test that UnpackResult prints same error output as diagnostic.
2024-04-09 15:00:21 +02:00
Igor Anić
9e47857ae9 fetch: make failing test 2024-04-09 15:00:21 +02:00
Ian Johnson
129de47a71 Fix first-time package fetches
Closes #19557
Closes #19561

Previously, `build.zig` was not being detected correctly by
`computeHash` for packages where there is a containing root directory.
2024-04-07 09:07:37 +02:00
Igor Anić
34bb670bb6 package manager: set executable bit
Based on file content. Detects elf magic header or shebang line.

Executable bit is ignored in hash calculation, as it was before this. So
packages hashes are not changed.

Reference:
https://github.com/ziglang/zig/issues/17463#issuecomment-1984798880

Fixes: 17463

Test is here:
7c4600d7bb/src/main.zig (L307)
(if #19500 got accepted I'll move this test to the Fetch.zig)
2024-04-06 13:00:57 -07:00
Igor Anić
a60b7af2c1 fetch: fix manifest included paths filtering
Filter should be applied on path where package root folder (if
there is any) is stripped. Manifest is inside package root and has paths
relative to package root not temporary directory root.
2024-04-04 01:59:15 +02:00
Igor Anić
8545cb0147 fetch: use package root detection pipeToFileSystem
Based on:
https://github.com/ziglang/zig/pull/19111#discussion_r1548620985

In pipeToFileSystem we are already iterating over all files in tarball.
Inspecting them there for existence of single root folder saves two
syscalls later.
2024-04-03 21:20:39 +02:00
Igor Anić
363acf4991 fetch: update comments 2024-04-03 21:17:57 +02:00
Igor Anić
1431e34cb9 fetch: remove one openDir in runResource
Based on comment:
https://github.com/ziglang/zig/pull/19111#discussion_r1548640939

computeHash finds all files in temporary directory. There is no
difference on what path are they. When calculating hash normalized_path
must be set relative to package root. That's the place where we strip
root if needed.
2024-04-03 21:01:29 +02:00
Igor Anić
24304a4385 fetch: save syscall, and add comment
From review comments: https://github.com/ziglang/zig/pull/19111
2024-04-03 17:08:41 +02:00
Igor Anić
c4261bf562 fetch: find package root only for archives 2024-04-03 17:06:20 +02:00
Igor Anić
15ca0c9471 fix typo 2024-04-03 17:06:20 +02:00
Igor Anić
a5a928b966 package manager: don't strip components in tar
Unpack tar without removing leading root folder. Then find package root
in unpacked tmp folder.
2024-04-03 17:06:20 +02:00
Igor Anić
bc5076715b fetch: fix failing test
Prior to
[this](ef9966c985 (diff-08c935ef8c633bb630641d44230597f1cff5afb0e736d451e2ba5569fa53d915R805))
commit tar was not a valid extension. After that this one is valid case.
2024-04-03 17:06:20 +02:00
Andrew Kelley
7bc0b74b6d move Package.Path to std.Build.Cache.Path 2024-03-21 16:16:47 -07:00
Andrew Kelley
cd62005f19 extract std.posix from std.os
closes #5019
2024-03-19 11:45:09 -07:00
Christofer Nolander
57cfe0778c zig fetch: resolve branch/tag names to commit SHA 2024-03-18 19:48:11 +01:00
Andrew Kelley
f60c24c73c
Merge pull request #19155 from ianic/tar_max_file_size
std.tar: error on insufficient buffers provided to iterator
2024-03-11 17:03:44 -07:00
Igor Anić
f8f43ca3e1 package: remove git fetch zlib lookahead fix
My first zlib implementation broke git fetch because it introduce
[lookahead](https://github.com/ziglang/zig/issues/18967). That resulted
in workarounds [1](80f3ef6e14)
[2](d00faa2407)

After [fixing](https://github.com/ziglang/zig/pull/19163) lookahead in
zlib decompressor this fixes are no longer necessary.
2024-03-11 16:49:21 -07:00
Tristan Ross
9d70d614ae
std.builtin: make link mode fields lowercase 2024-03-11 07:09:10 -07:00
Igor Anić
0cca7e732e std.tar: fix broken public interface 2024-03-11 12:25:51 +01:00
mlugg
38331b1cab
Package.Module: actually set File.path_digest for builtin modules 2024-03-07 06:26:25 +00:00
mlugg
b41a0b4768
Package.Module: deduplicate identical builtin modules
Previously, when multiple modules had builtin modules with identical
sources, two distinct `Module`s and `File`s were created pointing at the
same file path. This led to a bug later in the frontend. These modules
are now deduplicated with a simple hashmap on the builtin source.
2024-03-06 21:26:39 +00:00
Andrew Kelley
d661f0f35b compiler: JIT zig fmt
See #19063
2024-02-26 22:26:19 -07:00
Andrew Kelley
08e886b8fe package management: fix regression of printing expected hash
Regressed in ed4ccea7ba. The early exit
path was only supposed to happen in case of --system mode.
2024-02-25 09:27:39 -08:00
Andrew Kelley
c44a902836 fix zstd compilation errors from previous commit 2024-02-23 02:37:11 -07:00