Commit graph

15919 commits

Author SHA1 Message Date
pennae
2e0321912a use aligned flex tables
~2% speedup on parsing without eval, less (but still significant) on
system eval. having flex generate faster parsers leads to very strange
misparses. maybe re2c is worth investigating.

before:

  Time (mean ± σ):      4.260 s ±  0.003 s    [User: 3.754 s, System: 0.505 s]
  Range (min … max):    4.257 s …  4.266 s    10 runs

after:

  Time (mean ± σ):      4.231 s ±  0.004 s    [User: 3.725 s, System: 0.504 s]
  Range (min … max):    4.226 s …  4.240 s    10 runs
2023-12-19 19:32:16 +01:00
pennae
cc4038d541 use std::tie() for macro-generated operators
as written the comparisons generate copies, even though it looks as
though they shouldn't.

before:

  Time (mean ± σ):      4.396 s ±  0.002 s    [User: 3.894 s, System: 0.501 s]
  Range (min … max):    4.393 s …  4.399 s    10 runs

after:

  Time (mean ± σ):      4.260 s ±  0.003 s    [User: 3.754 s, System: 0.505 s]
  Range (min … max):    4.257 s …  4.266 s    10 runs
2023-12-19 19:32:16 +01:00
pennae
74c134914c compare string values with strcmp
string_view()ification calls strlen() first, which we don't need here.
2023-12-19 19:32:16 +01:00
pennae
78353deb02 encode black holes as tApp values
checking for isBlackhole in the forceValue hot path is rather more
expensive than necessary, and with a little bit of trickery we can move
such handling into the isApp case. small performance benefit, but under
some circumstances we've seen 2% improvement as well.

〉 nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'

before:

  Time (mean ± σ):      4.429 s ±  0.002 s    [User: 3.929 s, System: 0.500 s]
  Range (min … max):    4.427 s …  4.433 s    10 runs

after:

  Time (mean ± σ):      4.396 s ±  0.002 s    [User: 3.894 s, System: 0.501 s]
  Range (min … max):    4.393 s …  4.399 s    10 runs
2023-12-19 19:32:16 +01:00
pennae
0218e4e6c3 memset less in addToStoreFromDump
resizing a std::string clears the newly added bytes, which is not
necessary here and comes with a ~1.4% slowdown on our test nixos config.

〉 nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'

before:

  Time (mean ± σ):      4.486 s ±  0.003 s    [User: 3.978 s, System: 0.507 s]
  Range (min … max):    4.482 s …  4.492 s    10 runs

after:

  Time (mean ± σ):      4.429 s ±  0.002 s    [User: 3.929 s, System: 0.500 s]
  Range (min … max):    4.427 s …  4.433 s    10 runs
2023-12-19 19:32:16 +01:00
Eelco Dolstra
7cfd6c0efe
Merge pull request #9325 from NixOS/accessor-add-to-store
Content addressing and adding to store cleanup
2023-12-19 15:10:31 +01:00
Théophane Hufschmitt
83a32969ad
Merge pull request #9633 from cole-h/fix-sysconfdir
package: don't set sysconfdir in devShells
2023-12-19 10:14:48 +01:00
Théophane Hufschmitt
94a03a810e
Merge pull request #9637 from 9999years/fix-logging-test-on-macos
Fix `logging.sh` test on macOS
2023-12-19 10:13:08 +01:00
Théophane Hufschmitt
b077d545ad
Merge pull request #9638 from NixOS/dependabot/github_actions/zeebe-io/backport-action-2.3.0
build(deps): bump zeebe-io/backport-action from 2.2.0 to 2.3.0
2023-12-19 10:11:37 +01:00
Théophane Hufschmitt
e85fd92816
Merge pull request #9639 from awakesecurity/mz/darwin-UID-override
installer: allow overriding of NIX_FIRST_BUILD_ID on darwin
2023-12-19 10:11:24 +01:00
Théophane Hufschmitt
c6d7013583
Merge pull request #9641 from amarshall/darwin-sandbox-stderr
Allow access to /dev/stderr in Darwin sandbox
2023-12-19 10:09:38 +01:00
Andrew Marshall
7526b7ded6 Allow access to /dev/stderr in Darwin sandbox
We allow /dev/stdout, so why not this? Since it is process-local,
anyway, should not be possible to escape sandbox using it.
2023-12-18 19:33:20 -05:00
Mel Zuser
fa4bbe53e8 installer: allow overriding of NIX_FIRST_BUILD_ID on darwin
because there are often already users in the 300 range and it's painful
to work around.

revives #6466
2023-12-18 15:02:26 -08:00
dependabot[bot]
23fb19cb18
build(deps): bump zeebe-io/backport-action from 2.2.0 to 2.3.0
Bumps [zeebe-io/backport-action](https://github.com/zeebe-io/backport-action) from 2.2.0 to 2.3.0.
- [Release notes](https://github.com/zeebe-io/backport-action/releases)
- [Commits](https://github.com/zeebe-io/backport-action/compare/v2.2.0...v2.3.0)

---
updated-dependencies:
- dependency-name: zeebe-io/backport-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-12-18 22:36:31 +00:00
Rebecca Turner
0cee56db1a
Fix logging.sh test on macOS
On macOS in the `nix develop` shell, `make
tests/functional/logging.sh.test` errors:

    ++(logging.sh:18) mktemp
    +(logging.sh:18) builder=/var/folders/z5/fclwwdms3r1gq4k4p3pkvvc00000gn/T/tmp.StuabKUhMh
    +(logging.sh:19) echo -e '#!/bin/sh\nmkdir $out'
    +++(logging.sh:22) mktemp -d
    ++(logging.sh:22) nix-build -E 'with import ./config.nix; mkDerivation { name = "fnord"; builder = /var/folders/z5/fclwwdms3r1gq4k4p3pkvvc00000gn/T/tmp.StuabKUhMh; }' --out-link /var/folders/z5/fclwwdms3r1gq4k4p3pkvvc00000gn/T/tmp.oaKcy0NXqC/result
    error:
           … while calling the 'derivationStrict' builtin
             at <nix/derivation-internal.nix>:9:12:
                8|
                9|   strict = derivationStrict drvAttrs;
                 |            ^
               10|

           … while evaluating derivation 'fnord'
             whose name attribute is located at «string»:1:42

           … while evaluating attribute 'args' of derivation 'fnord'
             at /Users/wiggles/nix/tests/functional/config.nix:23:7:
               22|       builder = shell;
               23|       args = ["-e" args.builder or (builtins.toFile "builder-${args.name}.sh" ''
                 |       ^
               24|         if [ -e "$NIX_ATTRS_SH_FILE" ]; then source $NIX_ATTRS_SH_FILE; fi;

           error: path '/var' is a symlink
    +(logging.sh:22) outp=
    ++(logging.sh:22) onError
    ++(/Users/wiggles/nix/tests/functional/common/vars-and-functions.sh:237) set +x
    logging.sh: test failed at:
      main in logging.sh:22

This is because `mktemp` returns a path like
`/var/folders/z5/fclwwdms3r1gq4k4p3pkvvc00000gn/T/tmp.qDY24l6bIM`,
where `/var` is a symlink to `/private/var`.

Then, we attempt to use that path as a `builder`, which errors because
symlinks are impure or whatever.

Anyways, we can fix this by using `realpath "$(mktemp)"` instead of
`mktemp` directly.

NB: This error doesn't seem to happen when I run the tests through `nix
flake check`. I'm not sure if Nix does something to `TMP` in that case.
2023-12-18 14:04:25 -08:00
Rebecca Turner
6f4930382b
Document more Makefile variables 2023-12-18 13:34:08 -08:00
Cole Helbling
ba0087316a package: don't set sysconfdir in devShells 2023-12-18 12:59:58 -08:00
Cole Helbling
123ef6a996 Fix warnings when running checks
`nix flake check` had these warnings:

    trace: warning: Module argument `nodes.client.config` is deprecated. Use `nodes.client` instead.
    trace: warning: Module argument `nodes.client.config` is deprecated. Use `nodes.client` instead.
    trace: warning: The option `services.openssh.permitRootLogin' defined in `/nix/store/3m3hfpmbjdf4w39qfjami7ljhvhczay1-source/tests/nixos/nix-copy.nix' has been renamed to `services.openssh.settings.PermitRootLogin'.
    trace: warning: Module argument `nodes.http_dns.config` is deprecated. Use `nodes.http_dns` instead.
    trace: warning: Module argument `nodes.github.config` is deprecated. Use `nodes.github` instead.
    trace: warning: Module argument `nodes.sourcehut.config` is deprecated. Use `nodes.sourcehut` instead.
2023-12-18 10:20:52 -08:00
Robert Hensing
5d5b25f2e3
Merge pull request #9611 from obsidiansystems/fix-cross-configure
Split `--disable-tests`, fix cross builds
2023-12-18 17:25:19 +01:00
John Ericson
7feabf7d44 Split --disable-tests, fix cross builds
It might seem obnoxious to have yet more configure flags, but I found
controlling both the unit and functional tests with one flag was quite
confusing because they are so different:

- unit tests depending on building, functional tests don't (e.g. when
  we test already-built Nix)

- unit tests can be installed, functional tests cannot

- unit tests neeed extra libraries (GTest, RapidCheck), functional
  tests need extra executables (jq).

- unit tests are run by `make check`, functional tests are run by `make
  installcheck`

Really on a technical level, they seem wholly independent. Only on a
human level ("they are both are tests") do they have anything in common.

I had messed up the logic in cross builds because of this. Now I
split the flag in two (and cleaned up a few other inconsistencies), and
the logic fixed itself.

Co-Authored-By: Robert Hensing <roberth@users.noreply.github.com>
2023-12-18 10:47:50 -05:00
tomberek
b1c559eabc
docs: add link to project board to PRs (#9630)
* docs: add link to project board to PRs

* Update .github/PULL_REQUEST_TEMPLATE.md

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>

* fix wording

* add note on the process

---------

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-12-18 15:45:57 +00:00
John Ericson
ed26b186fb Remove now-redundant text-hashing store methods
`addTextToStore` and `computeStorePathFromDump` are now redundant.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-12-18 10:44:10 -05:00
John Ericson
dfc876531f Organize content addressing, use SourceAccessor with Store::addToStore
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-12-18 10:41:54 -05:00
Valentin Gagarin
d19a667528
CODEOWNERS: unsubscribe fricklerhandwerk (#9614) 2023-12-18 13:54:40 +01:00
Théophane Hufschmitt
ebfbc5a6a5
Merge pull request #9628 from fricklerhandwerk/add-links
add cross-reference
2023-12-18 13:07:39 +01:00
Robert Hensing
cea8354436
Merge pull request #9619 from 9999years/remove-blank-lines-in-errors
Remove some blank lines from stack traces
2023-12-18 12:50:50 +01:00
Valentin Gagarin
4f95800854 add cross-reference 2023-12-18 11:41:52 +01:00
David Arnold
c05d4fadd5
fix: valid branch name 2023-12-16 23:14:33 +01:00
Rebecca Turner
7f5ed330e4
Document Makefile variables in hacking.md (#9620) 2023-12-16 12:05:31 +00:00
Rebecca Turner
5cb98095ba
Remove some blank lines from stack traces
This keeps hint messages, source location information, and source code
snippets grouped together, while making stack traces shorter (so that
more stack frames can be viewed on the same terminal).

Before:

    error:
           … while evaluating the attribute 'body'

             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:4:3:

                3|
                4|   body = x "x";
                 |   ^
                5| }

           … from call site

             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:4:10:

                3|
                4|   body = x "x";
                 |          ^
                5| }

           … while calling 'x'

             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:2:7:

                1| let {
                2|   x = arg: assert arg == "y"; 123;
                 |       ^
                3|

           error: assertion '(arg == "y")' failed

           at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:2:12:

                1| let {
                2|   x = arg: assert arg == "y"; 123;
                 |            ^
                3|

After:

    error:
           … while evaluating the attribute 'body'
             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:4:3:
                3|
                4|   body = x "x";
                 |   ^
                5| }

           … from call site
             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:4:10:
                3|
                4|   body = x "x";
                 |          ^
                5| }

           … while calling 'x'
             at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:2:7:
                1| let {
                2|   x = arg: assert arg == "y"; 123;
                 |       ^
                3|

           error: assertion '(arg == "y")' failed
           at /Users/wiggles/nix/tests/functional/lang/eval-fail-assert.nix:2:12:
                1| let {
                2|   x = arg: assert arg == "y"; 123;
                 |            ^
                3|
2023-12-15 23:57:26 -08:00
John Ericson
419a64f9e0
Merge pull request #9613 from jlesquembre/jl/clang-stdenv
Fix clang devshell
2023-12-15 09:08:41 -05:00
José Luis Lafuente
66d37b7338
Move clang dev deps to the nix devshell override 2023-12-15 12:41:38 +01:00
John Ericson
071dbbee33
Merge pull request #4093 from matthewbauer/eval-system
Add eval-system option
2023-12-14 20:41:29 -05:00
John Ericson
228e995cde Add release not for eval-system 2023-12-14 19:55:50 -05:00
John Ericson
70f50cbb2a Functional Test for builtins.storeDir and builtins.currentSystem
These were under-tested. This tests the status quo and especially
previous commit of this PR better.
2023-12-14 19:47:10 -05:00
Matthew Bauer
bcbdb09ccf Add eval-system option
`eval-system` option overrides just the value of `builtins.currentSystem`.
This is more useful than overriding `system` since you can build these
derivations on remote builders which can work on the given system.

Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-12-14 19:04:00 -05:00
José Luis Lafuente
8d39c0c196
Fix clang devshell
Issue introduced in https://github.com/NixOS/nix/pull/9535
2023-12-14 23:14:59 +01:00
Eelco Dolstra
fc0accf3dc
Merge pull request #9567 from obsidiansystems/fix-sys-xattr-h
Fix `sys/xattr.h` check
2023-12-14 18:24:24 +01:00
John Ericson
e13fc0bbdb Fix sys/xattr.h check
I wrote the `configure.ac` wrong, and so we just got no builds
supporting ACLs.

Also, it needs to be more precise because Darwin puts other stuff in
that same header, evidently.
2023-12-14 10:03:48 -05:00
Eelco Dolstra
790cf13c26
Merge pull request #9608 from NixOS/default-lowdown
Remove custom lowdown
2023-12-14 13:18:32 +01:00
John Ericson
e8ebcfd665
Merge pull request #9607 from 9999years/macos-boost
Use `--with-boost` on macOS to fix local builds
2023-12-13 23:15:44 -05:00
Rebecca Turner
6ed803737c
Use --with-boost on macOS
`configureFlags` only included `--with-boost` on Linux, which makes
local builds as outlined in `doc/manual/src/contributing/hacking.md`
fail when performed on macOS.
2023-12-13 14:02:52 -08:00
Ramses
1e3d811840
worker protocol: serialise cgroup stats in BuildResult (#9598)
By doing so, they get reported when building through the daemon via either `unix://` or `ssh-ng://`.
2023-12-13 16:37:17 -05:00
John Ericson
7b29b44d8e Remove custom lowdown
This was last upgraded in 788008385e, but
the version in Nixpkgs is a now a lot newer. I think the custom was
added to get ahead of Nixpkgs before, and so now that we are in fact
behind, it is no longer needed.
2023-12-13 16:25:18 -05:00
John Ericson
e6515bd47b
Merge pull request #9535 from tfc/package-nix
Simplify flake.nix by creating generic callPackage-able package.nix
2023-12-13 16:22:03 -05:00
John Ericson
2d24875fe4 package.nix: Avoid ${..} for conditional strings
Using `+` is Nixpkgs standard ideom for this, and helps avoid needless
rebuilds somewhat.
2023-12-13 20:49:31 +00:00
John Ericson
ed93e51f4b
Merge pull request #9604 from obsidiansystems/test-substitute-on-destination
Test `nix copy --substitute-on-destination`
2023-12-13 15:47:56 -05:00
John Ericson
28f2f3136d Delete stray install_name_tool call 2023-12-13 20:47:36 +00:00
John Ericson
bf5804d46a flake.nix: Delete uneeded attrs0 binding 2023-12-13 20:41:41 +00:00
John Ericson
f10f0f1b50 Move lowdown.nix to misc/ 2023-12-13 20:41:32 +00:00