Commit graph

11987 commits

Author SHA1 Message Date
regnat
5753f6efbb Fix the rendering of the example directory tree 2022-01-18 10:55:00 +01:00
Eelco Dolstra
52ee7ec002 StringSource: Use std::string_view 2022-01-17 22:20:23 +01:00
Eelco Dolstra
776eb97a43 serialise.hh: Use std::string_view 2022-01-17 22:20:23 +01:00
Eelco Dolstra
fc2443a67c
Merge pull request #5812 from pennae/small-perf-improvements
improve parser performance a bit
2022-01-17 19:49:52 +01:00
Alexander Bantyev
34b66aab00
Update documentation for paths on command line
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2022-01-17 20:00:04 +03:00
Alexander Bantyev
8cf54f754d
Show build and substitution information when not connected to a TTY
When stderr is not connected to a tty, show "building" and
"substituting" messages, a-la nix-build et al.

Closes https://github.com/NixOS/nix/issues/4402

Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2022-01-17 19:57:29 +03:00
Eelco Dolstra
008ddef4b0
Merge pull request #5915 from NixOS/disallow-boost-reference
Forbid runtime references to boost
2022-01-17 10:14:45 +01:00
Eelco Dolstra
bb24d1edd7
Merge pull request #5914 from NixOS/5911-fix-nix-why-depends-pager
Start the pager early-enough in `nix why-depends`
2022-01-17 10:10:31 +01:00
Rok Garbas
e1720b8e9d
Merge pull request #5919 from johnae/var-tmp-in-docker-image
docker: also create var/tmp as some tools rely on it
2022-01-16 21:01:47 +01:00
Farid Zakaria
61f02f7f20 Make queryJSON not bail immediately on an assertion or error 2022-01-15 19:36:07 -08:00
Farid Zakaria
6ff2ce8caf Added result and .vscode to gitignore 2022-01-15 19:17:40 -08:00
John Axel Eriksson
84507daaaa docker: var/tmp make add -p option to mkdir to also create parent dirs 2022-01-15 14:11:37 +01:00
regnat
dd3aa1e515 Remove the references to boost on darwin 2022-01-15 11:34:15 +01:00
John Axel Eriksson
3fff0196cd docker: also create var/tmp as some tools rely on it 2022-01-15 10:20:18 +01:00
Rok Garbas
34f2eebd5b
Merge pull request #5916 from NixOS/5896-git-in-docker-image
Add git to the docker image
2022-01-14 21:28:39 +01:00
Alexander Bantyev
b9f5dccdbe
Check that we don't search past a git repo 2022-01-14 18:03:47 +03:00
Alexander Bantyev
f055cc5a0b
Document searching upwards and fix documentation for installables 2022-01-14 17:58:03 +03:00
regnat
1dace02866 Add git to the docker image
Fix #5896

See https://github.com/NixOS/docker/issues/33
2022-01-14 15:55:36 +01:00
regnat
de4489a672 Forbid runtime references to boost
We explicitly hack around to remove them, so might as well check that
the hack is useful.

(Introduced because I feared that the changes of
https://github.com/NixOS/nix/pull/5906#discussion_r784810238 would bring
back some runtime references)
2022-01-14 15:41:14 +01:00
Alexander Bantyev
e3690ab393
Add more tests for flake upward searching 2022-01-14 17:21:18 +03:00
tomberek
2dead20924
Apply suggestions from code review
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2022-01-14 09:16:34 -05:00
pennae
ad60dfde2a also cache split regexes, not just match regexes
gives about 1% improvement on system eval, a bit less on nix search.

 # before

  nix search --no-eval-cache --offline ../nixpkgs hello
    Time (mean ± σ):      7.419 s ±  0.045 s    [User: 6.362 s, System: 0.794 s]
    Range (min … max):    7.335 s …  7.517 s    20 runs

  nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
    Time (mean ± σ):      2.921 s ±  0.023 s    [User: 2.626 s, System: 0.210 s]
    Range (min … max):    2.883 s …  2.957 s    20 runs

 # after

  nix search --no-eval-cache --offline ../nixpkgs hello
    Time (mean ± σ):      7.370 s ±  0.059 s    [User: 6.333 s, System: 0.791 s]
    Range (min … max):    7.286 s …  7.541 s    20 runs

  nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
    Time (mean ± σ):      2.891 s ±  0.033 s    [User: 2.606 s, System: 0.210 s]
    Range (min … max):    2.823 s …  2.958 s    20 runs
2022-01-14 14:04:17 +01:00
pennae
c9fc975259 optimize removeAttrs builtin
use a sorted array of symbols to be removed instead of a set. this saves a lot
of memory allocations and slightly speeds up removal.
2022-01-14 14:01:52 +01:00
regnat
81cd0a113b Start the pager early-enough in nix why-depends
`nix why-depends` is piping its output into a pager by default.
However the pager was only started after the first path is printed,
causing it to be excluded from the pager output.

(Actually the pager was started *inside* the recursive function that was
printing the dependency chain, so a new instance was started at each
level. It’s a little miracle that it worked at all).

Fix #5911
2022-01-14 13:41:45 +01:00
pennae
72f42093e7 optimize unescapeStr
mainly to avoid an allocation and a copy of a string that can be
modified in place (ever since EvalState holds on to the buffer, not the
generated parser itself).

 # before

Benchmark 1: nix search --offline nixpkgs hello
  Time (mean ± σ):     571.7 ms ±   2.4 ms    [User: 563.3 ms, System: 8.0 ms]
  Range (min … max):   566.7 ms … 579.7 ms    50 runs

Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
  Time (mean ± σ):     376.6 ms ±   1.0 ms    [User: 345.8 ms, System: 30.5 ms]
  Range (min … max):   374.5 ms … 379.1 ms    50 runs

Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      2.922 s ±  0.006 s    [User: 2.707 s, System: 0.215 s]
  Range (min … max):    2.906 s …  2.934 s    50 runs

 # after

Benchmark 1: nix search --offline nixpkgs hello
  Time (mean ± σ):     570.4 ms ±   2.8 ms    [User: 561.3 ms, System: 8.6 ms]
  Range (min … max):   564.6 ms … 578.1 ms    50 runs

Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
  Time (mean ± σ):     375.4 ms ±   1.3 ms    [User: 343.2 ms, System: 31.7 ms]
  Range (min … max):   373.4 ms … 378.2 ms    50 runs

Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      2.925 s ±  0.006 s    [User: 2.704 s, System: 0.219 s]
  Range (min … max):    2.910 s …  2.942 s    50 runs
2022-01-13 18:06:15 +01:00
pennae
34e3bd10e3 avoid copies of parser input data
when given a string yacc will copy the entire input to a newly allocated
location so that it can add a second terminating NUL byte. since the
parser is a very internal thing to EvalState we can ensure that having
two terminating NUL bytes is always possible without copying, and have
the parser itself merely check that the expected NULs are present.

 # before

Benchmark 1: nix search --offline nixpkgs hello
  Time (mean ± σ):     572.4 ms ±   2.3 ms    [User: 563.4 ms, System: 8.6 ms]
  Range (min … max):   566.9 ms … 579.1 ms    50 runs

Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
  Time (mean ± σ):     381.7 ms ±   1.0 ms    [User: 348.3 ms, System: 33.1 ms]
  Range (min … max):   380.2 ms … 387.7 ms    50 runs

Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      2.936 s ±  0.005 s    [User: 2.715 s, System: 0.221 s]
  Range (min … max):    2.923 s …  2.946 s    50 runs

 # after

Benchmark 1: nix search --offline nixpkgs hello
  Time (mean ± σ):     571.7 ms ±   2.4 ms    [User: 563.3 ms, System: 8.0 ms]
  Range (min … max):   566.7 ms … 579.7 ms    50 runs

Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
  Time (mean ± σ):     376.6 ms ±   1.0 ms    [User: 345.8 ms, System: 30.5 ms]
  Range (min … max):   374.5 ms … 379.1 ms    50 runs

Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      2.922 s ±  0.006 s    [User: 2.707 s, System: 0.215 s]
  Range (min … max):    2.906 s …  2.934 s    50 runs
2022-01-13 18:06:15 +01:00
pennae
eee0bcee22 avoid allocations in SymbolTable::create
speeds up parsing by ~3%, system builds by a bit more than 1%

 # before

Benchmark 1: nix search --offline nixpkgs hello
  Time (mean ± σ):     574.7 ms ±   2.8 ms    [User: 566.3 ms, System: 8.0 ms]
  Range (min … max):   569.2 ms … 580.7 ms    50 runs

Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
  Time (mean ± σ):     394.4 ms ±   0.8 ms    [User: 361.8 ms, System: 32.3 ms]
  Range (min … max):   392.7 ms … 395.7 ms    50 runs

Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      2.976 s ±  0.005 s    [User: 2.757 s, System: 0.218 s]
  Range (min … max):    2.966 s …  2.990 s    50 runs

 # after

Benchmark 1: nix search --offline nixpkgs hello
  Time (mean ± σ):     572.4 ms ±   2.3 ms    [User: 563.4 ms, System: 8.6 ms]
  Range (min … max):   566.9 ms … 579.1 ms    50 runs

Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
  Time (mean ± σ):     381.7 ms ±   1.0 ms    [User: 348.3 ms, System: 33.1 ms]
  Range (min … max):   380.2 ms … 387.7 ms    50 runs

Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      2.936 s ±  0.005 s    [User: 2.715 s, System: 0.221 s]
  Range (min … max):    2.923 s …  2.946 s    50 runs
2022-01-13 18:06:15 +01:00
pennae
61a9d16d5c don't strdup tokens in the lexer
every stringy token the lexer returns is turned into a Symbol and not
used further, so we don't have to strdup. using a string_view is
sufficient, but due to limitations of the current parser we have to use
a POD type that holds the same information.

gives ~2% on system build, 6% on search, 8% on parsing alone

 # before

Benchmark 1: nix search --offline nixpkgs hello
  Time (mean ± σ):     610.6 ms ±   2.4 ms    [User: 602.5 ms, System: 7.8 ms]
  Range (min … max):   606.6 ms … 617.3 ms    50 runs

Benchmark 2: nix eval -f hackage-packages.nix
  Time (mean ± σ):     430.1 ms ±   1.4 ms    [User: 393.1 ms, System: 36.7 ms]
  Range (min … max):   428.2 ms … 434.2 ms    50 runs

Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      3.032 s ±  0.005 s    [User: 2.808 s, System: 0.223 s]
  Range (min … max):    3.023 s …  3.041 s    50 runs

 # after

Benchmark 1: nix search --offline nixpkgs hello
  Time (mean ± σ):     574.7 ms ±   2.8 ms    [User: 566.3 ms, System: 8.0 ms]
  Range (min … max):   569.2 ms … 580.7 ms    50 runs

Benchmark 2: nix eval -f hackage-packages.nix
  Time (mean ± σ):     394.4 ms ±   0.8 ms    [User: 361.8 ms, System: 32.3 ms]
  Range (min … max):   392.7 ms … 395.7 ms    50 runs

Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
  Time (mean ± σ):      2.976 s ±  0.005 s    [User: 2.757 s, System: 0.218 s]
  Range (min … max):    2.966 s …  2.990 s    50 runs
2022-01-13 18:06:14 +01:00
Eelco Dolstra
5e9653c370 Tweak 2022-01-13 14:33:41 +01:00
Eelco Dolstra
66424cd29f Merge branch 'patch-1' of https://github.com/TheodorRene/nix 2022-01-13 14:31:49 +01:00
pennae
6401e443a4 move strings in derivationStrict
the temporary will be discarded anyway, so we can move out of it and save many
small allocations and copies.
2022-01-13 14:00:20 +01:00
pennae
ef45787aae avoid string copies in attrNames sort comparison
symbols can also be cast to string_view, which compares the same but doesn't
require a copy of both symbol names on every comparison.
2022-01-13 14:00:19 +01:00
pennae
1bebb1095a cache more often-used symbols for primops
there's a few symbols in primops we can create once and pick them out of
EvalState afterwards instead of creating them every time we need them. this
gives almost 1% speedup to an uncached nix search.
2022-01-13 13:58:33 +01:00
pennae
44c92a1667 use more string_view in utils
there's a couple places that can be easily converted from using strings to using
string_views instead. gives a slight (~1%) boost to system eval.

 # before

  nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
    Time (mean ± σ):      2.946 s ±  0.026 s    [User: 2.655 s, System: 0.209 s]
    Range (min … max):    2.905 s …  2.995 s    20 runs

 # after

    Time (mean ± σ):      2.928 s ±  0.024 s    [User: 2.638 s, System: 0.211 s]
    Range (min … max):    2.893 s …  2.970 s    20 runs
2022-01-13 13:51:29 +01:00
Eelco Dolstra
a2811f8499
Merge pull request #5903 from Profpatsch/release-notes-2.4-add-indirect-noop
Release Notes 2.4: add `--indirect` no-op change
2022-01-13 11:18:41 +01:00
Profpatsch
a51c457204 Release Notes 2.4: add --indirect no-op change
Since
00d25e8457
which was first included in nix 2.4.

It is a backwards-compatible change since the flag will just be
ignored.
2022-01-12 23:19:15 +01:00
Théophane Hufschmitt
e61c4bc25a
Merge pull request #5887 from pennae/avoid-streams
avoid std::?stream overhead when it's not helpful
2022-01-12 10:52:40 +01:00
pennae
73fcc40fa4 use boost::lexical_cast for string2*
this avoids one copy from `s` into `str`, and possibly another copy needed to
construct `s` at the call site. lexical_cast is also more efficient in general.
2022-01-12 10:07:21 +01:00
pennae
5838354d34 optimize ExprConcatStrings::eval
constructing an ostringstream for non-string concats (like integer addition) is
a small constant cost that we can avoid. for string concats we can keep all the
string temporaries we get from coerceToString and concatenate them in one go,
which saves a lot of intermediate temporaries and copies in ostringstream. we
can also avoid copying the concatenated string again by directly allocating it
in GC memory and moving ownership of the concatenated string into the target
value.

saves about 2% on system eval.

before:

  Benchmark 1: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
    Time (mean ± σ):      2.837 s ±  0.031 s    [User: 2.562 s, System: 0.191 s]
    Range (min … max):    2.796 s …  2.892 s    20 runs

after:

  Benchmark 1: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
    Time (mean ± σ):      2.790 s ±  0.035 s    [User: 2.532 s, System: 0.187 s]
    Range (min … max):    2.722 s …  2.836 s    20 runs
2022-01-12 10:07:21 +01:00
Rok Garbas
2267c773f0
Push docker image (only x86_64-linux right now) to hub.docker.com 2022-01-11 18:51:58 +01:00
Rok Garbas
9aa486c4be
Rename github workflow from Tests to CI 2022-01-11 18:51:34 +01:00
Theodor René Carlsen
89b4df8d92
Add link to explanation when introducing a new operator
The logical implication operator is included in this section but never explained. It might stump new readers with a pretty uncommon operator, and it's never referenced explicitly.
2022-01-11 17:01:43 +01:00
Théophane Hufschmitt
d023903b6f
Merge pull request #5883 from fzakaria/faridzakaria/check-meta
Check for `--meta` before emitting it for `--json`
2022-01-11 16:57:28 +01:00
Eelco Dolstra
8a446aff75
Merge pull request #5898 from layus/repair-path-links
Make --repair-path also repair corrupt optimised links
2022-01-11 14:14:44 +01:00
Guillaume Maudoux
9f9f39a24b Prefer RepairFlag over bool when applicable 2022-01-11 13:38:28 +01:00
Guillaume Maudoux
e9a4abdb5d Make --repair-path also repair corrupt optimised links
There already existed a smoke test for the link content length,
but it appears that there exists some corruptions pernicious enough
to replace the file content with zeros, and keeping the same length.

--repair-path now goes as far as checking the content of the link,
making it true to its name and actually repairing the path for such
coruption cases.
2022-01-11 11:57:45 +01:00
Eelco Dolstra
f39d94a55b Merge branch 'readfile' of https://github.com/abbradar/nix 2022-01-10 16:55:28 +01:00
Eelco Dolstra
19fd6e585d 'target' points to 'source' 2022-01-10 16:52:25 +01:00
Eelco Dolstra
df11e75d0e Merge branch 'recursive-nix-fix' of https://github.com/L-as/nix 2022-01-10 16:51:59 +01:00
Nikolay Amiantov
120ca245d1 Add test for readFile keeping context 2022-01-09 13:42:36 +03:00