Commit graph

11333 commits

Author SHA1 Message Date
pennae
26a8b220eb avoid ostream sentries per json string character
we don't have to create an ostream sentry object for every character of a JSON
string we write. format a bunch of characters and flush them to the stream all
at once instead.

this doesn't affect small numbers of string characters, but larger numbers of
total JSON string characters written gain a lot. at 1MB of total string written
we gain almost 30%, at 16MB it's almost a factor of 3x. large numbers of JSON
string characters do occur naturally in a nixos system evaluation to generate
documentation (though this is now somewhat mitigated by caching the largest part
of nixos option docs).

benchmarked with

  hyperfine 'nix eval --raw --expr "let s = __concatStringsSep \"\" (__genList (_: \"c\") 256); in __toJSON (__genList (_: s) {e})"' --warmup 1 -L e 1,4,256,4096,65536

before:

  Benchmark 1: nix eval --raw --expr "let s = __concatStringsSep \"\" (__genList (_: \"c\") 256); in __toJSON (__genList (_: s) 1)"
    Time (mean ± σ):      12.5 ms ±   0.2 ms    [User: 9.2 ms, System: 4.0 ms]
    Range (min … max):    11.9 ms …  13.1 ms    223 runs

  Benchmark 2: nix eval --raw --expr "let s = __concatStringsSep \"\" (__genList (_: \"c\") 256); in __toJSON (__genList (_: s) 4)"
    Time (mean ± σ):      12.5 ms ±   0.2 ms    [User: 9.3 ms, System: 3.8 ms]
    Range (min … max):    11.9 ms …  13.2 ms    220 runs

  Benchmark 3: nix eval --raw --expr "let s = __concatStringsSep \"\" (__genList (_: \"c\") 256); in __toJSON (__genList (_: s) 256)"
    Time (mean ± σ):      13.2 ms ±   0.3 ms    [User: 9.8 ms, System: 4.0 ms]
    Range (min … max):    12.6 ms …  14.3 ms    205 runs

  Benchmark 4: nix eval --raw --expr "let s = __concatStringsSep \"\" (__genList (_: \"c\") 256); in __toJSON (__genList (_: s) 4096)"
    Time (mean ± σ):      24.0 ms ±   0.4 ms    [User: 19.4 ms, System: 5.2 ms]
    Range (min … max):    22.7 ms …  25.8 ms    119 runs

  Benchmark 5: nix eval --raw --expr "let s = __concatStringsSep \"\" (__genList (_: \"c\") 256); in __toJSON (__genList (_: s) 65536)"
    Time (mean ± σ):     196.0 ms ±   3.7 ms    [User: 171.2 ms, System: 25.8 ms]
    Range (min … max):   190.6 ms … 201.5 ms    14 runs

after:

  Benchmark 1: nix eval --raw --expr "let s = __concatStringsSep \"\" (__genList (_: \"c\") 256); in __toJSON (__genList (_: s) 1)"
    Time (mean ± σ):      12.4 ms ±   0.3 ms    [User: 9.1 ms, System: 4.0 ms]
    Range (min … max):    11.7 ms …  13.3 ms    204 runs

  Benchmark 2: nix eval --raw --expr "let s = __concatStringsSep \"\" (__genList (_: \"c\") 256); in __toJSON (__genList (_: s) 4)"
    Time (mean ± σ):      12.4 ms ±   0.2 ms    [User: 9.2 ms, System: 3.9 ms]
    Range (min … max):    11.8 ms …  13.0 ms    214 runs

  Benchmark 3: nix eval --raw --expr "let s = __concatStringsSep \"\" (__genList (_: \"c\") 256); in __toJSON (__genList (_: s) 256)"
    Time (mean ± σ):      12.6 ms ±   0.2 ms    [User: 9.5 ms, System: 3.8 ms]
    Range (min … max):    12.1 ms …  13.3 ms    209 runs

  Benchmark 4: nix eval --raw --expr "let s = __concatStringsSep \"\" (__genList (_: \"c\") 256); in __toJSON (__genList (_: s) 4096)"
    Time (mean ± σ):      15.9 ms ±   0.2 ms    [User: 11.4 ms, System: 5.1 ms]
    Range (min … max):    15.2 ms …  16.4 ms    171 runs

  Benchmark 5: nix eval --raw --expr "let s = __concatStringsSep \"\" (__genList (_: \"c\") 256); in __toJSON (__genList (_: s) 65536)"
    Time (mean ± σ):      69.0 ms ±   0.9 ms    [User: 44.3 ms, System: 25.3 ms]
    Range (min … max):    67.2 ms …  70.9 ms    42 runs
2022-01-07 06:53:47 +01:00
Farid Zakaria
e36fdbbfd9 Check for --meta before emitting it for --json
Check that the meta flag is present when emitting JSON query information
for `nix-env`.

fixes #5882
2022-01-06 14:40:02 -08:00
Eelco Dolstra
f71d84672b Merge branch 'install-nlohmann-json-headers' of https://github.com/hercules-ci/nix 2022-01-06 22:34:03 +01:00
Eelco Dolstra
5c7d3b351f
Merge pull request #5878 from fzakaria/faridzakaria/json-outputs
Add outputs to JSON query
2022-01-06 22:33:11 +01:00
Eelco Dolstra
3ec02c6743
Merge pull request #5546 from NixOS/allow-paths-in-flake-local-options
allow paths in flake local options
2022-01-06 21:55:20 +01:00
Farid Zakaria
65257614ea Add outputs to JSON query
Emit output information when printing JSON information
and `--out-paths` is requested.

fixes #5877
2022-01-06 09:13:18 -08:00
Domen Kožar
bdc577936f
Merge pull request #5749 from GavinRay97/patch-1
Better diagnostics if no valid signature found
2022-01-06 08:39:58 +01:00
Eelco Dolstra
6d7844695b
Merge pull request #5864 from edolstra/attr-sets-cleanup
Attrset improvements
2022-01-05 10:11:02 +01:00
Eelco Dolstra
2b4c944823 Remove EvalState::mkAttrs() 2022-01-04 20:29:17 +01:00
Eelco Dolstra
17daec0b83 Move empty attrset optimisation 2022-01-04 19:23:11 +01:00
Eelco Dolstra
ca5baf2392 Turn mkString(Symbol) into a method 2022-01-04 19:09:40 +01:00
Eelco Dolstra
ed93aec3c3 Remove non-method mkPath() 2022-01-04 18:45:16 +01:00
Eelco Dolstra
263a8d293c Remove non-method mk<X> functions 2022-01-04 18:40:39 +01:00
Eelco Dolstra
cc08364315 Remove non-method mkString() 2022-01-04 18:24:42 +01:00
Eelco Dolstra
6d9a6d2cc3 Ensure that attrsets are sorted
Previously you had to remember to call value->attrs->sort() after
populating value->attrs. Now there is a BindingsBuilder helper that
wraps Bindings and ensures that sort() is called before you can use
it.
2022-01-04 18:00:33 +01:00
Eelco Dolstra
8f214a3771
Merge pull request #5862 from SuperSandro2000/patch-1
Add iana-etc for /etc/protocols to default packages
2022-01-04 15:19:35 +01:00
Sandro
9dd45dfc3d
Add iana-etc for /etc/protocols to default packages 2022-01-04 12:49:02 +01:00
Eelco Dolstra
1ffacad8a5
Merge pull request #5830 from pennae/zipAttrsWith
add zipAttrsWith primop
2022-01-04 11:16:50 +01:00
pennae
00c993f48b add zipAttrsWith primop
nixpkgs can save a good bit of eval memory with this primop. zipAttrsWith is
used quite a bit around nixpkgs (eg in the form of recursiveUpdate), but the
most costly application for this primop is in the module system. it improves
the implementation of zipAttrsWith from nixpkgs by not checking an attribute
multiple times if it occurs more than once in the input list, allocates less
values and set elements, and just avoids many a temporary object in general.

nixpkgs has a more generic version of this operation, zipAttrsWithNames, but
this version is only used once so isn't suitable for being the base of a new
primop. if it were to be used more we should add a second primop instead.
2022-01-03 21:05:53 +01:00
Eelco Dolstra
96d08fcd66
Merge pull request #5839 from tweag/balsoft/yet-another-follows-bugfix
flake.cc: computeLocks: Only verify overrides when they could change
2022-01-03 20:51:58 +01:00
Eelco Dolstra
70dfcbbb37
Merge pull request #5840 from tweag/balsoft/nix-repl-show-trace
nix repl: fix --show-trace and add the ability to set trace display
2022-01-03 20:41:59 +01:00
Eelco Dolstra
d7b286fc77
Merge pull request #5844 from Kha/fix-ifd-chroot-once-more
Fix IFD with chroot store
2022-01-03 20:41:29 +01:00
Eelco Dolstra
1c099dee67
Merge pull request #5852 from autc04/docker-tmp-permissions
docker.nix: set correct permissions on /tmp (fix #5851)
2022-01-03 14:35:31 +01:00
Wolfgang Thaller
d9aabb7acb docker.nix: set correct permissions on /tmp 2022-01-01 16:47:05 +01:00
Sebastian Ullrich
d0c8e9254e Fix IFD with chroot store 2021-12-29 19:00:02 +01:00
Alexander Bantyev
a26351da02
Add ability to toggle show-trace from within the repl 2021-12-29 02:54:36 +03:00
Eelco Dolstra
ed3bc63666
Merge pull request #5841 from lilyinstarlight/fix-fromtoml-attrs
Sort attrs from tables in fromTOML
2021-12-29 00:03:10 +01:00
Lily Foster
92c7d33ee3 Sort attrs from tables in fromTOML
This was dropped in 10a8b5d for the migration from cpptoml to toml11 but
seems to be necessary for the attrsets to work correctly.

Fixes #5833
2021-12-28 10:11:00 -05:00
Alexander Bantyev
2dc29e0d93
Add a test that nix repl --show-trace actually shows the trace 2021-12-28 15:54:46 +03:00
Alexander Bantyev
581f774284
BaseError::calcWhat: take loggerSettings.showTrace into account
Text representation for errors should include the trace if
--show-trace is passed.
2021-12-28 15:53:21 +03:00
Alexander Bantyev
2664a216e5
flake.cc: computeLocks: Only verify overrides when they could change
When we check for disappeared overrides, we can get "false positives"
for follows and overrides which are defined in the dependencies of the
flake we are locking, since they are not parsed by
parseFlakeInputs. However, at that point we already know that the
overrides couldn't have possible been changed if the input itself
hasn't changed (since we check that oldLock->originalRef == *input.ref
for the input's parent). So, to prevent this, only perform this check
when it was possible that the flake changed (e.g. the flake we're
locking, or a new input, or the input has changed and mustRefetch ==
true).
2021-12-28 15:30:06 +03:00
Eelco Dolstra
0e90b13ab1
Merge pull request #5835 from yorickvP/fast-repl-load
Fix accidental O(n^2 * log n) performance in NixRepl::addAttrsToScope
2021-12-27 16:56:25 +01:00
Yorick van Pelt
a4ab0a74d9
Fix accidental O(n^2 * log n) performance in NixRepl::addAttrsToScope
Only sort once, after adding all of the attrs first. This reduces my
`nix repl '<nixpkgs>'` loading time from 1.07s to 103ms.

Fixes #5823
2021-12-27 13:18:55 +01:00
tomberek
f404189368
Merge branch 'master' into flake_search 2021-12-23 15:22:52 -05:00
Eelco Dolstra
af553b2090 Merge branch 'nm-int64' of https://github.com/nmattia/nix 2021-12-23 14:27:24 +01:00
Nicolas Mattia
b16643b6fc Use int64_t and NixFloat in fromTOML types
This makes sure that values parsed from TOML have a proper size. Using
e.g. `double` caused issues on i686 where the size of `double` (32bit)
was too small to accommodate some values.
2021-12-23 14:12:49 +01:00
Eelco Dolstra
6ca2db2dad
Merge pull request #5826 from NixOS/fixup-5807
Properly return false on `builtins.pathExists /someNonAllowedPath`
2021-12-23 11:50:37 +01:00
regnat
dc89dfa7b3 Properly return false on builtins.pathExists /someNonAllowedPath
Follow-up from https://github.com/NixOS/nix/pull/5807 to fix https://github.com/NixOS/nix/pull/5807#issuecomment-1000135394
2021-12-23 10:49:33 +01:00
Eelco Dolstra
7feb741e00
Merge pull request #5821 from edolstra/remove-affinity
Remove CPU locking
2021-12-22 20:31:19 +01:00
Eelco Dolstra
b666a2ca8c
Merge pull request #5820 from ncfavier/completion-nospace
Don't insert spaces when completing attribute paths
2021-12-22 17:55:05 +01:00
Naïm Favier
c4a03bc4ae
Fix attr path completion after a dot 2021-12-22 16:37:58 +01:00
Naïm Favier
1da1b2b345
Don't insert spaces when completing attribute paths 2021-12-22 16:17:01 +01:00
Eelco Dolstra
9747ea84b4 Remove CPU locking
This was already accidentally disabled in ba87b08. It also no longer
appears to be beneficial, and in fact slow things down, e.g. when
evaluating a NixOS system configuration:

  elapsed time:       median =      3.8170  mean =      3.8202  stddev =      0.0195  min =      3.7894  max =      3.8600  [rejected, p=0.00000, Δ=0.36929±0.02513]
2021-12-22 15:56:25 +01:00
Eelco Dolstra
ddd7839154
Merge pull request #5819 from edolstra/devshell-completion
Make bash completion for 'nix' work in a devshell
2021-12-22 14:44:40 +01:00
Eelco Dolstra
5b5f68f1dd Make bash completion work when nix is invoked as ~/.../nix 2021-12-22 13:26:35 +01:00
Eelco Dolstra
0a7c1da9f3 Make bash completion for 'nix' work in a devshell 2021-12-22 13:21:45 +01:00
Eelco Dolstra
dc83298449
Merge pull request #5814 from NixOS/docker-ssl-certs-in-etc
Add cacert to /etc/ssl/certs in the docker image
2021-12-21 21:44:30 +01:00
Rok Garbas
534bc5a3d1
Add cacert to /etc/ssl/certs in the docker image
Fixes #5797
2021-12-21 21:17:29 +01:00
Eelco Dolstra
89faff93e2
Merge pull request #5807 from NixOS/5805-ca-ifd
Fix IFD with CA derivations
2021-12-21 18:47:34 +01:00
Eelco Dolstra
881b610266 Update release notes 2021-12-21 14:19:21 +01:00