On Nix 2.6 the output of `nix why-depends --all` seems to be somewhat
off:
$ nix why-depends /nix/store/kn47hayxab8gc01jhr98dwyywbx561aq-nixos-system-roflmayr-21.11.20220207.6c202a9.drv /nix/store/srn5jbs1q30jpybdmxqrwskyny659qgc-nix-2.6.drv --derivation --extra-experimental-features nix-command --all
/nix/store/kn47hayxab8gc01jhr98dwyywbx561aq-nixos-system-roflmayr-21.11.20220207.6c202a9.drv
└───/nix/store/g8bpgfjhh5vxrdq0w6r6s64f9kkm9z6c-etc.drv
│ └───/nix/store/hm0jmhp8shbf3cl846a685nv4f5cp3fy-nspawn-inst.drv
| [...]
└───/nix/store/2d6q3ygiim9ijl5d4h0qqx6vnjgxywyr-system-units.drv
└───/nix/store/dil014y1b8qyjhhhf5fpaah5fzdf0bzs-unit-systemd-nspawn-hydra.service.drv
└───/nix/store/a9r72wwx8qrxyp7hjydyg0gsrwnn26zb-activate.drv
└───/nix/store/99hlc7i4gl77wq087lbhag4hkf3kvssj-nixos-system-hydra-21.11pre-git.drv
Please note that `[...]-system-units.drv` is supposed to be a direct
child of `[...]-etc.drv`.
The reason for that is that each new level printed by `printNode` is
four spaces off in comparison to `nix why-depends --precise` because the
recursive `printNode()` only prints the path and not the `tree*`-chars in
the case of `--precise` and in this format the path is four spaces further
indented, i.e. on a newline, but on the same level as the path's children, i.e.
/nix/store/kn47hayxab8gc01jhr98dwyywbx561aq-nixos-system-roflmayr-21.11.20220207.6c202a9.drv
└───/: …1-p8.drv",["out"]),("/nix/store/g8bpgfjhh5vxrdq0w6r6s64f9kkm9z6c-etc.drv",["out"]),("/nix/store/…
→ /nix/store/g8bpgfjhh5vxrdq0w6r6s64f9kkm9z6c-etc.drv
As you can see `[...]-etc.drv` is a direct child of the root, but four
spaces indented. This logic was directly applied to the code-path with
`precise=false` which resulted in `tree*` being printed four spaces too
deep.
In case of no `--precise`, `hits[hash]` is empty and the path itself
should be printed rather than hits using the same logic as for `hits[hash]`.
With this fix, the output looks correct now:
/nix/store/kn47hayxab8gc01jhr98dwyywbx561aq-nixos-system-roflmayr-21.11.20220207.6c202a9.drv
└───/nix/store/g8bpgfjhh5vxrdq0w6r6s64f9kkm9z6c-etc.drv
├───/nix/store/hm0jmhp8shbf3cl846a685nv4f5cp3fy-nspawn-inst.drv
| [...]
└───/nix/store/2d6q3ygiim9ijl5d4h0qqx6vnjgxywyr-system-units.drv
└───/nix/store/dil014y1b8qyjhhhf5fpaah5fzdf0bzs-unit-systemd-nspawn-hydra.service.drv
└───/nix/store/a9r72wwx8qrxyp7hjydyg0gsrwnn26zb-activate.drv
└───/nix/store/99hlc7i4gl77wq087lbhag4hkf3kvssj-nixos-system-hydra-21.11pre-git.drv
Bundlers are now responsible for correctly handling their inputs which
are no longer constrained to be (Drv->Drv)->Drv->Drv, but can be of
type (attrset->Drv)->attrset->Drv.
we'll retain the old coerceToString interface that returns a string, but callers
that don't need the returned value to outlive the Value it came from can save
copies by using the new interface instead. for values that weren't stringy we'll
pass a new buffer argument that'll be used for storage and shouldn't be
inspected.
- Make passing the position to `forceValue` mandatory,
this way we remember people that the position is
important for better error messages
- Add pos to all `forceValue` calls
if we defer the duplicate argument check for lambda formals we can use more
efficient data structures for the formals set, and we can get rid of the
duplication of formals names to boot. instead of a list of formals we've seen
and a set of names we'll keep a vector instead and run a sort+dupcheck step
before moving the parsed formals into a newly created lambda. this improves
performance on search and rebuild by ~1%, pure parsing gains more (about 4%).
this does reorder lambda arguments in the xml output, but the output is still
stable. this shouldn't be a problem since argument order is not semantically
important anyway.
before
nix search --no-eval-cache --offline ../nixpkgs hello
Time (mean ± σ): 8.550 s ± 0.060 s [User: 6.470 s, System: 1.664 s]
Range (min … max): 8.435 s … 8.666 s 20 runs
nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
Time (mean ± σ): 346.7 ms ± 2.1 ms [User: 312.4 ms, System: 34.2 ms]
Range (min … max): 343.8 ms … 353.4 ms 20 runs
nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
Time (mean ± σ): 2.720 s ± 0.031 s [User: 2.415 s, System: 0.231 s]
Range (min … max): 2.662 s … 2.780 s 20 runs
after
nix search --no-eval-cache --offline ../nixpkgs hello
Time (mean ± σ): 8.462 s ± 0.063 s [User: 6.398 s, System: 1.661 s]
Range (min … max): 8.339 s … 8.542 s 20 runs
nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
Time (mean ± σ): 329.1 ms ± 1.4 ms [User: 296.8 ms, System: 32.3 ms]
Range (min … max): 326.1 ms … 330.8 ms 20 runs
nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
Time (mean ± σ): 2.687 s ± 0.035 s [User: 2.392 s, System: 0.228 s]
Range (min … max): 2.626 s … 2.754 s 20 runs
Unless `--precise` is passed, make `nix why-depends` only show the
dependencies between the store paths, without introspecting them to
find the actual references.
This also makes it ~3x faster
This is needed to get the path of a derivation that might not exist
(e.g. for 'nix store copy-log').
InstallableStorePath::toDerivedPaths() cannot be used for this because
it calls readDerivation(), so it fails if the store doesn't have the
derivation.
When stderr is not connected to a tty, show "building" and
"substituting" messages, a-la nix-build et al.
Closes https://github.com/NixOS/nix/issues/4402
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
`nix why-depends` is piping its output into a pager by default.
However the pager was only started after the first path is printed,
causing it to be excluded from the pager output.
(Actually the pager was started *inside* the recursive function that was
printing the dependency chain, so a new instance was started at each
level. It’s a little miracle that it worked at all).
Fix#5911
when given a string yacc will copy the entire input to a newly allocated
location so that it can add a second terminating NUL byte. since the
parser is a very internal thing to EvalState we can ensure that having
two terminating NUL bytes is always possible without copying, and have
the parser itself merely check that the expected NULs are present.
# before
Benchmark 1: nix search --offline nixpkgs hello
Time (mean ± σ): 572.4 ms ± 2.3 ms [User: 563.4 ms, System: 8.6 ms]
Range (min … max): 566.9 ms … 579.1 ms 50 runs
Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
Time (mean ± σ): 381.7 ms ± 1.0 ms [User: 348.3 ms, System: 33.1 ms]
Range (min … max): 380.2 ms … 387.7 ms 50 runs
Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
Time (mean ± σ): 2.936 s ± 0.005 s [User: 2.715 s, System: 0.221 s]
Range (min … max): 2.923 s … 2.946 s 50 runs
# after
Benchmark 1: nix search --offline nixpkgs hello
Time (mean ± σ): 571.7 ms ± 2.4 ms [User: 563.3 ms, System: 8.0 ms]
Range (min … max): 566.7 ms … 579.7 ms 50 runs
Benchmark 2: nix eval -f ../nixpkgs/pkgs/development/haskell-modules/hackage-packages.nix
Time (mean ± σ): 376.6 ms ± 1.0 ms [User: 345.8 ms, System: 30.5 ms]
Range (min … max): 374.5 ms … 379.1 ms 50 runs
Benchmark 3: nix eval --raw --impure --expr 'with import <nixpkgs/nixos> {}; system'
Time (mean ± σ): 2.922 s ± 0.006 s [User: 2.707 s, System: 0.215 s]
Range (min … max): 2.906 s … 2.934 s 50 runs
Previously you had to remember to call value->attrs->sort() after
populating value->attrs. Now there is a BindingsBuilder helper that
wraps Bindings and ensures that sort() is called before you can use
it.
This was already accidentally disabled in ba87b08. It also no longer
appears to be beneficial, and in fact slow things down, e.g. when
evaluating a NixOS system configuration:
elapsed time: median = 3.8170 mean = 3.8202 stddev = 0.0195 min = 3.7894 max = 3.8600 [rejected, p=0.00000, Δ=0.36929±0.02513]
When a variable is assigned in the REPL, make sure to remove any possible reference to the old one so that we correctly pick the new one afterwards
Fix#5706
Doing it as a side-effect of calling LocalStore::makeStoreWritable()
is very ugly.
Also, make sure that stopping the progress bar joins the update
thread, otherwise that thread should be unshared as well.
When running a `:b` command in the repl, after building the derivations
query the store for its outputs rather than just assuming that they are
known in the derivation itself (which isn’t true for CA derivations)
Fix#5328
Rather than having them plain strings scattered through the whole
codebase, create an enum containing all the known experimental features.
This means that
- Nix can now `warn` when an unkwown experimental feature is passed
(making it much nicer to spot typos and spot deprecated features)
- It’s now easy to remove a feature altogether (once the feature isn’t
experimental anymore or is dropped) by just removing the field for the
enum and letting the compiler point us to all the now invalid usages
of it.
When I stop a download with Ctrl-C in a `nix repl` of a flake, the REPL
refuses to do any other downloads:
nix-repl> builtins.getFlake "nix-serve"
[0.0 MiB DL] downloading 'https://api.github.com/repos/edolstra/nix-serve/tarball/e9828a9e01a14297d15ca41 error: download of 'e9828a9e01' was interrupted
[0.0 MiB DL]
nix-repl> builtins.getFlake "nix-serve"
error: interrupted by the user
[0.0 MiB DL]
To fix this issue, two changes were necessary:
* Reset the global `_isInterrupted` variable: only because a single
operation was aborted, it should still be possible to continue the
session.
* Recreate a `fileTransfer`-instance if the current one was shut down by
an abort.
The boolean is only used to determine if the formals are set to a
non-null pointer in all our cases. We can get rid of that allocation and
instead just compare the pointer value with NULL. Saving up to
sizeof(bool) + platform specific alignment per ExprLambda instace.
Probably not a lot of memory but perhaps a few kilobyte with nixpkgs?
This also gets rid of a potential issue with dereferencing formals based on
the value of the boolean that didn't have to be aligned with the formals
pointer but was in all our cases.
I had started the trend of doing `std::visit` by value (because a type
error once mislead me into thinking that was the only form that
existed). While the optomizer in principle should be able to deal with
extra coppying or extra indirection once the lambdas inlined, sticking
with by reference is the conventional default. I hope this might even
improve performance.
It's now disabled by default for the following:
* 'nix search' (this was already implied by read-only mode)
* 'nix flake show'
* 'nix flake check', but only on the hydraJobs output
In dry run mode, new derivations can't be create, so running the command on anything that has not been evaluated before results in an error message of the form `don't know how to build these paths (may be caused by read-only store access)`.
For comparison, the classical `nix-build --dry-run` doesn't use read-only mode.
Closes#1795
(cherry picked from commit 54525682df707742e58311c32e9c9cb18de1e31f)
“packages” was probably meant to be “packages.${system}.” but that
is already listed in `getDefaultFlakeAttrPathPrefixes` in `installables`,
which is probably why no one noticed it was broken.
It currently fails with the following error:
error: flake 'git+file://…' does not provide attribute 'devShells.x86_64-linuxhaskell', 'packages.x86_64-linux.haskell', 'legacyPackages.x86_64-linux.haskell' or 'haskell'
For git+file and path flakes, chdir to flake directory so that phases
that expect to be in the flake directory can run
Fixes https://github.com/NixOS/nix/issues/3976
With this, we don't have to copy the entire .drv closure to the
destination store ahead of time (or at all). Instead, buildPaths()
reads .drv files from the eval store and copies inputSrcs to the
destination store if it needs to build a derivation.
Issue #5025.
In particular, this now works:
$ nix path-info --eval-store auto --store https://cache.nixos.org nixpkgs#hello
Previously this would fail as it would try to upload the hello .drv to
cache.nixos.org. Now the .drv is instantiated in the local store, and
then we check for the existence of the outputs in cache.nixos.org.
It supports functions as well. Also change `package` to
`derivation` because it operates at the language level and does
not open the derivation (which would be useful but not nearly
as much).
It does not operate on a derivation and does not return a
derivation path. Instead it works at the language level,
where a distinct term "package" is more appropriate to
distinguish the parent object of `meta.position`; an
attribute which doesn't even make it into the derivation.
Fill `NIX_CONFIG` with the value of the current Nix configuration before
calling the nix subprocesses in the repl
That way the whole configuration (including the possible
`experimental-features`, a possibly `--store` option or whatever) will
be made available.
This is required for example to make `nix repl` work with a custom
`--store`
Fill `NIX_CONFIG` with the value of the current Nix configuration before
calling the post-build-hook.
That way the whole configuration (including the possible
`experimental-features`, a possibly `--store` option or whatever) will
be made available to the hook
Pass the current experimental features using `NIX_CONFIG` to the various
Nix subprocesses that `nix repl` invokes.
This is quite a hack, but having `nix repl` call Nix with a subprocess
is a hack already, so I guess that’s fine.
`nix develop` is getting bash from an (assumed existing) `nixpkgs`
flake. However, when doing so, it reuses the `lockFlags` passed to the
current flake, including the `--input-overrides` and `--input-update`
which generally don’t make sense anymore at that point (and trigger a
warning because of that)
Clear these overrides before getting the nixpkgs flake to get rid of the
warning.
Useful when we're using a daemon with a chroot store, e.g.
$ NIX_DAEMON_SOCKET_PATH=/tmp/chroot/nix/var/nix/daemon-socket/socket nix-daemon --store /tmp/chroot
Then the client can now connect with
$ nix build --store unix:///tmp/chroot/nix/var/nix/daemon-socket/socket?root=/tmp/chroot nixpkgs#hello
When the `keep-going` option is set to `true`, make `nix flake check`
continue as much as it can before failing.
The UI isn’t perfect as-it-is as all the lines currently start with a
mostly useless `error (ignored): error:` prefix, but I’m not sure what
the best output would be, so I’ll leave it as-it-is for the time being
(This is a bit hijacking the `keep-going` flag as it’s supposed to be a
build-time only thing. But I think it’s faire to reuse it here).
Fix https://github.com/NixOS/nix/issues/4450
Accidentally removed in ca96f52194. This
caused `nix run` to systematically fail with
```
error: app program '/nix/store/…' is not in the Nix store
```
~/.bashrc should be sourced first in the rc script so that PATH &
other env vars give precedence over the bashrc PATH.
Also, in my bashrc I alias rm as:
alias rm='rm -Iv'
To avoid running this alias (which shows ‘removed '/tmp/nix-shell.*'),
we can just prefix rm with command.
This was previously done in https://github.com/NixOS/nix/pull/4515 but
got clobbered away in https://github.com/NixOS/nix/pull/4594.
--------------------------------------------------------------------------------
This fixes an issue where derivations with a primary output that is
not "out" would fail with:
$ nix profile install nixpkgs#sqlite
error: opening directory '/nix/store/2a2ydlgyydly5czcc8lg12n6qqkfz863-sqlite-3.34.1-bin': No such file or directory
This happens because while derivations produce every output when
built, you might not have them if you didn't build the derivation
yourself (for instance, the store path was fetch from a binary cache).
This uses outputName provided from DerivationInfo which appears to
match the first output of the derivation.
This avoids an ambiguity where the `StorePathWithOutputs { drvPath, {}
}` could mean "build `brvPath`" or "substitute `drvPath`" depending on
context.
It also brings the internals closer in line to the new CLI, by
generalizing the `Buildable` type is used there and makes that
distinction already.
In doing so, relegate `StorePathWithOutputs` to being a type just for
backwards compatibility (CLI and RPC).
This is probably what most people expect it to do. Fixes#3781.
There is a new command 'nix flake lock' that has the old behaviour of
'nix flake update', i.e. it just adds missing lock file entries unless
overriden using --update-input.
This is technically a breaking change, since attempting to set plugin
files after the first non-flag argument will now throw an error. This
is acceptable given the relative lack of stability in a plugin
interface and the need to tie the knot somewhere once plugins can
actually define new subcommands.
This makes nix search always go through the first level of an
attribute set, even if it's not a top level attribute. For instance,
you can now list all GHC compilers with:
$ nix search nixpkgs#haskell.compiler
...
This is similar to how nix-env works when you pass in -A.
This fixes an issue where derivations with a primary output that is
not "out" would fail with:
$ nix profile install nixpkgs#sqlite
error: opening directory '/nix/store/2a2ydlgyydly5czcc8lg12n6qqkfz863-sqlite-3.34.1-bin': No such file or directory
This happens because while derivations produce every output when
built, you might not have them if you didn't build the derivation
yourself (for instance, the store path was fetch from a binary cache).
This uses outputName provided from DerivationInfo which appears to
match the first output of the derivation.
Where a `RealisedPath` is a store path with its history, meaning either
an opaque path for stuff that has been directly added to the store, or a
`Realisation` for stuff that has been built by a derivation
This is a low-level refactoring that doesn't bring anything by itself
(except a few dozen extra lines of code :/ ), but raising the
abstraction level a bit is important on a number of levels:
- Commands like `nix build` have to query for the realisations after the
build is finished which is fragile (see
27905f12e4a7207450abe37c9ed78e31603b67e1 for example). Having them
oprate directly at the realisation level would avoid that
- Others like `nix copy` currently operate directly on (built) store
paths, but need a bit more information as they will need to register
the realisations on the remote side
Changes:
* The divider lines are gone. These were in practice a bit confusing,
in particular with --show-trace or --keep-going, since then there
were multiple lines, suggesting a start/end which wasn't the case.
* Instead, multi-line error messages are now indented to align with
the prefix (e.g. "error: ").
* The 'description' field is gone since we weren't really using it.
* 'hint' is renamed to 'msg' since it really wasn't a hint.
* The error is now printed *before* the location info.
* The 'name' field is no longer printed since most of the time it
wasn't very useful since it was just the name of the exception (like
EvalError). Ideally in the future this would be a unique, easily
googleable error ID (like rustc).
* "trace:" is now just "…". This assumes error contexts start with
something like "while doing X".
Example before:
error: --- AssertionError ---------------------------------------------------------------------------------------- nix
at: (7:7) in file: /home/eelco/Dev/nixpkgs/pkgs/applications/misc/hello/default.nix
6|
7| x = assert false; 1;
| ^
8|
assertion 'false' failed
----------------------------------------------------- show-trace -----------------------------------------------------
trace: while evaluating the attribute 'x' of the derivation 'hello-2.10'
at: (192:11) in file: /home/eelco/Dev/nixpkgs/pkgs/stdenv/generic/make-derivation.nix
191| // (lib.optionalAttrs (!(attrs ? name) && attrs ? pname && attrs ? version)) {
192| name = "${attrs.pname}-${attrs.version}";
| ^
193| } // (lib.optionalAttrs (stdenv.hostPlatform != stdenv.buildPlatform && !dontAddHostSuffix && (attrs ? name || (attrs ? pname && attrs ? version)))) {
Example after:
error: assertion 'false' failed
at: (7:7) in file: /home/eelco/Dev/nixpkgs/pkgs/applications/misc/hello/default.nix
6|
7| x = assert false; 1;
| ^
8|
… while evaluating the attribute 'x' of the derivation 'hello-2.10'
at: (192:11) in file: /home/eelco/Dev/nixpkgs/pkgs/stdenv/generic/make-derivation.nix
191| // (lib.optionalAttrs (!(attrs ? name) && attrs ? pname && attrs ? version)) {
192| name = "${attrs.pname}-${attrs.version}";
| ^
193| } // (lib.optionalAttrs (stdenv.hostPlatform != stdenv.buildPlatform && !dontAddHostSuffix && (attrs ? name || (attrs ? pname && attrs ? version)))) {
Move clearValue inside Value
mkInt instead of setInt
mkBool instead of setBool
mkString instead of setString
mkPath instead of setPath
mkNull instead of setNull
mkAttrs instead of setAttrs
mkList instead of setList*
mkThunk instead of setThunk
mkApp instead of setApp
mkLambda instead of setLambda
mkBlackhole instead of setBlackhole
mkPrimOp instead of setPrimOp
mkPrimOpApp instead of setPrimOpApp
mkExternal instead of setExternal
mkFloat instead of setFloat
Add note that the static mk* function should be removed eventually
This makes it possible to have per-project configuration in flake.nix,
e.g. binary caches and other stuff:
nixConfig.bash-prompt-suffix = "[1;35mngi# [0m";
nixConfig.substituters = [ "https://cache.ngi0.nixos.org/" ];
This is primarily useful if you're hacking simultaneously on a package
and one of its dependencies. E.g. if you're hacking on Hydra and Nix,
you would start a dev shell for Nix, and then a dev shell for Hydra as
follows:
$ nix develop \
--redirect .#hydraJobs.build.x86_64-linux.nix ~/Dev/nix/outputs/out \
--redirect .#hydraJobs.build.x86_64-linux.nix.dev ~/Dev/nix/outputs/dev
(This assumes hydraJobs.build.x86_64-linux has a passthru.nix
attribute. You can also use a store path.)
This causes all references in the environment to those store paths to
be rewritten to ~/Dev/nix/outputs/{out,dev}. Note: unfortunately, you
may need to set LD_LIBRARY_PATH=~/Dev/nix/outputs/out/lib because
Nixpkgs' ld-wrapper only adds -rpath entries for -L flags that point
to the Nix store.
Fix#3975: Currently if Ctrl-C is pressed during a phase, the interactive subshell
is not exited. Removing --rcfile when --phase is present makes bash
non-interactive
Make nix output completions in the form `completion\tdescription`.
This can't be used by bash (afaik), but other shells like zsh or fish
can display it along the completion choices
Directly register the store classes rather than a function to build an
instance of them.
This gives the possibility to introspect static members of the class or
choose different ways of instantiating them.
This allows interactively inspecting the state of the evaluator at the
point of failure.
Example:
$ nix eval path:///home/eelco/Dev/nix/flake2#modules.hello-closure._final --start-repl-on-eval-errors
error: --- TypeError -------------------------------------------------------------------------------------------------------------------------------------------------------------------- nix
at: (20:53) in file: /nix/store/4264z41dxfdiqr95svmpnxxxwhfplhy0-source/flake.nix
19|
20| _final = builtins.foldl' (xs: mod: xs // (mod._module.config { config = _final; })) _defaults _allModules;
| ^
21| };
attempt to call something which is not a function but a set
Starting REPL to allow you to inspect the current state of the evaluator.
The following extra variables are in scope: arg, fun
Welcome to Nix version 2.4. Type :? for help.
nix-repl> fun
error: --- EvalError -------------------------------------------------------------------------------------------------------------------------------------------------------------------- nix
at: (150:28) in file: /nix/store/4264z41dxfdiqr95svmpnxxxwhfplhy0-source/flake.nix
149|
150| tarballClosure = (module {
| ^
151| extends = [ self.modules.derivation ];
attribute 'derivation' missing
nix-repl> :t fun
a set
nix-repl> builtins.attrNames fun
[ "tarballClosure" ]
nix-repl>
Occasionally, `nix-build --check` is fairly helpful and I'd like to be
able to use this feature for flakes that need to be built with `nix
build` as well.
This adds a ‘nix export’ command which hooks into nix-bundle. It can
be used in a similar way as nix-bundle, with the benefit of hooking
into the new “app” functionality. For instance,
$ nix export nixpkgs#jq
$ ./jq --help
jq - commandline JSON processor [version 1.6]
...
$ scp jq machine-without-nix:
$ ssh machine-without-nix ./jq
jq - commandline JSON processor [version 1.6]
...
Note that nix-bundle currently requires Linux to run. Other exporters
might not have that requirement.
“exporters” are meant to be reusable, so that, other repos can
implement their own bundling.
Fixes#3705
match_continuous limits the search to the current start position,
instead of searching the entire file.
On libc++, this improves performance dramatically:
$ time /nix/store/70ai68dfm6xbzwn26j5n4li9di52ylia-nix-3.0pre20200728_c159f48/bin/nix print-dev-env >/dev/null
/nix/store/70ai68dfm6xbzwn26j5n4li9di52ylia-nix-3.0pre20200728_c159f48/bin/ni 2.39s user 0.19s system 64% cpu 4.032 total
$ time /nix/store/cwjfxxlp83zln4mfyy1d2dbsx7f6s962-nix-3.0pre20200728_dirty/bin/nix print-dev-env >/dev/null
/nix/store/cwjfxxlp83zln4mfyy1d2dbsx7f6s962-nix-3.0pre20200728_dirty/bin/nix 0.09s user 0.05s system 65% cpu 0.204 total
Fixes#3874
I think this better captures the intent of what's going on: we either
have an opaque store path, or a drv path with some outputs.
Having this structure will also help us support CA derivations: we'll
have to allow the outpath paths to be optional, so the structure we gain
now makes up for the structure we loose then.
to each Store implementation. The generic regStore implementation will
only be for the ambiguous shorthands, like "" and "auto".
This also could get us close to simplifying the daemon command.
Currently resizing of the terminal doesn't play nicely with
nix edit when using kakoune as the editor, as it relies on the
SIGWINCH signal which is trapped by nix. How this is not a problem
with e.g. vim is beyond me.
Virtually all other exec* calls are following a call to
restoreSignals(). This commit adds this behavior to nix edit
as well.
That is, the commands 'nix path-info nixpkgs#hello' and 'nix path-info
/nix/store/00ls0qi49qkqpqblmvz5s1ajl3gc63lr-hello-2.10.drv' now do the
same thing (i.e. build the derivation and operate on the output store
path, rather than the .drv path).
We've added the variant to `DerivationOutput` to support them, but made
`DerivationOutput::path` partial to avoid actually implementing them.
With this chage, we can all collaborate on "just" removing
`DerivationOutput::path` calls to implement CA derivations.
This allows you to refer to an input from another flake. For example,
$ nix run --inputs-from /path/to/hydra nixpkgs#hello
runs 'hello' from the 'nixpkgs' inputs of the 'hydra' flake.
Fixes#3769.
'nix run' will try to run $out/bin/<name>, where <name> is the
derivation name (excluding the version). This often works well:
$ nix run nixpkgs#hello
Hello, world!
$ nix run nix -- --version
nix (Nix) 2.4pre20200626_adf2fbb
$ nix run patchelf -- --version
patchelf 0.11.20200623.e61654b
$ nix run nixpkgs#firefox -- --version
Mozilla Firefox 77.0.1
$ nix run nixpkgs#gimp -- --version
GNU Image Manipulation Program version 2.10.14
though not always:
$ nix run nixpkgs#git
error: unable to execute '/nix/store/kp7wp760l4gryq9s36x481b2x4rfklcy-git-2.25.4/bin/git-minimal': No such file or directory
E.g.
$ nix run nixpkgs#hello
error: --- Error ---------- nix
flake 'flake:nixpkgs' does not provide attribute 'apps.x86_64-linux.hello' or 'hello'
instead of
$ nix run nixpkgs#hello
error: --- Error ---------- nix
flake 'flake:nixpkgs' does not provide attribute 'hello'
This further continues with the dependency inverstion. Also I just went
ahead and exposed `parseDerivation`: it seems like the more proper
building block, and not a bad thing to expose if we are trying to be
less wedded to drv files on disk anywas.
On nix-env -qa -f '<nixpkgs>', this reduces maximum RSS by 20970 KiB
and runtime by 0.8%. This is mostly because we're not parsing the hash
part as a hash anymore (just validating that it consists of base-32
characters).
Also, replace storePathToHash() by StorePath::hashPart().
This fixes an issue where lockfile generation was not idempotent:
after updating a lockfile, a "follows" node would end up pointing to a
new copy of the node, rather than to the original node.
Needed so that we can include it as a logger in loggers.cc without
adding a dependency on nix
This also requires moving names.hh to libutil to prevent a circular
dependency between libmain and libexpr
Make the printing of the build logs systematically go through the
logger, and replicate the behavior of `no-build-output` by having two
different loggers (one that prints the build logs and one that doesn't)
Add a new `--log-format` cli argument to change the format of the logs.
The possible values are
- raw (the default one for old-style commands)
- bar (the default one for new-style commands)
- bar-with-logs (equivalent to `--print-build-logs`)
- internal-json (the internal machine-readable json format)
The initial contents of the flake is specified by the
'templates.<name>' or 'defaultTemplate' output of another flake. E.g.
outputs = { self }: {
templates = {
nixos-container = {
path = ./nixos-container;
description = "An example of a NixOS container";
};
};
};
allows
$ nix flake init -t templates#nixos-container
Also add a command 'nix flake new', which is identical to 'nix flake
init' except that it initializes a specified directory rather than the
current directory.
Instead, `Hash` uses `std::optional<HashType>`. In the future, we may
also make `Hash` itself require a known hash type, encoraging people to
use `std::optional<Hash>` instead.
The attributes previously stored in TreeInfo (narHash, revCount,
lastModified) are now stored in Input. This makes it less arbitrary
what attributes are stored where.
As a result, the lock file format has changed. An entry like
"info": {
"lastModified": 1585405475,
"narHash": "sha256-bESW0n4KgPmZ0luxvwJ+UyATrC6iIltVCsGdLiphVeE="
},
"locked": {
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "b88ff468e9850410070d4e0ccd68c7011f15b2be",
"type": "github"
},
is now stored as
"locked": {
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "b88ff468e9850410070d4e0ccd68c7011f15b2be",
"type": "github",
"lastModified": 1585405475,
"narHash": "sha256-bESW0n4KgPmZ0luxvwJ+UyATrC6iIltVCsGdLiphVeE="
},
The 'Input' class is now a dumb set of attributes. All the fetcher
implementations subclass InputScheme, not Input. This simplifies the
API.
Also, fix substitution of flake inputs. This was broken since lazy
flake fetching started using fetchTree internally.
The idea is it's always more flexible to consumer a `Source` than a
plain string, and it might even reduce memory consumption.
I also looked at `addToStoreFromDump` with its `// FIXME: remove?`, but
the worked needed for that is far more up for interpretation, so I
punted for now.
This makes 'nix flake' less cluttered and more consistent (it's only
subcommands that operator on a flake). Also, the registry is not
inherently flake-related (e.g. fetchTree could also use it to remap
inputs).
This completes flakerefs using the registry (e.g. 'nix<TAB>' => 'nix
nixpkgs') and flake output attributes by evaluating the flake
(e.g. 'dwarffs#nix<TAB>' => 'dwarffs#nixosModules').
InstallableValue has children InstallableFlake and InstallableAttrPath, but InstallableFlake was overriding toDerivations, and usage was changed so that InstallableFlake didn't need cmd. So these changes were made:
InstallableValue::toDerivations() -> InstalllableAttrPath::toDerivations()
InstallableValue::cmd -> InstallableAttrPath::cmd
InstallableValue uses state instead of cmd
toBuildables() and toDerivations() were made abstract
- result list will be always empty if --json is passed
- for scripts an empty search result is not really an error,
we rather want to distinguish between evaluation errors and empty results
This means you now get an error message *before* stuff gets built:
$ nix copy .#hydraJobs.vendoredCrates
error: you must pass '--from' and/or '--to'
Try 'nix --help' for more information.
In particular, doing 'nix build /path/to/dir' now works if
/path/to/dir is not a Git tree (it only has to contain a flake.nix
file).
Also, 'nix flake init' no longer requires a Git tree (but it will do a
'git add flake.nix' if it's a Git tree)
In particular, we store whether an attribute failed to evaluate (threw
an exception) or was an unsupported type. This is to ensure that a
repeated 'nix flake show' never has to evaluate anything, so it can
execute without fetching the flake.
With this, 'nix flake show nixpkgs/nixos-20.03 --legacy' executes in
0.6s (was 3.4s).
This speeds up the creation of the cache for the nixpkgs flake from
21.2s to 10.2s. Oddly, it also speeds up querying the cache
(i.e. running 'nix flake show nixpkgs/nixos-20.03 --legacy') from 4.2s
to 3.4s.
(For comparison, running with --no-eval-cache takes 9.5s, so the
overhead of building the SQLite cache is only 0.7s.)
In the fully cached case for the 'nixpkgs' flake, it went from 101s to
4.6s. Populating the cache went from 132s to 17.4s (which could
probably be improved further by combining INSERTs).
Usually this just writes to stdout, but for ProgressBar, we need to
clear the current line, write the line to stdout, and then redraw the
progress bar.
(cherry picked from commit 696c026006)
Usually this just writes to stdout, but for ProgressBar, we need to
clear the current line, write the line to stdout, and then redraw the
progress bar.
Previously the memory would occasionally be collected during eval since
the GC doesn't consider the member variable as alive / doesn't scan the
region of memory where the pointer lives.
By using the traceable_allocator<T> allocator provided by Boehm GC we
can ensure the memory isn't collected. It should be properly freed when
SourceExprCommand goes out of scope.
Future editions of flakes or the Nix language can be supported by
renaming flake.nix (e.g. flake-v2.nix). This avoids a bootstrap
problem where we don't know which grammar to use to parse
flake*.nix. It also allows a project to support multiple flake
editions, in theory.
This provides a pluggable mechanism for defining new fetchers. It adds
a builtin function 'fetchTree' that generalizes existing fetchers like
'fetchGit', 'fetchMercurial' and 'fetchTarball'. 'fetchTree' takes a
set of attributes, e.g.
fetchTree {
type = "git";
url = "https://example.org/repo.git";
ref = "some-branch";
rev = "abcdef...";
}
The existing fetchers are just wrappers around this. Note that the
input attributes to fetchTree are the same as flake input
specifications and flake lock file entries.
All fetchers share a common cache stored in
~/.cache/nix/fetcher-cache-v1.sqlite. This replaces the ad hoc caching
mechanisms in fetchGit and download.cc (e.g. ~/.cache/nix/{tarballs,git-revs*}).
This also adds support for Git worktrees (c169ea5904).
This is useful for finding out what a registry lookup resolves to, e.g
$ nix flake info patchelf
Resolved URL: github:NixOS/patchelf
Locked URL: github:NixOS/patchelf/cd7955af31698c571c30b7a0f78e59fd624d0229
