Don't pass --timeout / --max-silent-time to the remote builder.
Instead, let the local Nix process terminate the build if it exceeds a
timeout. The remote builder will be killed as a side-effect. This
gives better error reporting (since the timeout message from the
remote side wasn't properly propagated) and handles non-Nix problems
like SSH hangs.
I'm not sure if it has ever worked correctly. The line "lastWait =
after;" seems to mean that the timer was reset every time a build
produced log output.
Note that the timeout is now per build, as documented ("the maximum
number of seconds that a builder can run").
It is surprisingly impossible to check if a mountpoint is a bind mount
on Linux, and in my previous commit I forgot to check if /nix/store was
even a mountpoint at all. statvfs.f_flag is not populated with MS_BIND
(and even if it were, my check was wrong in the previous commit).
Luckily, the semantics of mount with MS_REMOUNT | MS_BIND make both
checks unnecessary: if /nix/store is not a mountpoint, then mount will
fail with EINVAL, and if /nix/store is not a bind-mount, then it will
not be made writable. Thus, if /nix/store is not a mountpoint, we fail
immediately (since we don't know how to make it writable), and if
/nix/store IS a mountpoint but not a bind-mount, we fail at first write
(see below for why we can't check and fail immediately).
Note that, due to what is IMO buggy behavior in Linux, calling mount
with MS_REMOUNT | MS_BIND on a non-bind readonly mount makes the
mountpoint appear writable in two places: In the sixth (but not the
10th!) column of mountinfo, and in the f_flags member of struct statfs.
All other syscalls behave as if the mount point were still readonly (at
least for Linux 3.9-rc1, but I don't think this has changed recently or
is expected to soon). My preferred semantics would be for MS_REMOUNT |
MS_BIND to fail on a non-bind mount, as it doesn't make sense to remount
a non bind-mount as a bind mount.
/nix/store could be a read-only bind mount even if it is / in its own filesystem, so checking the 4th field in mountinfo is insufficient.
Signed-off-by: Shea Levy <shea@shealevy.com>
It turns out that in multi-user Nix, a builder may be able to do
ln /etc/shadow $out/foo
Afterwards, canonicalisePathMetaData() will be applied to $out/foo,
causing /etc/shadow's mode to be set to 444 (readable by everybody but
writable by nobody). That's obviously Very Bad.
Fortunately, this fails in NixOS's default configuration because
/nix/store is a bind mount, so "ln" will fail with "Invalid
cross-device link". It also fails if hard-link restrictions are
enabled, so a workaround is:
echo 1 > /proc/sys/fs/protected_hardlinks
The solution is to check that all files in $out are owned by the build
user. This means that innocuous operations like "ln
${pkgs.foo}/some-file $out/" are now rejected, but that already failed
in chroot builds anyway.
Wacky string coercion semantics caused expressions like
exec = "${./my-script} params...";
to evaluate to a path (‘/path/my-script params’), because
anti-quotations are desuged to string concatenation:
exec = ./my-script + " params...";
By constrast, adding a space at the start would yield a string as
expected:
exec = " ${./my-script} params...";
Now the first example also evaluates to a string.
...where <XX> is the first two characters of the derivation.
Otherwise /nix/var/log/nix/drvs may become so large that we run into
all sorts of weird filesystem limits/inefficiences. For instance,
ext3/ext4 filesystems will barf with "ext4_dx_add_entry:1551:
Directory index full!" once you hit a few million files.
So if a path is not garbage solely because it's reachable from a root
due to the gc-keep-outputs or gc-keep-derivations settings, ‘nix-store
-q --roots’ now shows that root.
But this time it's *obviously* correct! No more segfaults due to
infinite recursions for sure, etc.
Also, move directories to /nix/store/trash instead of renaming them to
/nix/store/bla-gc-<pid>. Then we can just delete /nix/store/trash at
the end.
This prevents zillions of derivations from being kept, and fixes an
infinite recursion in the garbage collector (due to an obscure cycle
that can occur with fixed-output derivations).
We now print all output paths of a package, e.g.
openssl-1.0.0i bin=/nix/store/gq2mvh0wb9l90djvsagln3aqywqmr6vl-openssl-1.0.0i-bin;man=/nix/store/7zwf5r5hsdarl3n86dasvb4chm2xzw9n-openssl-1.0.0i-man;/nix/store/cj7xvk7fjp9q887359j75pw3pzjfmqf1-openssl-1.0.0i
or (in XML mode)
<item attrPath="openssl" name="openssl-1.0.0i" system="x86_64-linux">
<output name="bin" path="/nix/store/gq2mvh0wb9l90djvsagln3aqywqmr6vl-openssl-1.0.0i-bin" />
<output name="man" path="/nix/store/7zwf5r5hsdarl3n86dasvb4chm2xzw9n-openssl-1.0.0i-man" />
<output name="out" path="/nix/store/cj7xvk7fjp9q887359j75pw3pzjfmqf1-openssl-1.0.0i" />
</item>
This allows adding attributes like
attr = if stdenv.system == "bla" then something else null;
without changing the resulting derivation on non-<bla> platforms.
We once considered adding a special "ignore" value for this purpose,
but using null seems more elegant.
The integer constant ‘langVersion’ denotes the current language
version. It gets increased every time a language feature is
added/changed/removed. It's currently 1.
The string constant ‘nixVersion’ contains the current Nix version,
e.g. "1.2pre2980_9de6bc5".
If a derivation has multiple outputs, then we only want to download
those outputs that are actuallty needed. So if we do "nix-build -A
openssl.man", then only the "man" output should be downloaded.
Likewise if another package depends on ${openssl.man}.
The tricky part is that different derivations can depend on different
outputs of a given derivation, so we may need to restart the
corresponding derivation goal if that happens.
For example, given a derivation with outputs "out", "man" and "bin":
$ nix-build -A pkg
produces ./result pointing to the "out" output;
$ nix-build -A pkg.man
produces ./result-man pointing to the "man" output;
$ nix-build -A pkg.all
produces ./result, ./result-man and ./result-bin;
$ nix-build -A pkg.all -A pkg2
produces ./result, ./result-man, ./result-bin and ./result-2.
This flag causes paths that do not have a known substitute to be
quietly ignored. This is mostly useful for Charon, allowing it to
speed up deployment by letting a machine use substitutes for all
substitutable paths, instead of uploading them. The latter is
frequently faster, e.g. if the target machine has a fast Internet
connection while the source machine is on a slow ADSL line.
This reverts commit 2980d1fba9. It
causes a regression in NixOS evaluation:
string `/nix/store/ya3s5gmj3b28170fpbjhgsk8wzymkpa1-pommed-1.39/etc/pommed.conf' cannot refer to other paths
vfork() is just too weird. For instance, in this build:
http://hydra.nixos.org/build/3330487
the value fromHook.writeSide becomes corrupted in the parent, even
though the child only reads from it. At -O0 the problem goes away.
Probably the child is overriding some spilled temporary variable.
If I get bored I may implement using posix_spawn() instead.
I.e. do what git does. I'm too lazy to keep the builtin help text up
to date :-)
Also add ‘--help’ to various commands that lacked it
(e.g. nix-collect-garbage).
With this flag, if any valid derivation output is missing or corrupt,
it will be recreated by using a substitute if available, or by
rebuilding the derivation. The latter may use hash rewriting if
chroots are not available.
This operation allows fixing corrupted or accidentally deleted store
paths by redownloading them using substituters, if available.
Since the corrupted path cannot be replaced atomically, there is a
very small time window (one system call) during which neither the old
(corrupted) nor the new (repaired) contents are available. So
repairing should be used with some care on critical packages like
Glibc.
In Nixpkgs, the attribute in all-packages.nix corresponding to a
package is usually equal to the package name. However, this doesn't
work if the package contains a dash, which is fairly common. The
convention is to replace the dash with an underscore (e.g. "dbus-lib"
becomes "dbus_glib"), but that's annoying. So now dashes are valid in
variable / attribute names, allowing you to write:
dbus-glib = callPackage ../development/libraries/dbus-glib { };
and
buildInputs = [ dbus-glib ];
Since we don't have a negation or subtraction operation in Nix, this
is unambiguous.
Using the immutable bit is problematic, especially in conjunction with
store optimisation. For instance, if the garbage collector deletes a
file, it has to clear its immutable bit, but if the file has
additional hard links, we can't set the bit afterwards because we
don't know the remaining paths.
So now that we support having the entire Nix store as a read-only
mount, we may as well drop the immutable bit. Unfortunately, we have
to keep the code to clear the immutable bit for backwards
compatibility.
It turns out that the immutable bit doesn't work all that well. A
better way is to make the entire Nix store a read-only bind mount,
i.e. by doing
$ mount --bind /nix/store /nix/store
$ mount -o remount,ro,bind /nix/store
(This would typically done in an early boot script, before anything
from /nix/store is used.)
Since Nix needs to be able to write to the Nix store, it now detects
if /nix/store is a read-only bind mount and then makes it writable in
a private mount namespace.
