Commit graph

8042 commits

Author SHA1 Message Date
Eelco Dolstra
7ef2645f45
Merge pull request #2921 from matthewbauer/handle-sigwinch
Handle SIGWINCH in main thread
2019-08-28 21:48:14 +02:00
Matthew Bauer
693e68e09c Set maximum name length in Nix
Previously we allowed any length of name for Nix derivations. This is
bad because different file systems have different max lengths. To make
things predictable, I have picked a max. This was done by trying to
build this derivation:

  derivation {
    name = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa";
    builder = "/no-such-path";
    system = "x86_64-linux";
  }

Take off one a and it will not lead to file name too long. That ends
up being 212 a’s. An even smaller max could be picked if we want to
support more file systems.

Working backwards, this is why:

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa-${name}.drv.chroot

> 255 - 32 - 1 - 4 - 7 = 211
2019-08-28 12:32:54 -04:00
Eelco Dolstra
7298a38a07
Don't send certain setting overrides to the daemon
These are already handled separately. This fixes warnings like

  warning: ignoring the user-specified setting 'max-jobs', because it is a restricted setting and you are not a trusted user

when using the -j flag.
2019-08-28 16:29:44 +02:00
Eelco Dolstra
ad03159e25
Merge pull request #2745 from samueldr/install/detect-systemd-separately
install-multi-user: Detect and fail lack of systemd separately
2019-08-28 11:34:23 +02:00
Eelco Dolstra
bd285849ed
Merge pull request #3054 from matthewbauer/nix-dir-macos
Allow empty /nix directory in multi-user installer
2019-08-28 11:29:43 +02:00
Eelco Dolstra
5fa8b3f965 Update the libboost hack
This cuts about 46 MiB from the closure.
2019-08-27 22:38:48 +02:00
Eelco Dolstra
ceefddafe8 Compress binary tarballs using xz
Fixes https://github.com/NixOS/nix/issues/240.

Apparently 'tar -xf' can decompress xz files on macOS nowadays.
2019-08-27 22:18:34 +02:00
Eelco Dolstra
787015fec0 Merge branch 'master' of github.com:NixOS/nix 2019-08-27 21:18:20 +02:00
Eelco Dolstra
fdff96501f Update release notes 2019-08-27 21:18:00 +02:00
Eelco Dolstra
e5b397b2c7 Merge branch 'test-sandboxing' of https://github.com/matthewbauer/nix 2019-08-27 20:58:47 +02:00
Eelco Dolstra
177e5742fa
Merge pull request #3056 from grahamc/operators
operators: document exact precedence, split up similar operators
2019-08-27 17:12:36 +02:00
Eelco Dolstra
73728874ab Hopefully fix post-hook test on macOS
https://hydra.nixos.org/build/99262744
2019-08-27 17:01:54 +02:00
Eelco Dolstra
45b3dc325a Add 2.3 release notes 2019-08-27 17:00:04 +02:00
Matthew Bauer
800fba1037 Use wait4path on org.nixos.nix-daemon.plist
When using a volume, the nix-daemon path may not exist. To avoid this
issue, we must use the wait4path tool. This should solve one of the
issues in multi-user on macOS Catalina.
2019-08-27 10:58:48 -04:00
Graham Christensen
171d784404
docs: operators: Make OR and AND capitalized 2019-08-27 06:55:22 -04:00
Eelco Dolstra
ee07ce7554
Merge pull request #3064 from pszubiak/systemd-unit-service-fix
nix-daemon.service: add install section.
2019-08-27 10:50:07 +02:00
Piotr Szubiakowski
d459224724 nix-daemon.service: add install section.
Signed-off-by: Piotr Szubiakowski <pszubiak@eso.org>
2019-08-27 10:35:35 +02:00
Graham Christensen
15ee2bc2fe
Merge pull request #2946 from vmandela/proxy
installer: handle network proxy in systemd multi-user install
2019-08-24 14:55:19 -04:00
Graham Christensen
057af1dbd8
docs: document the installer's use of proxy env vars 2019-08-24 09:08:44 -04:00
Venkateswara Rao Mandela
6dab42a551
installer: handle network proxy in systemd install
If a network proxy configuration is detected, setup an override
systemd unit file for nix-daemon service with the non-empty
proxy variables.

Proxy detection is performed by looking for http/https/ftp proxy and no
proxy variables in user environment
2019-08-24 09:08:41 -04:00
Matthew Bauer
5c06a8d328 Reset tmpDirInSandbox for unsandboxed 2019-08-23 20:24:39 -04:00
Graham Christensen
92ddce4f46
operators: document exact precedenc, split up similar operators 2019-08-23 15:50:54 -04:00
Matthew Bauer
0463d5e36f Allow empty /nix directory in multi-user installer
With macOS catalina, we can no longer modify the root system
volume (#2925). macOS provides a system configuration file in
synthetic.conf(5) to create empty root directories. This can be used
to mount /nix to a separate volume. As a result, this directory will
need to already exist prior to installation. Instead, check for
/nix/store and /nix/var for a live Nix installation.
2019-08-22 23:38:52 -04:00
Toon Nolten
1dbaf11948 Fix nix-env documentation for --delete-generations
The documentation for `--delete-generations` had an erroneous fullstop
and as it turns out inaccurate information on the `+No.` syntax.
2019-08-17 16:33:35 +02:00
Eelco Dolstra
f435634a29
Merge pull request #3041 from zimbatm/nix-store-error-13
Fix for `unknown serve command 13`
2019-08-16 16:15:05 +02:00
zimbatm
b226b5cd97
nix-store: fix out of sync protocol
If a NAR is already in the store, addToStore doesn't read the source
which makes the protocol go out of sync. This happens for example when
two client try to nix-copy-closure the same derivation at the same time.
2019-08-16 15:05:45 +02:00
zimbatm
91b00b145f
libutil: add SizedSource
Introduce the SizeSource which allows to bound how much data is being
read from a source. It also contains a drainAll() function to discard
the rest of the source, useful to keep the nix protocol in sync.
2019-08-16 15:05:40 +02:00
Eelco Dolstra
b7ea98bf34
Merge pull request #2945 from danidiaz/doc001
Expanded documentation for .nix-defexpr
2019-08-15 15:03:21 +02:00
Eelco Dolstra
477f82e5a7
Merge pull request #2782 from grahamc/flames
Track function start and end
2019-08-15 14:20:42 +02:00
Daniel Diaz
653c407784
Expanded documentation for .nix-defexpr 2019-08-15 08:05:22 -04:00
Graham Christensen
ee9c988a1b
Track function start and ends for flame graphs
With this patch, and this file I called `log.py`:

    #!/usr/bin/env nix-shell
    #!nix-shell -i python3 -p python3 --pure

    import sys
    from pprint import pprint

    stack = []
    timestack = []

    for line in open(sys.argv[1]):
        components = line.strip().split(" ", 2)
        if components[0] != "function-trace":
            continue

        direction = components[1]
        components = components[2].rsplit(" ", 2)

        loc = components[0]
        _at = components[1]
        time = int(components[2])

        if direction == "entered":
            stack.append(loc)
            timestack.append(time)
        elif direction == "exited":
            dur = time - timestack.pop()
            vst = ";".join(stack)
            print(f"{vst} {dur}")
            stack.pop()

and:

    nix-instantiate --trace-function-calls -vvvv ../nixpkgs/pkgs/top-level/release.nix -A unstable > log.matthewbauer 2>&1
    ./log.py ./log.matthewbauer > log.matthewbauer.folded
    flamegraph.pl --title matthewbauer-post-pr log.matthewbauer.folded > log.matthewbauer.folded.svg

I can make flame graphs like: http://gsc.io/log.matthewbauer.folded.svg

---

Includes test cases around function call failures and tryEval. Uses
RAII so the finish is always called at the end of the function.
2019-08-14 16:09:35 -04:00
Eelco Dolstra
662db921e2
nix dev-shell: Set dontAddDisableDepTrack 2019-08-09 18:51:52 +02:00
Eelco Dolstra
35ebae198f
Merge pull request #3031 from grahamc/low-speed-limit
conf: stalled-download-timeout: make tunable
2019-08-08 22:06:26 +02:00
Graham Christensen
a02457db71
conf: stalled-download-timeout: make tunable
Make curl's low speed limit configurable via stalled-download-timeout.
Before, this limit was five minutes without receiving a single byte.
This is much too long as if the remote end may not have even
acknowledged the HTTP request.
2019-08-08 10:22:13 -04:00
Eelco Dolstra
ab16b3d076
Fix gc-auto test 2019-08-08 15:49:19 +02:00
Eelco Dolstra
1d750e0587
Merge remote-tracking branch 'origin/master' into flakes 2019-08-08 15:49:13 +02:00
Eelco Dolstra
05a10dd835
tests/post-hook.sh: Don't put result link in cwd 2019-08-08 15:47:56 +02:00
Eelco Dolstra
2053ac7747
Rename file for consistency 2019-08-08 12:18:46 +02:00
Eelco Dolstra
f9021c4c6c
Merge pull request #3030 from dtzWill/fix/missing-include-ocloexec
pathlocks: add include to fcntl.h for O_CLOEXEC
2019-08-07 22:03:09 +02:00
Graham Christensen
1eeaf99cf8
fixup: docs for post-build-hook 2019-08-07 14:53:11 -04:00
Eelco Dolstra
56df30cd3f
Merge pull request #2995 from tweag/post-build-hook
Add a post build hook
2019-08-07 15:02:29 +02:00
Will Dietz
c3fefd1a6e
pathlocks: add include to fcntl.h for O_CLOEXEC 2019-08-07 07:41:22 -05:00
Graham Christensen
363a2f6826
post-build-hook: docs fixup 2019-08-06 14:26:43 -04:00
Eelco Dolstra
399b6f3c46
nix-store --verify: Don't repair while holding the GC lock 2019-08-02 18:48:26 +02:00
Eelco Dolstra
a2597d5f27
Simplify
With BSD locks we don't have to guard against reading our own
temproots.
2019-08-02 18:39:16 +02:00
Eelco Dolstra
e349f2c0a3
Use BSD instead of POSIX file locks
POSIX file locks are essentially incompatible with multithreading. BSD
locks have much saner semantics. We need this now that there can be
multiple concurrent LocalStore::buildPaths() invocations.
2019-08-02 18:39:16 +02:00
Eelco Dolstra
ec415d7166
Add a test for auto-GC
This currently fails because we're using POSIX file locks. So when the
garbage collector opens and closes its own temproots file, it causes
the lock to be released and then deleted by another GC instance.
2019-08-02 18:39:16 +02:00
regnat
7c5596734f
Add a post-build-hook
Passing `--post-build-hook /foo/bar` to a nix-* command will cause
`/foo/bar` to be executed after each build with the following
environment variables set:

    DRV_PATH=/nix/store/drv-that-has-been-built.drv
    OUT_PATHS=/nix/store/...build /nix/store/...build-bin /nix/store/...build-dev

This can be useful in particular to upload all the builded artifacts to
the cache (including the ones that don't appear in the runtime closure
of the final derivation or are built because of IFD).

This new feature prints the stderr/stdout output to the `nix-build`
and `nix build` client, and the output is printed in a Nix 2
compatible format:

    [nix]$ ./inst/bin/nix-build ./test.nix
    these derivations will be built:
      /nix/store/ishzj9ni17xq4hgrjvlyjkfvm00b0ch9-my-example-derivation.drv
    building '/nix/store/ishzj9ni17xq4hgrjvlyjkfvm00b0ch9-my-example-derivation.drv'...
    hello!
    bye!
    running post-build-hook '/home/grahamc/projects/github.com/NixOS/nix/post-hook.sh'...
    post-build-hook: + sleep 1
    post-build-hook: + echo 'Signing paths' /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation
    post-build-hook: Signing paths /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation
    post-build-hook: + sleep 1
    post-build-hook: + echo 'Uploading paths' /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation
    post-build-hook: Uploading paths /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation
    post-build-hook: + sleep 1
    post-build-hook: + printf 'very important stuff'
    /nix/store/qr213vjmibrqwnyp5fw678y7whbkqyny-my-example-derivation

    [nix-shell:~/projects/github.com/NixOS/nix]$ ./inst/bin/nix build -L -f ./test.nix
    my-example-derivation> hello!
    my-example-derivation> bye!
    my-example-derivation (post)> + sleep 1
    my-example-derivation (post)> + echo 'Signing paths' /nix/store/c263gzj2kb2609mz8wrbmh53l14wzmfs-my-example-derivation
    my-example-derivation (post)> Signing paths /nix/store/c263gzj2kb2609mz8wrbmh53l14wzmfs-my-example-derivation
    my-example-derivation (post)> + sleep 1
    my-example-derivation (post)> + echo 'Uploading paths' /nix/store/c263gzj2kb2609mz8wrbmh53l14wzmfs-my-example-derivation
    my-example-derivation (post)> Uploading paths /nix/store/c263gzj2kb2609mz8wrbmh53l14wzmfs-my-example-derivation
    my-example-derivation (post)> + sleep 1
    my-example-derivation (post)> + printf 'very important stuff'
    [1 built, 0.0 MiB DL]

Co-authored-by: Graham Christensen <graham@grahamc.com>
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2019-08-02 10:48:15 -04:00
Eelco Dolstra
320126aeeb
Tweak min-free/max-free descriptions 2019-08-02 14:04:09 +02:00
Matthew Bauer
9a0855bbb6 Don’t rely on EPERM
startProcess does not appear to send the exit code to the helper
correctly. Not sure why this is, but it is probably safe to just
fallback on all sandbox errors.
2019-07-30 17:53:37 -04:00