Commit graph

4716 commits

Author SHA1 Message Date
Eelco Dolstra
e682a8e138 Fix assertion failure in ThreadPool::enqueue() 2016-07-21 18:14:16 +02:00
Eelco Dolstra
d57981bac4 Store::queryMissing(): Use a thread pool
For one particular NixOS configuration, this cut the runtime of
"nix-store -r --dry-run" from 6m51s to 3.4s. It also fixes a bug in
the size calculation that was causing certain paths to be counted
twice, e.g. before:

  these paths will be fetched (1249.98 MiB download, 2995.74 MiB unpacked):

and after:

  these paths will be fetched (1219.56 MiB download, 2862.17 MiB unpacked):
2016-07-21 17:50:39 +02:00
Eelco Dolstra
6f70fcd1c5 Merge branch 'fix/remove-install' of https://github.com/siddharthist/nix 2016-07-21 15:03:16 +02:00
Langston Barrett
70a2381953 remove outdated install instructions 2016-07-21 14:56:38 +02:00
Eelco Dolstra
0f55dd5b0e Shut up warning 2016-07-21 14:25:06 +02:00
Eelco Dolstra
a728780fbd Store::computeFSClosure(): Use thread pool
This speeds up queries against the binary cache.
2016-07-21 14:13:35 +02:00
Eelco Dolstra
db459458c3 Merge pull request #961 from OptiverTimAll/fix-spec-file
Fix spec file
2016-07-21 11:33:41 +02:00
Eelco Dolstra
77c2739c25 Grammar/spelling 2016-07-21 11:30:01 +02:00
Matthias Beyer
2ec9d2fb02 NIX_REMOTE_SYSTEMS path must be absolute (#957)
Once upon a time, I wrote my bachelors thesis about functional
deployment mechanisms.

I had to evaluate several szenarios where package management and
deployment were relevant. One szenario was to do distributed builds
over several machines.

I told myself: Weee, nix can do this! And with nix, this is actually
save, as you do not have side effects when building!

So I started. I use a cloud to set up four virtual machines where I
wanted to do the build. A fifth machine was used as master to distribute
the builds. All was good.

I created the necessary SSH keys, made sure every machine was reachable
by the master and configured the build in my remotes.conf.

When I started to try to build weechat from source, the build failed. It
failed, telling me

    error: unable to start any build; either increase ‘--max-jobs’ or enable distributed builds

And I started to dig around. I digged long and good. But I wasn't able
to find the issue.

I double and triple checked my environment variables, my settings, the
SSH key and everything.

I reached out to fellow Nixers by asking on the nixos IRC channel. And I
got help. But we weren't able to find the issue, either.

So I became frustrated. I re-did all the environment variables.

And suddenly,... it worked! What did I change? Well... I made the
environment variables which contained pathes contain absolute pathes
rather than relatives.

And because I like to share my knowledge, this should be put into the
documentation, so others do not bang their heads against the wall
because something is not documented somewhere.
2016-07-21 11:19:21 +02:00
Eelco Dolstra
2d5a99baf0 Capitalize Nix 2016-07-21 11:08:13 +02:00
Marc Scholten
3614d55aa1 Add link to hacking section of the manual (#973) 2016-07-20 23:17:55 +02:00
Domen Kožar
c660eb61f1 Dockerfile: specify NIX_PATH 2016-07-20 16:58:01 +02:00
Dmitry Kalinkin
2a788bf9e7 documentation: mention required openssl dependency (#974)
Fixes: a6ca68a7 ("Require OpenSSL")
2016-07-20 15:02:29 +02:00
Shea Levy
06068b353d FdSink: Restore move constructor/assignment 2016-07-13 06:27:41 -04:00
Shea Levy
b33e85229d Make Buffered{Source,Sink} move-safe 2016-07-13 06:03:37 -04:00
Shea Levy
cb5e7254b6 Modernize AutoCloseFD 2016-07-11 15:44:44 -04:00
Eelco Dolstra
8a41792d43 Merge pull request #959 from aszlig/fix-docbook-xsl-ns
Don't hardcode docbook XSL namespace URL
2016-07-04 21:26:29 +02:00
aszlig
72fb2ccfa1
Don't hardcode docbook XSL namespace URL
Docbook XSL got updated to version 1.79.1 in NixOS/nixpkgs@fb893a8 and
we're still referring to the hardcoded previous version.

So instead of just updating this to 1.79.1 we're going to use "current"
in the hope that this won't happen again.

I have tested this by building the manual under Nix(OS) but I haven't
tested this in a non-Nix environment, so I'm not sure whether this could
have implications.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
Cc: @edolstra
2016-07-04 20:13:20 +02:00
Timothy Allen
91bec54016 nix.spec.in: RHEL 7/CentOS 7 supports systemd. 2016-07-01 14:09:45 +10:00
Timothy Allen
3a535abbc7 nix.spec.in: Create /nix/store at install time.
Since we now chmod /nix/store at install time, we don't need to do it in the
post install script. We still chgrp in the post-install, because the nixbld
group doesn't exist at install time.
2016-07-01 14:09:42 +10:00
Timothy Allen
0610486d61 nix.spec.in: RHEL 7/CentOS 7 do not require a group for each package. 2016-07-01 14:09:40 +10:00
Timothy Allen
f32c6ed873 nix.spec.in: Build from the .tar.xz file.
The Nix download page only mentions the .xz source tarball, so that's what
people are likely to have available. This means that somebody who downloads a
Nix source tarball can turn it directly into an RPM with `rpmbuild -ta
nix-*.tar.xz`.
2016-07-01 14:09:37 +10:00
Timothy Allen
85ec6a6a44 nix.spec.in: Use the default build group name.
Nix expects build users to be in the "nixbld" group. You can change that in the
config file, but `nix.spec` does not ship with a config file, so we should
use the defaults.
2016-07-01 14:09:33 +10:00
Eelco Dolstra
74dd603495 Re-implement negative binary cache lookup caching 2016-06-20 17:39:05 +02:00
Eelco Dolstra
fffacd7c78 Fix manual build
http://hydra.nixos.org/build/36944270
2016-06-20 16:55:22 +02:00
Eelco Dolstra
f94a804ced Restore instructions on how to install a binary tarball
Looks like these were accidentally commented out in
9ffc4f4363.
2016-06-17 15:26:10 +02:00
Eelco Dolstra
e95f3c4443 Fix test
http://hydra.nixos.org/build/36631898
2016-06-17 15:26:10 +02:00
Eelco Dolstra
6a5f9c86f6 Merge pull request #939 from discordianfish/patch-1
Remove nix-copy-closure reference note from nix-store docs
2016-06-16 10:25:45 +02:00
Johannes 'fish' Ziemke
df7e9b6d68 Remove nix-copy-closure reference note from nix-store docs
nix-copy-closure is not using nix-store directly anymore.
2016-06-15 15:58:13 +02:00
Eelco Dolstra
c68e5913c7 Run builds in a user namespace
This way, all builds appear to have a uid/gid of 0 inside the
chroot. In the future, this may allow using programs like
systemd-nspawn inside builds, but that will require assigning a larger
UID/GID map to the build.

Issue #625.
2016-06-09 18:30:39 +02:00
Eelco Dolstra
202683a4fc Use O_CLOEXEC in most places 2016-06-09 16:37:08 +02:00
Eelco Dolstra
9bdd949cfd Fix "creating statement: table ValidPaths has no column named ultimate" 2016-06-03 16:06:42 +02:00
Eelco Dolstra
5e51ffb1c2 Support sandbox builds by non-root users
This allows an unprivileged user to perform builds on a diverted store
(i.e. where the physical store location differs from the logical
location).

Example:

  $ NIX_LOG_DIR=/tmp/log NIX_REMOTE="local?real=/tmp/store&state=/tmp/var" nix-build -E \
    'with import <nixpkgs> {}; runCommand "foo" { buildInputs = [procps nettools]; } "id; ps; ifconfig; echo $out > $out"'

will do a build in the Nix store physically in /tmp/store but
logically in /nix/store (and thus using substituters for the latter).
2016-06-03 15:59:28 +02:00
Eelco Dolstra
2f8b0e557b Consistent quotes 2016-06-03 14:03:09 +02:00
Eelco Dolstra
080d607975 Don't call shared_from_this() from a destructor 2016-06-03 14:02:44 +02:00
Eelco Dolstra
30c8c3ba2d Fix some more references to storeDir 2016-06-03 14:02:44 +02:00
Eelco Dolstra
a8dfdc52b8 Merge pull request #925 from abbradar/master
Show both cycle ends
2016-06-03 13:58:51 +02:00
Nikolay Amiantov
c87a56f4d0 Show both cycle ends 2016-06-03 14:25:37 +03:00
Eelco Dolstra
75361b6dce Fix narrowing conversion from int64_t to bool
http://hydra.nixos.org/build/36613774
2016-06-03 13:20:46 +02:00
Eelco Dolstra
f8a8b4d8f8 nix run: Set a reasonable uid/gid 2016-06-02 19:04:09 +02:00
Eelco Dolstra
d64e0c1b64 Make derivationFromPath work on diverted stores 2016-06-02 18:43:36 +02:00
Eelco Dolstra
eda2aaae92 nix run: Handle the case where the /nix/store mount point doesn't exist 2016-06-02 18:24:51 +02:00
Eelco Dolstra
a424ab0444 createTempDir(): Don't do a chown on Linux
It's not needed and can cause problems in a user namespace.
2016-06-02 18:17:30 +02:00
Eelco Dolstra
a24f2c9b84 nix run: Mount the Nix store in a private namespace
This is a convenience command to allow users who are not privileged to
create /nix/store to use Nix with regular binary caches. For example,

  $ NIX_REMOTE="local?state=$HOME/nix/var&real=/$HOME/nix/store" nix run firefox bashInteractive

will download Firefox and bash from cache.nixos.org, then start a
shell in which $HOME/nix/store is mounted on /nix/store.
2016-06-02 16:51:43 +02:00
Eelco Dolstra
6f2d51287c Add basic "nix run" command 2016-06-02 16:29:49 +02:00
Eelco Dolstra
4494000e04 LocalStore: Allow the physical and logical store directories to differ
This is primarily to subsume the functionality of the
copy-from-other-stores substituter. For example, in the NixOS
installer, we can now do (assuming we're in the target chroot, and the
Nix store of the installation CD is bind-mounted on /tmp/nix):

  $ nix-build ... --option substituters 'local?state=/tmp/nix/var&real=/tmp/nix/store'

However, unlike copy-from-other-stores, this also allows write access
to such a store. One application might be fetching substitutes for
/nix/store in a situation where the user doesn't have sufficient
privileges to create /nix, e.g.:

  $ NIX_REMOTE="local?state=/home/alice/nix/var&real=/home/alice/nix/store" nix-build ...
2016-06-02 16:02:48 +02:00
Eelco Dolstra
064816ab98 Respect build-use-substitutes 2016-06-02 16:02:48 +02:00
Eelco Dolstra
812c0dfbe2 Allow setting the state directory as a store parameter
E.g. "local?store=/tmp/store&state=/tmp/var".
2016-06-02 16:02:48 +02:00
Eelco Dolstra
f2682e6e18 <nix/fetchurl.nix>: Use tarballs.nixos.org
This ensures that the disappearance of upstream bootstrap tarballs
(e.g. https://github.com/NixOS/nixpkgs-channels/pull/1) doesn't break
stdenv rebuilds.
2016-06-01 17:11:51 +02:00
Eelco Dolstra
dead8189ac Typo 2016-06-01 16:56:22 +02:00