Our use of boost::coroutine2 depends on -lboost_context,
which in turn depends on `-lboost_thread`, which in turn depends
on `-lboost_system`.
I suspect that this builds on nix only because of low-level hacks
like NIX_LDFLAGS.
This commit passes the proper linker flags, thus fixing bootstrap
builds on non-nix distributions like Ubuntu 16.04.
With these changes, I can build Nix on Ubuntu 16.04 using:
./bootstrap.sh
./configure --prefix=$HOME/editline-prefix \
--disable-doc-gen \
CXX=g++-7 \
--with-boost=$HOME/boost-prefix \
EDITLINE_CFLAGS=-I$HOME/editline-prefix/include \
EDITLINE_LIBS=-leditline \
LDFLAGS=-L$HOME/editline-prefix/lib
make
where
* g++-7 comes from gcc-7 from
https://launchpad.net/~ubuntu-toolchain-r/+archive/ubuntu/test,
* editline 1.14 from https://github.com/troglobit/editline/releases/tag/1.14.0
was installed into `$HOME/editline-prefix`
(because Ubuntu 16.04's `editline` is too old to have the function nix uses),
* boost 1.66 from
https://www.boost.org/doc/libs/1_66_0/more/getting_started/unix-variants.html
was installed into $HOME/boost-prefix (because Ubuntu 16.04 only has 1.58)
That was incorrect, because checking the dirent type already requires
a working compiler.
It had the effect that setting e.g. `: ${CFLAGS=""}` before `AC_PROG_CC`
as per `AC_PROG_CC`'s documentation would have no effect, because
`AC_STRUCT_DIRENT_D_TYPE` would automatically set CFLASGS.
(In a followup commit `: ${CFLAGS=""}` will be used, so it's important
to get this working first.)
And probably many other distributions.
Until now, ./configure would fail silently printing a warning
./configure: line 4621: AX_CXX_COMPILE_STDCXX_17: command not found
and then continuing, later failing with a C++ #error saying that some C++11
feature isn't supported (it didn't even get to the C++17 features).
This is because older distributions don't come with the
`AX_CXX_COMPILE_STDCXX_17` m4 macro.
This commit vendors that macro accordingly.
Now ./configure complains correctly:
configure: error: *** A compiler with support for C++17 language features is required.
On Ubuntu 16.04, ./configure completes if a newer compiler is used, e.g. with
gcc-7 from https://launchpad.net/~ubuntu-toolchain-r/+archive/ubuntu/test
using:
./bootstrap.sh
./configure CXX=g++-7 --disable-doc-gen --with-boost=$(nix-build --no-link '<nixpkgs>' -A boost.dev)
And probably other Linux distributions with long-term support releases.
Also update manual stating what version is needed;
I checked that 1.14 is the oldest version with which current nix compiles,
and added autoconf feature checks for some functions added in that release
that nix uses.
This turns previous compiler errors complaining about missing files
into proper ./configure time errors telling the user which version
of boost is required.
The unbalanced single-quotes cause many editor syntax highlighters
to interpret the rest of the file as a string literal, making it easier
to make syntax mistakes in absence of proper highlighting.
"configure.ac" is used to generate "configure", which is supposed to be
run with /bin/sh (as suggested by the shebang line of "configure"). As a
result it is a good idea to remove any /bin/sh-incompatible syntax from
configure.ac. Otherwise, systems that do not use Bash as their /bin/sh
are unable to run "configure" due to syntax errors.
Happily the failing tests should prevent anyone from using such a Nix
in situations where they expect sandboxing to be on,
which would otherwise be a risk.
* Look for both 'brotli' and 'bro' as external command,
since upstream has renamed it in newer versions.
If neither are found, current runtime behavior
is preserved: try to find 'bro' on PATH.
* Limit amount handed to BrotliEncoderCompressStream
to ensure interrupts are processed in a timely manner.
Testing shows negligible performance impact.
(Other compression sinks don't seem to require this)
This prevents builders from setting the S_ISUID or S_ISGID bits,
preventing users from using a nixbld* user to create a setuid/setgid
binary to interfere with subsequent builds under the same nixbld* uid.
This is based on aszlig's seccomp code
(47f587700d).
Reported by Linus Heckemann.
And add a 116 KiB ash shell from busybox to the release build. This
helps to make sandbox builds work out of the box on non-NixOS systems
and with diverted stores.
Build logs on cache.nixos.org are compressed using Brotli (since this
allows them to be decompressed automatically by Chrome and Firefox),
so it's handy if "nix log" can decompress them.
We're going to use libseccomp instead of creating the raw BPF program,
because we have different syscall numbers on different architectures.
Although our initial seccomp rules will be quite small it really doesn't
make sense to generate the raw BPF program because we need to duplicate
it and/or make branches on every single architecture we want to suuport.
Signed-off-by: aszlig <aszlig@redmoonstudios.org>
Alpine seems to use this, and it results in a wrong
builtins.currentSystem. Big-endian ARM systems have triples starting
with armv6eb- or armv7eb-, so this doesn't change any systems that
already worked.
This reverts commit 80ebd60e7c. The
reason why we cleared CFLAGS/CXXFLAGS was because otherwise we get a
default value of -O2, which interferes with the defaults set in the
Makefile. (E.g. "make OPTIMIZE=0" should not pass -O2.)
chroot only changes the process root directory, not the mount namespace root
directory, and it is well-known that any process with chroot capability can
break out of a chroot "jail". By using pivot_root as well, and unmounting the
original mount namespace root directory, breaking out becomes impossible.
Non-root processes typically have no ability to use chroot() anyway, but they
can gain that capability through the use of clone() or unshare(). For security
reasons, these syscalls are limited in functionality when used inside a normal
chroot environment. Using pivot_root() this way does allow those syscalls to be
put to their full use.
Sodium's Ed25519 signatures are much shorter than OpenSSL's RSA
signatures. Public keys are also much shorter, so they're now
specified directly in the nix.conf option ‘binary-cache-public-keys’.
The new command ‘nix-store --generate-binary-cache-key’ generates and
prints a public and secret key.
On a system with multiple CPUs, running Nix operations through the
daemon is significantly slower than "direct" mode:
$ NIX_REMOTE= nix-instantiate '<nixos>' -A system
real 0m0.974s
user 0m0.875s
sys 0m0.088s
$ NIX_REMOTE=daemon nix-instantiate '<nixos>' -A system
real 0m2.118s
user 0m1.463s
sys 0m0.218s
The main reason seems to be that the client and the worker get moved
to a different CPU after every call to the worker. This patch adds a
hack to lock them to the same CPU. With this, the overhead of going
through the daemon is very small:
$ NIX_REMOTE=daemon nix-instantiate '<nixos>' -A system
real 0m1.074s
user 0m0.809s
sys 0m0.098s
/nix/store could be a read-only bind mount even if it is / in its own filesystem, so checking the 4th field in mountinfo is insufficient.
Signed-off-by: Shea Levy <shea@shealevy.com>
vfork() is just too weird. For instance, in this build:
http://hydra.nixos.org/build/3330487
the value fromHook.writeSide becomes corrupted in the parent, even
though the child only reads from it. At -O0 the problem goes away.
Probably the child is overriding some spilled temporary variable.
If I get bored I may implement using posix_spawn() instead.
XZ compresses significantly better than bzip2. Here are the
compression ratios and execution times (using 4 cores in parallel) on
my /var/run/current-system (3.1 GiB):
bzip2: total compressed size 849.56 MiB, 30.8% [2m08]
xz -6: total compressed size 641.84 MiB, 23.4% [6m53]
xz -7: total compressed size 621.82 MiB, 22.6% [7m19]
xz -8: total compressed size 599.33 MiB, 21.8% [7m18]
xz -9: total compressed size 588.18 MiB, 21.4% [7m40]
Note that compression takes much longer. More importantly, however,
decompression is much faster:
bzip2: 1m47.274s
xz -6: 0m55.446s
xz -7: 0m54.119s
xz -8: 0m52.388s
xz -9: 0m51.842s
The only downside to using -9 is that decompression takes a fair
amount (~65 MB) of memory.
Nix needs SQLite's foreign key constraint feature, which was
introduced in 3.6.19. Without it, the database won't be cleaned up
correctly when paths are deleted. See
e.g. http://hydra.nixos.org/build/2494142.
Nix now requires SQLite and bzip2 to be pre-installed. SQLite is
detected using pkg-config. We required DBD::SQLite anyway, so
depending on SQLite is not a big problem.
The --with-bzip2, --with-openssl and --with-sqlite flags are gone.
I was bitten one time too many by Python modifying the Nix store by
creating *.pyc files when run as root. On Linux, we can prevent this
by setting the immutable bit on files and directories (as in ‘chattr
+i’). This isn't supported by all filesystems, so it's not an error
if setting the bit fails. The immutable bit is cleared by the garbage
collector before deleting a path. The only tricky aspect is in
optimiseStore(), since it's forbidden to create hard links to an
immutable file. Thus optimiseStore() temporarily clears the immutable
bit before creating the link.
scripts.
* Include the version and architecture in the -I flag so that there is
at least a chance that a Nix binary built for one Perl version will
run on another version.
bindings to be used in Nix's own Perl scripts.
The only downside is that Perl XS and Automake/libtool don't really
like each other, so building is a bit tricky.
little RAM. Even if the memory isn't actually used, it can cause
problems with the overcommit heuristics in the kernel. So use a VM
space of 25% of RAM, up to 384 MB.
because Berkeley DB needed it on some platforms, but we don't use
BDB anymore.
On FreeBSD, if you link against pthreads, then the main thread gets
a 2 MB stack which cannot be overriden (it ignores "ulimit -s"):
http://www.mail-archive.com/freebsd-hackers@freebsd.org/msg62445.html
This is not enough for Nix. For instance, the garbage collector can
fail if there is a pathologically deep chain of references
(http://hydra.nixos.org/build/556199). 2 MB is also not enough for
many Nix expressions.
Arguably the garbage collector shouldn't use recursion, because in
NixOS unprivileged users can DOS the garbage collector by creating a
sufficiently deeply nested chain of references. But getting rid of
recursion is a bit harder.
faster than the old mode when fsyncs are enabled, because it only
performs an fsync() when doing a checkpoint, rather than at every
commit. Some timings for doing a "nix-instantiate /etc/nixos/nixos
-A system" after modifying the stdenv setup script:
42.5s - SQLite 3.6.23 with truncate mode and fsync
3.4s - SQLite 3.6.23 with truncate mode and no fsync
32.1s - SQLite 3.7.0 with truncate mode and fsync
16.8s - SQLite 3.7.0 with WAL mode and fsync, auto-checkpoint
every 1000 pages
8.3s - SQLite 3.7.0 with WAL mode and fsync, auto-checkpoint
every 8192 pages
1.7s - SQLite 3.7.0 with WAL mode and no fsync
The default is now to use WAL mode with fsyncs. Because WAL doesn't
work on remote filesystems such as NFS (as it uses shared memory),
truncate mode can be re-enabled by setting the "use-sqlite-wal"
option to false.
