Commit graph

191 commits

Author SHA1 Message Date
Robert Hensing
d2696cdd1e Fix build hook error for libstore library users
A library shouldn't require changes to the caller's argument handling,
especially if it doesn't have to, and indeed we don't have to.

This changes the lookup order to prioritize the hardcoded path to nix
if it exists. The static executable still finds itself through /proc
and the like.
2023-06-15 14:32:00 +02:00
Yorick van Pelt
2c462486fe
create pathAccessible, use it to infer default dirs 2023-05-26 15:36:47 +02:00
Yorick van Pelt
b7cde90c6b
Call getDefaultSSLCertFile() only when none is specified
This does pathExists on various paths, which crashes on EPERM in the
macOS sandbox.
2023-05-26 15:36:45 +02:00
John Ericson
2c8475600d Fix some issues with experimental config settings
Issues:

1. Features gated on disabled experimental settings should warn and be
   ignored, not silently succeed.

2. Experimental settings in the same config "batch" (file or env var)
   as the enabling of the experimental feature should work.

3. For (2), the order should not matter.

These are analogous to the issues @roberth caught with my changes for
arg handling, but they are instead for config handling.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-04-17 12:41:04 -04:00
Robert Hensing
1c0b680ef9 libstore: Remove lockCPU dead code
Left over from 9747ea84b, https://github.com/NixOS/nix/pull/5821
2023-04-07 17:50:40 +02:00
Robert Hensing
781d3dceb3 Move initLibUtil() from initNix to initLibStore
libutil is a dependency of libstore, so it should always be
initialized as such.
libutil is also a dependency of libmain. Being explicit about this
dependency might be good, but not worth the slight code complexity
until the library structure gets more advanced.

Part of an effort to make it easier to initialize the right things,
by moving code into the appropriate libraries.
2023-04-07 16:24:18 +02:00
Robert Hensing
52d6ce6515 Move macOS TMPDIR hack from initNix to initLibStore
This code is bad. We shouldn't unset variables in programs whose
children may need them. Fixing one issue at a time, so postponing.
See https://github.com/NixOS/nix/issues/7731

Part of an effort to make it easier to initialize the right things,
by moving code into the appropriate libraries.
2023-04-07 16:24:18 +02:00
Robert Hensing
e706ffa007 Move preloadNSS() from initNix to initLibStore
It is required for the sandbox, which is a libstore responsibility;
not just libmain.

Part of an effort to make it easier to initialize the right things,
by moving code into the appropriate libraries.
2023-04-07 16:24:18 +02:00
Robert Hensing
a58be39476 Move sodium_init() to initLibStore()
Part of an effort to make it easier to initialize the right things,
by moving code into the appropriate libraries.
2023-04-07 16:24:18 +02:00
Robert Hensing
a692c43729 Move loadConfFile() to initLibStore
Part of an effort to make it easier to initialize the right things,
by moving code into the appropriate libraries.

Using libstore without loading the config file is risky, as sqlite
may then be misconfigured. See https://github.com/cachix/cachix/issues/475
2023-04-07 16:24:18 +02:00
Eelco Dolstra
5a0f5b5c34
Merge pull request #8062 from edolstra/ssl-cert-file
Add a setting for configuring the SSL certificates file
2023-03-27 15:37:15 +02:00
John Ericson
296831f641 Move enabled experimental feature to libutil struct
This is needed in subsequent commits to allow the settings and CLI args
infrastructure itself to read this setting.
2023-03-20 11:05:22 -04:00
Eelco Dolstra
e53e5c38d4 Add a setting for configuring the SSL certificates file
This provides a platform-independent way to configure the SSL
certificates file in the Nix daemon. Previously we provided
instructions for overriding the environment variable in launchd, but
that obviously doesn't work with systemd. Now we can just tell users
to add

  ssl-cert-file = /etc/ssl/my-certificate-bundle.crt

to their nix.conf.
2023-03-17 18:32:18 +01:00
Félix Baylac Jacqué
25300c0ecd
Treat empty env var paths as unset
We make sure the env var paths are actually set (ie. not "") before
sending them to the canonicalization function. If we forget to do so,
the user will end up facing a puzzled failed assertion internal error.

We issue a non-failing warning as a stop-gap measure. We could want to
revisit this to issue a detailed failing error message in the future.
2023-03-01 20:50:07 +01:00
Eelco Dolstra
14b0b9ea5a
Merge pull request #7203 from graham33/feature/cpp20
Proposal: Use C++20
2023-02-01 15:41:04 +01:00
Robert Hensing
aba6eb348e libstore: Make sure that initNix has been called
Prevent bugs like https://github.com/cachix/cachix/pull/477
2022-12-24 14:39:30 +01:00
Eelco Dolstra
2fde7e0108 Split auto UID allocation from cgroups
Cgroups are now only used for derivations that require the uid-range
range feature. This allows auto UID allocation even on systems that
don't have cgroups (like macOS).

Also, make things work on modern systems that use cgroups v2 (where
there is a single hierarchy and no "systemd" controller).
2022-11-08 16:03:42 +01:00
Graham Bennett
4563e80363 Fix C++20 warnings 2022-10-22 15:16:46 +01:00
Eelco Dolstra
285277a61a Remove useless debug statements
We haven't parsed the '-v' command line flags yet when this code executes,
so we can't actually get debug output here.
2022-10-14 00:35:33 -07:00
Eelco Dolstra
ddd5503950 Use /usr/bin/true 2022-10-14 00:34:31 -07:00
Eelco Dolstra
96eb5ef156 Improve Rosetta detection
Turns out that one of those *.plist files can exist even if Rosetta is
not installed. So let's just try to run an x86_64-darwin binary
directly.
2022-10-13 11:46:16 -07:00
Alex Wied
722de8ddcc libstore/globals.cc: Move cgroup detection to libutil 2022-07-19 16:25:53 -04:00
Alex Wied
1af5d798a4 libstore/globals.cc: Automatically set cores based on cgroup CPU limit
By default, Nix sets the "cores" setting to the number of CPUs which are
physically present on the machine. If cgroups are used to limit the CPU
and memory consumption of a large Nix build, the OOM killer may be
invoked.

For example, consider a GitLab CI pipeline which builds a large software
package. The GitLab runner spawns a container whose CPU is limited to 4
cores and whose memory is limited to 16 GiB. If the underlying machine
has 64 cores, Nix will invoke the build with -j64. In many cases, that
level of parallelism will invoke the OOM killer and the build will
completely fail.

This change sets the default value of "cores" to be
ceil(cpu_quota / cpu_period), with a fallback to
std:🧵:hardware_concurrency() if cgroups v2 is not detected.
2022-07-19 16:03:58 -04:00
Eelco Dolstra
184f4e40de Remove NIX_LIBEXEC_DIR 2022-06-23 01:32:46 +02:00
Eelco Dolstra
d3176ce076 Fix build-remote in nix-static
'build-remote' is now executed via /proc/self/exe so it always works.
2022-06-23 01:32:46 +02:00
Eelco Dolstra
df552ff53e Remove std::string alias (for real this time)
Also use std::string_view in a few more places.
2022-02-25 16:13:02 +01:00
John Ericson
286eb81143 "recursive-nix" system feature only with experimental feature 2021-11-23 20:19:04 +00:00
regnat
af99941279 Make experimental-features a proper type
Rather than having them plain strings scattered through the whole
codebase, create an enum containing all the known experimental features.

This means that
- Nix can now `warn` when an unkwown experimental feature is passed
  (making it much nicer to spot typos and spot deprecated features)
- It’s now easy to remove a feature altogether (once the feature isn’t
  experimental anymore or is dropped) by just removing the field for the
  enum and letting the compiler point us to all the now invalid usages
  of it.
2021-10-26 07:02:31 +02:00
yvt
a9d9e55551 Add another path where a Rosetta 2 configuration file is possibly located 2021-10-15 09:48:15 +09:00
Eelco Dolstra
01a4fa3b6e Advise using --extra-experimental-features instead of --experimental-features
Fixes #3737.
2021-09-14 13:56:36 +02:00
Eelco Dolstra
4638bcfb2c Fix some typos
Fixes #4671.
2021-03-26 16:14:38 +01:00
regnat
f67ff1f575 Don't crash when copying realisations to a non-ca remote
Rather throw a proper exception, and catch&log it on the client side
2021-02-25 17:19:59 +01:00
Shea Levy
98d1b64400
Initialize plugins after handling initial command line flags
This is technically a breaking change, since attempting to set plugin
files after the first non-flag argument will now throw an error. This
is acceptable given the relative lack of stability in a plugin
interface and the need to tie the knot somewhere once plugins can
actually define new subcommands.
2021-02-24 08:22:17 -05:00
Shea Levy
ec3497c1d6
Bail if plugin-files is set after plugins have been loaded.
We know the flag will be ignored but the user wants it to take effect.
2021-02-24 08:20:48 -05:00
Daniël de Kok
2de232d2b3 Add x86_64 compute levels as additional system types
When performing distributed builds of machine learning packages, it
would be nice if builders without the required SIMD instructions can
be excluded as build nodes.

Since x86_64 has accumulated a large number of different instruction
set extensions, listing all possible extensions would be unwieldy.
AMD, Intel, Red Hat, and SUSE have recently defined four different
microarchitecture levels that are now part of the x86-64 psABI
supplement and will be used in glibc 2.33:

https://gitlab.com/x86-psABIs/x86-64-ABI
https://lwn.net/Articles/844831/

This change uses libcpuid to detect CPU features and then uses them to
add the supported x86_64 levels to the additional system types. For
example on a Ryzen 3700X:

$ ~/aps/bin/nix -vv --version | grep "Additional system"
Additional system types: i686-linux, x86_64-v1-linux, x86_64-v2-linux, x86_64-v3-linux
2021-02-22 09:11:15 +01:00
Eelco Dolstra
6548b89cc4 string2Int(): Return std::optional 2021-01-08 12:22:21 +01:00
Matthew Bauer
3c9b7029ba Use com.apple.oahd.plist for rosetta 2 detection 2020-12-04 13:26:53 -06:00
Matthew Bauer
9b1824ecbd Add extraPlatforms for Rosetta 2 macOS
macOS systems with ARM64 can utilize a translation layer at
/Library/Apple/usr/libexec/oah to run x86_64 binaries. This change
makes Nix recognize that and it to "extra-platforms". Note that there
are two cases here since Nix could be built for either x86_64 or
aarch64. In either case, we can switch to the other architecture.
Unfortunately there is not a good way to prevent aarch64 binaries from
being run in x86_64 contexts or vice versa - programs can always
execute programs for the other architecture.
2020-12-03 15:41:43 -06:00
Eelco Dolstra
ff4dea63c9 Generalize extra-* settings
This removes the extra-substituters and extra-sandbox-paths settings
and instead makes every array setting extensible by setting
"extra-<name> = <value>" in the configuration file or passing
"--<name> <value>" on the command line.
2020-10-29 18:17:39 +01:00
Christian Kampka
461cf2b856
Add NIX_CONFIG env var for applying nix.conf overrides 2020-10-21 13:41:26 +02:00
Eelco Dolstra
85c8be6286 Remove static variable name clashes
This was useful for an experiment with building Nix as a single
compilation unit. It's not very useful otherwise but also doesn't
hurt...
2020-10-06 13:49:20 +02:00
Eelco Dolstra
5b107f2c5f
Merge pull request #4038 from maljub01/master
Add a nix.conf option for allowing a symlinked store
2020-09-22 13:19:22 +02:00
Cole Helbling
ba37299a03
Serialize SandboxMode enum to string for JSON
Rather than showing an integer as the default, instead show the boolean
referenced in the description.

The nix.conf.5 manpage used to show "default: 0", which is unnecessarily
opaque and confusing (doesn't 0 mean false, even though the default is
true?); now it properly shows that the default is true.
2020-09-21 10:36:45 -07:00
Eelco Dolstra
0716adaa8b abstractsettingtojson.hh -> abstract-setting-to-json.hh 2020-09-21 18:49:43 +02:00
Eelco Dolstra
340ca382c4 Don't include nlohmann/json.hpp in globals.hh
This reduces compilation time by 207s.

Issue #4045.
2020-09-21 18:47:18 +02:00
Marwan Aljubeh
f80ffeb8c9 Update the variable name accordingly 2020-09-21 17:29:08 +01:00
Marwan Aljubeh
c00e078343 Add a nix.conf option for allowing a symlinked store 2020-09-18 17:10:39 +01:00
regnat
3c525d1590 Complete the toJSON instance for Setting<T>
Don't let it just contain the value, but also the other fields of the
setting (description, aliases, etc..)
2020-09-16 13:53:08 +02:00
Eelco Dolstra
acb99f03f9
Config: Use nlohmann/json 2020-08-20 11:02:16 +02:00
regnat
223fbe644a Shorten the path to the test root
Fix a socket length failure on the OSX builders
2020-07-03 09:20:01 +02:00