Commit graph

9141 commits

Author SHA1 Message Date
Eelco Dolstra
fea7d3b1cd
Merge pull request #8681 from inclyc/libexpr/parser-move-noeffect
libexpr: remove std::move() for `basePath` in parser, it has no effect
2023-07-10 13:30:42 +02:00
Bader AlAttar
3fa0266e7a
Fix some grammar in installables doc (#8682) 2023-07-10 09:33:04 +00:00
Yingchi Long
3d74e7b811 libexpr: remove std::move() for basePath in parser, it has no effect 2023-07-10 12:02:29 +08:00
John Ericson
028b26a77f
Merge pull request #8370 from hercules-ci/fetchClosure-input-addressed
`fetchClosure`: input addressed and pure
2023-07-09 23:41:22 -04:00
John Ericson
be518e73ae Clean up SearchPath
- Better types

- Own header / C++ file pair

- Test factored out methods

- Pass parsed thing around more than strings

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-07-09 23:22:22 -04:00
John Ericson
87dcd09047 Clean up resolveSearchPathElem
We should use `std::optional<std::string>` not `std::pair<bool,
std::string>` for an optional string.
2023-07-09 23:13:30 -04:00
John Ericson
2add230939 Fix build 2023-07-09 21:53:06 -04:00
John Ericson
28398e6d02 Merge remote-tracking branch 'upstream/master' into overlayfs-store 2023-07-09 20:30:23 -04:00
John Ericson
903700c5e1 Simplify ContentAddress
Whereas `ContentAddressWithReferences` is a sum type complex because different
varieties support different notions of reference, and
`ContentAddressMethod` is a nested enum to support that,
`ContentAddress` can be a simple pair of a method and hash.

`ContentAddress` does not need to be a sum type on the outside because
the choice of method doesn't effect what type of hashes we can use.

Co-Authored-By: Cale Gibbard <cgibbard@gmail.com>
2023-07-07 07:30:01 -04:00
Robert Hensing
b4b02d084f fetchClosure: Interleave the examples in the docs 2023-07-07 11:40:40 +02:00
Robert Hensing
537e8beb77
fetchClosure: Apply suggestions from code review
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-07-07 11:00:40 +02:00
Eelco Dolstra
6db66ebfc5
Merge pull request #8631 from iFreilicht/profile-list
Profile list improvements
2023-07-05 16:06:17 +02:00
Théophane Hufschmitt
82d6699976
Document the path flakeref format (#8640)
* Document the path flakeref format

Fix https://github.com/NixOS/nix/issues/8482

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-07-04 08:39:08 +00:00
Eelco Dolstra
a353412c43 nix profile list: Add --json flag
This just dumps the profile manifest to stdout.
2023-07-03 12:23:57 +02:00
Eelco Dolstra
b8e8f27159 Rename 'resolvedRef' to 'lockedRef'
'resolvedRef' was incorrect, since a resolved ref is one after
registry resolution, which may still be unlocked (e.g. 'nixpkgs' ->
'github:NixOS/nixpkgs').
2023-07-03 12:23:57 +02:00
Yingchi Long
3c90340fe6 libexpr: use thread_local to make the parser thread-safe
If we call `adjustLoc`, the global variable `prev_yylloc` is shared
between threads and racy.

Currently, nix itself does not concurrently parsing files, but this is
helpful for libexpr users. (The parser is thread-safe except this.)
2023-07-03 16:05:43 +08:00
Eelco Dolstra
87b82db881 nix profile list: Improve readability of the output 2023-07-02 16:17:09 +02:00
Robert Hensing
7b39a388b3
Merge pull request #8566 from inclyc/nixd/value-print-depth
libexpr: extend `Value::print` to allow limited depth
2023-07-01 20:08:52 +02:00
Robert Hensing
40052c7613 fetchClosure: Docs and error message improvements
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-06-30 18:23:42 +02:00
Robert Hensing
50de11d662 doc: Improve fetchClosure documentation 2023-06-30 18:23:24 +02:00
Robert Hensing
dc79636007 fetchClosure: Refactor: replace enableRewriting
A single variable is nice and self-contained.
2023-06-30 18:22:47 +02:00
Robert Hensing
5bdca46117 fetchClosure: Split into three cases 2023-06-30 18:22:47 +02:00
Robert Hensing
55888633dd makeContentAddressed: Add single path helper 2023-06-30 18:22:47 +02:00
Robert Hensing
8dca95386c fetchClosure: Disallow toPath for inputAddressed = true 2023-06-30 18:22:47 +02:00
Robert Hensing
508aa58e67 fetchClosure: Always check that inputAddressed matches the result 2023-06-30 18:22:47 +02:00
Robert Hensing
ea30f152b7 fetchClosure: Allow input addressed paths in pure mode
When explicitly requested by the caller, as suggested in the meeting
(https://github.com/NixOS/nix/pull/8090#issuecomment-1531139324)

> @edolstra: { toPath } vs { fromPath } is too implicit

I've opted for the `inputAddressed = true` requirement, because it
we did not agree on renaming the path attributes.

> @roberth: more explicit
> @edolstra: except for the direction; not immediately clear in which direction the rewriting happens

This is in fact the most explicit syntax and a bit redundant, which is
good, because that redundancy lets us deliver an error message that
reminds expression authors that CA provides a better experience to
their users.
2023-06-30 18:22:43 +02:00
Robert Hensing
7e5b6d2c45 fetchClosure: Refactor: rename toCA -> enableRewriting 2023-06-30 18:19:56 +02:00
Robert Hensing
0f6d596df5 fetchClosure: Factor out attribute hint 2023-06-30 18:19:56 +02:00
Yueh-Shun Li
eebfe989a5 linkOrCopy: Fallback upon cross-device link error (EXDEV)
Fix building derivations in local chroot store on OpenAFS,
where hard linking accross directories causes cross-device link error
(EXDEV).
2023-06-30 21:12:26 +08:00
Eelco Dolstra
a0c617348b
Merge pull request #8589 from jfroche/sign-paths-as-allowed-user
Allow to sign path as unprivileged user
2023-06-30 13:13:42 +02:00
Robert Hensing
1632f08ea2
Merge pull request #8600 from inclyc/libexpr/fix-leaking-in-stripIndentation
libexpr: fix leaking `es2` in stripIndentation (parser.y)
2023-06-29 11:31:53 +02:00
Yingchi Long
3468cbaf47 libexpr: fix leaking es2 in stripIndentation (parser.y) 2023-06-28 22:38:44 +08:00
Jean-François Roche
80c9259756 Allow to sign path as unprivileged user
User can now sign path as unprivileged/allowed user

refs #1708
2023-06-27 18:31:31 +02:00
John Ericson
22b278e011 Automatically document builtin constants
This is done in roughly the same way builtin functions are documented.

Also auto-link experimental features for primops, subsuming PR #8371.

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-06-27 09:37:54 -04:00
John Ericson
e8067daf09 Generialize showType 2023-06-27 09:11:42 -04:00
Maximilian Bosch
559fd7ffe7
nix flake check: improve error message if overlay is not a lambda (#8582)
* nix flake check: improve error message if overlay is not a lambda

Suppose you have an overlay like this

    {
      inputs = { /* ... */ };
      outputs = { flake-utils, ... }: flake-utils.lib.eachDefaultSystem
        (system: {
          overlays.default = final: prev: {

          };
        });
    }

then `nix flake check` (correctly) fails because `overlays` are supposed
to have the structure `overlays.<name> = final: prev: exp`. However, the
error-message is a little bit counter-intuitive:

    error: overlay does not take an argument named 'final'

While one might guess where the error actually comes from because the
trace above says `… while checking the overlay 'overlays.x86_64-linux'`
this is still pretty confusing because it complains about an argument
not being named `final` even though that's evidently the case.

With this change, the error-message actually makes it clear what's
wrong:

    [ma27@carsten:~/Projects/nix/tmp]$ nix flake check --extra-experimental-features 'nix-command flakes' path:$(pwd)
    error:
           … while checking flake output 'overlays'

             at /nix/store/clgblnxx003hyrq8qkz5ab6kgqkck6qc-source/flake.nix:4:5:

                3|   outputs = { ... }: {
                4|     overlays.x86_64-linux.snens = final: prev: {
                 |     ^
                5|       kek = throw "snens";

           … while checking the overlay 'overlays.x86_64-linux'

             at /nix/store/clgblnxx003hyrq8qkz5ab6kgqkck6qc-source/flake.nix:4:5:

                3|   outputs = { ... }: {
                4|     overlays.x86_64-linux.snens = final: prev: {
                 |     ^
                5|       kek = throw "snens";

           error: overlay is not a lambda, but a set instead
2023-06-27 14:58:29 +02:00
Eelco Dolstra
8e770dac9f
Merge pull request #8576 from obsidiansystems/findPath-cleanup
Use a struct not `std::pair` for `SearchPathElem`
2023-06-26 12:37:06 +02:00
Théophane Hufschmitt
60f06a1714
Merge pull request #5385 from Enzime/add/dirty-rev
Add `dirtyRev` and `dirtyShortRev` to `fetchGit`
2023-06-24 14:55:31 +02:00
Michael Hoang
a7b49086c7 Add dirtyRev and dirtyShortRev to fetchGit
Fixes #4682
2023-06-24 14:17:25 +10:00
John Ericson
484290a9e0 Use a struct not std::pair for SearchPathElem
I got very confused trying to keep all the `first` and `second` straight
reading the code, *especially* as there is also another `(boolean,
string)` pair type also being used.

Named fields is much better.

There are other cleanups that we can do (for example, the existing
TODO), but we can do them later. Doing them now would just make this
harder to review.
2023-06-23 12:01:10 -04:00
Robert Hensing
fd4f03b8fd
Merge pull request #8519 from fricklerhandwerk/reword-trusted-users
reword documentation on trusted users and substituters
2023-06-23 13:08:46 +02:00
Eelco Dolstra
8350f06d6c
Merge pull request #8574 from inclyc/nixd/remove-unused-token
libexpr: remove unused token `ATTRPATH` in token declaration
2023-06-23 12:02:40 +02:00
Yingchi Long
9d8c4ac446 libexpr: remove unused token ATTRPATH in token declaration 2023-06-23 13:35:41 +08:00
John Ericson
97df060588 Better document build failure exit codes
- Improved API docs from comment

- Exit codes are for `nix-build`, not just `nix-store --release`

- Make note in tests so the magic numbers are not surprising

Picking up where #8387 left off.
2023-06-22 14:29:45 -04:00
Yingchi Long
1400fde144 libexpr: extend Value::print to allow limited depth 2023-06-22 18:28:30 +08:00
Yingchi Long
c48277c1c1 libexpr: add tests for nix::Value::print 2023-06-22 18:28:30 +08:00
John Ericson
48fe0ed554
Merge pull request #8374 from obsidiansystems/improve-path-setting
Split `OptionalPathSetting` from `PathSetting`
2023-06-21 15:40:43 -04:00
John Ericson
9c0473120f Merge remote-tracking branch 'upstream/master' into overlayfs-store 2023-06-21 10:15:13 -04:00
Valentin Gagarin
e91d19db5f
be more serious about security risks with trusted users 2023-06-20 12:23:53 +02:00
Eelco Dolstra
b1352700c4
Merge pull request #8552 from edolstra/fix-eagain
GC server: Clear O_NONBLOCK on the right file descriptor
2023-06-20 12:12:53 +02:00
Ben Radford
6ae35534b7
Support opening local store with database on read-only filesystem (#8356)
Previously it was not possible to open a local store when its database is on a read-only filesystem. Obviously a store on a read-only filesystem cannot be modified, but it would still be useful to be able to query it.

This change adds a new read-only setting to LocalStore. When set to true, Nix will skip operations that fail when the database is on a read-only filesystem (acquiring big-lock, schema migration, etc), and the store database will be opened in immutable mode.

Co-authored-by: Ben Radford <benradf@users.noreply.github.com>
Co-authored-by: cidkidnix <cidkidnix@protonmail.com>
Co-authored-by: Dylan Green <67574902+cidkidnix@users.noreply.github.com>
Co-authored-by: John Ericson <git@JohnEricson.me>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-06-20 11:34:09 +02:00
Eelco Dolstra
a6a75ecad8 GC server: Clear O_NONBLOCK on the right file descriptor
The bug fix in 6d30f9e6fe erroneously
cleared O_NONBLOCK on the server rather than client FD (leaving both
in an incorrect state).

Fixes #8551.
2023-06-20 11:19:14 +02:00
John Ericson
3859cf6b21 Remove unused #include from local-derivation-goal.cc
These were never needed for this file, and date back to before this was
split from `derivation-goal.cc`.
2023-06-19 12:18:04 -04:00
John Ericson
9f69b7dee9 Create worker_proto::{Read,Write}Conn
Pass this around instead of `Source &` and `Sink &` directly. This will
give us something to put the protocol version on once the time comes.

To do this ergonomically, we need to expose `RemoteStore::Connection`,
so do that too. Give it some more API docs while we are at it.
2023-06-19 12:08:23 -04:00
John Ericson
4e8b495ad7 Likewise namespace and enum struct-ify ServeCommand
The motivation is exactly the same as for the last commit. In addition,
this anticipates us formally defining separate serialisers for the serve
protocol.
2023-06-19 12:08:23 -04:00
John Ericson
95eae0c002 Put worker protocol items inside a WorkerProto struct
See API docs on that struct for why. The pasing as as template argument
doesn't yet happen in that commit, but will instead happen in later
commit.

Also make `WorkerOp` (now `Op`) and enum struct. This led us to catch
that two operations were not handled!

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-06-19 12:08:23 -04:00
John Ericson
469d06f9bc Split out worker protocol template definitions from declarations
This is generally a fine practice: Putting implementations in headers
makes them harder to read and slows compilation. Unfortunately it is
necessary for templates, but we can ameliorate that by putting them in a
separate header. Only files which need to instantiate those templates
will need to include the header with the implementation; the rest can
just include the declaration.

This is now documenting in the contributing guide.

Also, it just happens that these polymorphic serializers are the
protocol agnostic ones. (Worker and serve protocol have the same logic
for these container types.) This means by doing this general template
cleanup, we are also getting a head start on better indicating which
code is protocol-specific and which code is shared between protocols.
2023-06-19 11:45:59 -04:00
Ben Radford
71f3bad749
Merge branch 'read-only-local-store' into overlayfs-store 2023-06-19 16:12:04 +01:00
Ben Radford
4e72b8483e
Update src/libstore/sqlite.hh
Co-authored-by: John Ericson <git@JohnEricson.me>
2023-06-19 16:01:43 +01:00
Ben Radford
0b3fa995e5
Merge branch 'master' into read-only-local-store 2023-06-19 14:09:20 +01:00
Ben Radford
ba492a98ba
Update src/libstore/local-store.hh
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-06-19 14:07:31 +01:00
Ben Radford
b09baa3bc3
Link to LocalStore section of nix3-help-stores section. 2023-06-19 13:57:10 +01:00
Ben Radford
ef40448b1c
Remove redundant description on experimental flag. 2023-06-19 13:52:32 +01:00
Ben Radford
feb8d552ae
Update src/libstore/local-store.hh
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-06-19 13:22:41 +01:00
Eelco Dolstra
f5e620bf2b
Merge pull request #8483 from edolstra/save-root
restoreMountNamespace(): Restore the original root directory
2023-06-19 12:54:05 +02:00
Eelco Dolstra
8f6255b86a
Merge pull request #8524 from amjoseph-nixpkgs/pr/doc/linkify-allowed-uris
src/libexpr/eval.hh: add link for allowed-uris option
2023-06-19 06:59:07 +02:00
John Ericson
c404623a1d
Clean up a few things related to profiles (#8526)
- Greatly expand API docs

- Clean up code in misc ways

  - Instead of a complicated single loop on generations, do different
    operations in successive subsequent steps.

  - Avoid `ref` in one place where `&` is fine

  - Just return path instead of mutating an argument in `makeName`

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-06-19 04:04:59 +00:00
Adam Joseph
6b06e97bde src/libexpr/eval.hh: add link for allowed-uris option
This commit adds a link to the documentation for `--option
allowed-uris` where that option is mentioned while describing
`restrict-eval`.
2023-06-18 23:36:32 -04:00
John Ericson
d2ce2e89b1 Split OptionalPathSetting from PathSetting
Rather than doing `allowEmpty` as boolean, have separate types and use
`std::optional`. This makes it harder to forget the possibility of an
empty path.

The `build-hook` setting was categorized as a `PathSetting`, but
actually it was split into arguments. No good! Now, it is
`Setting<Strings>` which actually reflects what it means and how it is
used.

Because of the subtyping, we now also have support for
`Setting<std::optional<String>>` in general. I imagine this can be used
to clean up many more settings also.
2023-06-18 23:31:18 -04:00
John Ericson
c8825e9d8c Create nlohmann serializers for std::optional and use
This is somewhat tricky.
2023-06-18 23:31:10 -04:00
Eelco Dolstra
e503eadafc
Merge pull request #8477 from edolstra/tarball-flake-redirects
Tarball flake improvements
2023-06-16 18:03:50 +02:00
Valentin Gagarin
f695a74751
Update src/libstore/globals.hh 2023-06-16 17:58:01 +02:00
Valentin Gagarin
baef05e6fe
fix typo
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-06-16 14:37:08 +02:00
Valentin Gagarin
126eea48e3
do not refer to trusted-users another time 2023-06-16 14:36:53 +02:00
Valentin Gagarin
1a8ca85d48
use "store URLs" consistently 2023-06-16 14:34:11 +02:00
Valentin Gagarin
4a33d5fe35
fix link text
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2023-06-16 14:33:31 +02:00
Eelco Dolstra
713836112c
Merge pull request #8517 from hercules-ci/fix-build-hook-error-for-lib-users
Fix build hook error for libstore library users
2023-06-16 13:20:50 +02:00
Théophane Hufschmitt
b2247ef4f6 Don't assume the type of string::size_type
The code accidentally conflated `std::string::size_type` and `long unsigned int`.
This was fine on 64bits machines where they are apparently the same in
practice, but not on 32bits. Fix that by using `std::string::size_type`
everywhere.
2023-06-15 21:24:14 +02:00
Valentin Gagarin
2ceacce484
Update src/libstore/globals.hh 2023-06-15 15:57:54 +02:00
Ben Radford
abb3bb7133
Merge branch 'master' into read-only-local-store 2023-06-15 13:42:14 +01:00
Ben Radford
04d8f202a7
Merge branch 'read-only-local-store' into overlayfs-store 2023-06-15 13:37:57 +01:00
Ben Radford
f5d83a8029
One line per sentence in markdown docs. 2023-06-15 13:36:28 +01:00
Ben Radford
f2fe9822c1
Comment explaining what schema version 0 means. 2023-06-15 13:34:42 +01:00
Ben Radford
4642b60afe
Update src/libstore/local-store.hh
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-06-15 13:33:26 +01:00
Ben Radford
a7b1b92d81
Update src/libstore/local-store.hh
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-06-15 13:32:56 +01:00
Ben Radford
984b01924a
Update src/libstore/local-store.cc
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-06-15 13:32:35 +01:00
Ben Radford
78e2f931d0
Update src/libstore/local-store.cc
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-06-15 13:32:16 +01:00
Robert Hensing
d2696cdd1e Fix build hook error for libstore library users
A library shouldn't require changes to the caller's argument handling,
especially if it doesn't have to, and indeed we don't have to.

This changes the lookup order to prioritize the hardcoded path to nix
if it exists. The static executable still finds itself through /proc
and the like.
2023-06-15 14:32:00 +02:00
Ben Radford
264b644c53
More detail on why read-only mode disables locking. 2023-06-15 13:22:17 +01:00
John Ericson
e1fa48f17c
Update src/nix/daemon.cc
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2023-06-15 07:41:37 -04:00
Adam Joseph
098fbf6273 src/libexpr/eval.hh: fix typo
The option name is `allowed-uris`, not `allowed-uri`.
2023-06-14 21:47:58 -07:00
Valentin Gagarin
b7d47e1d22
fix wording 2023-06-15 04:58:07 +02:00
Valentin Gagarin
e09b40e0d0 reword documentation on trusted users and substituters
this is to make it slightly easier to scan over
2023-06-15 02:19:13 +02:00
John Ericson
ca9f544160
Merge pull request #8516 from wentasah/remove-registerprimop
Remove RegisterPrimOp constructor without support for documentation
2023-06-15 01:13:46 +02:00
Valentin Gagarin
a1cf16563f
Fixup description of substituters (#8291)
Introduce what substituters "are" in the configuration option entry.
Remove arbitrary line breaks for easier editing in the future.
Link glossary some more.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: John Ericson <git@JohnEricson.me>
2023-06-14 20:49:58 +00:00
Michal Sojka
a0c4d58549 Remove RegisterPrimOp constructor without support for documentation
The remaining constructor RegisterPrimOp::RegisterPrimOp(Info && info)
allows specifying the documentation in .args and .doc members of the
Info structure.

Commit 8ec1ba0210 removed all uses of the removed constructor in the
nix binary. Here, we remove the constructor completely as well as its
use in a plugin test. According to #8515, we didn't promis to maintain
compatibility with external plugins.

Fixes #8515
2023-06-14 22:37:52 +02:00
John Ericson
05eb06a1de
Merge pull request #8490 from flox/stdin_handling
fix: Do not apply default installables when using --stdin
2023-06-14 20:41:22 +02:00
John Ericson
e377746842
Merge pull request #8491 from wentasah/builtins-doc
Document several undocumented builtin functions
2023-06-14 20:32:37 +02:00
Daniel Asaturov
468add5aa0
Remove dead code (#8504)
`filesystem.cc` is the only place where `createSymlink()` is used with three arguments:
in the definition of `replaceSymlink()` with three parameters that _is not used at all_.

Closes #8495
2023-06-14 14:09:11 -04:00
John Ericson
ff905cb796
Merge pull request #4803 from ShamrockLee/nix-channel-list-generations
Add `nix-channel --list-generations`
2023-06-14 18:30:35 +02:00