Commit graph

431 commits

Author SHA1 Message Date
John Ericson
ac89bb064a Split up util.{hh,cc}
All OS and IO operations should be moved out, leaving only some misc
portable pure functions.

This is useful to avoid copious CPP when doing things like Windows and
Emscripten ports.

Newly exposed functions to break cycles:

 - `restoreSignals`
 - `updateWindowSize`
2023-11-05 12:20:02 -05:00
Eelco Dolstra
b0455e9931 Fix uninitialized variable warnings on i686-linux
https://hydra.nixos.org/build/239849607
2023-11-03 12:04:20 +01:00
John Ericson
90de958637
Merge pull request #9265 from obsidiansystems/better-parse-sink
Make `ParseSink` a bit better
2023-11-02 09:28:29 -04:00
Eelco Dolstra
1a902f5fa7 Merge FSAccessor into SourceAccessor 2023-11-01 17:09:28 +01:00
John Ericson
1093d6585f Make ParseSink a bit better
I wouldn't call it *good* yet, but this will do for now.

- `RetrieveRegularNARSink` renamed to `RegularFileSink` and moved
  accordingly because it actually has nothing to do with NARs in
  particular.

  - its `fd` field is also marked private

- `copyRecursive` introduced to dump a `SourceAccessor` into a
  `ParseSink`.

- `NullParseSink` made so `ParseSink` no longer has sketchy default
  methods.

This was done while updating #8918 to work with the new
`SourceAccessor`.
2023-11-01 02:36:43 -04:00
John Ericson
8b68bbb777
Merge pull request #6223 from obsidiansystems/worker-proto-with-version
Give `nix daemon` and `nix-store --serve` protocols separate serializers with version info
2023-10-23 09:16:23 -04:00
Eelco Dolstra
955bbe53c5
Merge pull request #9177 from edolstra/input-accessors
Backport FSInputAccessor and MemoryInputAccessor from lazy-trees
2023-10-23 11:42:04 +02:00
John Ericson
596bd469cc Move ValidPathInfo serialization code to worker-protocol.{cc.hh}
It does not belong with the data type itself.

This also materializes the fact that `copyPath` does not do any version
negotiation just just hard-codes "16".

The non-standard interface of these serializers makes it harder to test,
but this is fixed in the next commit which then adds those tests.
2023-10-20 15:20:48 -04:00
Yueh-Shun Li
e026f3e1ae treewide: Reference HashFormat members with scope
Base* -> HashFormat::Base*
2023-10-19 00:56:41 +08:00
Yueh-Shun Li
838c70f621 treewide: Rename hashBase to hashFormat
hashBase is ambiguous, since it's not about the digital bases, but about
the format of hashes. Base16, Base32 and Base64 are all character maps
for binary encoding.

Rename the enum Base to HashFormat.

Rename variables of type HashFormat from [hash]Base to hashFormat,
including CmdHashBase::hashFormat and CmdToBase::hashFormat.
2023-10-19 00:38:32 +08:00
Eelco Dolstra
ea38605d11 Introduce FSInputAccessor and use it
Backported from the lazy-trees branch. Note that this doesn't yet use
the access control features of FSInputAccessor.
2023-10-18 17:37:32 +02:00
John Ericson
3b592c880a Add infra for experimental store implemenations
This is analogous to that for experimental settings and flags that we
have also added as of late.
2023-08-02 15:46:38 -04:00
John Ericson
fe1fbdb5a1
Merge pull request #8724 from obsidiansystems/queryPartialDerivationOutputMap-evalStore
Give `queryPartialDerivationOutputMap` an `evalStore` parameter
2023-07-21 08:53:18 -04:00
John Ericson
6bc98c7fba Give queryPartialDerivationOutputMap an evalStore parameter
This makes it more useful. In general, the derivation will be in one
store, and the realisation info is in another.

This also helps us avoid duplication. See how `resolveDerivedPath` is
now simpler because it uses `queryPartialDerivationOutputMap`. In #8369
we get more flavors of derived path, and need more code to resolve them
all, and this problem only gets worse.

The fact that we need a new method to deal with the multiple dispatch is
unfortunate, but this generally relates to the fact that `Store` is a
sub-par interface, too bulky/unwieldy and conflating separate concerns.
Solving that is out of scope of this PR.

This is part of the RFC 92 work. See tracking issue #6316
2023-07-20 15:59:52 -04:00
John Ericson
903700c5e1 Simplify ContentAddress
Whereas `ContentAddressWithReferences` is a sum type complex because different
varieties support different notions of reference, and
`ContentAddressMethod` is a nested enum to support that,
`ContentAddress` can be a simple pair of a method and hash.

`ContentAddress` does not need to be a sum type on the outside because
the choice of method doesn't effect what type of hashes we can use.

Co-Authored-By: Cale Gibbard <cgibbard@gmail.com>
2023-07-07 07:30:01 -04:00
John Ericson
fd21f9d76e Merge remote-tracking branch 'upstream/master' into path-info 2023-04-07 20:39:04 -04:00
John Ericson
9383520b75 Move querySubstitutablePathInfos from LocalStore to Store
The code is not local-store-specific, so we should share it with all
stores. More uniform behavior is better, and a less store-specific
functionality is more maintainable.

This fixes a FIXME added in f73d911628 by @edolstra himself.
2023-04-02 20:32:01 -04:00
Patrick Jackson
2b905d1d35 Replace unnecessary Sync<uint64_t> with std::atomic<uint64_t> 2023-03-31 17:56:25 -07:00
Patrick Jackson
804180ad52 Only lock once 2023-03-30 18:06:27 -07:00
Patrick Jackson
309753ebb5 Fix data race in copyPaths 2023-03-30 16:59:34 -07:00
John Ericson
c51d554c93 Use "raw pattern" for content address types
We weren't because this ancient PR predated it!

This is actually a new version of the pattern which addresses some
issues identified in #7479.
2023-03-30 17:12:49 -04:00
John Ericson
aa99005004 Merge remote-tracking branch 'upstream/master' into path-info
Also improve content-address.hh API docs.
2023-03-30 16:28:53 -04:00
John Ericson
296831f641 Move enabled experimental feature to libutil struct
This is needed in subsequent commits to allow the settings and CLI args
infrastructure itself to read this setting.
2023-03-20 11:05:22 -04:00
Théophane Hufschmitt
99af56cd0d
Merge pull request #8015 from tweag/progress-during-nix-copy
Display progress when running copyPaths (nix copy)
2023-03-10 16:23:22 +01:00
Alexander Bantyev
4bef2016a1
Display progress when running copyPaths (nix copy)
`nix copy` operations did not show progress. This is quite confusing.

Add a `progressSink` which displays the progress during `copyPaths`,
pretty much copied from `copyStorePath`.

Fixes https://github.com/NixOS/nix/issues/8000
2023-03-09 16:03:48 +04:00
Eelco Dolstra
29abc8e764 Remove FormatOrString and remaining uses of format() 2023-03-02 15:57:54 +01:00
Matthew Kenigsberg
f86f2b973f Always set valid in path-info --json output
Currently the valid key is only present when the path is invalid, which
makes checking path validity more complex than it should be. With this
change, the valid key can always be used to check if a path is valid
2023-02-28 16:04:41 -07:00
John Ericson
d381248ec0 No inheritance for TextInfo and FixedOutputInfo 2023-02-28 12:14:11 -05:00
John Ericson
85bb865d20 Revert "Remove some designated initializers"
This reverts commit ee9eb83a84.
2023-02-28 11:57:20 -05:00
John Ericson
123b11ff83 Clarify store path grammar and improve comment on makeType 2023-02-28 11:49:13 -05:00
John Ericson
c36b584f8e Fix typo in the method name 2023-02-28 11:34:18 -05:00
John Ericson
87ca46263d Merge branch 'master' into path-info 2023-02-02 09:55:07 -05:00
Eelco Dolstra
57a4258426 Remove an unused capture 2023-02-01 20:27:35 +01:00
John Ericson
ee9eb83a84 Remove some designated initializers
With the switch to C++20, the rules became more strict, and we can no
longer initialize base classes. Make them comments instead.

(BTW
https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2021/p2287r1.html
this offers some new syntax for this use-case. Hopefully this will be
adopted and we can eventually use it.)
2023-02-01 11:25:56 -05:00
John Ericson
f3e272ba02 Avoid some StorePath <-> Path round trips
Avoid needless work and throwing away invariants.

These conversions date back to when `StorePath` was in Rust and there
were issues with it missing utility methods.
2023-01-30 09:37:57 -05:00
John Ericson
4540e7b940 Don't add StorePathDescriptor for now
We don't need it yet, we can add it back later.
2023-01-23 12:58:27 -05:00
John Ericson
b3d91239ae Make ValidPathInfo have plain StorePathSet references like before
This change can wait for another PR.
2023-01-14 16:42:03 -05:00
John Ericson
056cc1c1b9 Merge remote-tracking branch 'upstream/master' into path-info 2023-01-14 14:27:28 -05:00
John Ericson
a416476217 Move ValidPathInfo defintions to path-info.cc
Originally there was no `path-info.*`, then there was `path-info.hh`,
then there was `path-info.cc`, but only for new things. Moving this
stuff over makes everything consistent.
2023-01-13 15:39:19 -05:00
John Ericson
2e7be46e73 Move new ValidPathInfo methods to path-info.cc
We'll move the old ones separately, so as not to clutter the diff.
2023-01-13 15:06:07 -05:00
Théophane Hufschmitt
bdeb6de889
Merge pull request #7430 from tweag/ca/fix-nix-log
Ca/fix nix log
2023-01-13 11:00:56 +01:00
John Ericson
46e942ff9e Do big rename to clean up code
- `PathReferences` -> `References`

- `PathReferences<StorePath>` -> `StoreReference`

- `references` -> `others`

- `hasSelfReference` -> `self`

And get rid of silly subclassing
2023-01-06 15:36:05 -05:00
John Ericson
9cfa78e58a Optimize ValidPathInfo construction a bit better 2023-01-06 12:26:15 -05:00
John Ericson
6a168254ce Use named field initialization for references 2023-01-06 12:24:20 -05:00
John Ericson
e9fc1e4fdb Merge remote-tracking branch 'upstream/master' into path-info 2023-01-06 10:35:20 -05:00
Robert Hensing
aba6eb348e libstore: Make sure that initNix has been called
Prevent bugs like https://github.com/cachix/cachix/pull/477
2022-12-24 14:39:30 +01:00
Taeer Bar-Yam
3b27181ee5 fix missing function after rebase 2022-12-08 16:59:21 -05:00
regnat
04b113f6cb Fix nix log with CA derivations
Fix #6209

When trying to run `nix log <installable>`, try first to resolve the derivation pointed to
by `<installable>` as it is the resolved one that holds the build log.

This has a couple of shortcomings:
1. It’s expensive as it requires re-reading the derivation
2. It’s brittle because if the derivation doesn’t exist anymore or can’t
   be resolved (which is the case if any one of its build inputs is missing),
   then we can’t access the log anymore

However, I don’t think we can do better (at least not right now).
The alternatives I see are:
1. Copy the build log for the un-resolved derivation. But that means a
   lot of duplication
2. Store the results of the resolving in the db. Which might be the best
   long-term solution, but leads to a whole new class of potential
   issues.
2022-12-08 16:03:20 -05:00
Eelco Dolstra
703d863a48 Trivial changes from the lazy-trees branch 2022-12-07 14:06:34 +01:00
Yorick van Pelt
09f00dd4d0
Replace src/libutil/json.cc with nlohmann json generation 2022-11-16 16:50:50 +01:00