Commit graph

18669 commits

Author SHA1 Message Date
Ben Radford
33ebae75ca Reuse deletion logic for optimiseStore and rename method. 2023-07-26 16:01:20 +01:00
Ben Radford
11c493f8fa Avoid creating whiteout for duplicate store paths. 2023-07-26 16:01:20 +01:00
Ben Radford
cc6f8aa91a Test that delete works for duplicate file edge case. 2023-07-26 16:01:20 +01:00
Ben Radford
d9688ba708 Add new remount-hook store parameter. 2023-07-26 16:01:20 +01:00
John Ericson
b0877ad3c9 Give test a more specific name 2023-07-26 09:50:31 -04:00
John Ericson
07b34edc44 Fix deletion test
Lower layer references are ignored for deleting just in the upper layer.
2023-07-25 18:09:02 -04:00
John Ericson
19c43c5d78 Write test for deleting objects referenced from below
Currently fails, as expected.
2023-07-25 18:09:02 -04:00
Alex Ameen
2d1d81114d
Add parseFlakeRef and flakeRefToString builtins (#8670)
Over the last year or so I've run into several use cases where I need to
parse and/or serialize URLs for use by `builtins.fetchTree` or
`builtins.getFlake`, largely in order to produce _lockfile-like_ files
for lang2nix frameworks or tools which use `nix` internally to drive
builds.

I've gone through the painstaking process of emulating
`nix::FlakeRef::fromAttrs` and `nix::parseFlakeRef` several times with
mixed success; but these are difficult to create and even harder to
maintain if I hope to stay aligned with changes to the real
parser/serializer.

I understand why adding new `builtins` isn't something we want to do
flagrantly. I'm recommending this addition simply because I keep
encountering use cases where I need to parse/serialize these URIs in
`nix` expressions, and I want a reliable solution.

Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
Co-authored-by: John Ericson <git@JohnEricson.me>
2023-07-25 17:43:33 +00:00
John Ericson
a1a7f3156d
Merge pull request #9 from NixLayeredStore/gc
Add a GC test, fix hardlinking issue
2023-07-25 11:44:45 -04:00
John Ericson
484c820c48
Merge pull request #8741 from ncfavier/clarify-rl-notes-nested-attrs-merge
doc: clarify release notes about nested attribute merges
2023-07-25 11:11:17 -04:00
John Ericson
9ef0a9e8aa Fix hard linking issue causing overlay fs copy-ups 2023-07-25 10:28:11 -04:00
Naïm Favier
1b756e300f
doc: clarify release notes about nested attribute merges 2023-07-25 16:09:56 +02:00
John Ericson
83e703e7a8 Merge branch 'overlayfs-store' into gc 2023-07-25 09:55:31 -04:00
John Ericson
272cfd6aed Merge branch 'overlayfs-store' of github.com:NixLayeredStore/nix into overlayfs-store 2023-07-25 09:54:50 -04:00
John Ericson
73a55e4005 Merge remote-tracking branch 'layered-store/overlayfs-store' into gc 2023-07-25 09:48:06 -04:00
Ben Radford
497464f494
Extend verify test to check that repair is supported. 2023-07-25 13:33:37 +01:00
John Ericson
3731208dc1 Adopt GC test for local-overlay store
Doesn't yet pass. Fixes are needed.
2023-07-24 18:14:12 -04:00
Eelco Dolstra
ae3a7d6eba
Merge pull request #8736 from NixOS/bump-2.18.0
Bump version to 2.18
2023-07-24 22:13:09 +02:00
John Ericson
0e595a52a3 Remove trailing whitespace 2023-07-24 15:39:59 -04:00
John Ericson
2cabf85b53 Merge remote-tracking branch 'upstream/master' into overlayfs-store 2023-07-24 15:39:36 -04:00
Eelco Dolstra
0c275558e7 Bump version 2023-07-24 21:30:33 +02:00
Eelco Dolstra
e6784f0ca6
Merge pull request #8734 from NixOS/release-notes
2.17 release notes
2023-07-24 18:07:43 +02:00
Eelco Dolstra
c51be0345e Release notes 2023-07-24 17:19:31 +02:00
John Ericson
40c77f3514
Merge pull request #8243 from obsidiansystems/indirect-root-store
Refactor `Store` hierarchy with a new `IndirectRootStore` interface
2023-07-24 10:03:34 -04:00
John Ericson
60d8dd7aea Clean up store hierarchy with IndirectRootStore
See the API doc comments for details.
2023-07-24 09:19:44 -04:00
John Ericson
13269ba93b Make RemoteStore::ConnectionHandle part of class and expose
Will need to do subclass-specific implementations in the next commit.
This isn't because there will be multiple variations of the daemon
protocol (whew!) but because different clients pick and choose different
parts to use.
2023-07-24 09:17:01 -04:00
John Ericson
0a30b07277 Move Store::Params typedef to StoreConfig::Params
This is because `StoreConfig` also uses it.
2023-07-24 09:03:44 -04:00
John Ericson
4685c9b55f
Merge pull request #8664 from ncfavier/merge-dynamic-attrs
parser: merge nested dynamic attributes
2023-07-23 11:03:38 -04:00
Naïm Favier
570a1a3ad7
parser: merge nested dynamic attributes
Fixes https://github.com/NixOS/nix/issues/7115
2023-07-21 17:14:03 +02:00
John Ericson
fe1fbdb5a1
Merge pull request #8724 from obsidiansystems/queryPartialDerivationOutputMap-evalStore
Give `queryPartialDerivationOutputMap` an `evalStore` parameter
2023-07-21 08:53:18 -04:00
Eelco Dolstra
7ac24d9525
Merge pull request #8650 from obsidiansystems/content-address-simpler
Simplify `ContentAddress`
2023-07-21 13:46:53 +02:00
John Ericson
6bc98c7fba Give queryPartialDerivationOutputMap an evalStore parameter
This makes it more useful. In general, the derivation will be in one
store, and the realisation info is in another.

This also helps us avoid duplication. See how `resolveDerivedPath` is
now simpler because it uses `queryPartialDerivationOutputMap`. In #8369
we get more flavors of derived path, and need more code to resolve them
all, and this problem only gets worse.

The fact that we need a new method to deal with the multiple dispatch is
unfortunate, but this generally relates to the fact that `Store` is a
sub-par interface, too bulky/unwieldy and conflating separate concerns.
Solving that is out of scope of this PR.

This is part of the RFC 92 work. See tracking issue #6316
2023-07-20 15:59:52 -04:00
John Ericson
f62543fe1c Remove unneeded copy
It appeared in 8eb73a8724 (by me!) without
justification.
2023-07-20 15:42:06 -04:00
John Ericson
7b30293d38 Tighten #includes: DerivedPath doesn't care about Realisation 2023-07-20 15:42:06 -04:00
Valentin Gagarin
85d0eb6316
fix broken links (#8722) 2023-07-20 17:58:14 +02:00
John Ericson
ae0eb7424f
Merge pull request #8 from NixLayeredStore/overlayfs-store-more-tests
Implement deduplication and add more test cases
2023-07-20 11:55:31 -04:00
Eelco Dolstra
ab363cd494
Merge pull request #8723 from fricklerhandwerk/doc-constants
revert anchor prefix for builtin constants
2023-07-20 17:13:07 +02:00
Ben Radford
2fc00ec19f
Fix unbound variable error in optimise test. 2023-07-20 11:27:41 +01:00
Ben Radford
2c66a093e0
Define storeBRoot variable distinct from storeB URI. 2023-07-20 11:03:41 +01:00
Ben Radford
878c84d5ee
Fix errors about NIX_STORE_DIR being unset. 2023-07-20 10:42:40 +01:00
Ben Radford
9769a0ae7d
Ensure all overlay tests use new tmpfs store paths. 2023-07-20 10:42:40 +01:00
Ben Radford
7fda19e2f1
Mount tmpfs first to ensure overlayfs works consistently. 2023-07-20 10:42:40 +01:00
Ben Radford
44f855d14e
Missing addTextToStore function. 2023-07-20 10:42:39 +01:00
Ben Radford
d1c77b201a
Explicitly exec shell to fix ENOENT errors. 2023-07-20 10:42:39 +01:00
Ben Radford
8ddbcb736a
Implement overlay store deduplication. 2023-07-20 10:42:39 +01:00
Ben Radford
a9510f9502
Implement test for store path deduplication. 2023-07-20 10:42:39 +01:00
Ben Radford
614efc1240
Add test for store optimise path deduplication. 2023-07-20 10:42:39 +01:00
Ben Radford
d5cd74a401
Override verifyStore to always pass NoRepair for LocalOverlayStore. 2023-07-20 10:42:39 +01:00
Ben Radford
58085e4eff
Have verify test exercise check-contents too. 2023-07-20 10:42:38 +01:00
Ben Radford
0ccf6382af
Add test for verifying overlay store. 2023-07-20 10:42:38 +01:00