max/nix-super
1
0
Fork 0
mirror of https://github.com/privatevoid-net/nix-super.git synced 2024-11-10 08:16:15 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions
nix-super/tests/functional/fetchGit.sh

299 lines
13 KiB
Bash
Raw Normal View History

fetchGit: Add a test
2017-11-03 14:33:50 +02:00
source common.sh
Cleanup test skipping - Try not to put cryptic "99" in many places Factor out `exit 99` into `skipTest` function - Alows make sure skipping a test is done with a reason `skipTest` takes a mandatory argument - Separate pure conditionals vs side-effectful test skipping. "require daemon" already had this, but "sandbox support" did not.
2023-03-16 22:00:20 +02:00
requireGit
fetchGit: Add a test
2017-11-03 14:33:50 +02:00
clearStore
Test fetchgit with path containing a `.` segment
2022-04-07 18:31:12 +03:00
# Intentionally not in a canonical form
# See https://github.com/NixOS/nix/issues/6195
repo=$TEST_ROOT/./git
fetchGit: Add a test
2017-11-03 14:33:50 +02:00
exportGit: Don't clone local repositories This ensures that commands like 'nix flake info /my/nixpkgs' don't copy a gigabyte of crap to ~/.cache/nix. Fixes #60.
2019-04-19 15:06:27 +03:00
export _NIX_FORCE_HTTP=1
git fetcher: invoke diff instead of diff-index diff-index operates on the view that git has of the working tree, which might be outdated. The higher-level diff command does this automatically. This change also adds handling for submodules. fixes #4140 Alternative fixes would be invoking update-index before diff-index or matching more closely what require_clean_work_tree from git-sh-setup.sh does, but both those options make it more difficult to reason about correctness.
2021-12-13 22:31:20 +02:00
rm -rf $repo ${repo}-tmp $TEST_HOME/.cache/nix $TEST_ROOT/worktree $TEST_ROOT/shallow $TEST_ROOT/minimal
fetchGit: Add a test
2017-11-03 14:33:50 +02:00
git init $repo
git -C $repo config user.email "foobar@example.com"
git -C $repo config user.name "Foobar"
echo utrecht > $repo/hello
fetchGit/fetchMercurial: Fix directory inclusion check E.g. the existence of .gitignore would cause .git to be included.
2017-11-21 20:12:47 +02:00
touch $repo/.gitignore
git -C $repo add hello .gitignore
fetchGit: Add a test
2017-11-03 14:33:50 +02:00
git -C $repo commit -m 'Bla1'
rev1=$(git -C $repo rev-parse HEAD)
Add test for issue 7146
2022-10-15 01:27:13 +03:00
git -C $repo tag -a tag1 -m tag1
fetchGit: Add a test
2017-11-03 14:33:50 +02:00
echo world > $repo/hello
git -C $repo commit -m 'Bla2' -a
builtins.fetchGit: Fix build when fetching a git worktree Worktrees[1] are a feature of git which allow you to check out a ref in a different directory. While playing around with flakes I realized that git repositories in a worktree checkout break when trying to build a flake: ``` $ git worktree add ../nixpkgs-flakes nixpkgs-flakes $ cd ../nixpkgs-flakes $ nix build .#hello error: opening directory '/home/ma27/Projects/nixpkgs-flakes/.git/refs/heads': Not a directory ``` This issue has been fixed by determining with `git rev-parse --git-common-dir` where the actual `.git` directory is. Please note that this issue only exists on the `flakes` branch, fetching worktree checkouts with Nix master seems to work fine. [1] https://git-scm.com/docs/git-worktree
2019-11-15 14:04:42 +02:00
git -C $repo worktree add $TEST_ROOT/worktree
echo hello >> $TEST_ROOT/worktree/hello
fetchGit: Add a test
2017-11-03 14:33:50 +02:00
rev2=$(git -C $repo rev-parse HEAD)
Add test for issue 7146
2022-10-15 01:27:13 +03:00
git -C $repo tag -a tag2 -m tag2
fetchGit: Add a test
2017-11-03 14:33:50 +02:00
fetchToStore(): Don't always respect settings.readOnlyMode It's now up to the caller whether readOnlyMode should be applied. In some contexts (like InputScheme::fetch()), we always need to fetch.
2024-02-20 11:36:36 +02:00
# Check whether fetching in read-only mode works.
nix-instantiate --eval -E "builtins.readFile ((builtins.fetchGit file://$TEST_ROOT/worktree) + \"/hello\") == \"utrecht\\n\""
# Fetch a worktree.
builtins.fetchGit: Fix build when fetching a git worktree Worktrees[1] are a feature of git which allow you to check out a ref in a different directory. While playing around with flakes I realized that git repositories in a worktree checkout break when trying to build a flake: ``` $ git worktree add ../nixpkgs-flakes nixpkgs-flakes $ cd ../nixpkgs-flakes $ nix build .#hello error: opening directory '/home/ma27/Projects/nixpkgs-flakes/.git/refs/heads': Not a directory ``` This issue has been fixed by determining with `git rev-parse --git-common-dir` where the actual `.git` directory is. Please note that this issue only exists on the `flakes` branch, fetching worktree checkouts with Nix master seems to work fine. [1] https://git-scm.com/docs/git-worktree
2019-11-15 14:04:42 +02:00
unset _NIX_FORCE_HTTP
path0=$(nix eval --impure --raw --expr "(builtins.fetchGit file://$TEST_ROOT/worktree).outPath")
Merge legacy `fetchGit`-builtin with the generic `fetchTree`-function The original idea was to implement a git-fetcher in Nix's core that supports content hashes[1]. In #3549[2] it has been suggested to actually use `fetchTree` for this since it's a fairly generic wrapper over the new fetcher-API[3] and already supports content-hashes. This patch implements a new git-fetcher based on `fetchTree` by incorporating the following changes: * Removed the original `fetchGit`-implementation and replaced it with an alias on the `fetchTree` implementation. * Ensured that the `git`-fetcher from `libfetchers` always computes a content-hash and returns an "empty" revision on dirty trees (the latter one is needed to retain backwards-compatibility). * The hash-mismatch error in the fetcher-API exits with code 102 as it usually happens whenever a hash-mismatch is detected by Nix. * Removed the `flakes`-feature-flag: I didn't see a reason why this API is so tightly coupled to the flakes-API and at least `fetchGit` should remain usable without any feature-flags. * It's only possible to specify a `narHash` for a `git`-tree if either a `ref` or a `rev` is given[4]. * It's now possible to specify an URL without a protocol. If it's missing, `file://` is automatically added as it was the case in the original `fetchGit`-implementation. [1] https://github.com/NixOS/nix/pull/3216 [2] https://github.com/NixOS/nix/pull/3549#issuecomment-625194383 [3] https://github.com/NixOS/nix/pull/3459 [4] https://github.com/NixOS/nix/pull/3216#issuecomment-553956703
2020-04-29 23:39:58 +03:00
path0_=$(nix eval --impure --raw --expr "(builtins.fetchTree { type = \"git\"; url = file://$TEST_ROOT/worktree; }).outPath")
[[ $path0 = $path0_ ]]
fetchTree cleanup Two changes: * The (probably unintentional) hack to handle paths as tarballs has been removed. This is almost certainly not what users expect and is inconsistent with flakeref handling everywhere else. * The hack to support scp-style Git URLs has been moved to the Git fetcher, so it's now supported not just by fetchTree but by flake inputs.
2023-09-28 17:52:28 +03:00
path0_=$(nix eval --impure --raw --expr "(builtins.fetchTree git+file://$TEST_ROOT/worktree).outPath")
[[ $path0 = $path0_ ]]
builtins.fetchGit: Fix build when fetching a git worktree Worktrees[1] are a feature of git which allow you to check out a ref in a different directory. While playing around with flakes I realized that git repositories in a worktree checkout break when trying to build a flake: ``` $ git worktree add ../nixpkgs-flakes nixpkgs-flakes $ cd ../nixpkgs-flakes $ nix build .#hello error: opening directory '/home/ma27/Projects/nixpkgs-flakes/.git/refs/heads': Not a directory ``` This issue has been fixed by determining with `git rev-parse --git-common-dir` where the actual `.git` directory is. Please note that this issue only exists on the `flakes` branch, fetching worktree checkouts with Nix master seems to work fine. [1] https://git-scm.com/docs/git-worktree
2019-11-15 14:04:42 +02:00
export _NIX_FORCE_HTTP=1
[[ $(tail -n 1 $path0/hello) = "hello" ]]
fetchGit: Add a test
2017-11-03 14:33:50 +02:00
# Fetch the default branch.
nix: Add --expr flag This replaces the '(...)' installable syntax, which is not very discoverable. The downside is that you can't have multiple expressions or mix expressions and other installables.
2019-11-27 01:05:30 +02:00
path=$(nix eval --impure --raw --expr "(builtins.fetchGit file://$repo).outPath")
fetchGit: Add a test
2017-11-03 14:33:50 +02:00
[[ $(cat $path/hello) = world ]]
tests/fetchGit: test behavior of `allRefs = true;`
2020-12-22 13:18:10 +02:00
# Fetch a rev from another branch
git -C $repo checkout -b devtest
echo "different file" >> $TEST_ROOT/git/differentbranch
git -C $repo add differentbranch
git -C $repo commit -m 'Test2'
git -C $repo checkout master
devrev=$(git -C $repo rev-parse devtest)
Fetch specific Git revisions This is more efficient, and necessary when using shallow=1 with a rev.
2023-11-15 17:15:47 +02:00
nix eval --impure --raw --expr "builtins.fetchGit { url = file://$repo; rev = \"$devrev\"; }"
tests/fetchGit: test behavior of `allRefs = true;`
2020-12-22 13:18:10 +02:00
[[ $(nix eval --raw --expr "builtins.readFile (builtins.fetchGit { url = file://$repo; rev = \"$devrev\"; allRefs = true; } + \"/differentbranch\")") = 'different file' ]]
Add pure evaluation mode In this mode, the following restrictions apply: * The builtins currentTime, currentSystem and storePath throw an error. * $NIX_PATH and -I are ignored. * fetchGit and fetchMercurial require a revision hash. * fetchurl and fetchTarball require a sha256 attribute. * No file system access is allowed outside of the paths returned by fetch{Git,Mercurial,url,Tarball}. Thus 'nix build -f ./foo.nix' is not allowed. Thus, the evaluation result is completely reproducible from the command line arguments. E.g. nix build --pure-eval '( let nix = fetchGit { url = https://github.com/NixOS/nixpkgs.git; rev = "9c927de4b179a6dd210dd88d34bda8af4b575680"; }; nixpkgs = fetchGit { url = https://github.com/NixOS/nixpkgs.git; ref = "release-17.09"; rev = "66b4de79e3841530e6d9c6baf98702aa1f7124e4"; }; in (import (nix + "/release.nix") { inherit nix nixpkgs; }).build.x86_64-linux )' The goal is to enable completely reproducible and traceable evaluation. For example, a NixOS configuration could be fully described by a single Git commit hash. 'nixos-rebuild' would do something like nix build --pure-eval '( (import (fetchGit { url = file:///my-nixos-config; rev = "..."; })).system ') where the Git repository /my-nixos-config would use further fetchGit calls or Git externals to fetch Nixpkgs and whatever other dependencies it has. Either way, the commit hash would uniquely identify the NixOS configuration and allow it to reproduced.
2018-01-16 19:50:38 +02:00
# In pure eval mode, fetchGit without a revision should fail.
nix: Add --expr flag This replaces the '(...)' installable syntax, which is not very discoverable. The downside is that you can't have multiple expressions or mix expressions and other installables.
2019-11-27 01:05:30 +02:00
[[ $(nix eval --impure --raw --expr "builtins.readFile (fetchGit file://$repo + \"/hello\")") = world ]]
(! nix eval --raw --expr "builtins.readFile (fetchGit file://$repo + \"/hello\")")
Add pure evaluation mode In this mode, the following restrictions apply: * The builtins currentTime, currentSystem and storePath throw an error. * $NIX_PATH and -I are ignored. * fetchGit and fetchMercurial require a revision hash. * fetchurl and fetchTarball require a sha256 attribute. * No file system access is allowed outside of the paths returned by fetch{Git,Mercurial,url,Tarball}. Thus 'nix build -f ./foo.nix' is not allowed. Thus, the evaluation result is completely reproducible from the command line arguments. E.g. nix build --pure-eval '( let nix = fetchGit { url = https://github.com/NixOS/nixpkgs.git; rev = "9c927de4b179a6dd210dd88d34bda8af4b575680"; }; nixpkgs = fetchGit { url = https://github.com/NixOS/nixpkgs.git; ref = "release-17.09"; rev = "66b4de79e3841530e6d9c6baf98702aa1f7124e4"; }; in (import (nix + "/release.nix") { inherit nix nixpkgs; }).build.x86_64-linux )' The goal is to enable completely reproducible and traceable evaluation. For example, a NixOS configuration could be fully described by a single Git commit hash. 'nixos-rebuild' would do something like nix build --pure-eval '( (import (fetchGit { url = file:///my-nixos-config; rev = "..."; })).system ') where the Git repository /my-nixos-config would use further fetchGit calls or Git externals to fetch Nixpkgs and whatever other dependencies it has. Either way, the commit hash would uniquely identify the NixOS configuration and allow it to reproduced.
2018-01-16 19:50:38 +02:00
fetchGit: Add a test
2017-11-03 14:33:50 +02:00
# Fetch using an explicit revision hash.
nix: Add --expr flag This replaces the '(...)' installable syntax, which is not very discoverable. The downside is that you can't have multiple expressions or mix expressions and other installables.
2019-11-27 01:05:30 +02:00
path2=$(nix eval --raw --expr "(builtins.fetchGit { url = file://$repo; rev = \"$rev2\"; }).outPath")
fetchGit: Add a test
2017-11-03 14:33:50 +02:00
[[ $path = $path2 ]]
Add pure evaluation mode In this mode, the following restrictions apply: * The builtins currentTime, currentSystem and storePath throw an error. * $NIX_PATH and -I are ignored. * fetchGit and fetchMercurial require a revision hash. * fetchurl and fetchTarball require a sha256 attribute. * No file system access is allowed outside of the paths returned by fetch{Git,Mercurial,url,Tarball}. Thus 'nix build -f ./foo.nix' is not allowed. Thus, the evaluation result is completely reproducible from the command line arguments. E.g. nix build --pure-eval '( let nix = fetchGit { url = https://github.com/NixOS/nixpkgs.git; rev = "9c927de4b179a6dd210dd88d34bda8af4b575680"; }; nixpkgs = fetchGit { url = https://github.com/NixOS/nixpkgs.git; ref = "release-17.09"; rev = "66b4de79e3841530e6d9c6baf98702aa1f7124e4"; }; in (import (nix + "/release.nix") { inherit nix nixpkgs; }).build.x86_64-linux )' The goal is to enable completely reproducible and traceable evaluation. For example, a NixOS configuration could be fully described by a single Git commit hash. 'nixos-rebuild' would do something like nix build --pure-eval '( (import (fetchGit { url = file:///my-nixos-config; rev = "..."; })).system ') where the Git repository /my-nixos-config would use further fetchGit calls or Git externals to fetch Nixpkgs and whatever other dependencies it has. Either way, the commit hash would uniquely identify the NixOS configuration and allow it to reproduced.
2018-01-16 19:50:38 +02:00
# In pure eval mode, fetchGit with a revision should succeed.
nix: Add --expr flag This replaces the '(...)' installable syntax, which is not very discoverable. The downside is that you can't have multiple expressions or mix expressions and other installables.
2019-11-27 01:05:30 +02:00
[[ $(nix eval --raw --expr "builtins.readFile (fetchGit { url = file://$repo; rev = \"$rev2\"; } + \"/hello\")") = world ]]
Add pure evaluation mode In this mode, the following restrictions apply: * The builtins currentTime, currentSystem and storePath throw an error. * $NIX_PATH and -I are ignored. * fetchGit and fetchMercurial require a revision hash. * fetchurl and fetchTarball require a sha256 attribute. * No file system access is allowed outside of the paths returned by fetch{Git,Mercurial,url,Tarball}. Thus 'nix build -f ./foo.nix' is not allowed. Thus, the evaluation result is completely reproducible from the command line arguments. E.g. nix build --pure-eval '( let nix = fetchGit { url = https://github.com/NixOS/nixpkgs.git; rev = "9c927de4b179a6dd210dd88d34bda8af4b575680"; }; nixpkgs = fetchGit { url = https://github.com/NixOS/nixpkgs.git; ref = "release-17.09"; rev = "66b4de79e3841530e6d9c6baf98702aa1f7124e4"; }; in (import (nix + "/release.nix") { inherit nix nixpkgs; }).build.x86_64-linux )' The goal is to enable completely reproducible and traceable evaluation. For example, a NixOS configuration could be fully described by a single Git commit hash. 'nixos-rebuild' would do something like nix build --pure-eval '( (import (fetchGit { url = file:///my-nixos-config; rev = "..."; })).system ') where the Git repository /my-nixos-config would use further fetchGit calls or Git externals to fetch Nixpkgs and whatever other dependencies it has. Either way, the commit hash would uniquely identify the NixOS configuration and allow it to reproduced.
2018-01-16 19:50:38 +02:00
tests/functional/fetchGit.sh: Test fetchGit/fetchTree error message Follow-up for https://github.com/NixOS/nix/pull/9626 176dcd5c617367dbff6d5455856a25518326f79d
2024-01-24 14:19:02 +02:00
# But without a hash, it fails
libexpr: Support structured error classes While preparing PRs like #9753, I've had to change error messages in dozens of code paths. It would be nice if instead of EvalError("expected 'boolean' but found '%1%'", showType(v)) we could write TypeError(v, "boolean") or similar. Then, changing the error message could be a mechanical refactor with the compiler pointing out places the constructor needs to be changed, rather than the error-prone process of grepping through the codebase. Structured errors would also help prevent the "same" error from having multiple slightly different messages, and could be a first step towards error codes / an error index. This PR reworks the exception infrastructure in `libexpr` to support exception types with different constructor signatures than `BaseError`. Actually refactoring the exceptions to use structured data will come in a future PR (this one is big enough already, as it has to touch every exception in `libexpr`). The core design is in `eval-error.hh`. Generally, errors like this: state.error("'%s' is not a string", getAttrPathStr()) .debugThrow<TypeError>() are transformed like this: state.error<TypeError>("'%s' is not a string", getAttrPathStr()) .debugThrow() The type annotation has moved from `ErrorBuilder::debugThrow` to `EvalState::error`.
2024-01-23 03:08:29 +02:00
expectStderr 1 nix eval --expr 'builtins.fetchGit "file:///foo"' | grepQuiet "fetchGit requires a locked input"
tests/functional/fetchGit.sh: Test fetchGit/fetchTree error message Follow-up for https://github.com/NixOS/nix/pull/9626 176dcd5c617367dbff6d5455856a25518326f79d
2024-01-24 14:19:02 +02:00
fetchGit: Add a test
2017-11-03 14:33:50 +02:00
# Fetch again. This should be cached.
mv $repo ${repo}-tmp
nix: Add --expr flag This replaces the '(...)' installable syntax, which is not very discoverable. The downside is that you can't have multiple expressions or mix expressions and other installables.
2019-11-27 01:05:30 +02:00
path2=$(nix eval --impure --raw --expr "(builtins.fetchGit file://$repo).outPath")
fetchGit: Add a test
2017-11-03 14:33:50 +02:00
[[ $path = $path2 ]]
nix: Add --expr flag This replaces the '(...)' installable syntax, which is not very discoverable. The downside is that you can't have multiple expressions or mix expressions and other installables.
2019-11-27 01:05:30 +02:00
[[ $(nix eval --impure --expr "(builtins.fetchGit file://$repo).revCount") = 2 ]]
[[ $(nix eval --impure --raw --expr "(builtins.fetchGit file://$repo).rev") = $rev2 ]]
builtins.fetchGit: Fix shortRev attribute for dirty trees
2020-11-26 22:45:28 +02:00
[[ $(nix eval --impure --raw --expr "(builtins.fetchGit file://$repo).shortRev") = ${rev2:0:7} ]]
fetchGit: Add a test
2017-11-03 14:33:50 +02:00
# Fetching with a explicit hash should succeed.
Git: Use unified caching system
2020-03-17 23:32:26 +02:00
path2=$(nix eval --refresh --raw --expr "(builtins.fetchGit { url = file://$repo; rev = \"$rev2\"; }).outPath")
fetchGit: Add a test
2017-11-03 14:33:50 +02:00
[[ $path = $path2 ]]
Git: Use unified caching system
2020-03-17 23:32:26 +02:00
path2=$(nix eval --refresh --raw --expr "(builtins.fetchGit { url = file://$repo; rev = \"$rev1\"; }).outPath")
fetchGit: Add a test
2017-11-03 14:33:50 +02:00
[[ $(cat $path2/hello) = utrecht ]]
mv ${repo}-tmp $repo
# Using a clean working tree should produce the same result.
nix: Add --expr flag This replaces the '(...)' installable syntax, which is not very discoverable. The downside is that you can't have multiple expressions or mix expressions and other installables.
2019-11-27 01:05:30 +02:00
path2=$(nix eval --impure --raw --expr "(builtins.fetchGit $repo).outPath")
fetchGit: Add a test
2017-11-03 14:33:50 +02:00
[[ $path = $path2 ]]
# Using an unclean tree should yield the tracked but uncommitted changes.
fetchGit/fetchMercurial: Filter out directories with untracked files
2017-11-03 14:48:02 +02:00
mkdir $repo/dir1 $repo/dir2
echo foo > $repo/dir1/foo
fetchGit: Add a test
2017-11-03 14:33:50 +02:00
echo bar > $repo/bar
fetchGit/fetchMercurial: Filter out directories with untracked files
2017-11-03 14:48:02 +02:00
echo bar > $repo/dir2/bar
git -C $repo add dir1/foo
fetchGit: Add a test
2017-11-03 14:33:50 +02:00
git -C $repo rm hello
Pluggable fetchers Flakes are now fetched using an extensible mechanism. Also lots of other flake cleanups.
2020-01-21 17:27:53 +02:00
unset _NIX_FORCE_HTTP
nix: Add --expr flag This replaces the '(...)' installable syntax, which is not very discoverable. The downside is that you can't have multiple expressions or mix expressions and other installables.
2019-11-27 01:05:30 +02:00
path2=$(nix eval --impure --raw --expr "(builtins.fetchGit $repo).outPath")
fetchGit: Add a test
2017-11-03 14:33:50 +02:00
[ ! -e $path2/hello ]
[ ! -e $path2/bar ]
fetchGit/fetchMercurial: Filter out directories with untracked files
2017-11-03 14:48:02 +02:00
[ ! -e $path2/dir2/bar ]
fetchGit/fetchMercurial: Fix directory inclusion check E.g. the existence of .gitignore would cause .git to be included.
2017-11-21 20:12:47 +02:00
[ ! -e $path2/.git ]
fetchGit/fetchMercurial: Filter out directories with untracked files
2017-11-03 14:48:02 +02:00
[[ $(cat $path2/dir1/foo) = foo ]]
fetchGit: Add a test
2017-11-03 14:33:50 +02:00
Restore backwards-compat for current `builtins.fetchGit` If a repo is dirty, it used to return a `rev` object with an "empty" sha1 (0000000000000000000000000000000000000000). Please note that this only applies for `builtins.fetchGit` and *not* for `builtins.fetchTree{ type = "git"; }`.
2020-07-28 23:46:39 +03:00
[[ $(nix eval --impure --raw --expr "(builtins.fetchGit $repo).rev") = 0000000000000000000000000000000000000000 ]]
Add `dirtyRev` and `dirtyShortRev` to `fetchGit` Fixes #4682
2021-10-14 15:44:45 +03:00
[[ $(nix eval --impure --raw --expr "(builtins.fetchGit $repo).dirtyRev") = "${rev2}-dirty" ]]
[[ $(nix eval --impure --raw --expr "(builtins.fetchGit $repo).dirtyShortRev") = "${rev2:0:7}-dirty" ]]
Restore backwards-compat for current `builtins.fetchGit` If a repo is dirty, it used to return a `rev` object with an "empty" sha1 (0000000000000000000000000000000000000000). Please note that this only applies for `builtins.fetchGit` and *not* for `builtins.fetchTree{ type = "git"; }`.
2020-07-28 23:46:39 +03:00
fetchGit: Add a test
2017-11-03 14:33:50 +02:00
# ... unless we're using an explicit ref or rev.
nix: Add --expr flag This replaces the '(...)' installable syntax, which is not very discoverable. The downside is that you can't have multiple expressions or mix expressions and other installables.
2019-11-27 01:05:30 +02:00
path3=$(nix eval --impure --raw --expr "(builtins.fetchGit { url = $repo; ref = \"master\"; }).outPath")
fetchGit: Add a test
2017-11-03 14:33:50 +02:00
[[ $path = $path3 ]]
nix: Add --expr flag This replaces the '(...)' installable syntax, which is not very discoverable. The downside is that you can't have multiple expressions or mix expressions and other installables.
2019-11-27 01:05:30 +02:00
path3=$(nix eval --raw --expr "(builtins.fetchGit { url = $repo; rev = \"$rev2\"; }).outPath")
fetchGit: Add a test
2017-11-03 14:33:50 +02:00
[[ $path = $path3 ]]
# Committing should not affect the store path.
git -C $repo commit -m 'Bla3' -a
Git: Use unified caching system
2020-03-17 23:32:26 +02:00
path4=$(nix eval --impure --refresh --raw --expr "(builtins.fetchGit file://$repo).outPath")
fetchGit: Add a test
2017-11-03 14:33:50 +02:00
[[ $path2 = $path4 ]]
fetchGit: Ignore tarballTtl if rev is set and not in the repo. Fixes #1697.
2017-11-24 04:50:01 +02:00
Add `dirtyRev` and `dirtyShortRev` to `fetchGit` Fixes #4682
2021-10-14 15:44:45 +03:00
[[ $(nix eval --impure --expr "builtins.hasAttr \"rev\" (builtins.fetchGit $repo)") == "true" ]]
[[ $(nix eval --impure --expr "builtins.hasAttr \"dirtyRev\" (builtins.fetchGit $repo)") == "false" ]]
[[ $(nix eval --impure --expr "builtins.hasAttr \"dirtyShortRev\" (builtins.fetchGit $repo)") == "false" ]]
Trivial changes from the lazy-trees branch
2022-12-07 13:58:58 +02:00
status=0
Merge legacy `fetchGit`-builtin with the generic `fetchTree`-function The original idea was to implement a git-fetcher in Nix's core that supports content hashes[1]. In #3549[2] it has been suggested to actually use `fetchTree` for this since it's a fairly generic wrapper over the new fetcher-API[3] and already supports content-hashes. This patch implements a new git-fetcher based on `fetchTree` by incorporating the following changes: * Removed the original `fetchGit`-implementation and replaced it with an alias on the `fetchTree` implementation. * Ensured that the `git`-fetcher from `libfetchers` always computes a content-hash and returns an "empty" revision on dirty trees (the latter one is needed to retain backwards-compatibility). * The hash-mismatch error in the fetcher-API exits with code 102 as it usually happens whenever a hash-mismatch is detected by Nix. * Removed the `flakes`-feature-flag: I didn't see a reason why this API is so tightly coupled to the flakes-API and at least `fetchGit` should remain usable without any feature-flags. * It's only possible to specify a `narHash` for a `git`-tree if either a `ref` or a `rev` is given[4]. * It's now possible to specify an URL without a protocol. If it's missing, `file://` is automatically added as it was the case in the original `fetchGit`-implementation. [1] https://github.com/NixOS/nix/pull/3216 [2] https://github.com/NixOS/nix/pull/3549#issuecomment-625194383 [3] https://github.com/NixOS/nix/pull/3459 [4] https://github.com/NixOS/nix/pull/3216#issuecomment-553956703
2020-04-29 23:39:58 +03:00
nix eval --impure --raw --expr "(builtins.fetchGit { url = $repo; rev = \"$rev2\"; narHash = \"sha256-B5yIPHhEm0eysJKEsO7nqxprh9vcblFxpJG11gXJus1=\"; }).outPath" || status=$?
[[ "$status" = "102" ]]
path5=$(nix eval --impure --raw --expr "(builtins.fetchGit { url = $repo; rev = \"$rev2\"; narHash = \"sha256-Hr8g6AqANb3xqX28eu1XnjK/3ab8Gv6TJSnkb1LezG9=\"; }).outPath")
[[ $path = $path5 ]]
fetchGit: Ignore tarballTtl if rev is set and not in the repo. Fixes #1697.
2017-11-24 04:50:01 +02:00
# tarball-ttl should be ignored if we specify a rev
echo delft > $repo/hello
git -C $repo add hello
git -C $repo commit -m 'Bla4'
rev3=$(git -C $repo rev-parse HEAD)
nix: Add --expr flag This replaces the '(...)' installable syntax, which is not very discoverable. The downside is that you can't have multiple expressions or mix expressions and other installables.
2019-11-27 01:05:30 +02:00
nix eval --tarball-ttl 3600 --expr "builtins.fetchGit { url = $repo; rev = \"$rev3\"; }" >/dev/null
fetchGit: Fix handling of local repo when not using 'master' branch Add tests checking this behavior.
2017-12-22 22:18:29 +02:00
# Update 'path' to reflect latest master
nix: Add --expr flag This replaces the '(...)' installable syntax, which is not very discoverable. The downside is that you can't have multiple expressions or mix expressions and other installables.
2019-11-27 01:05:30 +02:00
path=$(nix eval --impure --raw --expr "(builtins.fetchGit file://$repo).outPath")
fetchGit: Fix handling of local repo when not using 'master' branch Add tests checking this behavior.
2017-12-22 22:18:29 +02:00
# Check behavior when non-master branch is used
git -C $repo checkout $rev2 -b dev
echo dev > $repo/hello
Pluggable fetchers Flakes are now fetched using an extensible mechanism. Also lots of other flake cleanups.
2020-01-21 17:27:53 +02:00
# File URI uses dirty tree unless specified otherwise
nix: Add --expr flag This replaces the '(...)' installable syntax, which is not very discoverable. The downside is that you can't have multiple expressions or mix expressions and other installables.
2019-11-27 01:05:30 +02:00
path2=$(nix eval --impure --raw --expr "(builtins.fetchGit file://$repo).outPath")
Pluggable fetchers Flakes are now fetched using an extensible mechanism. Also lots of other flake cleanups.
2020-01-21 17:27:53 +02:00
[ $(cat $path2/hello) = dev ]
fetchGit: Fix handling of local repo when not using 'master' branch Add tests checking this behavior.
2017-12-22 22:18:29 +02:00
# Using local path with branch other than 'master' should work when clean or dirty
nix: Add --expr flag This replaces the '(...)' installable syntax, which is not very discoverable. The downside is that you can't have multiple expressions or mix expressions and other installables.
2019-11-27 01:05:30 +02:00
path3=$(nix eval --impure --raw --expr "(builtins.fetchGit $repo).outPath")
fetchGit: Fix handling of local repo when not using 'master' branch Add tests checking this behavior.
2017-12-22 22:18:29 +02:00
# (check dirty-tree handling was used)
Restore backwards-compat for current `builtins.fetchGit` If a repo is dirty, it used to return a `rev` object with an "empty" sha1 (0000000000000000000000000000000000000000). Please note that this only applies for `builtins.fetchGit` and *not* for `builtins.fetchTree{ type = "git"; }`.
2020-07-28 23:46:39 +03:00
[[ $(nix eval --impure --raw --expr "(builtins.fetchGit $repo).rev") = 0000000000000000000000000000000000000000 ]]
builtins.fetchGit: Fix shortRev attribute for dirty trees
2020-11-26 22:45:28 +02:00
[[ $(nix eval --impure --raw --expr "(builtins.fetchGit $repo).shortRev") = 0000000 ]]
git fetcher: invoke diff instead of diff-index diff-index operates on the view that git has of the working tree, which might be outdated. The higher-level diff command does this automatically. This change also adds handling for submodules. fixes #4140 Alternative fixes would be invoking update-index before diff-index or matching more closely what require_clean_work_tree from git-sh-setup.sh does, but both those options make it more difficult to reason about correctness.
2021-12-13 22:31:20 +02:00
# Making a dirty tree clean again and fetching it should
# record correct revision information. See: #4140
echo world > $repo/hello
[[ $(nix eval --impure --raw --expr "(builtins.fetchGit $repo).rev") = $rev2 ]]
fetchGit: Fix handling of local repo when not using 'master' branch Add tests checking this behavior.
2017-12-22 22:18:29 +02:00
# Committing shouldn't change store path, or switch to using 'master'
git fetcher: invoke diff instead of diff-index diff-index operates on the view that git has of the working tree, which might be outdated. The higher-level diff command does this automatically. This change also adds handling for submodules. fixes #4140 Alternative fixes would be invoking update-index before diff-index or matching more closely what require_clean_work_tree from git-sh-setup.sh does, but both those options make it more difficult to reason about correctness.
2021-12-13 22:31:20 +02:00
echo dev > $repo/hello
fetchGit: Fix handling of local repo when not using 'master' branch Add tests checking this behavior.
2017-12-22 22:18:29 +02:00
git -C $repo commit -m 'Bla5' -a
nix: Add --expr flag This replaces the '(...)' installable syntax, which is not very discoverable. The downside is that you can't have multiple expressions or mix expressions and other installables.
2019-11-27 01:05:30 +02:00
path4=$(nix eval --impure --raw --expr "(builtins.fetchGit $repo).outPath")
fetchGit: Fix handling of local repo when not using 'master' branch Add tests checking this behavior.
2017-12-22 22:18:29 +02:00
[[ $(cat $path4/hello) = dev ]]
[[ $path3 = $path4 ]]
Test fetchGit with non-'master' remote repo
2021-04-20 14:52:50 +03:00
# Using remote path with branch other than 'master' should fetch the HEAD revision.
# (--tarball-ttl 0 to prevent using the cached repo above)
export _NIX_FORCE_HTTP=1
path4=$(nix eval --tarball-ttl 0 --impure --raw --expr "(builtins.fetchGit $repo).outPath")
[[ $(cat $path4/hello) = dev ]]
[[ $path3 = $path4 ]]
unset _NIX_FORCE_HTTP
fetchGit: Fix handling of local repo when not using 'master' branch Add tests checking this behavior.
2017-12-22 22:18:29 +02:00
# Confirm same as 'dev' branch
nix: Add --expr flag This replaces the '(...)' installable syntax, which is not very discoverable. The downside is that you can't have multiple expressions or mix expressions and other installables.
2019-11-27 01:05:30 +02:00
path5=$(nix eval --impure --raw --expr "(builtins.fetchGit { url = $repo; ref = \"dev\"; }).outPath")
fetchGit: Fix handling of local repo when not using 'master' branch Add tests checking this behavior.
2017-12-22 22:18:29 +02:00
[[ $path3 = $path5 ]]
fetchGit.sh: Test we don't "corrupt" cache if invoke w/o git avail
2018-01-09 16:58:19 +02:00
# Nuke the cache
Git: Use unified caching system
2020-03-17 23:32:26 +02:00
rm -rf $TEST_HOME/.cache/nix
fetchGit.sh: Test we don't "corrupt" cache if invoke w/o git avail
2018-01-09 16:58:19 +02:00
GitInputScheme: Use libgit2 This replaces most calls to the "git" binary with libgit2.
2023-10-25 19:55:08 +03:00
# Try again. This should work.
nix: Add --expr flag This replaces the '(...)' installable syntax, which is not very discoverable. The downside is that you can't have multiple expressions or mix expressions and other installables.
2019-11-27 01:05:30 +02:00
path5=$(nix eval --impure --raw --expr "(builtins.fetchGit { url = $repo; ref = \"dev\"; }).outPath")
fetchGit.sh: Test we don't "corrupt" cache if invoke w/o git avail
2018-01-09 16:58:19 +02:00
[[ $path3 = $path5 ]]
Add a test for shallow Git clones Also, don't return a revCount anymore for shallow or dirty Git trees, since it's incorrect. Closes #2988.
2020-03-16 14:20:32 +02:00
git fetcher: improve check for valid repository The .git/refs/heads directory might be empty for a valid usable git repository. This often happens in CI environments, which might only fetch commits, not branches. Therefore instead we let git itself check if HEAD points to something that looks like a commit. fixes #5302
2021-12-13 22:31:15 +02:00
# Fetching from a repo with only a specific revision and no branches should
# not fall back to copying files and record correct revision information. See: #5302
mkdir $TEST_ROOT/minimal
git -C $TEST_ROOT/minimal init
git -C $TEST_ROOT/minimal fetch $repo $rev2
git -C $TEST_ROOT/minimal checkout $rev2
[[ $(nix eval --impure --raw --expr "(builtins.fetchGit { url = $TEST_ROOT/minimal; }).rev") = $rev2 ]]
Require shallow clones to be requested explicitly If you do a fetchTree on a Git repository, whether the result contains a revCount attribute should not depend on whether that repository happens to be a shallow clone or not. That would complicate caching a lot and would be semantically messy. So applying fetchTree/fetchGit to a shallow repository is now an error unless you pass the attribute 'shallow = true'. If 'shallow = true', we don't return revCount, even if the repository is not actually shallow. Note that Nix itself is not doing shallow clones at the moment. But it could do so as an optimisation if the user specifies 'shallow = true'. Issue #2988.
2020-03-17 22:34:38 +02:00
# Fetching a shallow repo shouldn't work by default, because we can't
# return a revCount.
Add a test for shallow Git clones Also, don't return a revCount anymore for shallow or dirty Git trees, since it's incorrect. Closes #2988.
2020-03-16 14:20:32 +02:00
git clone --depth 1 file://$repo $TEST_ROOT/shallow
Require shallow clones to be requested explicitly If you do a fetchTree on a Git repository, whether the result contains a revCount attribute should not depend on whether that repository happens to be a shallow clone or not. That would complicate caching a lot and would be semantically messy. So applying fetchTree/fetchGit to a shallow repository is now an error unless you pass the attribute 'shallow = true'. If 'shallow = true', we don't return revCount, even if the repository is not actually shallow. Note that Nix itself is not doing shallow clones at the moment. But it could do so as an optimisation if the user specifies 'shallow = true'. Issue #2988.
2020-03-17 22:34:38 +02:00
(! nix eval --impure --raw --expr "(builtins.fetchGit { url = $TEST_ROOT/shallow; ref = \"dev\"; }).outPath")
# But you can request a shallow clone, which won't return a revCount.
path6=$(nix eval --impure --raw --expr "(builtins.fetchTree { type = \"git\"; url = \"file://$TEST_ROOT/shallow\"; ref = \"dev\"; shallow = true; }).outPath")
Add a test for shallow Git clones Also, don't return a revCount anymore for shallow or dirty Git trees, since it's incorrect. Closes #2988.
2020-03-16 14:20:32 +02:00
[[ $path3 = $path6 ]]
Require shallow clones to be requested explicitly If you do a fetchTree on a Git repository, whether the result contains a revCount attribute should not depend on whether that repository happens to be a shallow clone or not. That would complicate caching a lot and would be semantically messy. So applying fetchTree/fetchGit to a shallow repository is now an error unless you pass the attribute 'shallow = true'. If 'shallow = true', we don't return revCount, even if the repository is not actually shallow. Note that Nix itself is not doing shallow clones at the moment. But it could do so as an optimisation if the user specifies 'shallow = true'. Issue #2988.
2020-03-17 22:34:38 +02:00
[[ $(nix eval --impure --expr "(builtins.fetchTree { type = \"git\"; url = \"file://$TEST_ROOT/shallow\"; ref = \"dev\"; shallow = true; }).revCount or 123") == 123 ]]
tests: check that builtins.fetchGit { ..., ref = "HEAD"; } works
2021-03-30 12:39:37 +03:00
libexpr: Support structured error classes While preparing PRs like #9753, I've had to change error messages in dozens of code paths. It would be nice if instead of EvalError("expected 'boolean' but found '%1%'", showType(v)) we could write TypeError(v, "boolean") or similar. Then, changing the error message could be a mechanical refactor with the compiler pointing out places the constructor needs to be changed, rather than the error-prone process of grepping through the codebase. Structured errors would also help prevent the "same" error from having multiple slightly different messages, and could be a first step towards error codes / an error index. This PR reworks the exception infrastructure in `libexpr` to support exception types with different constructor signatures than `BaseError`. Actually refactoring the exceptions to use structured data will come in a future PR (this one is big enough already, as it has to touch every exception in `libexpr`). The core design is in `eval-error.hh`. Generally, errors like this: state.error("'%s' is not a string", getAttrPathStr()) .debugThrow<TypeError>() are transformed like this: state.error<TypeError>("'%s' is not a string", getAttrPathStr()) .debugThrow() The type annotation has moved from `ErrorBuilder::debugThrow` to `EvalState::error`.
2024-01-23 03:08:29 +02:00
expectStderr 1 nix eval --expr 'builtins.fetchTree { type = "git"; url = "file:///foo"; }' | grepQuiet "fetchTree requires a locked input"
tests/functional/fetchGit.sh: Test fetchGit/fetchTree error message Follow-up for https://github.com/NixOS/nix/pull/9626 176dcd5c617367dbff6d5455856a25518326f79d
2024-01-24 14:19:02 +02:00
tests: check that builtins.fetchGit { ..., ref = "HEAD"; } works
2021-03-30 12:39:37 +03:00
# Explicit ref = "HEAD" should work, and produce the same outPath as without ref
path7=$(nix eval --impure --raw --expr "(builtins.fetchGit { url = \"file://$repo\"; ref = \"HEAD\"; }).outPath")
path8=$(nix eval --impure --raw --expr "(builtins.fetchGit { url = \"file://$repo\"; }).outPath")
[[ $path7 = $path8 ]]
# ref = "HEAD" should fetch the HEAD revision
rev4=$(git -C $repo rev-parse HEAD)
rev4_nix=$(nix eval --impure --raw --expr "(builtins.fetchGit { url = \"file://$repo\"; ref = \"HEAD\"; }).rev")
[[ $rev4 = $rev4_nix ]]
Allow `fetchGit` to take a `name` argument Fix #3388
2021-07-05 14:09:46 +03:00
# The name argument should be handled
path9=$(nix eval --impure --raw --expr "(builtins.fetchGit { url = \"file://$repo\"; ref = \"HEAD\"; name = \"foo\"; }).outPath")
[[ $path9 =~ -foo$ ]]
git fetcher: distinguish errors more precisely
2022-01-01 22:26:41 +02:00
Add test for issue 7146
2022-10-15 01:27:13 +03:00
# Specifying a ref without a rev shouldn't pick a cached rev for a different ref
export _NIX_FORCE_HTTP=1
rev_tag1_nix=$(nix eval --impure --raw --expr "(builtins.fetchGit { url = \"file://$repo\"; ref = \"refs/tags/tag1\"; }).rev")
rev_tag1=$(git -C $repo rev-parse refs/tags/tag1)
[[ $rev_tag1_nix = $rev_tag1 ]]
rev_tag2_nix=$(nix eval --impure --raw --expr "(builtins.fetchGit { url = \"file://$repo\"; ref = \"refs/tags/tag2\"; }).rev")
rev_tag2=$(git -C $repo rev-parse refs/tags/tag2)
[[ $rev_tag2_nix = $rev_tag2 ]]
unset _NIX_FORCE_HTTP
fetchTree/fetchGit: add test for .gitattributes ...with the intention to prevent future regressions in fetchGit
2023-11-19 15:37:42 +02:00
# Ensure .gitattributes is respected
touch $repo/not-exported-file
git fetcher: Add exportIgnore parameter Enabled for fetchGit, which historically had this behavior, among other behaviors we do not want in fetchGit. fetchTree disables this parameter by default. It can choose the simpler behavior, as it is still experimental. I am not confident that the filtering implementation is future proof. It should reuse a source filtering wrapper, which I believe Eelco has already written, but not merged yet.
2023-11-27 23:34:41 +02:00
touch $repo/exported-wonky
fetchTree/fetchGit: add test for .gitattributes ...with the intention to prevent future regressions in fetchGit
2023-11-19 15:37:42 +02:00
echo "/not-exported-file export-ignore" >> $repo/.gitattributes
git fetcher: Add exportIgnore parameter Enabled for fetchGit, which historically had this behavior, among other behaviors we do not want in fetchGit. fetchTree disables this parameter by default. It can choose the simpler behavior, as it is still experimental. I am not confident that the filtering implementation is future proof. It should reuse a source filtering wrapper, which I believe Eelco has already written, but not merged yet.
2023-11-27 23:34:41 +02:00
echo "/exported-wonky export-ignore=wonk" >> $repo/.gitattributes
git -C $repo add not-exported-file exported-wonky .gitattributes
fetchTree/fetchGit: add test for .gitattributes ...with the intention to prevent future regressions in fetchGit
2023-11-19 15:37:42 +02:00
git -C $repo commit -m 'Bla6'
rev5=$(git -C $repo rev-parse HEAD)
path12=$(nix eval --impure --raw --expr "(builtins.fetchGit { url = file://$repo; rev = \"$rev5\"; }).outPath")
[[ ! -e $path12/not-exported-file ]]
git fetcher: Add exportIgnore parameter Enabled for fetchGit, which historically had this behavior, among other behaviors we do not want in fetchGit. fetchTree disables this parameter by default. It can choose the simpler behavior, as it is still experimental. I am not confident that the filtering implementation is future proof. It should reuse a source filtering wrapper, which I believe Eelco has already written, but not merged yet.
2023-11-27 23:34:41 +02:00
[[ -e $path12/exported-wonky ]]
fetchTree/fetchGit: add test for .gitattributes ...with the intention to prevent future regressions in fetchGit
2023-11-19 15:37:42 +02:00
git fetcher: distinguish errors more precisely
2022-01-01 22:26:41 +02:00
# should fail if there is no repo
rm -rf $repo/.git
(! nix eval --impure --raw --expr "(builtins.fetchGit \"file://$repo\").outPath")
# should succeed for a repo without commits
git init $repo
GitInputScheme: Use libgit2 This replaces most calls to the "git" binary with libgit2.
2023-10-25 19:55:08 +03:00
git -C $repo add hello # need to add at least one file to cause the root of the repo to be visible
git fetcher: distinguish errors more precisely
2022-01-01 22:26:41 +02:00
path10=$(nix eval --impure --raw --expr "(builtins.fetchGit \"file://$repo\").outPath")
tests/fetchGit: add regression test for #7707
2023-02-27 16:33:54 +02:00
# should succeed for a path with a space
# regression test for #7707
repo="$TEST_ROOT/a b"
git init "$repo"
git -C "$repo" config user.email "foobar@example.com"
git -C "$repo" config user.name "Foobar"
echo utrecht > "$repo/hello"
touch "$repo/.gitignore"
git -C "$repo" add hello .gitignore
git -C "$repo" commit -m 'Bla1'
cd "$repo"
path11=$(nix eval --impure --raw --expr "(builtins.fetchGit ./.).outPath")
Support empty Git repositories / workdirs Fixes #10039.
2024-02-20 12:40:02 +02:00
# Test a workdir with no commits.
empty="$TEST_ROOT/empty"
git init "$empty"
emptyAttrs='{ lastModified = 0; lastModifiedDate = "19700101000000"; narHash = "sha256-pQpattmS9VmO3ZIQUFn66az8GSmB4IvYhTTCFn6SUmo="; rev = "0000000000000000000000000000000000000000"; revCount = 0; shortRev = "0000000"; submodules = false; }'
[[ $(nix eval --impure --expr "builtins.removeAttrs (builtins.fetchGit $empty) [\"outPath\"]") = $emptyAttrs ]]
echo foo > "$empty/x"
[[ $(nix eval --impure --expr "builtins.removeAttrs (builtins.fetchGit $empty) [\"outPath\"]") = $emptyAttrs ]]
git -C "$empty" add x
[[ $(nix eval --impure --expr "builtins.removeAttrs (builtins.fetchGit $empty) [\"outPath\"]") = '{ lastModified = 0; lastModifiedDate = "19700101000000"; narHash = "sha256-wzlAGjxKxpaWdqVhlq55q5Gxo4Bf860+kLeEa/v02As="; rev = "0000000000000000000000000000000000000000"; revCount = 0; shortRev = "0000000"; submodules = false; }' ]]
# Test a repo with an empty commit.
git -C "$empty" rm -f x
git -C "$empty" config user.email "foobar@example.com"
git -C "$empty" config user.name "Foobar"
git -C "$empty" commit --allow-empty --allow-empty-message --message ""
nix eval --impure --expr "let attrs = builtins.fetchGit $empty; in assert attrs.lastModified != 0; assert attrs.rev != \"0000000000000000000000000000000000000000\"; assert attrs.revCount == 1; true"
Reference in a new issue Copy permalink