Commit graph

339 commits

Author SHA1 Message Date
John Ericson
4540e7b940 Don't add StorePathDescriptor for now
We don't need it yet, we can add it back later.
2023-01-23 12:58:27 -05:00
John Ericson
6a168254ce Use named field initialization for references 2023-01-06 12:24:20 -05:00
John Ericson
e9fc1e4fdb Merge remote-tracking branch 'upstream/master' into path-info 2023-01-06 10:35:20 -05:00
Benoit de Chezelles
a456630a5a Allow to disable global flake-registry with "" 2022-12-12 15:32:02 +01:00
Eelco Dolstra
703d863a48 Trivial changes from the lazy-trees branch 2022-12-07 14:06:34 +01:00
Théophane Hufschmitt
90ed041677
Merge pull request #7039 from Mic92/libfetchers-variant
libfetchers: avoid api.github.com ratelimit if no github token is set
2022-10-31 11:48:51 +01:00
Andrew Brooks
a259084c50 Fix #7146
When fetching a non-local git repo by ref (and no rev), don't consider unrelated
cached revs for the same repository.
2022-10-14 18:04:47 -05:00
Jörg Thalheim
d9abce4ad4 libfetchers: avoid api.github.com ratelimit if no github token is set
If we don't have any github token, we won't be able to fetch private
repos, but we are also more likely to run into API limits since
we don't have a token. To mitigate this only ever use the github api
if we actually have a token.
2022-10-01 10:38:09 +02:00
Eelco Dolstra
c80a74b7d5 Don't pass --force to 'git add'
Fixes #5810.
2022-08-17 16:59:02 +02:00
Guillaume Girol
e8109cf405 fetchGit: document shallow argument 2022-06-26 12:00:00 +00:00
Eelco Dolstra
4b6cc3da62 Fetch flake-registry.json from channels.nixos.org
Using fastly is slightly faster, provides some resilience due to a
high stale TTL, and allows some usage metrics.
2022-06-23 23:56:26 +02:00
Gabriel Fontes
9f6b4639c2
fix sourcehut brach/tag resolving regression
nixos/nix#6290 introduced a regex pattern to account for tags when
resolving sourcehut refs. nixos/nix#4638 reafactored the code,
accidentally treating the pattern as a regular string, causing all
non-HEAD ref resolving to break.

This fixes the regression and adds more test cases to avoid future
breakage.
2022-06-11 16:52:20 -03:00
Naïm Favier
da8f8668ca
libfetchers/git: add missing --git-dir flags 2022-06-10 12:57:13 +02:00
Théophane Hufschmitt
027fd45230 Fix a segfault in the git fetcher
The git fetcher code used to dereference the (potentially empty) `ref`
input attribute. This was magically working, probably because the
compiler somehow outsmarted us, but is now blowing up with newer nixpkgs
versions.

Fix that by not trying to access this field while we don't know for sure
that it has been defined.

Fix #6554
2022-05-27 16:15:28 +02:00
Eelco Dolstra
d8398d33c9
Typo 2022-05-25 15:29:27 +02:00
Eelco Dolstra
89a8955e79
Merge remote-tracking branch 'origin/file-fetcher' 2022-05-25 15:15:16 +02:00
Maximilian Bosch
b916c08feb
libfetchers: drop getGitDir and hardcode .git
As discussed[1] this is most likely not desirable.

[1] https://github.com/NixOS/nix/pull/6440#issuecomment-1120876248
2022-05-24 14:20:48 +02:00
Tony Olagbaiye
5b8c1deb18 fetchTree: Allow fetching plain files
Add a new `file` fetcher type, which will fetch a plain file over
http(s), or from the local file.

Because plain `http(s)://` or `file://` urls can already correspond to
`tarball` inputs (if the path ends-up with a know archive extension),
the URL parsing logic is a bit convuluted in that:

- {http,https,file}:// urls will be interpreted as either a tarball or a
  file input, depending on the extensions of the path part (so
  `https://foo.com/bar` will be a `file` input and
  `https://foo.com/bar.tar.gz` as a `tarball` input)
- `file+{something}://` urls will be interpreted as `file` urls (with
  the `file+` part removed)
- `tarball+{something}://` urls will be interpreted as `tarball` urls (with
  the `tarball+` part removed)

Fix #3785

Co-Authored-By: Tony Olagbaiye <me@fron.io>
2022-05-19 18:24:49 +02:00
Norbert Melzer
831e2743ea fix GitHub URL template 2022-05-12 00:56:39 +02:00
Eelco Dolstra
eb957ad6d8
Merge pull request #6497 from danielfullmer/ghe-fetcher-url
Use correct URL for GitHub Enterprise
2022-05-10 19:23:22 +02:00
Daniel Fullmer
7a3d5b2ff0 Use correct URL for GitHub Enterprise
For GitHub Enterprise, the API is accessed through a slightly different
URL. See [1], where it says:

> Use http(s)://[hostname]/api/v3 to access the API for GitHub
> Enterprise Server.

Also tested working on a GHE instance.

[1] https://docs.github.com/en/enterprise-server@3.3/rest/guides/getting-started-with-the-rest-api
2022-05-06 13:13:11 -07:00
Théophane Hufschmitt
e68676e6c8 Fix the parsing of the sourcehut refs file
Since a26be9f3b8, the same parser is used
to parse the result of sourcehut’s `HEAD` endpoint (coming from [git
dumb protocol]) and the output of `git ls-remote`. However, they are very
slightly different (the former doesn’t specify the current reference
since it’s implied to be `HEAD`).

Unify both, and make the parser a bit more robust and understandable (by
making it more typed and adding tests for it)

[git dumb protocol]: https://git-scm.com/book/en/v2/Git-Internals-Transfer-Protocols#_the_dumb_protocol
2022-05-04 14:38:59 +02:00
Eelco Dolstra
61289ceee3 Style fixes 2022-05-02 13:37:53 +02:00
Eelco Dolstra
564faa6b4e
Merge pull request #6470 from Ma27/git-followup
libfetchers/git: fix every occasion of a permission error
2022-05-02 13:33:08 +02:00
Maximilian Bosch
1849e6a1f6
libfetchers/git: fix every occasion of a permission error
I'm afraid I missed a few problematic `git(1)`-calls while implementing
PR #6440, sorry for that! Upon investigating what went wrong, I realized
that I only tested against the "cached"-case by accident because my
git-checkout with my system's flake was apparently cached during my
debugging.

I managed to trigger the original issue again by running:

    $ git commit --allow-empty -m "tmp"
    $ sudo nixos-rebuild switch --flake .# -L --builders ''

Since `repoDir` points to the checkout that's potentially owned by
another user, I decided to add `--git-dir` to each call affecting
`repoDir`.

Since the `tmpDir` for the temporary submodule-checkout is created by
Nix itself, it doesn't seem to be an issue.

Sorry for that, it should be fine now.
2022-04-30 15:56:12 +02:00
Kjetil Orbekk
9bf296c970 Extract git reference parsing to a shared library
These utility functions can be shared between the git and github fetchers.
2022-04-29 18:46:21 -04:00
Kjetil Orbekk
1203e48926 Store cached head in cached git repo
The previous head caching implementation stored two paths in the local
cache; one for the cached git repo and another textfile containing the
resolved HEAD ref. This commit instead stores the resolved HEAD by
setting the HEAD ref in the local cache appropriately.
2022-04-29 18:46:21 -04:00
Kjetil Orbekk
de54e1cd3f Refactor fetching of dirty workdir
Extract the handling of a local dirty workdir to a helper function.
2022-04-29 18:46:17 -04:00
Kjetil Orbekk
401e60f289 Resolve reference for remote repository
Resolves the HEAD reference from the remote repository instead
of assuming "master".
2022-04-29 18:42:28 -04:00
Maximilian Bosch
d1f5356311
libfetchers/git: fix for nixos-rebuild
The `--git-dir=` must be `.` in some cases (for cached repos that are
"bare" repos in `~/.cache/nix/gitv3`). With this fix we can add
`--git-dir` to each `git`-invokation needed for `nixos-rebuild`.
2022-04-24 18:14:24 +02:00
Maximilian Bosch
0256e5578e
libfetchers/git: hardcode --git-dir
To demonstrate the problem:

* You need a `git` at 2.33.3 in your $PATH
* An expression like this in a git repository:

  ``` nix
  {
    outputs = { self, nixpkgs }: {
      packages.foo.x86_64-linux = with nixpkgs.legacyPackages.x86_64-linux;
        runCommand "snens" { } ''
          echo ${(builtins.fetchGit ./.).lastModifiedDate} > $out
        '';
    };
  }
  ```

Now, when instantiating the package via `builtins.getFlake`, it fails on
Nix 2.7 like this:

    $ nix-instantiate -E '(builtins.getFlake "'"$(pwd)"'").packages.foo.x86_64-linux'
    fatal: unsafe repository ('/nix/store/a7j3125km4h8l0p71q6ssfkxamfh5d61-source' is owned by someone else)
    To add an exception for this directory, call:

    	git config --global --add safe.directory /nix/store/a7j3125km4h8l0p71q6ssfkxamfh5d61-source
    error: program 'git' failed with exit code 128
    (use '--show-trace' to show detailed location information)

This breaks e.g. `nixops`-deployments using flakes with similar
expressions as shown above.

The cause for this is that `git(1)` tries to find the highest
`.git`-directory in the directory tree and if it finds a such a
directory, but with another owning user (root vs. the user who evaluates
the expression), it fails as above. This was changed recently to fix
CVE-2022-24765[1].

By explicitly specifying `--git-dir`, Git assumes to be in the top-level
directory and doesn't attempt to look for a `.git`-directory in the
parent directories and thus the code-path leading to said error is never
reached.

[1] https://lore.kernel.org/git/xmqqv8veb5i6.fsf@gitster.g/
2022-04-23 23:20:17 +02:00
John Ericson
55caef36ed Merge remote-tracking branch 'upstream/master' into path-info 2022-04-19 22:27:21 +00:00
John Ericson
75b62e5260 Avoid fmt when constructor already does it
There is a correctnes issue here, but #3724 will fix that. This is just
a cleanup for brevity's sake.
2022-04-19 01:44:11 +00:00
Eelco Dolstra
d6effddd3b
Merge pull request #6387 from Uthar/fix
assert hash types for Git and Mercurial
2022-04-14 14:55:27 +02:00
Kasper Gałkowski
2769e43f61 assert hash types for Git and Mercurial 2022-04-12 21:13:14 +02:00
Eelco Dolstra
f7276bc948
Merge pull request #6392 from danpls/fix-actualUrl-mercurial
libfetchers: Fix assertion (Mercurial)
2022-04-11 11:16:04 +02:00
Daniel Pauls
d6b7529579 libfetchers: Fix assertion (Mercurial)
See commit 1e1cd6e7a for more information.
2022-04-09 19:10:23 +02:00
Daniel Pauls
770f7371f3 libfetchers: Replace regex to clarify intent 2022-04-09 17:00:14 +02:00
Théophane Hufschmitt
646af7325d
Merge pull request #6376 from Uthar/master
don't assume that rev is a SHA1 hash
2022-04-08 17:56:27 +02:00
Kasper Gałkowski
2c2fd4946f don't assume that rev is a SHA1 hash
This was a problem when writing a fetcher that uses e.g. sha256 hashes
for revisions. This doesn't actually do anything new, but allows for
creating such fetchers in the future (perhaps when support for Git's
SHA256 object format gains more popularity).
2022-04-07 19:49:47 +02:00
Daniel Pauls
1e1cd6e7a9 libfetchers: Fix assertion
The filter expects all paths to have a prefix of the raw `actualUrl`, but
`Store::addToStore(...)` provides absolute canonicalized paths.
To fix this create an absolute and canonicalized path from the `actualUrl` and
use it instead.

Fixes #6195.
2022-04-06 17:33:23 +02:00
John Ericson
0dc2974930 Merge remote-tracking branch 'upstream/master' into path-info 2022-03-25 19:25:08 +00:00
Gabriel Fontes
9720797f69
Don't partial match sourcehut refs 2022-03-19 11:04:04 -03:00
Gabriel Fontes
345a8ee0cb
Fix sourcehut tag ref resolving 2022-03-19 10:56:13 -03:00
Maximilian Bosch
244baff2c7
libfetchers: remove obsolete filesystem #include 2022-03-15 12:32:11 +01:00
Maximilian Bosch
34e20c164c
libfetchers/path: set lastModified to path's mtime
When importing e.g. a local `nixpkgs` in a flake to test a change like

    {
      inputs.nixpkgs.url = path:/home/ma27/Projects/nixpkgs;
      outputs = /* ... */
    }

then the input is missing a `lastModified`-field that's e.g. used in
`nixpkgs.lib.nixosSystem`. Due to the missing `lastMoified`-field, the
mtime is set to 19700101:

    result -> /nix/store/b7dg1lmmsill2rsgyv2w7b6cnmixkvc1-nixos-system-nixos-22.05.19700101.dirty

With this change, the `path`-fetcher now sets a `lastModified` attribute
to the `mtime` just like it's the case in the `tarball`-fetcher already.
When building NixOS systems with `nixpkgs` being a `path`-input and this
patch, the output-path now looks like this:

    result -> /nix/store/ld2qf9c1s98dxmiwcaq5vn9k5ylzrm1s-nixos-system-nixos-22.05.20220217.dirty
2022-03-15 12:32:11 +01:00
Eelco Dolstra
167766b65c Style 2022-03-11 11:19:21 +01:00
Théophane Hufschmitt
7ebd6f1093
Merge pull request #5758 from mschwaig/fix-git-workspace-dirty-detection
git fetcher: fix detection of dirty git workspaces
2022-03-11 09:35:01 +01:00
John Ericson
8ba089597f Merge remote-tracking branch 'upstream/master' into path-info 2022-03-10 15:48:14 +00:00
Nicholas Sielicki
314852a10e
Point to new github oauth docs url
Previous URL was 404'ing.
2022-03-06 17:01:14 -06:00
John Ericson
ea71da395f Move some stuff from Settings to a new FetchSettings.
Starting work on #5638

The exact boundary between `FetchSettings` and `EvalSettings` is not
clear to me, but that's fine. First lets clean out `libstore`, and then
worry about what, if anything, should be the separation between those
two.
2022-03-01 01:39:25 +00:00
Eelco Dolstra
df552ff53e Remove std::string alias (for real this time)
Also use std::string_view in a few more places.
2022-02-25 16:13:02 +01:00
Eelco Dolstra
8518cebfce libfetchers: Rename immutable -> locked
This is more consistent with flake terminology.
2022-02-24 18:09:00 +01:00
Théophane Hufschmitt
6524eb4b77
Merge pull request #5342 from Misterio77/add-sourcehut
Add support for sourcehut input scheme
2022-02-24 09:09:21 +01:00
Martin Schwaighofer
53523c0ab8 git fetcher: set locale for rev-parse 2022-02-19 22:58:23 +01:00
Martin Schwaighofer
9504445cab git fetcher: distinguish errors more precisely 2022-02-19 22:58:23 +01:00
Martin Schwaighofer
c7e527b82b git fetcher: invoke diff instead of diff-index
diff-index operates on the view that git has of the working tree,
which might be outdated. The higher-level diff command does this
automatically. This change also adds handling for submodules.

fixes #4140

Alternative fixes would be invoking update-index before diff-index or
matching more closely what require_clean_work_tree from git-sh-setup.sh
does, but both those options make it more difficult to reason about
correctness.
2022-02-19 22:58:23 +01:00
Martin Schwaighofer
0bfa0cdea1 git fetcher: improve check for valid repository
The .git/refs/heads directory might be empty for a valid
usable git repository. This often happens in CI environments,
which might only fetch commits, not branches.
Therefore instead we let git itself check if HEAD points to
something that looks like a commit.

fixes #5302
2022-02-19 22:58:22 +01:00
Eelco Dolstra
aa5b83d93c InputScheme::fetch(): Return a StorePath instead of a Tree 2022-02-16 11:14:01 +01:00
Gabriel Fontes
72e8f94081
add sourcehut input scheme 2022-02-14 23:53:01 -03:00
Eelco Dolstra
d62a9390fc Get rid of std::shared_ptr<std::string> and ref<std::string>
These were needed back in the pre-C++11 era because we didn't have
move semantics. But now we do.
2022-01-18 11:12:30 +01:00
Connor Baker
52f5fa948a Add support for .tar.gz shorthand (.tgz) to TarballInputScheme 2021-12-03 22:16:02 -05:00
David Arnold
7489811736
Fix parent path check boundary
- Previous to this commit the boundary was exclusive of the
  top level flake.
- This is wrong since the top level flake is still a valid
  relative reference.
- Now, the check boundary is inclusive of the top level flake.

Signed-off-by: Timothy DeHerrera <tim.deh@pm.me>
2021-12-01 17:01:48 -07:00
Eelco Dolstra
ec608e3def
Merge pull request #5452 from Kha/git-extend-lock
git: extend cache dir lock over all mutating operations
2021-11-16 12:55:12 +01:00
Finn Behrens
79d07d0980 libfetchers: set free gitlab headers 2021-11-14 12:23:46 +01:00
Sebastian Ullrich
b459a3e856 git: extend cache dir lock over all mutating operations 2021-11-01 09:14:56 +01:00
Timothy
8919b81dad Support building flakes from a Git repo url with submodules query parameter 2021-10-26 20:02:37 +07:00
John Ericson
f4f3203aa7 Merge remote-tracking branch 'upstream/master' into path-info 2021-09-30 22:41:53 +00:00
oxalica
81b8e910a0
Explicitly set initial branch name for git 2021-09-24 22:09:49 +08:00
Eelco Dolstra
d1bf7431bb Revert "Merge pull request #4922 from nrdxp/default-submodules"
This reverts commit 6678e98411, reversing
changes made to 90b2dd570c.
2021-09-22 17:25:25 +02:00
Eelco Dolstra
06557299b3 Allow relative paths anywhere into the parent's store path 2021-09-21 13:45:11 +02:00
Eelco Dolstra
5cbb9c5406 path fetcher: Fix relative path check 2021-09-21 13:19:26 +02:00
Eelco Dolstra
fda4efff87 Rename unnecessary git@ 2021-09-14 22:53:31 +02:00
Eelco Dolstra
07996c4810 github fetcher: Don't use HEAD branch when cloning
Fixes #4394.
2021-09-14 13:38:45 +02:00
Eelco Dolstra
d72d31d529 github fetcher: Use git+https for cloning
git+ssh only works if you have SSH access.
2021-09-14 13:38:05 +02:00
Eelco Dolstra
76e368a3b4 Fix macOS build 2021-09-14 10:07:31 +02:00
Eelco Dolstra
4ed66735b6 RunOptions: Use designated initializers
Also get rid of _killStderr because it wasn't actually checked
anywhere.
2021-09-13 23:31:04 +02:00
Eelco Dolstra
c3e9acd1c0 Remove tabs 2021-09-13 23:06:33 +02:00
Shea Levy
6678e98411
Merge pull request #4922 from nrdxp/default-submodules
libfetchers/git: fetch submodules by default
2021-09-07 05:48:23 -04:00
Eelco Dolstra
eda0fee160
Merge pull request #5175 from Pamplemousse/make
Don't overwrite user provided `lib*_LDFLAGS`
2021-08-30 12:44:29 +02:00
Pamplemousse
a4c6d319a8 Don't overwrite user provided lib*_LDFLAGS
Signed-off-by: Pamplemousse <xav.maso@gmail.com>
2021-08-25 08:59:19 -07:00
Eelco Dolstra
af94b54db3 Coding style 2021-08-23 13:05:42 +02:00
Alex Zero
b3c424f5a6
Fix follows paths in subordinate lockfiles 2021-08-17 15:25:42 +01:00
Michael Fellinger
5c99ec374c
fix git init race condition 2021-08-02 13:47:59 +02:00
Alyssa Ross
ae0c026fe9
Enable pthreads for new libraries
Otherwise the lack of pthread causes linking to fail for NetBSD.
2021-07-24 09:15:01 +00:00
Eelco Dolstra
8648143120
Merge pull request #4988 from NixOS/fetchgit-name-attribute
Add a name attribute to the fetchers
2021-07-08 14:33:49 +02:00
Eelco Dolstra
3b3e6bb1e5 Style tweaks 2021-07-07 10:02:55 +02:00
regnat
2c8240677e allow fetchFromGitHub to take a name argument 2021-07-06 08:43:19 +02:00
regnat
e4b082a52b allow fetchMercurial to take a name argument 2021-07-06 08:43:06 +02:00
regnat
a487a652ed allow fetchTarball to take a name argument 2021-07-06 08:42:47 +02:00
regnat
83615fcf8f Allow fetchGit to take a name argument
Fix #3388
2021-07-06 08:41:52 +02:00
Alexander Bantyev
811f3e8605
nix registry pin: add a way to pin to a custom locked 2021-06-30 22:16:40 +03:00
Timothy DeHerrera
769ca4e26d
libfetchers/git: fetch submodules by default 2021-06-17 12:00:26 -06:00
keke
50dc88a56c
fix error: 'optional' in namespace 'std' does not name a template type 2021-06-02 18:09:03 +08:00
Timothy Klim
4da9ec772c Add .tar.zst support for TarballInputScheme 2021-05-29 16:03:26 +07:00
Alyssa Ross
9ac6534f7c
Include sys/wait.h everywhere WIFEXITED etc is used
This is required on NetBSD, and I think FreeBSD too.
2021-04-19 18:31:58 +00:00
Eelco Dolstra
42f0246698
Revert "libfetchers/tarball: Lock on effectiveUrl"
This reverts commit fc6bfb261d.

Fixes #4672.
2021-04-09 13:51:57 +02:00
John Ericson
e12308dd63 Merge branch 'master' into path-info 2021-04-05 18:29:52 -04:00
Bjørn Forsman
edd606ae62 fetchGit: don't prefix "refs/heads/" on ref = "HEAD"
This fixes builtins.fetchGit { url = ...; ref = "HEAD"; }, that works in
stable nix (v2.3.10), but is broken in nix master:

  $ ./result/bin/nix repl
  Welcome to Nix version 2.4pre19700101_dd77f71. Type :? for help.

  nix-repl> builtins.fetchGit { url = "https://github.com/NixOS/nix"; ref = "HEAD"; }
  fetching Git repository 'https://github.com/NixOS/nix'fatal: couldn't find remote ref refs/heads/HEAD
  error: program 'git' failed with exit code 128

The documentation for builtins.fetchGit says ref = "HEAD" is the
default, so it should also be supported to explicitly pass it.

I came across this issue because poetry2nix can use ref = "HEAD" in some
situations.

Fixes #4674.
2021-03-30 11:21:38 +02:00
Samuel Dionne-Riel
66b857244f Use the appropriate config dir for the registry 2021-03-19 15:20:47 -04:00
Eelco Dolstra
3bb1becdbb
Merge pull request #4566 from orbekk/master
Add support for bare git repositories when using git+file
2021-03-11 10:38:07 +01:00
dramforever
fc6bfb261d libfetchers/tarball: Lock on effectiveUrl
Basically, if a tarball URL is used as a flake input, and the URL leads
to a redirect, the final redirect destination would be recorded as the
locked URL.

This allows tarballs under https://nixos.org/channels to be used as
flake inputs. If we, as before, lock on to the original URL it would
break every time the channel updates.
2021-03-02 21:56:50 +08:00
Kjetil Orbekk
9931f18c2d Add support for bare git repositories with git+file
Local git repositories are normally used directly instead of
cloning. This commit checks if a repo is bare and forces a
clone.

Co-authored-by: Théophane Hufschmitt <regnat@users.noreply.github.com>
2021-03-01 09:03:25 -05:00
John Ericson
ca0994819d Merge remote-tracking branch 'upstream/master' into path-info 2021-02-25 21:51:05 +00:00
Eelco Dolstra
c03f41055d Add traces to errors while updating flake lock file
Example:

$ nix build --show-trace
error: unable to download 'https://api.github.com/repos/NixOS/nixpkgs/commits/no-such-branch': HTTP error 422 ('')

       response body:

       {
         "message": "No commit found for SHA: no-such-branch",
         "documentation_url": "https://docs.github.com/rest/reference/repos#get-a-commit"
       }

       … while fetching the input 'github:NixOS/nixpkgs/no-such-branch'

       … while updating the flake input 'nixpkgs'

       … while updating the lock file of flake 'git+file:///home/eelco/Dev/nix'
2021-01-27 14:02:54 +01:00
Matthew Bauer
1ea5f0b66c Remove expectedETag assert in tarball.cc 2021-01-22 23:19:52 -06:00
Eelco Dolstra
1bbc66f865 Merge branch 'slashes-in-github-branches' of https://github.com/Ma27/nix 2021-01-18 12:55:06 +01:00
Eelco Dolstra
6548b89cc4 string2Int(): Return std::optional 2021-01-08 12:22:21 +01:00
Danila Fedorin
988dd0a65f Fix conversion from JSON to fetch attributes
It appears as through the fetch attribute, which
is simply a variant with 3 elements, implicitly
converts boolean arguments to integers. One must
use Explicit<bool> to correctly populate it with
a boolean. This was missing from the implementation,
and resulted in clearly boolean JSON fields being
treated as numbers.
2021-01-05 02:06:25 +00:00
Matthew Bauer
ede534a3a1
Merge branch 'master' into support-libcxx10 2020-12-24 14:16:09 -06:00
Matthew Bauer
d4870462f8 Cast variants fully for libc++10
libc++10 seems to be stricter on what it allows in variant conversion.
I'm not sure what the rules are here, but this is the minimal change
needed to get through the compilation errors.
2020-12-23 23:41:58 -06:00
Maximilian Bosch
2857b1baaf
Add explicit allRefs = true; argument to fetchGit
Sometimes it's necessary to fetch a git repository at a revision and
it's unknown which ref contains the revision in question. An example
would be a Cargo.lock which only provides the URL and the revision when
using a git repository as build input.

However it's considered a bad practice to perform a full checkout of a
repository since this may take a lot of time and can eat up a lot of
disk space. This patch makes a full checkout explicit by adding an
`allRefs` argument to `builtins.fetchGit` which fetches all refs if
explicitly set to true.

Closes #2409
2020-12-22 10:46:00 +01:00
Maximilian Bosch
629af83b2d
Provide a more meaningful error-message for builtins.fetchGit if a revision can't be checked out
A common pitfall when using e.g. `builtins.fetchGit` is the `fatal: not
a tree object`-error when trying to fetch a revision of a git-repository
that isn't on the `master` branch and no `ref` is specified.

In order to make clear what's the problem, I added a simple check
whether the revision in question exists and if it doesn't a more
meaningful error-message is displayed:

```
nix-repl> builtins.fetchGit { url = "https://github.com/owner/myrepo"; rev = "<commit not on master>"; }
moderror: --- Error -------------------------------------------------------------------- nix
Cannot find Git revision 'bf1cc5c648e6aed7360448a3745bb2fe4fbbf0e9' in ref 'master' of repository 'https://gitlab.com/Ma27/nvim.nix'! Please make sure that the rev exists on the ref you've specified or add allRefs = true; to fetchGit.
```

Closes #2431
2020-12-22 10:46:00 +01:00
Luke Granger-Brown
226116f482 fetchMercurial: set HGPLAIN when invoking hg
Without setting HGPLAIN, the user's environment leaks into
hg invocations, which means that the output may not be in the
expected format.

HGPLAIN is the Mercurial-recommended solution for this in that
it's intended for uses by scripts and programs which are looking
to parse Mercurial's output in a consistent manner.
2020-11-23 16:12:33 +00:00
Eelco Dolstra
4badb6943f Fix use of dirty Git/Mercurial inputs with chroot stores
Fixes:

  $ nix build --store /tmp/nix /home/eelco/Dev/patchelf#hydraJobs.build.x86_64-linux
  warning: Git tree '/home/eelco/Dev/patchelf' is dirty
  error: --- RestrictedPathError ------------------------------------------------------------------------------------------- nix
  access to path '/tmp/nix/nix/store/xmkvfmffk7xfnazykb5kx999aika8an4-source/flake.nix' is forbidden in restricted mode
  (use '--show-trace' to show detailed location information)
2020-11-10 23:22:45 +01:00
Matthew Kenigsberg
6a4bf535d8 Capitalize JSON for consistency 2020-10-28 17:54:28 -05:00
Eelco Dolstra
9d5e9ef0da Move Explicit 2020-10-26 17:01:20 +01:00
Eelco Dolstra
9635fb77bd
Merge pull request #4080 from kquick/kwq/flake-int-doc
Add some internal documentation for flake support objects.
2020-10-19 11:29:12 +02:00
John Ericson
a0f369aa3f Merge remote-tracking branch 'upstream/master' into path-info 2020-10-12 20:48:35 +00:00
John Ericson
6cc1541782 Split out local-fs-store.hh
This matches the already-existing `local-fs-store.cc`.
2020-10-09 20:18:08 +00:00
John Ericson
f8d562c0a7 Use PathReferences more widely 2020-10-07 15:00:10 +00:00
Maximilian Bosch
59f2dd8e8d
libfetchers/github: allow slashes in refs
Refs #4061
2020-10-06 20:08:51 +02:00
Eelco Dolstra
ad143c5b3b Shut up some clang warnings 2020-10-06 14:52:30 +02:00
Eelco Dolstra
85c8be6286 Remove static variable name clashes
This was useful for an experiment with building Nix as a single
compilation unit. It's not very useful otherwise but also doesn't
hurt...
2020-10-06 13:49:20 +02:00
Eelco Dolstra
20a1e20d91 Style 2020-09-30 13:35:56 +02:00
Eelco Dolstra
274357eb6a Simplify 2020-09-30 12:09:18 +02:00
Eelco Dolstra
002ce8449d Merge branch 'access-tokens' of https://github.com/kquick/nix 2020-09-30 11:35:15 +02:00
Kevin Quick
5e7838512e
Remove github-access-token in favor of access-token. 2020-09-29 16:26:34 -07:00
Eelco Dolstra
64e9b3c83b nix registry list: Show 'dir' attribute
Issue #4050.
2020-09-29 23:33:16 +02:00
Kevin Quick
66c3959e8c
Merge branch 'master' into access-tokens 2020-09-29 08:32:06 -07:00
Eelco Dolstra
5999978a05 Make Headers an optional argument 2020-09-29 13:05:19 +02:00
Eelco Dolstra
de86abbf3f Cleanup 2020-09-29 12:55:06 +02:00
Eelco Dolstra
cebd2fc35d Merge branch 'github-api-token' of https://github.com/imalsogreg/nix 2020-09-29 12:17:00 +02:00
Kevin Quick
887be7b6f2
Switch comment format from '// ...' to '/* ... */' for consistency. 2020-09-28 09:37:26 -07:00
Kevin Quick
bcb3da3b6b
Fix spelling error. 2020-09-28 08:58:14 -07:00
Kevin Quick
5885b0cfd8
Miscellaneous spelling fixes in comments. (#4071) 2020-09-27 20:04:06 +00:00
Kevin Quick
bd5328814f
Add some internal documentation for flake support objects. 2020-09-26 14:32:58 -07:00
Gregory Hale
faa5607f54 Merge remote-tracking branch 'origin/master' into github-api-token 2020-09-25 12:10:58 -04:00
Kevin Quick
5a35cc29bf
Re-add support for github-access-token, but mark as deprecated. 2020-09-25 08:09:56 -07:00
Kevin Quick
8fba2a8b54
Update to use access-tokens configuration for github/gitlab access.
This change provides support for using access tokens with other
instances of GitHub and GitLab beyond just github.com and
gitlab.com (especially company-specific or foundation-specific
instances).

This change also provides the ability to specify the type of access
token being used, where different types may have different handling,
based on the forge type.
2020-09-24 22:49:44 -07:00
Kevin Quick
c2f48cfcee
Complete conversion of "url" to "host" with associated variable renaming.
Completes the change begun in commit 56f1e0d to consistently use the
"host" attribute for "github" and "gitlab" inputs instead of a "url"
attribute.
2020-09-24 22:46:03 -07:00
Eelco Dolstra
e8e1d420f3 Don't include <regex> in header files
This reduces compilation time by ~15 seconds (CPU time).

Issue #4045.
2020-09-21 18:22:45 +02:00
Maximilian Bosch
56f1e0df05
libfetchers/github: rename url to host 2020-09-21 16:29:08 +02:00
Maximilian Bosch
2bcf8cbe7a
libfetchers/github: allow url attribute
Since 108debef6f we allow a
`url`-attribute for the `github`-fetcher to fetch tarballs from
self-hosted `gitlab`/`github` instances.

However it's not used when defining e.g. a flake-input

    foobar = {
        type = "github";
        url = "gitlab.myserver";
        /* ... */
    }

and breaks with an evaluation-error:

    error: --- Error --------------------------------------nix
    unsupported input attribute 'url'
    (use '--show-trace' to show detailed location information)

This patch allows flake-inputs to be fetched from self-hosted instances
as well.
2020-09-18 14:10:45 +02:00
Greg Hale
a303c0b6dc Fetch commits from github/gitlab using Auth header
`nix flake info` calls the github 'commits' API, which requires
authorization when the repository is private. Currently this request
fails with a 404.

This commit adds an authorization header when calling the 'commits' API.
It also changes the way that the 'tarball' API authenticates, moving the
user's token from a query parameter into the Authorization header.

The query parameter method is recently deprecated and will be disallowed
in November 2020. Using them today triggers a warning email.
2020-09-16 13:46:48 -04:00
regnat
7d5bdf8b56 Make the store plugins more introspectable
Directly register the store classes rather than a function to build an
instance of them.
This gives the possibility to introspect static members of the class or
choose different ways of instantiating them.
2020-09-16 13:53:08 +02:00
Eelco Dolstra
82b77a7726
addPermRoot(): Remove indirect flag 2020-09-03 11:29:34 +02:00
Eelco Dolstra
6d7f7efb89
github: Use access token when calling .../commits API 2020-09-01 15:29:04 +02:00
Eelco Dolstra
e849b19872
Merge pull request #3932 from chkno/no-show-signature
Don't try to parse signature check as commit timestamp
2020-08-17 11:00:24 +02:00
Chuck
ed026f7206 Don't try to parse signature check as commit timestamp
When the log.showSignature git setting is enabled, the output of
"git log" contains signature verification information in addition to the
timestamp GitInputScheme::fetch wants:

  $ git log -1 --format=%ct
  gpg: Signature made Sat 07 Sep 2019 02:02:03 PM PDT
  gpg:                using RSA key 0123456789ABCDEF0123456789ABCDEF01234567
  gpg:                issuer "user@example.com"
  gpg: Good signature from "User <user@example.com>" [ultimate] 1567890123
  1567890123

For folks that had log.showSignature set, this caused all nix operations
on flakes to fail:

  $ nix build
  error: stoull
2020-08-13 17:44:42 -07:00