Commit graph

19 commits

Author SHA1 Message Date
Eelco Dolstra
ba5929c7be Merge InputAccessor into SourceAccessor
After the removal of the InputAccessor::fetchToStore() method, the
only remaining functionality in InputAccessor was `fingerprint` and
`getLastModified()`, and there is no reason to keep those in a
separate class.
2024-05-03 12:14:01 +02:00
Bouke van der Bijl
1a76ca4161 Set the origin instead of hacking in the URL resolving 2024-04-11 14:22:12 +02:00
Eelco Dolstra
cabee98152 Tarball fetcher: Use the content-addressed Git cache
Backported from the lazy-trees branch.
2024-02-20 12:57:36 +01:00
Eelco Dolstra
78b8db72b5 Remove tarball-cache.{hh,cc}
TarballInfo is only used in github.cc, and getTarballCache() is a bit
too trivial to have its own file.
2024-02-15 21:58:08 +01:00
John Ericson
ed24baaec4 Finish separating concerns with tarball cache
There is no longer an `importTarball` method. Instead, there is a
`unpackTarfileToSink` function (back in libutil). The caller can use
thisw with the `getParseSink` method we added in the last commit easily
enough.

In addition, tarball cache functionality is separated from `git-utils`
and moved into `tarball-cache`. This ensures we are separating mechanism
and policy.
2024-02-15 10:34:01 -05:00
John Ericson
ba6a5f06ee Split GitRepoImpl::importTarball
There is now a separation of:

1. A `FileSystemObjectSink` for writing to git repos

2. Adapting libarchive to use that parse sink.

The prepares a proper separation of concerns.
2024-02-15 10:27:54 -05:00
Eelco Dolstra
54354eaecf Merge remote-tracking branch 'origin/master' into tarball-cache 2024-02-14 14:45:19 +01:00
John Ericson
4687beecef Get rid of CanonPath::fromCwd
As discussed in the last Nix team meeting (2024-02-95), this method
doesn't belong because `CanonPath` is a virtual/ideal absolute path
format, not used in file systems beyond the native OS format for which a
"current working directory" is defined.

Progress towards #9205
2024-02-08 11:01:41 -05:00
Robert Hensing
7774eff10e libfetchers/git: Move workdir accessor into GitRepo::getAccessor 2024-01-12 15:31:53 +01:00
Robert Hensing
ce6d58a97c git fetcher: Add exportIgnore parameter
Enabled for fetchGit, which historically had this behavior,
among other behaviors we do not want in fetchGit.

fetchTree disables this parameter by default. It can choose the
simpler behavior, as it is still experimental.

I am not confident that the filtering implementation is future
proof. It should reuse a source filtering wrapper, which I believe
Eelco has already written, but not merged yet.
2024-01-12 15:31:52 +01:00
Eelco Dolstra
b36857ac8d Add a Git-based content-addressed tarball cache
GitArchiveInputScheme now streams tarballs into a Git repository. This
deduplicates data a lot, e.g. when you're fetching different revisions
of the Nixpkgs repo. It also warns if the tree hash returned by GitHub
doesn't match the tree hash of the imported tarball.
2023-11-29 12:37:03 +01:00
Eelco Dolstra
7ab91e7238 Implement shallow fetching 2023-11-15 15:00:17 +01:00
Eelco Dolstra
2964a9f562 Fix relative submodule handling
Tested on

  nix flake prefetch 'git+https://github.com/blender/blender.git?rev=4ed8a360e956daf2591add4d3c9ec0719e2628fe&submodules=1'
2023-11-14 16:00:21 +01:00
Eelco Dolstra
98a120b8b8 Merge remote-tracking branch 'origin/master' into libgit2 2023-11-09 16:48:41 +01:00
Eelco Dolstra
0c5eac9c45 Git fetcher: Handle submodules for workdirs 2023-10-31 15:59:25 +01:00
Eelco Dolstra
d88106df24 Git fetcher: Improve submodule handling
Instead of making a complete copy of the repo, fetching the
submodules, and writing the result to the store (which is all
superexpensive), we now fetch the submodules recursively using the Git
fetcher, and return a union accessor that "mounts" the accessors for
the submodules on top of the root accessor.
2023-10-31 14:52:21 +01:00
Eelco Dolstra
ee36a44bf2 GitInputScheme: Use libgit2
This replaces most calls to the "git" binary with libgit2.
2023-10-31 14:52:21 +01:00
Théophane Hufschmitt
e68676e6c8 Fix the parsing of the sourcehut refs file
Since a26be9f3b8, the same parser is used
to parse the result of sourcehut’s `HEAD` endpoint (coming from [git
dumb protocol]) and the output of `git ls-remote`. However, they are very
slightly different (the former doesn’t specify the current reference
since it’s implied to be `HEAD`).

Unify both, and make the parser a bit more robust and understandable (by
making it more typed and adding tests for it)

[git dumb protocol]: https://git-scm.com/book/en/v2/Git-Internals-Transfer-Protocols#_the_dumb_protocol
2022-05-04 14:38:59 +02:00
Kjetil Orbekk
9bf296c970 Extract git reference parsing to a shared library
These utility functions can be shared between the git and github fetchers.
2022-04-29 18:46:21 -04:00