Commit graph

365 commits

Author SHA1 Message Date
Eelco Dolstra
d342de02b9
fetchFlake: Use infinite TTL when the revision is specified 2019-02-25 23:23:45 +08:00
Eelco Dolstra
529add316c
downloadCached: Return ETag
This allows fetchFlake() to return the Git revision of a GitHub
archive.
2019-02-25 23:20:50 +08:00
Eelco Dolstra
6e9182fbc2
Add basic flake lock file support 2019-02-12 22:43:22 +01:00
Eelco Dolstra
e38ec77de8
Interpret all file:// URIs as Git repositories 2019-02-12 22:06:19 +01:00
Eelco Dolstra
beab05851b
nix: Add --flake flag
This allows using an arbitrary "provides" attribute from the specified
flake. For example:

  nix build --flake nixpkgs packages.hello

(Maybe provides.packages should be used for consistency...)
2019-02-12 21:55:43 +01:00
Eelco Dolstra
272b58220d
Enforce use of immutable flakes in pure mode
... plus a temporary hack to allow impure flakes at top-level for the
default installation source.
2019-02-12 21:07:48 +01:00
Eelco Dolstra
ba05f29838
nix: Enable pure mode by default
We want to encourage a brave new world of hermetic evaluation for
source-level reproducibility, so flakes should not poke around in the
filesystem outside of their explicit dependencies.

Note that the default installation source remains impure in that it
can refer to mutable flakes, so "nix build nixpkgs.hello" still works
(and fetches the latest nixpkgs, unless it has been pinned by the
user).

A problem with pure evaluation is that builtins.currentSystem is
unavailable. For the moment, I've hard-coded "x86_64-linux" in the
nixpkgs flake. Eventually, "system" should be a flake function
argument.
2019-02-12 20:35:03 +01:00
Eelco Dolstra
91a6a47b0e
Improve flake references 2019-02-12 18:23:11 +01:00
Eelco Dolstra
c8a0b9d5cb
experimental/optional -> optional 2019-02-12 13:43:32 +01:00
Eelco Dolstra
52419f8db3
Add flake registry
This will eventually be moved to nixos.org.
2019-02-11 13:01:39 +01:00
Eelco Dolstra
dcae46ab14
Add github URIs
For example, github:edolstra/dwarffs is more-or-less equivalent to
https://github.com/edolstra/dwarffs.git. It's a much faster way to get
GitHub repositories: it fetches tarballs rather than entire Git
repositories. It also allows fetching specific revisions by hash
without specifying a ref (e.g. a branch name):

  github:edolstra/dwarffs/41c0c1bf292ea3ac3858ff393b49ca1123dbd553
2019-02-11 12:05:17 +01:00
Eelco Dolstra
ef4cf4e681
Introduce flake URIs 2019-02-11 12:05:13 +01:00
Eelco Dolstra
7a5cf31060
Initial flake support 2019-02-11 12:00:13 +01:00
Shea Levy
b30be6b450
Add builtins.appendContext.
A partner of builtins.getContext, useful for the same reasons.
2019-01-31 08:52:23 -05:00
Shea Levy
1d757292d0
Add builtins.getContext.
This can be very helpful when debugging, as well as enabling complex
black magic like surgically removing a single dependency from a
string's context.
2019-01-14 11:27:10 -05:00
Daiderd Jordan
7e35e914c1
fetchGit: allow fetching explicit refs
Trying to fetch refs that are not in refs/heads currently fails because
it looks for refs/heads/refs/foo instead of refs/foo.

eg.

	builtins.fetchGit {
	  url = https://github.com/NixOS/nixpkgs.git;
	  ref = "refs/pull/1024/head;
	}
2018-12-14 20:12:22 +01:00
CHEIKH Chawki
fa5143c722 Solve hg "abandoned transaction" issue 2018-12-06 13:57:59 +01:00
Eelco Dolstra
4aee93d5ce
fetchGit: Drop unnecessary localRef 2018-11-20 20:59:44 +01:00
Eelco Dolstra
3f4de91d80
Merge branch 'better-git-cache' of https://github.com/graham-at-target/nix 2018-11-20 20:41:19 +01:00
Eelco Dolstra
475a0a54a9
fetchGit/fetchMercurial: Don't absolutize paths
This is already done by coerceToString(), provided that the argument
is a path (e.g. 'fetchGit ./bla'). It fixes the handling of URLs like
git@github.com:owner/repo.git. It breaks 'fetchGit "./bla"', but that
was never intended to work anyway and is inconsistent with other
builtin functions (e.g. 'readFile "./bla"' fails).
2018-09-01 00:19:49 +02:00
Graham Christensen
02098d2073 fetchGit: use a better caching scheme
The current usage technically works by putting multiple different
repos in to the same git directory. However, it is very slow as
Git tries very hard to find common commits between the two
repositories. If the two repositories are large (like Nixpkgs and
another long-running project,) it is maddeningly slow.

This change busts the cache for existing deployments, but users
will be promptly repaid in per-repository performance.
2018-08-17 11:27:34 -04:00
Eelco Dolstra
c2de2ff385
Fix build on 32-bit systems and macOS
Apparently, on macOS, 'long' != 'int64_t'.

https://hydra.nixos.org/build/77100756
2018-07-11 21:12:18 +02:00
Eelco Dolstra
efd04888ca
Shup up a warning 2018-07-11 21:05:09 +02:00
Eelco Dolstra
ea3c9dab5f
Include cpptoml for build simplicity 2018-07-03 18:39:36 +02:00
Eelco Dolstra
3b1f54cf06
Add a fromTOML primop
This is primarily useful for processing Cargo.lock files.
2018-07-03 18:39:36 +02:00
Eelco Dolstra
1672bcd230
Move evaluator-specific settings out of libstore 2018-05-30 13:29:50 +02:00
Eelco Dolstra
138af2e554
Shut up signedness warning 2018-03-19 11:57:15 +01:00
Guillaume Maudoux
80735c4cc9
fetchGit: Fix debug message 2018-03-13 10:28:23 +01:00
Will Dietz
e89d02bf03 fetchGit: use "HEAD" as default ref 2018-02-28 16:34:34 -06:00
Eelco Dolstra
d4dcffd643
Add pure evaluation mode
In this mode, the following restrictions apply:

* The builtins currentTime, currentSystem and storePath throw an
  error.

* $NIX_PATH and -I are ignored.

* fetchGit and fetchMercurial require a revision hash.

* fetchurl and fetchTarball require a sha256 attribute.

* No file system access is allowed outside of the paths returned by
  fetch{Git,Mercurial,url,Tarball}. Thus 'nix build -f ./foo.nix' is
  not allowed.

Thus, the evaluation result is completely reproducible from the
command line arguments. E.g.

  nix build --pure-eval '(
    let
      nix = fetchGit { url = https://github.com/NixOS/nixpkgs.git; rev = "9c927de4b179a6dd210dd88d34bda8af4b575680"; };
      nixpkgs = fetchGit { url = https://github.com/NixOS/nixpkgs.git; ref = "release-17.09"; rev = "66b4de79e3841530e6d9c6baf98702aa1f7124e4"; };
    in (import (nix + "/release.nix") { inherit nix nixpkgs; }).build.x86_64-linux
  )'

The goal is to enable completely reproducible and traceable
evaluation. For example, a NixOS configuration could be fully
described by a single Git commit hash. 'nixos-rebuild' would do
something like

  nix build --pure-eval '(
    (import (fetchGit { url = file:///my-nixos-config; rev = "..."; })).system
  ')

where the Git repository /my-nixos-config would use further fetchGit
calls or Git externals to fetch Nixpkgs and whatever other
dependencies it has. Either way, the commit hash would uniquely
identify the NixOS configuration and allow it to reproduced.
2018-01-16 19:23:18 +01:00
Will Dietz
428680b307 fetchGit: fix creation of uninitialized cache dir, let git create it
fetchGit test (as modified in previous commit) now passes.
2018-01-09 09:05:18 -06:00
Will Dietz
2e6f06c37e fetchGit: Fix handling of local repo when not using 'master' branch
Add tests checking this behavior.
2017-12-22 15:29:52 -06:00
Shea Levy
eedbc4e06c
fetchGit: Ignore tarballTtl if rev is set and not in the repo.
Fixes #1697.
2017-11-24 06:09:24 -05:00
Eelco Dolstra
d7da6c9ea9
fetchGit/fetchMercurial: Fix directory inclusion check
E.g. the existence of .gitignore would cause .git to be included.
2017-11-21 19:34:46 +01:00
Eelco Dolstra
7a4d9574d9
fetchgit.cc -> fetchGit.cc 2017-11-03 13:55:31 +01:00
Eelco Dolstra
6cf7c6a6b0
Remove git:// support in NIX_PATH
This didn't support specifying a revision/branch, and was restricted
to git:// URIs (since https:// or ssh:// would be ambiguous).
2017-11-03 13:55:31 +01:00
Eelco Dolstra
ee6ac38848
fetchGit/fetchMercurial: Filter out directories with untracked files 2017-11-03 13:55:31 +01:00
Eelco Dolstra
4dee01da7c
fetchGit: Add a test 2017-11-03 13:55:30 +01:00
Eelco Dolstra
0e77aa3982
fetchGit: Don't do a remote fetch if we already have the rev 2017-11-03 13:55:30 +01:00
tv
5ab37f0e44
fetchMercurial: fix error message 2017-11-02 23:37:42 +01:00
Eelco Dolstra
212e72c609 Fix build
https://hydra.nixos.org/build/63172338
2017-11-01 21:32:30 +01:00
Eelco Dolstra
e026bc3b05
fetchMercurial: Don't fetch hashes we already have 2017-11-01 18:43:11 +01:00
Eelco Dolstra
1969f357b7
Add fetchMercurial primop
E.g.

  $ nix eval '(fetchMercurial https://www.mercurial-scm.org/repo/hello)'
  { branch = "default"; outPath = "/nix/store/alvb9y1kfz42bjishqmyy3pphnrh1pfa-source"; rev = "82e55d328c8ca4ee16520036c0aaace03a5beb65"; revCount = 1; shortRev = "82e55d328c8c"; }

  $ nix eval '(fetchMercurial { url = https://www.mercurial-scm.org/repo/hello; rev = "0a04b987be5ae354b710cefeba0e2d9de7ad41a9"; })'
  { branch = "default"; outPath = "/nix/store/alvb9y1kfz42bjishqmyy3pphnrh1pfa-source"; rev = "0a04b987be5ae354b710cefeba0e2d9de7ad41a9"; revCount = 0; shortRev = "0a04b987be5a"; }

  $ nix eval '(fetchMercurial /tmp/unclean-hg-tree)'
  { branch = "default"; outPath = "/nix/store/cm750cdw1x8wfpm3jq7mz09r30l9r024-source"; rev = "0000000000000000000000000000000000000000"; revCount = 0; shortRev = "000000000000"; }
2017-11-01 17:45:32 +01:00
Eelco Dolstra
72cd52c3cd
builtins.fetchgit: Support importing a working tree
For example, you can write

  src = fetchgit ./.;

and if ./. refers to an unclean working tree, that tree will be copied
to the Nix store. This removes the need for "cleanSource".
2017-10-30 19:59:25 +01:00
Eelco Dolstra
63c80ae26f
Make "fetchGit /path" work 2017-10-30 13:18:28 +01:00
Eelco Dolstra
a5c392a80e
fetchGit: Fix broken assertion
Different URIs can map to the same cache entry if they have the same
revision.
2017-10-30 12:55:46 +01:00
Eelco Dolstra
812e027e1d
Add option allowed-uris
This allows network access in restricted eval mode.
2017-10-30 12:41:49 +01:00
Eelco Dolstra
e38382895d
builtins.fetchGit: Return an attrset with revision info
This adds rev, shortRev and revCount attributes, equal to what Hydra
provides. E.g.

  $ nix eval '(fetchGit https://github.com/NixOS/patchelf.git)'
  { outPath = "/nix/store/ghigrkw02l440g8vfxa9wj4c3zpfmw99-source"; rev = "29c085fd9d3fc972f75b3961905d6b4ecce7eb2b"; revCount = 303; shortRev = "29c085f"; }
2017-10-30 11:49:03 +01:00
Eelco Dolstra
049322702b
fetchgit -> fetchGit
Almost all other primops are camelCase so no reason not to use that
here.
2017-10-30 10:25:08 +01:00
Eelco Dolstra
f25791c196
builtins.fetchgit: Don't mess up the progress indicator 2017-10-30 10:22:58 +01:00
Eelco Dolstra
65b5f177b5
builtins.fetchgit: Support a "name" attribute
The "name" attribute defaults to "source", which we should use for all
similar functions (e.g. fetchTarball and in Hydra) to ensure that we
get a consistent store path regardless of how the tree is fetched.

"source" is not necessarily a correct label, but using an empty name
is problematic: you get an ugly store path ending in a dash, and it's
impossible to have a fixed-output derivation that produces that path
because ".drv" is not a valid store name.

Fixes #904.
2017-10-30 10:22:07 +01:00
Shea Levy
cbc216911d
fetchgit: Remove incomplete/unneeded ref check.
This check fails for tags and branches, and is made redundant by the
checks git itself will do when fetching the repo.
2017-10-17 12:30:54 -04:00
Shea Levy
4e58294ae6
fetchgit: Remove incomplete/unneeded isURI check.
This check spuriously fails for e.g. git@github.com:NixOS/nixpkgs.git,
and even for ssh://git@github.com/NixOS/nixpkgs.git, and is made
redundant by the checks git itself will do when fetching the repo. We
instead pass a -- before passing the URI to git to avoid injection.
2017-10-16 14:35:28 -04:00
Jörg Thalheim
2fd8f8bb99 Replace Unicode quotes in user-facing strings by ASCII
Relevant RFC: NixOS/rfcs#4

$ ag -l | xargs sed -i -e "/\"/s/’/'/g;/\"/s/‘/'/g"
2017-07-30 12:32:45 +01:00
Eelco Dolstra
7480f4f9a4
builtins.fetchgit: Support specifying commit hashes
This adds an argument "rev" specififying the Git commit hash. The
existing argument "rev" is renamed to "ref". The default value for
"ref" is "master". When specifying a hash, it's necessary to specify a
ref since we're not cloning the entire repository but only fetching a
specific ref.

Example usage:

  builtins.fetchgit {
    url = https://github.com/NixOS/nixpkgs.git;
    ref = "release-16.03";
    rev = "c1c0484041ab6f9c6858c8ade80a8477c9ae4442";
  };
2017-07-27 18:08:23 +02:00
Eelco Dolstra
9f64cb89cb
builtins.fetchgit: Respect tarball-ttl
I.e. if the local ref is more recent than tarball-ttl seconds, then
don't check the remote.
2017-07-27 17:23:29 +02:00
Eelco Dolstra
69deca194e
builtins.fetchgit: Use proper refs locally 2017-07-27 17:02:25 +02:00
Eelco Dolstra
6d7de7f3de
builtins.fetchgit: Cache hash -> store path mappings
This prevents an expensive call to addToStore() in the cached case.
2017-07-27 16:16:08 +02:00
Eelco Dolstra
b01d62285c
Improve progress indicator 2017-05-16 16:09:57 +02:00
Eelco Dolstra
ffcf9d24a6
builtins.fetchgit: Support paths 2017-03-02 11:46:28 +01:00
Eelco Dolstra
fbf17f1ad7
builtins.fetchgit: Fix bad format string 2017-03-02 11:40:11 +01:00
Eelco Dolstra
215b70f51e
Revert "Get rid of unicode quotes (#1140)"
This reverts commit f78126bfd6. There
really is no need for such a massive change...
2016-11-26 00:38:01 +01:00
Guillaume Maudoux
f78126bfd6 Get rid of unicode quotes (#1140) 2016-11-25 15:48:27 +01:00
Eelco Dolstra
d8bf0d4859 Support Git repos in the Nix path
E.g.

  $ nix-build -I nixpkgs=git://github.com/NixOS/nixpkgs '<nixpkgs>' -A hello

This is not extremely useful yet because you can't specify a
branch/revision.
2016-04-29 21:07:47 +02:00
Eelco Dolstra
38539b943a Add fetchgit builtin
The function builtins.fetchgit fetches Git repositories at evaluation
time, similar to builtins.fetchTarball. (Perhaps the name should be
changed, being confusing with respect to Nixpkgs's fetchgit function,
with works at build time.)

Example:

  (import (builtins.fetchgit git://github.com/NixOS/nixpkgs) {}).hello

or

  (import (builtins.fetchgit {
    url = git://github.com/NixOS/nixpkgs-channels;
    rev = "nixos-16.03";
  }) {}).hello

Note that the result does not contain a .git directory.
2016-04-29 20:47:36 +02:00