Commit graph

63 commits

Author SHA1 Message Date
Eelco Dolstra
f3cf0436b5 Install bsdiff and bspatch in $(libexecdir)/nix 2013-12-02 20:21:31 +00:00
Eelco Dolstra
5116214343 Add support for uncompressed NARs in binary caches
Issue NixOS/hydra#102.
2013-07-01 21:03:14 +02:00
Eelco Dolstra
22144afa8d Don't keep "disabled" substituters running
For instance, it's pointless to keep copy-from-other-stores running if
there are no other stores, or download-using-manifests if there are no
manifests.  This also speeds things up because we don't send queries
to those substituters.
2013-06-20 11:55:15 +02:00
Eelco Dolstra
167e36a5c3 nix-push: Only generate and copy a NAR if it doesn't already exist
This prevents unnecessary and slow rebuilds of NARs that already exist
in the binary cache.
2012-10-17 16:58:05 -04:00
Eelco Dolstra
9958bd6992 Add operation ‘nix-store --repair-path’
This operation allows fixing corrupted or accidentally deleted store
paths by redownloading them using substituters, if available.

Since the corrupted path cannot be replaced atomically, there is a
very small time window (one system call) during which neither the old
(corrupted) nor the new (repaired) contents are available.  So
repairing should be used with some care on critical packages like
Glibc.
2012-10-02 14:08:59 -04:00
Eelco Dolstra
c1f91570b3 Style fix 2012-09-21 15:02:33 -04:00
Eelco Dolstra
20582e9ae3 Support xz compression in the download-using-manifests substituter 2012-09-19 17:33:42 -04:00
Eelco Dolstra
9cd63d2244 Do some validation of URLs 2012-07-30 17:09:36 -04:00
Eelco Dolstra
73acb8b836 Let build.cc verify the expected hash of a substituter's output
Since SubstitutionGoal::finished() in build.cc computes the hash
anyway, we can prevent the inefficiency of computing the hash twice by
letting the substituter tell Nix about the expected hash, which can
then verify it.
2012-07-27 12:16:02 -04:00
Eelco Dolstra
3a8f841612 download-using-manifests: Don't use nix-prefetch-url
Instead call curl directly and pipe it into ‘nix-store --restore’.
This saves I/O and prevents creating garbage in the Nix store.
2012-07-27 10:47:36 -04:00
Eelco Dolstra
f2bdc87595 Update the other substituters 2012-07-11 18:52:09 -04:00
Eelco Dolstra
1749a7b0ae * download-using-manifests: use the Perl bindings. 2011-11-29 13:01:24 +00:00
Eelco Dolstra
5090c34ee1 * Set the executable bit on scripts. 2011-10-10 22:40:17 +00:00
Eelco Dolstra
6fcdbcac20 * Install NixManifest.pm, NixConfig.pm and GeneratePatches.pm under
the Nix:: namespace.
2011-10-10 21:11:08 +00:00
Eelco Dolstra
e649f3168b * Fix concurrency issues in download-using-manifests' handling of the
SQLite manifest cache.  The DBI AutoCommit feature caused every
  process to have an active transaction at all times, which could
  indefinitely block processes wanting to update the manifest cache.

* Disable fsync() in the manifest cache because we don't need
  integrity (the cache can always be recreated if it gets corrupted).
2011-07-13 14:05:54 +00:00
Eelco Dolstra
3cbf680f5d * configure: detect whether DBD::SQLite is present. If necessary the
location to DBI and DBD::SQLite can be passed with --with-dbi and
  --with-dbd-sqlite.
2011-04-11 10:13:53 +00:00
Eelco Dolstra
5591fcc529 * Cache the manifests in /nix/var/nix/manifests in a SQLite database.
This significantly speeds up the download-using-manifests
  substituter, especially if manifests are very large.  For instance,
  one "nix-build -A geeqie" operation that updated four packages using
  binary patches went from 18.5s to 1.6s.  It also significantly
  reduces memory use.

  The cache is kept in /nix/var/nix/manifests/cache.sqlite.  It's
  updated automatically when manifests are added to or removed from
  /nix/var/nix/manifests.  It might be interesting to have nix-pull
  store manifests directly in the DB, rather than storing them as
  separate flat files, but then we would need a command line interface
  to delete manifests from the DB.
2011-04-10 23:22:46 +00:00
Eelco Dolstra
1e7e4f21ba * Remove the localPaths feature in manifests since it's no longer used
and redundant anyway.
2011-04-06 09:16:22 +00:00
Eelco Dolstra
0423d0692a * Print a better error message. 2011-03-16 15:55:57 +00:00
Eelco Dolstra
d6c8b995c5 * In Hydra manifests the Size field is missing, so don't rely on it.
This caused a lot of "Use of uninitialized value" warnings from
  Perl.
2010-12-22 10:32:32 +00:00
Eelco Dolstra
542fc69062 * When doing a query (e.g. `nix-store -r --dry-run'), don't make a lot
of expensive calls to `nix-store --check-validity'.
2010-12-13 08:39:10 +00:00
Eelco Dolstra
365f3028dd * Use CamelCase for the Perl modules. 2010-12-05 17:50:29 +00:00
Eelco Dolstra
1a211d812f * Oops. 2010-11-17 17:54:49 +00:00
Eelco Dolstra
a4f0365b2d * When checking whether a patch is applicable, for efficiency, use
`nix-store -q --hash' to get the hash of the base path rather than
  `nix-hash'.  However, only do this for estimating the size of a
  download, not for the actual substitution, because sometimes the
  contents of store paths are modified (which they shouldn't, of
  course).
2010-11-17 17:41:59 +00:00
Eelco Dolstra
3d38a49840 * In the download size indication, take binary patches into account.
Hopefully this doesn't slow things down too much.
2010-11-17 17:32:25 +00:00
Eelco Dolstra
bdf089f463 * Before a build, show the disk space that the downloaded store paths
will approximately require.
2010-11-17 14:31:42 +00:00
Eelco Dolstra
719cebcac2 * "Fix" incorrect help message. 2010-02-04 09:38:09 +00:00
Eelco Dolstra
e839802720 * Don't do a chdir to $tmpDir. It's not necessary, and Windows doesn't
support deleting the current directory.
2010-02-02 15:29:18 +00:00
Eelco Dolstra
7e05b8b75e * Future proofing: assume we can read manifests up to version 10
(which should therefore be backwards compatible).
2009-03-19 10:02:02 +00:00
Eelco Dolstra
1273d355ac * nix-install-package: don't pollute /nix/var/nix/manifests. 2009-02-27 14:06:38 +00:00
Eelco Dolstra
a7cee528c5 * Handle base-16 hashes in manifests. 2009-02-26 21:12:35 +00:00
Eelco Dolstra
041717eda3 * download-using-manifests: don't check the cryptographic hash of
downloaded files; rather, check the hash of the unpacked store
  path.

  When the server produces bzipped NAR archives on demand (like Hydra
  does), the hash of the file is not known in advance; it's streamed
  from the server.  Thus the manifest doesn't contain a hash for the
  bzipped NAR archive.  However, the server does know the hash of the
  *uncompressed* NAR archive (the "NarHash" field), since it's stored
  in the Nix database (nix-store -q --hash /nix/store/bla).  So we use
  that instead for checking the integrity of the download.
2009-02-19 23:46:37 +00:00
Eelco Dolstra
4213b8d8ec * Urgh. 2008-11-20 15:44:59 +00:00
Eelco Dolstra
2b2aa8a820 * Doh. 2008-09-08 11:02:15 +00:00
Eelco Dolstra
42043953c3 * Doh. 2008-08-04 13:15:47 +00:00
Eelco Dolstra
3c92ea399d * Make nix-env --dry-run print the paths to be substituted correctly
again.  (After the previous substituter mechanism refactoring I
  didn't update the code that obtains the references of substitutable
  paths.)  This required some refactoring: the substituter programs
  are now kept running and receive/respond to info requests via
  stdin/stdout.
2008-08-02 12:54:35 +00:00
Eelco Dolstra
989176c56e * Allow read-only access to the store (e.g., non-root users on NixOS
can do operations like "nix-store -qR <path>" even without the Nix
  daemon).
2008-07-18 15:34:46 +00:00
Eelco Dolstra
f8ed2e64a5 2007-10-22 12:05:18 +00:00
Eelco Dolstra
9e975458b4 * Get rid of the substitutes database table (NIX-47). Instead, if we
need any info on substitutable paths, we just call the substituters
  (such as download-using-manifests.pl) directly.  This means that
  it's no longer necessary for nix-pull to register substitutes or for
  nix-channel to clear them, which makes those operations much faster
  (NIX-95).  Also, we don't have to worry about keeping nix-pull
  manifests (in /nix/var/nix/manifests) and the database in sync with
  each other.

  The downside is that there is some overhead in calling an external
  program to get the substitutes info.  For instance, "nix-env -qas"
  takes a bit longer.

  Abolishing the substitutes table also makes the logic in
  local-store.cc simpler, as we don't need to store info for invalid
  paths.  On the downside, you cannot do things like "nix-store -qR"
  on a substitutable but invalid path (but nobody did that anyway).

* Never catch interrupts (the Interrupted exception).
2007-08-12 00:29:28 +00:00
Eelco Dolstra
bae75ca5a1 * New kind of manifest object: "localPath", which denotes that a store
path can be created by copying it from another location in the file
  system.  This is useful in the NixOS installation.
2007-01-23 16:50:19 +00:00
Eelco Dolstra
36d9258c0d * Successors have been gone for ages. 2007-01-23 16:05:59 +00:00
Eelco Dolstra
215505bb46 * Removed chroot support. 2007-01-13 17:54:01 +00:00
Eelco Dolstra
d98f750fd8 * tmpnam() -> File::Temp::tempdir(). 2006-10-04 18:58:11 +00:00
Eelco Dolstra
68ae953d8a * Clean up calls to system(). 2006-09-25 10:44:27 +00:00
Armijn Hemel
f1947cce93 prevent doing recursive chroots, by unsetting NIX_ROOT in the scripts. 2006-08-05 00:31:04 +00:00
Eelco Dolstra
981eff065b * Remove the dependency on `date', use strftime instead. 2006-08-04 11:45:50 +00:00
Eelco Dolstra
846b53bde4 * Set the current directory to something well-defined. Might help in
setuid installations.
2005-09-16 09:03:24 +00:00
Eelco Dolstra
7f384d9c1b * Use a proper temporary directory. 2005-09-15 15:21:35 +00:00
Eelco Dolstra
cbc8d083ac * Make unpacking of patch sequences much faster by not doing redundant
unpacking and repacking of intermediate paths.
2005-05-10 14:22:36 +00:00
Eelco Dolstra
9f3601a36c * Argh! The patch downloader was broken due to the renaming of the
`--isvalid' flag in nix-store.
2005-04-12 10:51:38 +00:00