Commit graph

491 commits

Author SHA1 Message Date
Eelco Dolstra
f56a039775 * Use SSH connection sharing in the remote build script.
* Removed the Cygwin password hack since the problem is apparently
  fixed in Visual Studio.
2010-02-03 20:12:18 +00:00
Eelco Dolstra
bc1e478db1 * nix-copy-closure: start only one SSH connection to the server, or
recycle an already existing connection (using  OpenSSH's connection
  sharing feature).
2010-02-03 15:34:52 +00:00
Eelco Dolstra
e839802720 * Don't do a chdir to $tmpDir. It's not necessary, and Windows doesn't
support deleting the current directory.
2010-02-02 15:29:18 +00:00
Eelco Dolstra
50e34891f0 * Disable gzip compression in build-remote.pl because it puts too much
load on the Hydra build farm (where it's unnecessary anyway because
  it has a fast connection to the build machines).  In any case,
  compression can be enabled by using the `-C' option to ssh.
2010-01-25 16:14:45 +00:00
Eelco Dolstra
7ca9972636 * When doing a nix-pull, remove old manifests downloaded from the same
URL.  This prevents lots of old cruft accumulating in
  /nix/var/nix/manifests.
2009-12-09 19:36:54 +00:00
Eelco Dolstra
c4c84d1edb * nix-build: be less verbose. 2009-12-09 18:08:28 +00:00
Eelco Dolstra
997db91e07 * Don't pass -K. It should really inherit the setting of the calling
Nix though.
2009-11-17 16:22:39 +00:00
Eelco Dolstra
3392d32e8b * In nix-pull/nix-channel, create the manifests directory if it
doesn't exist.  The Debian packages don't include the manifests
  directory, so nix-channel would silently skip doing a nix-pull,
  resulting in everything being built from source.  Thanks to Juan
  Pedro Bolívar Puente.
2009-11-13 10:08:31 +00:00
Eelco Dolstra
e1df4ef73c 2009-09-17 17:02:14 +00:00
Eelco Dolstra
86408b3f47 * build-remote.pl: Pick machines in a round-robin order, rather than
giving jobs to the first machine until it hits its job limit, then
  the second machine and so on.  This should improve utilisation of
  the Hydra build farm a lot.  Also take an optional speed factor
  into account to cause fast machines to be preferred over slower
  machines with a similar load.
2009-09-17 15:48:17 +00:00
Eelco Dolstra
57e0d73c77 * build-remote.pl: allow the system type to be a comma-separated list
of system types.  Don't treat the x86_64-linux system type
  specially.
2009-09-17 13:51:04 +00:00
Eelco Dolstra
20b6f94b65 * nix-build: pass the --show-trace flag. 2009-07-15 09:10:38 +00:00
Eelco Dolstra
f24cf5d303 * nix-build: if nix-instantiate or nix-store fails due to a signal
(e.g. out of memory or a segfault), print an error message.
  Otherwise the user doesn't see anything.
2009-06-10 11:30:34 +00:00
Eelco Dolstra
c710fe540e * Machines of type x86_64-linux can do i686-linux builds. 2009-04-21 12:42:47 +00:00
Eelco Dolstra
0460ea4c39 * Cleanup. 2009-04-16 09:27:33 +00:00
Eelco Dolstra
096affb55b * Update the mtime on the lock file to make it easy to see when a slot
was last used.
2009-03-29 18:40:44 +00:00
Eelco Dolstra
737423a89c * Use polling to wait for a remote build slot when using a build hook
(that is, call the build hook with a certain interval until it
  accepts the build).
* build-remote.pl was totally broken: for all system types other than
  the local system type, it would send all builds to the *first*
  machine of the appropriate type.
2009-03-29 18:00:45 +00:00
Eelco Dolstra
47706e3924 * nix-copy-closure: compute the closure in one call to nix-store,
which is much faster.
2009-03-28 21:10:29 +00:00
Eelco Dolstra
3a2bbe7f8a * Simplify communication with the hook a bit (don't use file
descriptors 3/4, just use stdin/stderr).
2009-03-28 19:29:55 +00:00
Eelco Dolstra
7fb548aa26 * Clean up the output a bit. 2009-03-28 16:12:46 +00:00
Eelco Dolstra
7e05b8b75e * Future proofing: assume we can read manifests up to version 10
(which should therefore be backwards compatible).
2009-03-19 10:02:02 +00:00
Eelco Dolstra
b88460bcbc * Disregard the Hash field in manifests. 2009-03-19 09:47:34 +00:00
Eelco Dolstra
2897286487 * Unify exportReferencesGraph and exportBuildReferencesGraph, and make
sure that it works as expected when you pass it a derivation.  That
  is, we have to make sure that all build-time dependencies are built,
  and that they are all in the input closure (otherwise remote builds
  might fail, for example).  This is ensured at instantiation time by
  adding all derivations and their sources to inputDrvs and inputSrcs.
2009-03-18 17:36:42 +00:00
Eelco Dolstra
1dcf208f56 * Clean up some tests (use nix-build where appropriate). 2009-03-18 13:15:55 +00:00
Eelco Dolstra
1273d355ac * nix-install-package: don't pollute /nix/var/nix/manifests. 2009-02-27 14:06:38 +00:00
Eelco Dolstra
d4ca5c3952 * Use the regular progress bar; the hash bar isn't very useful when
the size of the download isn't known in advance.
2009-02-27 13:55:11 +00:00
Eelco Dolstra
fd2e14b3c8 * nix-build: support --option. 2009-02-27 12:09:30 +00:00
Eelco Dolstra
8ab6bc5a49 * nix-channel: use nix-build. 2009-02-27 11:01:03 +00:00
Eelco Dolstra
f052c10eed * Check the manifest version. 2009-02-27 09:53:58 +00:00
Eelco Dolstra
a7cee528c5 * Handle base-16 hashes in manifests. 2009-02-26 21:12:35 +00:00
Eelco Dolstra
041717eda3 * download-using-manifests: don't check the cryptographic hash of
downloaded files; rather, check the hash of the unpacked store
  path.

  When the server produces bzipped NAR archives on demand (like Hydra
  does), the hash of the file is not known in advance; it's streamed
  from the server.  Thus the manifest doesn't contain a hash for the
  bzipped NAR archive.  However, the server does know the hash of the
  *uncompressed* NAR archive (the "NarHash" field), since it's stored
  in the Nix database (nix-store -q --hash /nix/store/bla).  So we use
  that instead for checking the integrity of the download.
2009-02-19 23:46:37 +00:00
Michael Raskin
621093cb1c Replace wrong (w.r.t. PATH) sed call with in-shell substitution 2009-02-19 20:46:45 +00:00
Eelco Dolstra
1bb0f1e84b * Build hook: compress the transferred data. 2009-02-03 10:34:15 +00:00
Eelco Dolstra
b682fae9d9 * Build hooks: use nix-store --import. This prevents a redundant
scan for runtime dependencies (i.e. the local machine shouldn't do a
  scan that the remote machine has already done).  Also pipe directly
  into `nix-store --import': don't use a temporary file.
2009-02-02 17:24:10 +00:00
Eelco Dolstra
019176137f * When using a build hook, distinguish between transient failures
(e.g. an SSH connection problem) and permanent failures (i.e. the
  builder failed).  This matters to Hydra (it wants to know whether it
  makes sense to retry a build).
2009-01-13 11:39:09 +00:00
Eelco Dolstra
4ce692df88 2009-01-13 10:42:13 +00:00
Eelco Dolstra
d91dc086bb 2008-12-04 16:51:36 +00:00
Eelco Dolstra
9ac3f5df9c * Propagate --max-silent-time to remote machines. 2008-12-04 16:51:16 +00:00
Eelco Dolstra
bcfe98acff * Prefer building on a remote machine over a local machine. This
makes more sense for the build farm, otherwise every nix-store
  invocation will lead to at least one local build.  Will come up with
  a better solution later...
2008-12-04 16:35:47 +00:00
Eelco Dolstra
909fbb9de1 * When using build hooks, for any nix-store -r build operation, it is
necessary that at least one build hook doesn't return "postpone",
  otherwise nix-store will barf ("waiting for a build slot, yet there
  are no running children").  So inform the build hook when this is
  the case, so that it can start a build even when that would exceed
  the maximum load on a machine.
2008-12-04 14:29:41 +00:00
Eelco Dolstra
5dfba0b4db * Force allocation of a pseudo-terminal to clean up the remote
nix-store process when the connection is interrupted.
2008-12-04 13:36:52 +00:00
Eelco Dolstra
9fd9c4c635 * Support multiple system types per remote machine, e.g. a machine
list like

    root@example.org x86_64-linux /root/.ssh/id_buildfarm 1
    root@example.org i686-darwin  /root/.ssh/id_buildfarm 1

  This is possible when the Nix installation on example.org itself has
  remote builds enabled.
2008-12-04 12:20:06 +00:00
Eelco Dolstra
f8713e1287 * Dirty hack to make nix-push work properly on derivations: the
derivation should be a source rather than a derivation dependency of
  the call to the NAR derivation.  Otherwise the derivation (and all
  its dependencies) will be built as a side-effect, which may not even
  succeed.
2008-12-04 10:40:41 +00:00
Eelco Dolstra
6bbff48079 2008-11-20 17:22:42 +00:00
Eelco Dolstra
4213b8d8ec * Urgh. 2008-11-20 15:44:59 +00:00
Eelco Dolstra
aab530e971 * Primop builtins.storePath for declaring a store path as a
dependency.  `storePath /nix/store/bla' gives exactly the same
  result as `toPath /nix/store/bla', except that the former includes
  /nix/store/bla in the dependency context of the string.

  Useful in some generated Nix expressions like nix-push, which now
  finally does the right thing wrt distributed builds.  (Previously
  the path to be packed wasn't an explicit dependency, so it wouldn't
  be copied to the remote machine.)
2008-11-19 23:26:19 +00:00
Eelco Dolstra
fa791116a3 * Get rid of nix-pack-closure / nix-unpack-closure, they're redundant. 2008-11-18 14:43:40 +00:00
Eelco Dolstra
2b2aa8a820 * Doh. 2008-09-08 11:02:15 +00:00
Michael Raskin
2a01d06da6 Added nix-http-export.cgi to extra distributed scripts; so it can be installed from releases, not only from SVN. 2008-08-29 08:34:38 +00:00
Michael Raskin
b455c4c45c Updates to nix-reduce-build
Common code in local build package sources refactored out in a function; before building the real set of derivations needed is found (slightly slower for only one build strategy, but less garbage on output and better performance for multiple build strategies).

Now you have full choice of best-effort build regardless of method (substituters or actual build), using substituters, building only fixed derivations (should get you all the downloads) and local build without even trying substituters. 

Some minor fix in the help text about behavior with no package sources.
2008-08-06 19:43:53 +00:00
Eelco Dolstra
7592f48c83 * nix-build: `--dry-run' flag. 2008-08-04 13:46:01 +00:00
Eelco Dolstra
42043953c3 * Doh. 2008-08-04 13:15:47 +00:00
Michael Raskin
b3c26180e3 Updates to nix-reduce-build: only realize fixed derivations if user asks so, or only use substituters. Oh, and add possibility to use : for things like /etc/nixos/nixpkgs:-A:gnused 2008-08-02 16:43:25 +00:00
Eelco Dolstra
3c92ea399d * Make nix-env --dry-run print the paths to be substituted correctly
again.  (After the previous substituter mechanism refactoring I
  didn't update the code that obtains the references of substitutable
  paths.)  This required some refactoring: the substituter programs
  are now kept running and receive/respond to info requests via
  stdin/stdout.
2008-08-02 12:54:35 +00:00
Eelco Dolstra
096198d11f * A quick hack to make nix-prefetch-url support mirror:// URLs. It
requires that $NIXPKGS_ALL points at a Nixpkgs tree.
2008-07-23 16:02:58 +00:00
Eelco Dolstra
660244f65f * Make sure that copy-from-other-stores.pl is built. 2008-07-23 09:38:38 +00:00
Eelco Dolstra
989176c56e * Allow read-only access to the store (e.g., non-root users on NixOS
can do operations like "nix-store -qR <path>" even without the Nix
  daemon).
2008-07-18 15:34:46 +00:00
Eelco Dolstra
8bc591a6f0 * Use the copy-from-other-stores substituter by default. Of course,
it only does something if $NIX_OTHER_STORES (not really a good
  name...) is set.
* Do globbing on the elements of $NIX_OTHER_STORES.  E.g. you could
  set it to /mnts/*/nix or something.
* Install substituters in libexec/nix/substituters.
2008-07-18 13:05:10 +00:00
Eelco Dolstra
15f39aba8c * Quick prototype of a substituter that copies paths from other Nix
stores (typically remote Nix stores mounted via e.g. NFS, or the Nix
  store on the NixOS installation CD).  Example use:

  $ sshfs foo@example.org:/ /mnt
  $ NIX_OTHER_STORES=/mnt/nix \
    NIX_SUBSTITUTERS=.../copy-from-other-stores.pl \
    nix-env -i foo

  This will be especially useful for the installation CD since it
  doesn't require a manifest for the CD contents.
2008-07-12 18:58:24 +00:00
Michael Raskin
955b8841cd Also trying to build derivers in case we cannot get substituters 2008-06-14 08:48:40 +00:00
Michael Raskin
18e27629d3 Added local best-effort builds (i.e. one failure does not ruin all packages you would like to see built) 2008-06-14 08:30:35 +00:00
Michael Raskin
194c66eeeb Stupid error in script 2008-06-13 14:34:19 +00:00
Michael Raskin
f903d86740 OK, I will believe that fix does no worse.. 2008-06-13 13:53:14 +00:00
Michael Raskin
ce85b55cf0 Updated help text 2008-06-12 17:45:38 +00:00
Michael Raskin
4532e4b90d Added verbosity for nix-reduce-build 2008-06-12 16:26:53 +00:00
Michael Raskin
9819bb20da Added support for file:// archive (.nar.gz) repositories to nix-reduce-build. /tmp/nix-export created by nix-http-export.cgi is OK. 2008-05-11 15:54:30 +00:00
Michael Raskin
b4bc8b7616 --proxy=proxy:3128 2008-05-07 14:18:28 +00:00
Michael Raskin
b1e321d6ce Added http alternative transport for nix-reduce-build 2008-04-29 04:03:54 +00:00
Eelco Dolstra
f8985d195e * Fix for NIX-101 (should use an absolute path for call to nix-hash). 2008-03-28 17:52:33 +00:00
Eelco Dolstra
f106868110 * Cleanup. 2008-03-20 10:16:36 +00:00
Eelco Dolstra
d7caac3e81 * Don't install nix-reduce-build by default yet please, I first want
to understand better what it does...
2008-02-28 21:27:47 +00:00
Michael Raskin
11d512e7a8 Added nix-reduce-build. You point it to some path you want to build and it fetches whatever it can from specified computers via nix-copy-closure. NOTE: You do want to set up RSA keys or ssh-agent or something... You really do want it. It will run separate ssh instances insane number of times. 2008-02-27 21:26:47 +00:00
Eelco Dolstra
de012e76e2 * Don't use "store expression", it's obsolete. 2007-12-30 23:35:45 +00:00
Eelco Dolstra
c370755583 * Flag `--no-build-hook' to disable distributed builds.
* queryDeriver in daemon mode: don't barf if the other side returns an
  empty string (which means there is no deriver).
2007-11-16 16:15:26 +00:00
Eelco Dolstra
6784b14241 * nix-build: pass --argstr to nix-instantiate. 2007-11-15 16:52:40 +00:00
Eelco Dolstra
ca210d2a58 * Doh! 2007-11-15 15:33:30 +00:00
Eelco Dolstra
bfec245efa * Add build-remote.pl to the Nix distribution. 2007-11-15 14:28:08 +00:00
Eelco Dolstra
5b74a59570 * nix-prefetch-url: don't fail if /tmp/nix-prefetch-url-<pid> exists,
instead use a counter just like we do for temporary build
  directories.
2007-11-05 18:12:29 +00:00
Eelco Dolstra
f8ed2e64a5 2007-10-22 12:05:18 +00:00
Eelco Dolstra
a8629de827 * New command `nix-store --optimise' to reduce Nix store disk space
usage by finding identical files in the store and hard-linking them
  to each other.  It typically reduces the size of the store by
  something like 25-35%.  This is what the optimise-store.pl script
  did, but the new command is faster and more correct (it's safe wrt
  garbage collection and concurrent builds).
2007-10-09 22:14:27 +00:00
Eelco Dolstra
26f981c2e5 * Remove garbage. 2007-09-18 08:26:55 +00:00
Eelco Dolstra
055608227f * nix-env: allow ~/.nix-defexpr to be a directory. If it is, then the
Nix expressions in that directory are combined into an attribute set
  {file1 = import file1; file2 = import file2; ...}, i.e. each Nix
  expression is an attribute with the file name as the attribute
  name.  Also recurses into directories.

* nix-env: removed the "--import" (-I) option which set the
  ~/.nix-defexpr symlink.

* nix-channel: don't use "nix-env --import", instead symlink
  ~/.nix-defexpr/channels.  So finally nix-channel --update doesn't
  override any default Nix expressions but combines with them.

  This means that you can have (say) a local Nixpkgs SVN tree and use
  it as a default for nix-env:

  $ ln -s .../path-to-nixpkgs-tree ~/.nix-defexpr/nixpkgs_svn

  and be subscribed to channels (including Nixpkgs) at the same time.
  (If there is any ambiguity, the -A flag can be used to
  disambiguate, e.g. "nix-env -i -A nixpkgs_svn.pan".)
2007-09-17 16:08:24 +00:00
Eelco Dolstra
9441550acb * nix-push / generate-patches: bzip the manifest. 2007-09-04 15:38:09 +00:00
Eelco Dolstra
7d5836b34d * nix-channel: supports users who don't have write permission to the
manifests directory.  In that case, we don't do a nix-pull, so the
  user gets pure source deployment.

  The directory /nix/var/nix/gcroots/per-user/$USER should be
  writable.  (It's created automatically if
  /nix/var/nix/gcroots/per-user is writable, e.g. if it has 1777
  permission.)
2007-08-22 14:52:22 +00:00
Eelco Dolstra
dbc0170ed9 * Show errors in nix-prefetch-url. 2007-08-15 09:24:06 +00:00
Eelco Dolstra
ffa1c61cd5 * Fix the tests. 2007-08-14 13:15:59 +00:00
Eelco Dolstra
9e975458b4 * Get rid of the substitutes database table (NIX-47). Instead, if we
need any info on substitutable paths, we just call the substituters
  (such as download-using-manifests.pl) directly.  This means that
  it's no longer necessary for nix-pull to register substitutes or for
  nix-channel to clear them, which makes those operations much faster
  (NIX-95).  Also, we don't have to worry about keeping nix-pull
  manifests (in /nix/var/nix/manifests) and the database in sync with
  each other.

  The downside is that there is some overhead in calling an external
  program to get the substitutes info.  For instance, "nix-env -qas"
  takes a bit longer.

  Abolishing the substitutes table also makes the logic in
  local-store.cc simpler, as we don't need to store info for invalid
  paths.  On the downside, you cannot do things like "nix-store -qR"
  on a substitutable but invalid path (but nobody did that anyway).

* Never catch interrupts (the Interrupted exception).
2007-08-12 00:29:28 +00:00
Eelco Dolstra
4695f4edd6 * nix-pull: support bzipped manifests: when doing a nix-pull on $url,
try $url.bz2 first.
2007-08-10 01:42:00 +00:00
Eelco Dolstra
911bc01454 * Enable nix-prefetch-url caching in nix-channel. 2007-08-10 00:28:44 +00:00
Eelco Dolstra
5e52df18fe * Don't rely on /dev/stdin. 2007-08-10 00:22:21 +00:00
Eelco Dolstra
c5e934dcc9 * nix-pull: using nix-prefetch-url (so that we get caching for free),
and store the manifests in the Nix store.  (So now
  /nix/var/nix/manifests/ just contains symlinks to the store and is
  searched for GC roots.)
2007-08-09 23:52:53 +00:00
Eelco Dolstra
f881f7a017 * nix-prefetch-url: support caching. If the environment variable
NIX_DOWNLOAD_CACHE is set, then nix-prefetch-url will store the hash
  and timestamp of downloaded files in the directory
  $NIX_DOWNLOAD_CACHE.  This allows it to figure out if the file is
  still in the Nix store.
2007-08-09 23:16:44 +00:00
Eelco Dolstra
9bff7ad728 * Check against creation of GC roots in the store. Those roots don't
work, because findRoots() stops when it encounters a symlink to the
  store.  And of course the store is supposed to be read-only.
2007-06-11 11:36:22 +00:00
Eelco Dolstra
f3ebd03bb1 * Remove debug statement. 2007-06-04 12:03:33 +00:00
Eelco Dolstra
34d4c9388b 2007-05-02 16:06:04 +00:00
Eelco Dolstra
93aefd9fc0 * Give unpacked channels more sensible names than 0, 1, ... They now
get the basename of the channel URL (e.g., nixpkgs-unstable).  The
  top-level Nix expression of the channel is now an attribute set, the
  attributes of which are the individual channels (e.g.,
  {nixpkgs_unstable = ...; strategoxt_unstable = ...}).  This makes
  attribute paths ("nix-env -qaA" and "nix-env -iA") more sensible,
  e.g., "nix-env -iA nixpkgs_unstable.subversion".
2007-05-01 23:16:38 +00:00
Eelco Dolstra
4caca58ff7 * Make the maximum patch size configurable. 2007-03-30 09:01:05 +00:00
Eelco Dolstra
efd31139df * Forgot a @bindir@. 2007-03-27 09:53:58 +00:00
Eelco Dolstra
d303b389a9 * `nix-copy-closure --from': copy from a remote machine instead of to
a remote machine.
2007-03-26 21:05:17 +00:00
Eelco Dolstra
7edd2e2cd2 * Refactoring. 2007-03-26 20:49:22 +00:00
Eelco Dolstra
a8ea4cbcc8 * Scan /proc/sys/kernel/modprobe for roots to prevent the kernel
modules for the running kernel from being garbage-collected.  Idem
  for /proc/sys/kernel/fbsplash.
2007-03-20 11:13:15 +00:00
Eelco Dolstra
eb2dd4815c * Remove old generations in all directories under
/nix/var/nix/profiles, not just in that directory itself.  (NixOS
  puts profiles in /nix/var/nix/profiles/per-user.)
2007-03-13 11:30:57 +00:00
Eelco Dolstra
df0283ae86 * Get rid of those stupid --login tricks, it's the responsibility of
the remote system to make sure that Nix is in the $PATH.
2007-03-01 13:55:47 +00:00
Eelco Dolstra
30394a4f3f * sh -> bash. 2007-03-01 13:49:20 +00:00
Eelco Dolstra
363e307fd3 * Error message to stdout. 2007-02-26 23:32:10 +00:00
Eelco Dolstra
fa2be32034 * nix-copy-closure: force a login shell on the remote machine to make
sure that nix-store is in the PATH.
* nix-copy-closure: option --gzip to compress data.
2007-02-22 16:42:01 +00:00
Eelco Dolstra
4c5e6d1a2f * nix-copy-closure: option --sign.
* nix-copy-closure: set SSH options through NIX_SSHOPTS..
2007-02-22 15:48:20 +00:00
Eelco Dolstra
024a8ed382 * New command `nix-copy-closure' to copy a closure to a Nix store on
another machine through ssh.  E.g.,

    $ nix-copy-closure xyzzy $(which svn)

  copies the closure of Subversion to machine `xyzzy'.  This is like
  `nix-pack-closure $(which svn) | ssh xyzzy', but it's much more
  efficient since it only copies those paths that are missing on the
  target machine.
2007-02-21 23:14:53 +00:00
Eelco Dolstra
8181a1c3bb * Close the file - just in case. 2007-02-20 22:49:08 +00:00
Eelco Dolstra
bae75ca5a1 * New kind of manifest object: "localPath", which denotes that a store
path can be created by copying it from another location in the file
  system.  This is useful in the NixOS installation.
2007-01-23 16:50:19 +00:00
Eelco Dolstra
36d9258c0d * Successors have been gone for ages. 2007-01-23 16:05:59 +00:00
Eelco Dolstra
7bc30e1ca8 * nix-prefetch-url: change the default hash to SHA-256 (in base-32). 2007-01-22 09:53:36 +00:00
Eelco Dolstra
afe23b5f38 * nix-pack-closure: store the top-level store paths in the closure.
* nix-unpack-closure: extract the top-level paths from the closure and
  print them on stdout.  This allows them to be installed, e.g.,
  "nix-env -i $(nix-unpack-closure)".  (NIX-64)
2007-01-13 19:50:42 +00:00
Eelco Dolstra
215505bb46 * Removed chroot support. 2007-01-13 17:54:01 +00:00
Eelco Dolstra
1b7840b949 2007-01-11 19:28:28 +00:00
Eelco Dolstra
69c8b5b8a7 * Install generate-patches into libexec. 2007-01-11 16:19:45 +00:00
Eelco Dolstra
1f3722bd4a * Reject patches that are larger than a certain fraction of the full archive
(currently 60%).  Large patches aren't very economical.
2007-01-08 15:32:15 +00:00
Eelco Dolstra
50bdec410a * Huge speedup in patch propagation (20 minutes or so to 3 seconds). 2007-01-08 15:17:18 +00:00
Eelco Dolstra
cafaceb707 * Handle weird cases when the server redirects us while setting a cookie. 2006-12-15 21:27:26 +00:00
Eelco Dolstra
9dbfe242e3 * Kill a build if it has gone for more than a certain number of
seconds without producing output on stdout or stderr (NIX-65).  This
  timeout can be specified using the `--max-silent-time' option or the
  `build-max-silent-time' configuration setting.  The default is
  infinity (0).

* Fix a tricky race condition: if we kill the build user before the
  child has done its setuid() to the build user uid, then it won't be
  killed, and we'll potentially lock up in pid.wait().  So also send a
  conventional kill to the child.
2006-12-08 15:44:00 +00:00
Eelco Dolstra
d941186289 * Show more progress. 2006-11-18 19:03:45 +00:00
Eelco Dolstra
b3f916995a * Oops, `nix-build --no-out-link' was broken. 2006-10-31 18:45:17 +00:00
Eelco Dolstra
8d17265ac4 * Don't use EPSV. 2006-10-28 22:07:09 +00:00
Eelco Dolstra
b3d3700e11 * nix-build: check the exit status of `nix-store -r'. 2006-10-19 17:30:09 +00:00
Eelco Dolstra
2a535689fe * Reduce the maximum archive size for patch generation to 100 MB to
prevent trashing on nix.cs.uu.nl.
2006-10-12 20:13:29 +00:00
Eelco Dolstra
d98f750fd8 * tmpnam() -> File::Temp::tempdir(). 2006-10-04 18:58:11 +00:00
Eelco Dolstra
3632019b73 * Quick hack to let nix-install-package set the package name properly
(e.g., "java-front-0.9pre15899" instead of "java-front";
  particularly important when doing upgrades later on).
2006-09-25 14:00:59 +00:00
Eelco Dolstra
d43565c3e8 * In `nix-channel --update', skip manifests that assume a Nix store at
a different location than the user's.  This makes channels usable as
  a source deployment mechanism for people who install Nix under
  non-standard prefixes.  (NIX-57)
2006-09-25 11:11:16 +00:00
Eelco Dolstra
68ae953d8a * Clean up calls to system(). 2006-09-25 10:44:27 +00:00
Eelco Dolstra
76c9710091 * Use builtins.toPath. 2006-09-25 10:29:25 +00:00
Eelco Dolstra
0bd5eb71a0 * `nix-install-package --url': install from a URL (NIX-12).
* `nix-install-package --help' (NIX-9).
* `nix-install-package --non-interactive': don't prompt or pause.
* Tests for nix-install-package.
* Security fixes: filter the values obtained from the nixpkg.
2006-09-21 18:54:08 +00:00
Eelco Dolstra
ee5040421f * Try konsole and gnome-terminal in addition to xterm. 2006-09-21 11:29:14 +00:00
Eelco Dolstra
1b804f88e4 * Absolute path to rm (NIX-51).
* Don't hardcore /nix/bin and /nix/store.
2006-09-20 16:23:14 +00:00
Eelco Dolstra
947e645789 * Hide warnings about a missing "lsof" (NIX-54). 2006-09-20 15:14:19 +00:00
Eelco Dolstra
a060adf165 * Use paths, not strings, when calling the function that generates
NARs.  Fixes the impurity of nix-push (NIX-21).
* Better help.
2006-09-20 15:04:04 +00:00
Eelco Dolstra
7dd342e482 * Doh. 2006-09-19 16:40:22 +00:00
Eelco Dolstra
ee6cf99660 * Doh! Of course we have to take execute permission into account.
* Restore the mtime on modified directories.
2006-09-19 16:14:15 +00:00
Eelco Dolstra
6dbed1bf38 * `optimise-store.pl' reduces disk space consumption by hard-linking
all identitical files in the Nix store to each other.  (Previously
  it only computed the size that would be saved by doing so.)
2006-09-19 14:58:35 +00:00
Eelco Dolstra
07cec27848 * Cleanups. 2006-09-19 14:27:28 +00:00
Eelco Dolstra
9488ae7357 * `show-duplication.pl', a small utility that shows the amount of
package duplication present in (e.g.) a profile.  It shows the
  number of instances of each package in a closure, along with the
  size in bytes of each instance as well as the "waste" (the
  difference between the sum of the sizes of all instances and the
  average size).

  $ ./show-duplication.pl /nix/var/nix/profiles/default
  gcc 11
    3.3.6 19293318
    3.4.4 21425257
    ...
    average 14942970, waste 149429707
  coreutils 6
  ...
  average package duplication 1.87628865979381, total size 3486330471, total waste 1335324237, 38.3017114443825% wasted

  This utility is useful for measuring the cost in terms of disk space
  of the Nix approach.
2006-09-19 13:53:35 +00:00
Eelco Dolstra
2e210b2387 * Convenience option `nix-collect-garbage -d' (--delete-old): removes
old generations of *all* profiles in /nix/var/nix/profiles, then
  runs the garbage collector.  Quick way to get rid of all old stuff.
  Of course, one cannot roll back to earlier points in time after
  this.
2006-09-06 14:23:04 +00:00
Eelco Dolstra
3e239a37ff * file:/ -> file:// 2006-08-09 19:37:23 +00:00
Armijn Hemel
c0bfcbdd45 rework the --target flag. If this flag is used, a URI should be given. Default
values are not changed.
2006-08-08 15:42:33 +00:00
Eelco Dolstra
5a6b45e252 * In nar.nix, path -> storePath, otherwise we get a collision between
environment variable names on Cygwin (where they are case
  insensitive).
2006-08-07 18:22:57 +00:00
Armijn Hemel
a61129c48c add coreutils to the default PATH for this scripts, so we know for sure we have tools like rm, mkdir, and so on 2006-08-05 00:33:52 +00:00
Armijn Hemel
f1947cce93 prevent doing recursive chroots, by unsetting NIX_ROOT in the scripts. 2006-08-05 00:31:04 +00:00
Eelco Dolstra
f4a502a09a * Use old-school pipe opens; the new style is Perl >= 5.8.0. 2006-08-04 11:51:46 +00:00
Eelco Dolstra
981eff065b * Remove the dependency on `date', use strftime instead. 2006-08-04 11:45:50 +00:00
Armijn Hemel
fcb784051f urgh...for some weird reason this one-liner was not in svn. PEBKAC! 2006-08-03 11:48:09 +00:00
Eelco Dolstra
a29b64a231 * Better error checking. 2006-08-01 13:50:10 +00:00
Armijn Hemel
4fde308ec0 add a flag --target, so we can override the URL in the MANIFEST file. This
is only for local copies (so file:///)
2006-08-01 13:15:55 +00:00
Eelco Dolstra
4661282fde * `nix-instantiate ... --arg NAME VALUE': allow arguments to be passed
to functions from the command line.
* nix-build: started removing backticks.
2006-07-28 16:03:28 +00:00
Eelco Dolstra
dcded7da47 * Don't try to do DNS lookups. 2006-07-21 12:28:25 +00:00
Eelco Dolstra
ebcccbd358 * Added a tool to find additional roots for the garbage collector,
such as open files, current directories, mmaped files, etc.  This is
  inherently unportable, but it's easy to adapt this script to other
  platforms.  Currently we call `lsof' and try to read various bits in
  /proc/NNN.

  The goal is to prevent the garbage collector from removing store
  paths that are no longer reachable from a permanent root but that
  are still in use (for instance, after the user has done "nix-env -e"
  on a running program).
2006-07-19 16:49:59 +00:00
Eelco Dolstra
04cf72287b * This may be useful in the future. 2006-05-31 09:24:54 +00:00
Eelco Dolstra
e3c07782d1 * Remove old manifests in `nix-channel --update'. 2006-05-08 20:00:28 +00:00
Eelco Dolstra
05bb644890 * Allow the resulting symlink of nix-build to be named, e.g.,
$ nix-build .../i686-linux.nix -A apacheHttpd -o apache
2006-03-14 16:35:01 +00:00
Eelco Dolstra
9811815429 * Write messages to stderr, not stdout. 2006-03-10 09:41:28 +00:00
Eelco Dolstra
120f00c04f * More online help. 2006-03-03 14:25:07 +00:00
Eelco Dolstra
2d54312f87 * Rewrote nix-build in Perl, since sh is just too limited (turns out
that arrays are a bash extension, so it didn't work on FreeBSD).
  Also fixes NIX-8 (readlink(1) dependency).
2006-03-03 14:15:02 +00:00
Eelco Dolstra
85793fa438 * Remove debug statement. 2006-02-24 17:10:38 +00:00
Eelco Dolstra
e2f16b9cae * A script to remove from a manifest those patches whose base or
target no longer applies to any available release.  This is a
  partial fix for NIX-34 (when producing linear patch sequences
  between releases, the number of patches grows without bound).
2006-02-24 16:05:55 +00:00
Eelco Dolstra
5168f9bb00 * Canonicalise manifests a bit by sorting them. 2006-02-24 16:02:36 +00:00
Eelco Dolstra
43fa1c20e7 * Separate the cache reading code. 2006-02-22 14:35:19 +00:00
Eelco Dolstra
928cce5922 * Directory for nix.cs.uu.nl manifest / cache maintenance scripts. 2006-02-22 14:13:02 +00:00
Eelco Dolstra
fcec32a2c5 * Use right directories for NAR files and patches.
* Print errors to STDERR.
2006-02-22 14:10:02 +00:00
Eelco Dolstra
982399bb14 * Enable the --attr in nix-build as well (and add -A as an alias).
Example:

    $ nix-build ./all-packages.nix -A xlibs.libX11

  So finally it's easy to perform a test build of a Nix expression!
2006-02-10 17:37:35 +00:00
Eelco Dolstra
151f10dbc7 * Add @bindir@. 2006-02-01 12:41:54 +00:00
Eelco Dolstra
83424eb71b * Don't force a build of derivations. 2006-01-26 23:18:46 +00:00
Eelco Dolstra
e4d4969ae9 * New tools nix-pack-closure and nix-unpack-closure. These provide a
useful way to transfer the closure of a store path to another
  machine.

  These commands provide functionality previously possible through
  `nix-push --copy'.  However, they are much more convenient in many
  situations (though possibly less efficient).
  
  Example:
  $ nix-pack-closure /nix/store/hj232g1r...-subversion-1.3.0 > svn.closure
  (on another machine:)
  $ nix-unpack-closure < svn.closure

  Note that Subversion is added to the store, but not installed into a
  user environment.  One should do `nix-env -i
  /nix/store/hj232g1r...-subversion-1.3.0' for that.

  Another example: copy the application Azureus to the machine
  `scratchy' through ssh:
  
  $ nix-pack-closure $(which azureus) | ssh scratchy nix-unpack-closure
2006-01-12 15:17:51 +00:00
Eelco Dolstra
ab5c6bb3a3 * Change referer' to referrer' throughout. In particular, the
nix-store query options `--referer' and `--referer-closure' have
  been changed to `--referrer' and `--referrer-closure' (but the old
  ones are still accepted for compatibility).
2005-12-13 21:04:48 +00:00
Eelco Dolstra
dc528128cc * FreeBSD compatibility fix. 2005-11-17 13:58:23 +00:00
Eelco Dolstra
221c79013f * Turn off build hooks in nix-push because of an impurity (NIX-21). 2005-10-29 18:17:45 +00:00
Armijn Hemel
15ff877438 add @coreutils@ to correctly use coreutils to create a profile. This is needed
for NixOS, where we might not know our PATH in advance.
2005-10-11 17:30:57 +00:00
Eelco Dolstra
607a2f01e6 * Remove other uses of IPC::Open2. 2005-09-21 17:14:52 +00:00
Eelco Dolstra
95304172a5 * Don't use IPC::Open2, it has a subtle race bug on Mac OS X 10.4. If
the parent runs before the child, it closes some pipe file
  descriptors which causes the child to fail due to a bad file
  descriptor.  So we just use the normal open() function instead.
  
  This fixes NIX-14 (intermittent nix-pull failures).
2005-09-21 17:06:06 +00:00
Eelco Dolstra
d34fa9a6cc * Remove dead file. 2005-09-16 09:05:18 +00:00
Eelco Dolstra
846b53bde4 * Set the current directory to something well-defined. Might help in
setuid installations.
2005-09-16 09:03:24 +00:00
Eelco Dolstra
7f384d9c1b * Use a proper temporary directory. 2005-09-15 15:21:35 +00:00
Eelco Dolstra
a5ceb5bc0b * nix-build: default to `./default.nix' if no paths are specified.
So when using Nix as a build tool, you can just say `nix-build' and
  it will build the top-level derivation defined in `default.nix'.
2005-07-13 17:39:10 +00:00
Eelco Dolstra
928a7c06dc * Don't create patches for archives >= 150 MB because bsdiff can't
handle it.  It crashed on the 234 MB tetex archive.  Probably we
  will never be able to handle archives of that size on 32-bit
  machines (because bsdiff does everything in memory requiring
  max(17*n,9*n+m)+O(1) bytes, so the address space simply isn't
  there).
2005-06-18 14:20:24 +00:00
Eelco Dolstra
cbc8d083ac * Make unpacking of patch sequences much faster by not doing redundant
unpacking and repacking of intermediate paths.
2005-05-10 14:22:36 +00:00
Eelco Dolstra
9f3601a36c * Argh! The patch downloader was broken due to the renaming of the
`--isvalid' flag in nix-store.
2005-04-12 10:51:38 +00:00
Eelco Dolstra
d5219a351a * Damn. Disable the USE heuristic for now, since the deriver in the
database isn't always in the manifest (so the reference graph cannot
  be reconstructed fully).
2005-04-12 10:07:02 +00:00
Eelco Dolstra
c702dfca3f * nix-store: --substitute' -> --register-substitutes'. 2005-04-08 13:48:41 +00:00
Eelco Dolstra
7d876f8fa7 * Get rid of fetchurl, we don't need it anymore. 2005-04-07 14:35:44 +00:00
Eelco Dolstra
c815aff21b * `nix-store --add-fixed' to preload the outputs of fixed-output
derivations.  This is mostly to simplify the implementation of
  nix-prefetch-{url, svn}, which now work properly in setuid
  installations.

* Enforce valid store names in `nix-store --add / --add-fixed'.
2005-04-07 14:01:51 +00:00
Eelco Dolstra
31e140d70b * I said it couldn't be done. I was wrong. 2005-04-04 15:18:19 +00:00
Eelco Dolstra
ebe342c9c1 * Better error checking. 2005-03-25 14:30:01 +00:00
Eelco Dolstra
7272c3f817 * Ignore hash conflicts in gc-releases.pl. 2005-03-18 09:43:25 +00:00
Eelco Dolstra
e52ae1c0ff * Use SHA-256 for nix-push. 2005-03-15 11:12:48 +00:00
Eelco Dolstra
5675d5f488 * Idem. 2005-03-14 18:55:46 +00:00
Eelco Dolstra
6fb5f7e532 * Pass `--base32' unless using MD5. 2005-03-14 18:55:29 +00:00
Eelco Dolstra
bb2e53699f * Parse new hash format properly. 2005-03-14 17:05:42 +00:00
Eelco Dolstra
5863f24722 * Print SHA-1 hashes in base-32 by default. 2005-03-14 17:05:20 +00:00
Eelco Dolstra
bd333b939c * Prefix hash algorithm in patch generator too. 2005-03-14 16:46:19 +00:00
Eelco Dolstra
8eff18cd43 * Set NAR name to content hash; previous nix-push names were not
unique.
* Drop `hashAlgo' attribute in manifests; prefix hashes with the hash
  algorithm instead.
2005-03-14 15:09:53 +00:00
Eelco Dolstra
1562dfe9ba * Script to garbage collect nix-push directories. It prints out all
file names in the directory not included in any of the manifests
  specified on the command line.
2005-03-14 14:03:41 +00:00
Eelco Dolstra
536f324177 * nix-install-package: install outPath, not drvPath, for now.
* nix-prefecth-url: print out in base-16.
2005-03-11 15:27:37 +00:00
Eelco Dolstra
9e6bca8765 * Channel fix. 2005-03-03 13:10:52 +00:00