Commit graph

5048 commits

Author SHA1 Message Date
Eelco Dolstra
8052aef486 * A test for multiple and/or failing substitutes. 2004-06-24 12:56:24 +00:00
Eelco Dolstra
66c7f34759 * Arghhhhhh 2004-06-22 17:07:32 +00:00
Eelco Dolstra
05a5362d63 * Some more diagnostics changes. 2004-06-22 17:04:10 +00:00
Eelco Dolstra
d051cd40e1 * Nix-instantiate can return multiple store expressions. 2004-06-22 15:17:04 +00:00
Eelco Dolstra
3093af58a7 * A utility script `nix-build' that builds Nix expressions and prints
their output paths (and only that) on standard output.
2004-06-22 15:12:34 +00:00
Eelco Dolstra
b302e5f63b * We don't really need this here. 2004-06-22 15:06:43 +00:00
Eelco Dolstra
83ae1723da * Well, it's better than printf. 2004-06-22 15:01:01 +00:00
Eelco Dolstra
72bc9a522f * Started making Nix's diagnostic messages a bit more useful. 2004-06-22 14:48:59 +00:00
Eelco Dolstra
5e2cf44a4d * Put WEXITSTATUS stuff somewhere else. 2004-06-22 11:03:41 +00:00
Eelco Dolstra
84007a0958 * Reduce gratuitous cut & pasting. 2004-06-22 10:21:44 +00:00
Eelco Dolstra
c9fbd2dfd5 * Wrapper class around pids. 2004-06-22 09:51:44 +00:00
Eelco Dolstra
155d7c8dfa * Substitutes should occupy a build slot. 2004-06-22 09:00:31 +00:00
Eelco Dolstra
c4cb6ea2bc * Refactoring. 2004-06-22 08:50:25 +00:00
Eelco Dolstra
88fb4f6e53 * Missing files added to `make dist'. 2004-06-21 12:20:47 +00:00
Eelco Dolstra
2db9748221 * Remove debug output. 2004-06-21 10:01:17 +00:00
Eelco Dolstra
37ee6cef99 * Adapted nix-pull to use the new substitute mechanism. 2004-06-21 09:51:23 +00:00
Eelco Dolstra
3f3a3ae87b * Acquire a lock on the output path when running a substitute. Also
delete obstructing invalid paths.
2004-06-21 09:35:50 +00:00
Eelco Dolstra
72c857f0eb * Ugh, nasty Heisenbug due to an uninitialiased variable. The bug
only caused a crash if the program was *not* invoked with a high
  verbosity level.
2004-06-21 08:51:55 +00:00
Eelco Dolstra
be1a917beb * Remove obstructing invalid store paths add[Text]ToStore(). 2004-06-21 07:46:02 +00:00
Eelco Dolstra
daf0a923c7 * Wrap calls to registerSubstitute() in a single transaction to
improve throughput.
* Don't build the `substitute-rev' table for now, since it caused
  Theta(N^2) time and log file consumption when adding N substitutes.
  Maybe we can do without it.
2004-06-21 07:38:17 +00:00
Eelco Dolstra
15c60ca1b6 * Disable calls to fsync() since Berkeley DB's DB_TXN_WRITE_NOSYNC
flag doesn't seem to work as advertised.
2004-06-21 07:36:01 +00:00
Eelco Dolstra
112ee89501 * Re-enable support for substitutes in the normaliser.
* A better substitute mechanism.

  Instead of generating a store expression for each store path for
  which we have a substitute, we can have a single store expression
  that builds a generic program that is invoked to build the desired
  store path, which is passed as an argument.

  This means that operations like `nix-pull' only produce O(1) files
  instead of O(N) files in the store when registering N substitutes.
  (It consumes O(N) database storage, of course, but that's not a
  performance problem).

* Added a test for the substitute mechanism.
  
* `nix-store --substitute' reads the substitutes from standard input,
  instead of from the command line.  This prevents us from running
  into the kernel's limit on command line length.
2004-06-20 19:17:54 +00:00
Eelco Dolstra
bafb2357d1 * README for the Emacs mode (written a while ago, but forgot to commit). 2004-06-20 19:08:59 +00:00
Eelco Dolstra
85ae781765 * Refactoring. 2004-06-20 13:37:51 +00:00
Eelco Dolstra
23bb902d1f * Re-enable build hooks. 2004-06-19 21:45:04 +00:00
Eelco Dolstra
41ec982f31 * Big refactoring. Move to a much more explicitly state machine based
approach.  This makes it much easier to add extra complexity in the
  normaliser / realiser (e.g., build hooks, substitutes).
2004-06-18 18:09:32 +00:00
Eelco Dolstra
3454c685ee * This is also useful. 2004-06-18 16:52:31 +00:00
Eelco Dolstra
6ba26f27c3 * Shared (garbage collecting) pointers. Copied from Boost. 2004-06-18 09:20:51 +00:00
Eelco Dolstra
0b70231b9d * Refactoring. 2004-06-15 13:49:42 +00:00
Eelco Dolstra
1bc6afefac * Cleanup. 2004-06-08 13:21:03 +00:00
Eelco Dolstra
e8411948ff * A Nix mode for Emacs. 2004-06-04 14:31:57 +00:00
Eelco Dolstra
5e4a2272bf * Drain the output of the build hook to show error messages. Ugly
hack.
2004-05-18 14:52:35 +00:00
Eelco Dolstra
19479899fb * Don't set the rpath here --- it's not portable. 2004-05-18 12:57:26 +00:00
Eelco Dolstra
1d08093b48 * Go back to the old way of generating the system name, and allow it
to be specified in configure (using `--with-system=SYSTEM').
2004-05-18 09:45:46 +00:00
Eelco Dolstra
8e9fd57ef9 * setpgrp() is not POSIX (and on Mac OS X it's different than on
Linux), so use setpgid().
2004-05-18 09:45:18 +00:00
Eelco Dolstra
ace8872706 * execl() requires a terminating 0.
* When a fast build wakes up a goal, try to start that goal in the
  same iteration of the startBuild() loop of run().  Otherwise no job
  might be started until the next job terminates.
2004-05-14 12:24:29 +00:00
Eelco Dolstra
4fc00cbec1 * Distributed builds and load balancing now seem to work pretty well.
(Though the `build-remote.pl' script has a gigantic race condition).
2004-05-13 22:52:37 +00:00
Eelco Dolstra
2fa3304933 * Set the executable bit. 2004-05-13 20:28:20 +00:00
Eelco Dolstra
25db622454 * Load balancing. `build-remote.pl' will only execute up to a
per-machine maximum number of parallel jobs on a remote machine.
2004-05-13 19:35:46 +00:00
Eelco Dolstra
3426d19547 * Perform all tests. 2004-05-13 19:16:48 +00:00
Eelco Dolstra
a8306cb98f * The build hooks used to implement distributed builds can now be run
in parallel.  Hooks are more efficient: locks on output paths are
  only acquired when the hook says that it is willing to accept a
  build job.  Hooks now work in two phases.  First, they should first
  tell Nix whether they are willing to accept a job.  Nix guarantuees
  that no two hooks will ever be in the first phase at the same time
  (this simplifies the implementation of hooks, since they don't have
  to perform locking (?)).  Second, if they accept a job, they are
  then responsible for building it (on the remote system), and copying
  the result back.  These can be run in parallel with other hooks and
  locally executed jobs.

  The implementation is a bit messy right now, though.  

* The directory `distributed' shows a (hacky) example of a hook that
  distributes build jobs over a set of machines listed in a
  configuration file.
2004-05-13 19:14:49 +00:00
Eelco Dolstra
5087c8f645 * Use `-j0'. 2004-05-12 14:30:57 +00:00
Eelco Dolstra
efa5fa1a91 * A switch `-j NUMBER' to set the maximum number of parallel jobs (0 =
no limit).
* Add missing file to distribution.
2004-05-12 14:20:32 +00:00
Eelco Dolstra
1f48aa0be7 * Broken test. 2004-05-12 13:49:10 +00:00
Eelco Dolstra
aa5a5084e4 * Pass to the build hook all sorts of information useful for
distributing a build action to another machine.  In particular, the
  paths in the input closures, the output paths, and successor mapping
  for sub-derivations.
2004-05-12 13:32:26 +00:00
Eelco Dolstra
8c0b42f857 * An quick and dirty hack to support distributed builds. 2004-05-12 09:35:51 +00:00
Eelco Dolstra
c8d3882cdc * True parallel builds. Nix can now run as many build jobs in
parallel as possible (similar to GNU Make's `-j' switch).  This is
  useful on SMP systems, but it is especially useful for doing builds
  on multiple machines.  The idea is that a large derivation is
  initiated on one master machine, which then distributes
  sub-derivations to any number of slave machines.  This should not
  happen synchronously or in lock-step, so the master must be capable
  of dealing with multiple parallel build jobs.  We now have the
  infrastructure to support this.

  TODO: substitutes are currently broken.
2004-05-11 18:05:44 +00:00
Eelco Dolstra
aea436503e * Ignore interrupt signals while handling an exception.
* Ignore EINTR in reads and writes.
2004-05-11 13:48:25 +00:00
Eelco Dolstra
a9858c9f26 * A test to verify that Nix executes build jobs in parallel, if
possible.

  This test fails right now because this hasn't been implemented right
  now.  Yes, I'm doing Test-Driven Development! ;-)
2004-05-04 17:04:17 +00:00
Eelco Dolstra
f044ccf702 * 1000th revision!
* A test to verify that locking of output paths (caused by concurrent
  invocations of Nix) works correctly.
2004-05-04 13:56:30 +00:00