max/nix-super
1
0
Fork 0
mirror of https://github.com/privatevoid-net/nix-super.git synced 2024-11-11 00:36:20 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 1
2883 commits 7 branches 0 tags 64 MiB
Commit graph

1629 commits

Author SHA1 Message Date
Eelco Dolstra
4ba6afaf48 * nix-env -ub' (--prebuilt-only') didn't really work because it 
checked too soon whether substitutes are available.  That is, it did
  so for every available package, rather than those matching installed
  packages.  This was very slow and subject to assertion failures.  So
  do the check much later.  Idem for `nix-env -qab' and `nix-env -ib'.
 2011-04-11 16:27:05 +00:00
Eelco Dolstra
538b7caab0 * Don't allocate a big initial GC address space on machines with 
little RAM.  Even if the memory isn't actually used, it can cause
  problems with the overcommit heuristics in the kernel.  So use a VM
  space of 25% of RAM, up to 384 MB.
 2011-02-10 14:31:04 +00:00
Eelco Dolstra
5a6b039802 * Don't call GC_expand_hp unless we're actually using the garbage 
collector.
 2011-02-09 22:59:50 +00:00
Ludovic Courtès
f1462c208b Use $BDW_GC_LIBS instead of a custom variable. 2011-02-09 21:23:13 +00:00
Eelco Dolstra
543988572e * Sync with the trunk. 2011-02-09 12:25:53 +00:00
Eelco Dolstra
aeae0beba4 * Clang compatibility fix. Clang actually checks template definitions 
when they are defined
 2011-02-05 16:29:10 +00:00
Ludovic Courtès
3854fc9b42 Remove useless <config.h> inclusion from public header. 2011-01-14 13:55:58 +00:00
Eelco Dolstra
9db190eb31 * builtins.substring: if "start" is beyond the end of the string, 
return the empty string.
 2011-01-14 12:47:10 +00:00
Eelco Dolstra
c931a7aec5 * Do a short sleep after SQLITE_BUSY. 2010-12-17 17:23:15 +00:00
Eelco Dolstra
b1eb252172 * Propagate the "100" exit status for failed builds through the Nix 
daemon.
 2010-12-17 11:28:26 +00:00
Eelco Dolstra
3dd02580e3 * I forgot to catch SQLiteBusy in registerValidPaths(). So 
registerValidPaths() now handles busy errors and registerValidPath()
  is simply a wrapper around it.
 2010-12-14 13:25:20 +00:00
Eelco Dolstra
d787285af9 * nix-instantiate: return exit status 100 to denote a permanent build 
failure.  The build hook can use this to distinguish between
  transient and permanent failures on the remote side.
 2010-12-13 16:53:23 +00:00
Eelco Dolstra
5833243c92 * Create /nix/var/nix/db if it's missing. 2010-12-13 13:42:34 +00:00
Eelco Dolstra
100becf8d1 * createDirs(path): if path already exists, make sure it's a 
directory.
* Provide a C++ wrapper around lstat().
 2010-12-13 13:32:58 +00:00
Eelco Dolstra
d7ca6f44eb * Update some comments. 2010-12-13 13:19:46 +00:00
Eelco Dolstra
e4720b1a79 * Ignore the result of sqlite3_reset(). 2010-12-08 18:19:15 +00:00
Eelco Dolstra
8062d3af30 * `nix-store --verify --check-contents': don't hold the global GC lock 
while checking the contents, since this operation can take a very
  long time to finish.  Also, fill in missing narSize fields in the DB
  while doing this.
 2010-12-06 15:29:38 +00:00
Eelco Dolstra
de79d23f76 * Retry a transaction if SQLite returns SQLITE_BUSY. This can happen 
even with a very long busy timeout, because SQLITE_BUSY is also
  returned to resolve deadlocks.  This should get rid of random
  "database is locked" errors.  This is kind of hard to test though.
* Fix a horrible bug in deleteFromStore(): deletePathWrapped() should
  be called after committing the transaction, not before, because the
  commit might not succeed.
 2010-12-05 18:23:19 +00:00
Eelco Dolstra
77f7a6d591 * Quick hack around one of the pathological cases in bsdiff 
(namely, when there is a long region of 0s).  On one 86 MiB
  case, this cut patch generation time from 44m to 2m.
 2010-11-29 17:07:04 +00:00
Eelco Dolstra
703e5a2ce2 * Sync with the trunk. 2010-11-25 14:09:13 +00:00
Eelco Dolstra
812fae424e * DrvInfo contains pointers to expressions, so DrvInfos should be 
traced by the garbage collector.  Otherwise "nix-env -u" can crash
  randomly.
 2010-11-25 13:47:34 +00:00
Eelco Dolstra
bdf089f463 * Before a build, show the disk space that the downloaded store paths 
will approximately require.
 2010-11-17 14:31:42 +00:00
Eelco Dolstra
e60c962fb8 * Add an operation `nix-store -q --size'. 2010-11-17 12:40:52 +00:00
Eelco Dolstra
1db6259076 * Implement RemoteStore::queryPathInfo(). 2010-11-17 12:08:01 +00:00
Eelco Dolstra
a3883cbd28 * Store the size of a store path in the database (to be precise, the 
size of the NAR serialisation of the path, i.e., `nix-store --dump
  PATH').  This is useful for Hydra.
 2010-11-16 17:11:46 +00:00
Eelco Dolstra
fb9368b5a0 * Sync with the trunk. 2010-11-16 12:49:47 +00:00
Eelco Dolstra
26def5392f * Document Boehm GC support. 2010-10-29 14:44:02 +00:00
Eelco Dolstra
14fbf85380 * Set libgc's initial heap size to 384 MiB to prevent garbage 
collection in most cases (and therefore its performance overhead).
 2010-10-29 13:11:50 +00:00
Eelco Dolstra
0c4828ea05 * new(UseGC) is inexplicably slower than GC_MALLOC, so prefer the 
latter.
 2010-10-28 12:50:01 +00:00
Eelco Dolstra
e11e6fb1c6 * Handle out of memory condition. 2010-10-28 12:29:40 +00:00
Eelco Dolstra
43535499f3 * When allocating an attribute set, reserve enough space for all 
elements.  This prevents the vector from having to resize itself.
 2010-10-24 20:09:37 +00:00
Eelco Dolstra
e0b7fb8f27 * Keep attribute sets in sorted order to speed up attribute lookups. 
* Simplify the representation of attributes in the AST.
* Change the behaviour of listToAttrs() in case of duplicate names.
 2010-10-24 19:52:33 +00:00
Eelco Dolstra
2dc6d50941 * Don't create thunks for variable lookups (if possible). This 
significantly reduces the number of values allocated (e.g. from 8.7m
  to 4.9m for the Bittorrent test).
 2010-10-24 14:20:02 +00:00
Eelco Dolstra
0b305c534f * Store attribute sets as a vector instead of a map (i.e. a red-black 
tree).  This saves a lot of memory.  The vector should be sorted so
  that names can be looked up using binary search, but this is not the
  case yet.  (Surprisingly, looking up attributes using linear search
  doesn't have a big impact on performance.)

  Memory consumption for

    $ nix-instantiate /etc/nixos/nixos/tests -A bittorrent.test --readonly-mode

  on x86_64-linux with GC enabled is now 185 MiB (compared to 946
  MiB on the trunk).
 2010-10-24 00:41:29 +00:00
Eelco Dolstra
a247d20604 * Fix compiling without Boehm. 
* Fix the stats.
 2010-10-23 22:58:24 +00:00
Eelco Dolstra
b2ba62170c * Optimise string constants by putting them in the symbol table. 2010-10-23 21:11:59 +00:00
Eelco Dolstra
8ac06726b9 * Make Value smaller by not storing redundant PrimOp info. 
* Clear pointers in Values after overwriting them to make sure that no
  objects are kept alive unnecessarily.
 2010-10-23 20:07:47 +00:00
Eelco Dolstra
3f66cfb96b * Remove allocValues(). 2010-10-23 18:18:07 +00:00
Eelco Dolstra
4dee289550 * In environments, store pointers to values rather than values. This 
improves GC effectiveness a bit more (because a live value doesn't
  keep other values in the environment plus the parent environments
  alive), and removes the need for copy nodes.
 2010-10-22 15:51:52 +00:00
Eelco Dolstra
41c45a9b31 * Store Value nodes outside of attribute sets. I.e., Attr now stores 
a pointer to a Value, rather than the Value directly.  This improves
  the effectiveness of garbage collection a lot: if the Value is
  stored inside the set directly, then any live pointer to the Value
  causes all other attributes in the set to be live as well.
 2010-10-22 14:47:42 +00:00
Eelco Dolstra
64c3325b0b * Make building against the Boehm GC a configure option. 2010-10-22 13:39:15 +00:00
Eelco Dolstra
76feaf016a * Keep some more stats. 2010-10-20 15:48:00 +00:00
Eelco Dolstra
e879a0371b * Use the Boehm garbage collector to reclaim unused memory in the Nix 
expression evaluator.
 2010-10-20 11:38:30 +00:00
Eelco Dolstra
64fd29855a * Wrap deleteFromStore() in a transaction. Otherwise there might be a 
race with other processes that add new referrers to a path,
  resulting in the garbage collector crashing with "foreign key
  constraint failed".  (Nix/4)
* Make --gc --print-dead etc. interruptible.
 2010-10-14 15:55:51 +00:00
Eelco Dolstra
705868a8a9 * Make sure that config.h is included before the system headers, 
because it defines _FILE_OFFSET_BITS.  Without this, on 
  OpenSolaris the system headers define it to be 32, and then 
  the 32-bit stat() ends up being called with a 64-bit "struct 
  stat", or vice versa.

  This also ensures that we get 64-bit file sizes everywhere.

* Remove the redundant call to stat() in parseExprFromFile().
  The file cannot be a symlink because that's the exit condition
  of the loop before.
 2010-10-04 17:55:38 +00:00
Eelco Dolstra
95f4f2cf61 * If std::tr1::unordered_set is unavailable, use std::set. 2010-10-04 16:16:19 +00:00
Eelco Dolstra
71dfe4b90b * Sync with the trunk. 2010-10-04 11:44:47 +00:00
Eelco Dolstra
450837bcc8 * In printMsg(), ignore failing writes to stderr if we're in an 
exception handler, otherwise throw an exception.  We need to ignore
  write errors in exception handlers to ensure that cleanup code runs
  to completion if the other side of stderr has been closed
  unexpectedly.
 2010-10-04 11:23:07 +00:00
Eelco Dolstra
4aa9245083 * Hack needed for GCC 4.3.2 on OpenSolaris. 2010-10-04 10:51:16 +00:00
Eelco Dolstra
923736df38 * Doh. Remove debug message. 2010-09-14 12:47:19 +00:00
Eelco Dolstra
e4907411c2 * Only do "pragma journal_mode = ..." if the current journal mode 
differs from the desired mode.  There is an open SQLite ticket
  `Executing "PRAGMA journal_mode" may delete journal file while it is
  in use.'
 2010-09-01 11:36:22 +00:00
Eelco Dolstra
bf0dde9597 * Always print hook output on stderr, even if --no-build-output is 
set.
* In the build hook, print a trace message to allow Hydra to pick up
  the name of the remote machine used for the build.
 2010-08-31 12:36:24 +00:00
Eelco Dolstra
e2e168f7c2 `nix-store --verify' improvements: 
* If a path has disappeared, check its referrers first, and don't try
  to invalidate paths that have valid referrers.  Otherwise we get a
  foreign key constraint violation.
* Read the whole Nix store directory instead of statting each valid
  path, which is slower.
* Acquire the global GC lock.
 2010-08-31 11:47:31 +00:00
Eelco Dolstra
80e722278c * When using the build hook, distinguish between the stderr of the 
hook script proper, and the stdout/stderr of the builder.  Only the
  latter should be saved in /nix/var/log/nix/drvs.
* Allow the verbosity to be set through an option.
* Added a flag --quiet to lower the verbosity level.
 2010-08-30 14:53:03 +00:00
Eelco Dolstra
20acd43c25 * Disable the GC reachability check for now (when creating new roots), 
as it's hopelessly inefficient.
 2010-08-30 14:11:57 +00:00
Eelco Dolstra
766f708418 * Experimental feature: allow a derivation to tell the build hook that 
it requires a certain feature on the build machine, e.g.

    requiredSystemFeatures = [ "kvm" ];

  We need this in Hydra to make sure that builds that require KVM
  support are forwarded to machines that have KVM support.  Probably
  this should also be enforced for local builds.
 2010-08-27 13:18:13 +00:00
Eelco Dolstra
df50916e46 * Oops - "null" was displayed as "true". 2010-08-27 12:10:56 +00:00
Eelco Dolstra
e437b08250 * Made the build hook mechanism more efficient. Rather than starting 
the hook every time we want to ask whether we can run a remote build
  (which can be very often), we now reuse a hook process for answering
  those queries until it accepts a build.  So if there are N
  derivations to be built, at most N hooks will be started.
 2010-08-25 20:44:28 +00:00
Eelco Dolstra
034f608e00 * Don't complain if the stored hash of a path is zero (unknown). 2010-08-24 14:25:33 +00:00
Eelco Dolstra
f58f51f380 * Handle the unlikely case where a derivation has no dependencies at 
all.
 2010-08-24 11:45:44 +00:00
Eelco Dolstra
955d11aae7 * Sync with the trunk. 2010-08-24 09:56:28 +00:00
Eelco Dolstra
6846ed8b44 * Make --cores work when building through the Nix daemon. 2010-08-12 09:21:50 +00:00
Eelco Dolstra
587dc8aa00 * Sync with the trunk. 2010-08-04 17:48:29 +00:00
Eelco Dolstra
fd9c77dfc7 * Use SQLite 3.7.0's write-ahead logging (WAL mode). This is a lot 
faster than the old mode when fsyncs are enabled, because it only
  performs an fsync() when doing a checkpoint, rather than at every
  commit.  Some timings for doing a "nix-instantiate /etc/nixos/nixos
  -A system" after modifying the stdenv setup script:

    42.5s - SQLite 3.6.23 with truncate mode and fsync
     3.4s - SQLite 3.6.23 with truncate mode and no fsync
    32.1s - SQLite 3.7.0 with truncate mode and fsync
    16.8s - SQLite 3.7.0 with WAL mode and fsync, auto-checkpoint
            every 1000 pages
     8.3s - SQLite 3.7.0 with WAL mode and fsync, auto-checkpoint
            every 8192 pages
     1.7s - SQLite 3.7.0 with WAL mode and no fsync

  The default is now to use WAL mode with fsyncs.  Because WAL doesn't
  work on remote filesystems such as NFS (as it uses shared memory),
  truncate mode can be re-enabled by setting the "use-sqlite-wal"
  option to false.
 2010-08-04 17:35:59 +00:00
Eelco Dolstra
750be19ae8 * Remove "auto" and "guess" as synonyms for 0 in the handling of 
build-cores and --cores.  They're superfluous and just complicate
  the parsing.
 2010-08-04 12:23:59 +00:00
Eelco Dolstra
7f893b7a43 * Allow derivations to hint that they should not be built remotely 
using the build hook mechanism, by setting the derivation attribute
  "preferLocalBuild" to true.  This has a few use cases:

  - The user environment builder.  Since it just creates a bunch of
    symlinks without much computation, there is no reason to do it
    remotely.  In fact, doing it remotely requires the entire closure
    of the user environment to be copied to the remote machine, which
    is extremely wasteful.

  - `fetchurl'.  Performing the download on a remote machine and then
    copying it to the local machine involves twice as much network
    traffic as performing the download locally, and doesn't save any
    CPU cycles on the local machine.
 2010-08-04 12:13:58 +00:00
Eelco Dolstra
315d8fbd75 * Set the default system filter to "*". This ensures that (for 
instance) "nix-env -i wine" works on x86_64-linux, even though Wine
  is built on i686-linux.  In the event that there are multiple
  matching derivations, prefer those built for the current system.
 2010-08-04 09:32:42 +00:00
Eelco Dolstra
6d6200f37a * Optimisation in the // operator: if one of the sets is empty, return 
the other set.
 2010-08-02 16:31:05 +00:00
Eelco Dolstra
7af6a2fd71 * intersectAttrs: optimise for the case where the second set is larger 
than the first set.  (That's usually the case with callPackage.)
 2010-08-02 11:54:44 +00:00
Peter Simons
7e043d28a6 src/bsdiff-4.3/Makefile.am: include the 'compat-include' directory in distribution tarballs 2010-07-15 14:35:20 +00:00
Peter Simons
af09fe12dd Fix build of bsdiff-4.3 on machines that don't have <err.h>, such as Solaris. 2010-06-24 17:51:31 +00:00
Peter Simons
bcec46057c src/libutil/util.cc: include <limit.h> to ensure that PATH_MAX is defined 2010-06-24 17:51:13 +00:00
Peter Simons
a17071fef1 Include <cstring> to ensure that strcpy(), strlen(), and memset() are declared. 
An "using namespace std" was added locally in those functions that refer to
names from <cstring>. That is not pretty, but it's a very portable solution,
because strcpy() and friends will be found in both the 'std' and in the global
namespace.
 2010-06-24 17:51:04 +00:00
Peter Simons
8b7f8b56f1 Added support for passing an (impure) NIX_BUILD_CORES variable to build expressions. 
This patch adds the configuration file variable "build-cores" and the
command line argument "--cores". These settings specify the number of
CPU cores to utilize for parallel building within a job, i.e. by passing
an appropriate "-j" flag to GNU Make. The default value is 1, which
means that parallel building is *disabled*. If the number of build cores
is specified as 0 (synonymously: "guess" or "auto"), then the actual
value is supposed to be auto-detected by builders at run-time, i.e by
calling the nproc(1) utility from coreutils.

The environment variable $NIX_BUILD_CORES is available to builders, but
the contents of that variable does *not* influence the hash that goes
into the $out store path, i.e. the number of build cores to be utilized
can be changed at will without requiring any re-builds.
 2010-06-23 14:34:08 +00:00
Eelco Dolstra
3e5e0faf9c * Okay, putting a lock on the temporary directory used by importPath() 
doesn't work because the garbage collector doesn't actually look at
  locks.  So r22253 was stupid.  Use addTempRoot() instead.  Also,
  locking the temporary directory in exportPath() was silly because it
  isn't even in the store.
 2010-06-21 11:08:09 +00:00
Eelco Dolstra
bf87cc44b4 * Sync with the trunk. 2010-06-21 07:55:38 +00:00
Eelco Dolstra
b57189174f * In importPath() and exportPath(), lock the temporary directory to 
prevent it from being deleted by the garbage collector.
 2010-06-14 08:34:48 +00:00
Eelco Dolstra
f16fe2af8d * builtins.toXML: propagate the string context. This is a regression 
from the old ATerm-based evaluator.
 2010-06-10 10:29:50 +00:00
Eelco Dolstra
d1f6c0cbe3 * Replacing ValidPath rows doesn't work because it causes a constraint 
violation of the Refs table.  So don't do that.
 2010-06-08 13:38:28 +00:00
Eelco Dolstra
07ca66cf24 * Applied a patch from David Brown to prevent `nix-store --optimise' 
from failing on rename() on BtrFS.
 2010-06-04 13:56:11 +00:00
Eelco Dolstra
1ab67cf437 2010-06-02 09:43:04 +00:00
Eelco Dolstra
89865da76d * Turn build errors during evaluation into EvalErrors. 2010-06-01 11:19:32 +00:00
Eelco Dolstra
a443c7573b * Hack to allow derivations to disable chroot builds by setting the 
attribute "__noChroot = true" (requested by Rob).
 2010-06-01 10:01:14 +00:00
Ludovic Courtès
8bcdd36f10 Add XML output to `nix-store'. 
* src/nix-store/Makefile.am (nix_store_SOURCES): Add `xmlgraph.cc' and
  `xmlgraph.hh'.

* src/nix-store/help.txt (Operations): Document `--xml'.

* src/nix-store/nix-store.cc (opQuery): Handle `--xml'.

* src/nix-store/xmlgraph.cc, src/nix-store/xmlgraph.hh: New files.
 2010-05-31 16:36:24 +00:00
Ludovic Courtès
da52f8bea0 Comment out dead code in `nix-store'. 
* src/nix-store/dotgraph.cc (pathLabel): Move within #if 0 section.
 2010-05-31 16:36:20 +00:00
Eelco Dolstra
7343e6c8ae * Remove an accidentally committed debug statement. 2010-05-30 20:29:56 +00:00
Eelco Dolstra
b92a2e5cc2 * Sync with the trunk. 2010-05-20 12:36:03 +00:00
Eelco Dolstra
93cd5a4a13 * The << operator on values should be const. 2010-05-18 10:36:37 +00:00
Eelco Dolstra
32539e41d5 * Sync with trunk. 2010-05-16 17:16:00 +00:00
Eelco Dolstra
b2235d81d1 * Restore the __overrides feature that was lost somewhere in the 
fast-eval branch.
 2010-05-15 08:10:12 +00:00
Eelco Dolstra
aa45027818 * Sync with the trunk. 2010-05-12 22:13:09 +00:00
Eelco Dolstra
bd25ac2260 * Print attributes in sorted order. 2010-05-12 12:15:49 +00:00
Eelco Dolstra
81a4b4e49b * Implemented tryEval, the last missing primop in the fast-eval 
branch.  Also added a test for tryEval.
 2010-05-12 11:23:44 +00:00
Eelco Dolstra
1a8eb6e3ec 2010-05-07 15:26:33 +00:00
Eelco Dolstra
83dfa89870 * Sync with the trunk. 2010-05-07 14:46:47 +00:00
Eelco Dolstra
01e58adce0 * Store position info for inherited attributes. 2010-05-07 12:43:57 +00:00
Eelco Dolstra
83d7b89660 * Updated addErrorContext. 2010-05-07 12:33:14 +00:00
Eelco Dolstra
e2d5e40f4f * Keep track of the source positions of attributes. 2010-05-07 12:11:05 +00:00
Eelco Dolstra
84ce7ac76f * Store attribute positions in the AST and report duplicate attribute 
errors with position info.
* For all positions, use the position of the first character of the
  first token, rather than the last character of the first token plus
  one.
 2010-05-06 16:46:48 +00:00
Eelco Dolstra
a0e3b84fac * Revert r15436. This was a workaround for a bug in btrfs which seems 
to have been fixed now.
 2010-05-04 12:42:58 +00:00
Eelco Dolstra
f92c9a0ac5 * Allow unprivileged users to do `nix-store --clear-failed-paths' and 
`nix-store --query-failed-paths'.
 2010-05-04 10:45:10 +00:00
Eelco Dolstra
c82782f9a5 2010-04-27 09:05:11 +00:00
Eelco Dolstra
ef337f7089 2010-04-26 12:58:12 +00:00
Eelco Dolstra
6199f9b93e * Added a command `nix-store --clear-failed-paths <PATHS>' to clear 
the "failed" status of the given store paths.  The special value `*'
  clears all failed paths.
 2010-04-26 12:56:42 +00:00
Eelco Dolstra
2398af13c5 * Add an command `nix-store --query-failed-paths' to list the cached 
failed paths (when using the `build-cache-failure' option).
 2010-04-26 12:43:42 +00:00
Eelco Dolstra
2be6118f4c * Don't need the test program anymore. 2010-04-23 09:09:32 +00:00
Eelco Dolstra
0bc468f195 * Simplify the implementation of `with'. This gives a 7% speedup in 
evaluating the NixOS system configuration.
 2010-04-22 15:08:09 +00:00
Eelco Dolstra
ebade9ff8b * Check for duplicate attribute names / function arguments. `make 
check' now succeeds :-)
* An attribute set such as `{ foo = { enable = true; };
  foo.port = 23; }' now parses.  It was previously rejected, but I'm
  too lazy to implement the check.  (The only reason to reject it is
  that the reverse, `{ foo.port = 23; foo = { enable = true; }; }', is
  rejected, which is kind of ugly.)
 2010-04-22 11:02:24 +00:00
Eelco Dolstra
2d7636529f * String equality tests should take the context into account. All the 
evaluation test cases now succeed.
 2010-04-22 09:54:11 +00:00
Eelco Dolstra
6bbfe95e30 * Don't use an ostringstream in unparseDerivation(), because it's 
slow.  A `nix-env -qa --drv-path \*' on Nixpkgs was spending 40% of
  its time in unparseDerivation() because of this (now 11%).
 2010-04-21 19:25:50 +00:00
Eelco Dolstra
6f0f16497a * Fix the interpretation of ''\<character> in indented strings. 2010-04-21 16:18:27 +00:00
Eelco Dolstra
0777448ca6 * Fixed builtins.genericClosure. 2010-04-21 15:57:11 +00:00
Eelco Dolstra
fe2d869e04 * Store user environment manifests as a Nix expression in 
$out/manifest.nix rather than as an ATerm.

  (Hm, I thought I committed this two days ago...)
 2010-04-21 15:08:58 +00:00
Eelco Dolstra
d66ea83a76 * Sync with the trunk. 2010-04-20 09:20:29 +00:00
Eelco Dolstra
f3b8833a48 * Drop the dependency on the ATerm library. 2010-04-19 14:51:58 +00:00
Eelco Dolstra
efc7a579e8 * Don't use the ATerm library for parsing/printing .drv files. 2010-04-19 13:46:58 +00:00
Eelco Dolstra
55b5ddd3ca * Added parsing of manifests in ATerm format. 2010-04-19 12:10:04 +00:00
Eelco Dolstra
b7ff69eb7c * Refactoring: move the user environment stuff into its own module. 2010-04-19 10:47:56 +00:00
Eelco Dolstra
5c31995bb8 * Updated some more primops. 2010-04-16 15:13:47 +00:00
Eelco Dolstra
8bb0210fea * _combineChannels shouldn't be an integer. 2010-04-16 14:07:52 +00:00
Eelco Dolstra
8ca4a001cb * Improve sharing a bit. 2010-04-16 14:03:26 +00:00
Eelco Dolstra
497e4ad126 * Remove some redundant tests. 2010-04-16 13:51:01 +00:00
Eelco Dolstra
02c1dac909 * In an nested with' where the inner with is a variable (with ...; 
with someVar; ...'), the contents of the variable would be
  clobbered.  (The attributes in the outer `with' were added to the
  variable.)
 2010-04-16 13:44:02 +00:00
Eelco Dolstra
04c4bd3624 * Store lists as lists of pointers to values rather than as lists of 
values.  This improves sharing and gives another speed up.
  Evaluation of the NixOS system attribute is now almost 7 times
  faster than the old evaluator.
 2010-04-15 00:37:36 +00:00
Eelco Dolstra
e41b5828db * Better stats. 2010-04-14 23:48:46 +00:00
Eelco Dolstra
d39d3c6264 * Implemented inherit. 2010-04-14 23:25:05 +00:00
Eelco Dolstra
267dc693d2 * Fix builtins. 2010-04-14 22:59:39 +00:00
Eelco Dolstra
81de12bc8f * Refactoring: move variable uses to a separate class. 2010-04-14 15:14:23 +00:00
Eelco Dolstra
110d155778 * Implemented withs. 2010-04-14 15:01:04 +00:00
Eelco Dolstra
9985230c00 * After parsing, compute level/displacement pairs for each variable 
use site, allowing environments to be stores as vectors of values
  rather than maps.  This should speed up evaluation and reduce the
  number of allocations.
 2010-04-14 14:42:32 +00:00
Eelco Dolstra
816dd3f061 * Remove more obsolete code. 2010-04-14 12:49:05 +00:00
Eelco Dolstra
011b5da0f4 * Get nix-env to compile again. 2010-04-14 09:39:06 +00:00
Eelco Dolstra
85d13c8f93 * Change the semantics of "with" so that inner "withs" take 
precedence, i.e. `with {x=1;}; with {x=2;}; x' evaluates to 2'.
  This has a simpler implementation and seems more natural.  There
  doesn't seem to be any code in Nixpkgs or NixOS that relies on the
  old behaviour.
 2010-04-14 08:37:08 +00:00
Eelco Dolstra
816f9c0f6f * Use std::tr1::unordered_set instead of std::set for the symbol 
table.  This gives a 10% speed increase on `nix-instantiate
  /etc/nixos/nixos -A system --readonly-mode'.
 2010-04-13 14:34:11 +00:00
Eelco Dolstra
7d47498b5e * Evaluate lets directly (i.e. without desugaring to `rec { attrs...; 
<let-body> = e; }.<let-body>).  This prevents the unnecessary
  allocation of an attribute set.
 2010-04-13 13:42:25 +00:00
Eelco Dolstra
ac1e8f40d4 * Use a symbol table to represent identifiers and attribute names 
efficiently.  The symbol table ensures that there is only one copy
  of each symbol, thus allowing symbols to be compared efficiently
  using a pointer equality test.
 2010-04-13 12:25:42 +00:00
Eelco Dolstra
10e8b1fd15 * Finished the ATerm-less parser. 2010-04-12 23:33:23 +00:00
Eelco Dolstra
0d272fca79 * Remove some obsolete functions. 2010-04-12 23:31:47 +00:00
Eelco Dolstra
d4f0b0fc6c * Indented strings. 2010-04-12 22:03:27 +00:00
Eelco Dolstra
a60317f20f * More missing constructs. 2010-04-12 21:21:24 +00:00
Eelco Dolstra
4d6ad5be17 * Don't use ATerms for the abstract syntax trees anymore. Not 
finished yet.
 2010-04-12 18:30:11 +00:00
Eelco Dolstra
ed711f73bc * Don't use ATerms to represent integers in the lexer. 2010-04-12 10:38:18 +00:00
Eelco Dolstra
db90b88e65 * Hack to support builderDefs expressions. 2010-04-12 09:50:20 +00:00
Eelco Dolstra
4e49002576 * Doh. 2010-04-12 09:45:00 +00:00
Eelco Dolstra
c3f228f296 2010-04-12 09:14:27 +00:00
Ludovic Courtès
aac5fcfbb5 Re-add drvPath' and outPath' attributes to <derivation> XML nodes. 
This fixes a regression introduced in r20882 ("Add source location
information to the XML output.").

* src/libexpr/expr-to-xml.cc (nix::printTermAsXML): Dereference the
  attribute RHS from "drvPath" and "outPath".
 2010-04-09 21:30:55 +00:00
Eelco Dolstra
f3dc7ab877 * Keep more statistics about stack space usage. 
* Reduce stack space usage.
 2010-04-09 12:00:49 +00:00
Eelco Dolstra
b7b3dd55f9 * Remove a lot of dead code. 2010-04-08 11:41:19 +00:00
Eelco Dolstra
7e048eddf5 * Fix blackholing. If evaluation fails due to an assertion failure, 
then the blackhole has to be removed to ensure that repeated
  evaluation of the same value gives an assertion failure again rather
  than an "infinite recursion" error.
 2010-04-08 11:25:14 +00:00
Eelco Dolstra
af2a372bb0 * Update autoCallFunction() and findAlongAttrPath(). 2010-04-07 15:47:06 +00:00
Eelco Dolstra
9a64454faa * expr-to-xml -> value-to-xml. 2010-04-07 13:59:45 +00:00
Eelco Dolstra
fc92244ba8 * Implemented the primops necessary for generating the NixOS manual. 2010-04-07 13:55:46 +00:00
Eelco Dolstra
a353aef0b1 * In eval(), don't use the target value `v' as a temporary. 
Overwriting `v' breaks when the expression evaluation to an
  assertion failure or throw.
 2010-04-06 14:15:29 +00:00
Eelco Dolstra
a5ece7d016 * Removed the `~' operator. 2010-04-01 16:59:07 +00:00
Eelco Dolstra
c172274e17 * Quick hack to make coerceToString work more or less correctly on 
nested lists.  `nix-instantiate' can now evaluate the NixOS system
  derivation attribute correctly (in 2.1s on my laptop vs. 6.2s for
  the trunk).
 2010-04-01 14:35:03 +00:00
Eelco Dolstra
7b851915bf * Improve sharing. 2010-04-01 12:04:57 +00:00
Eelco Dolstra
95cc417d76 * Functions are incomparable. 2010-04-01 10:55:36 +00:00
Eelco Dolstra
71f026292b * Make `derivation' lazy again for performance. It also turns out 
that there are some places in Nixpkgs (php_configurable /
  composableDerivation, it seems) that call `derivation' with
  incorrect arguments (namely, the `name' attribute missing) but get
  away with it because of laziness.
 2010-04-01 09:55:57 +00:00
Eelco Dolstra
dc31305b38 * Fixed the trace primop and path comparison. 
* Removed exprToString and stringToExpr because there is no ATerm
  representation to work on anymore (and exposing the internals of the
  evaluator like this is not a good idea anyway).
 2010-03-31 20:09:20 +00:00
Eelco Dolstra
979f163615 * Handle string contexts. `nix-instantiate' can now correctly compute 
the `firefoxWrapper' attribute in Nixpkgs, and it's about 3 times
  faster than the trunk :-)
 2010-03-31 19:52:29 +00:00
Eelco Dolstra
d8cd3115d8 * Get nix-env to compile. 2010-03-31 19:12:08 +00:00
Eelco Dolstra
55e207b2dc * Cache parse trees to prevent repeated parsing of imported Nix 
expressions.
 2010-03-31 16:14:32 +00:00
Eelco Dolstra
3d94be61ea * Implemented derivations. 2010-03-31 15:38:03 +00:00
Ludovic Courtès
09381cccff Make source location info in the XML output optional. 
* src/libexpr/expr-to-xml.cc (nix::showAttrs): Add `location'
  parameter.  Provide location XML attributes when it's true.  Update
  callers.
  (nix::printTermAsXML): Likewise.

* src/libexpr/expr-to-xml.hh (nix::printTermAsXML): Update prototype;
  have `location' default to `false'.

* src/nix-instantiate/nix-instantiate.cc (printResult, processExpr): Add
  `location' parameter; update callers.
  (run): Add support for `--no-location'.

* src/nix-instantiate/help.txt: Update accordingly.

* tests/lang.sh: Invoke `nix-instantiate' with `--no-location' for the
  XML tests.

* tests/lang/eval-okay-toxml.exp, tests/lang/eval-okay-to-xml.nix: New
  files.
 2010-03-31 12:38:31 +00:00
Eelco Dolstra
13c2adc897 * Implemented `rec { inherit ...; }'. 2010-03-31 11:05:39 +00:00
Eelco Dolstra
4c53ca2692 * Compare nulls. 2010-03-31 09:54:12 +00:00
Ludovic Courtès
471419d1fa Add source location information to the XML output. 
* src/libexpr/expr-to-xml.cc (nix::showAttrs): Dereference the attribute
  RHS.  Add "path", "line", and "column" XML attributes to the node when
  source location information is available.
  (nix::printTermAsXML): Likewise for functions.
 2010-03-31 08:29:05 +00:00
Ludovic Courtès
eb07a4f1ee Escape `>' signs in the XML output. 
* src/libutil/xml-writer.cc (nix::XMLWriter::writeAttrs): Escape `>'.
 2010-03-31 08:29:01 +00:00
Eelco Dolstra
7f19e03c65 * More primops. 2010-03-30 22:39:48 +00:00
Eelco Dolstra
47df476daa * More operators / primops. 2010-03-30 18:05:54 +00:00
Eelco Dolstra
c9170be2bd * More primops. 2010-03-30 15:18:20 +00:00
Eelco Dolstra
c3aa615a5f * More primops. 2010-03-30 14:39:27 +00:00
Eelco Dolstra
5b72d8a749 * Implemented `map'. 2010-03-30 13:47:59 +00:00
Eelco Dolstra
d78a05ab40 * Make `import' work. 2010-03-30 09:22:33 +00:00
Eelco Dolstra
31428c3a06 * Started integrating the new evaluator. 2010-03-29 14:37:56 +00:00
Eelco Dolstra
52090d2418 2010-03-29 10:13:51 +00:00
Eelco Dolstra
e3f32ac5af 2010-03-29 09:43:55 +00:00
Eelco Dolstra
807a67bc74 2010-03-29 09:43:39 +00:00
Eelco Dolstra
392811eb8f * Strings. 2010-03-28 18:27:07 +00:00
Eelco Dolstra
d96cdcea6b 2010-03-28 16:57:16 +00:00
Eelco Dolstra
3d2b835f30 * Implemented multi-argument primops. 2010-03-28 16:37:39 +00:00
Eelco Dolstra
45d822f29c * Primops (not yet finished). 2010-03-26 15:45:53 +00:00
Eelco Dolstra
cad8726b2c * Implemented the ==' and !=' operators. These now use a deep 
equality test, so they also work for (finite) attribute sets and
  lists.
 2010-03-26 13:27:26 +00:00
Eelco Dolstra
8da118e4d0 * Measure stack usage. 2010-03-25 16:35:24 +00:00
Eelco Dolstra
c2ba4313fb * Implemented lists. 2010-03-25 15:38:37 +00:00
Eelco Dolstra
25eedf085d * Quick and dirty implementation of with'. with e1; e2' is 
basically desugared to `let <with> = e1; e2', and `lookupVar' looks
  in each <with> in the environment chain for an attribute with the
  specified name.
 2010-03-25 14:51:04 +00:00
Eelco Dolstra
3c9f8fc9b6 * Don't convert variable names to strings. 2010-03-25 13:10:04 +00:00
Eelco Dolstra
f450384ded * Implement blackholing. 2010-03-25 12:51:14 +00:00
Eelco Dolstra
ef8bd919fc * Implement `...' and default function arguments. 2010-03-25 12:45:23 +00:00
Eelco Dolstra
8a10360c91 * Simplify @-patterns: only {attrs}@name' or name@{attrs}' are now 
allowed.  So `name1@name2', `{attrs1}@{attrs2}' and so on are now no
  longer legal.  This is no big loss because they were not useful
  anyway.

  This also changes the output of builtins.toXML for @-patterns
  slightly.
 2010-03-25 12:19:41 +00:00
Eelco Dolstra
7482349fe8 * Implemented attribute set pattern matches. 2010-03-24 23:40:00 +00:00
Eelco Dolstra
0fd3648d34 * Store values in environments. 2010-03-24 12:41:08 +00:00
Eelco Dolstra
b70bd8fe56 * Reduce the number of value allocations in eval() by moving 
responsibility for allocation of the result to the caller.
 2010-03-24 12:11:38 +00:00
Eelco Dolstra
d31c59eb17 * Plain lambdas. 2010-03-24 11:06:05 +00:00
Eelco Dolstra
e8f7978274 2010-03-23 19:19:52 +00:00
Eelco Dolstra
0910ae9568 * Start of an evaluator that uses call-by-need (with thunk updating) 
instead of (memoised) call-by-name.
 2010-03-23 17:30:50 +00:00
Eelco Dolstra
141294ff38 * Clean up error messages in killUser(). 2010-03-19 11:36:34 +00:00
Eelco Dolstra
f0c473c5f7 * Fix building on Cygwin (http://hydra.nixos.org/build/325071). 2010-03-16 13:01:52 +00:00
Nicolas Pierron
741b7577c1 Merge r20344 & r20346. 2010-03-14 11:58:07 +00:00
Lluís Batlle i Rossell
2fb0df83e9 Uh. somehow a Makefile tab got in as spaces. Fixing. 2010-03-11 21:22:52 +00:00
Lluís Batlle i Rossell
13cce8ec45 Making 'bin2c' to be built with the compiler for the local system. 
I copied the configure.ac code about CC_FOR_BUILD from libX11.
 2010-03-11 20:56:25 +00:00
Eelco Dolstra
e020d80e4e * Sync with the trunk. 2010-03-11 15:45:05 +00:00
Ludovic Courtès
05e15049a5 Show the build user's group in /etc/group in chroots. 
* src/libstore/build.cc (nix::DerivationGoal::startBuilder): Create
  /etc/group showing the build user's group.
 2010-03-11 14:47:04 +00:00
Eelco Dolstra
070057c1b9 * Sync with the trunk. 2010-03-11 10:52:52 +00:00
Ludovic Courtès
c752c9f41a Fix thinko in r20547. 
* src/libstore/build.cc (nix::DerivationGoal::startBuilder): Fix the GID
  of the build user in /etc/passwd.
 2010-03-11 10:33:04 +00:00
Ludovic Courtès
2e8eaca573 Clear supplementary groups of `nixbld' in /etc/passwd in chroots. 
* src/libstore/build.cc (nix::DerivationGoal::startBuilder): Don't
  display any supplementary groups for `nixbld' in /etc/passwd.
 2010-03-11 10:21:23 +00:00
Eelco Dolstra
1a65142ec4 * Remove a debug statement. 2010-03-10 12:46:25 +00:00
Eelco Dolstra
4c356acd04 * In `nix-store --export', abort if the contents of a path has 
changed.  This prevents corrupt paths from spreading to other
  machines.  Note that checking the hash is cheap because we're
  hashing anyway (because of the --sign feature).
 2010-03-09 14:32:03 +00:00
Eelco Dolstra
44f6e6de77 * Set gc-keep-outputs' or gc-keep-derivations' to false with 
`--delete --ignore-liveness'.
 2010-03-08 21:31:42 +00:00
Eelco Dolstra
2e4ef03aa3 * Increase the sqlite timeout. 2010-03-08 10:35:45 +00:00
Eelco Dolstra
04791840f4 * Emit warning='1' or error='1' attributes for lines marked as 
warnings or errors with \e[w or \e[e.
 2010-03-05 12:54:58 +00:00
Eelco Dolstra
bc6f7fc139 * Remove some unused functions. 2010-03-04 13:03:26 +00:00
Eelco Dolstra
fb6e223ddc * Synced with the trunk. 2010-03-04 12:15:00 +00:00
Eelco Dolstra
e14e2399ed * Prevent a potential memory corruption problem if an ATerm garbage 
collection happens during fixAttrs().
 2010-03-02 20:23:42 +00:00
Eelco Dolstra
56af8e86e3 * Protect the true', false' and `__overrides' constants. Without 
an ATprotect call, these could be garbage collected, leading to
  weird crashes or wrong results.
 2010-03-02 20:09:12 +00:00
Eelco Dolstra
3f9e647ae8 * checkInterrupt() shouldn't be called from a destructor. 2010-03-02 19:04:17 +00:00
Eelco Dolstra
594eaddd11 * When using the included sqlite/aterm libraries, build with 
--enable-shared.
* In libutil/libstore/libexpr etc., link against sqlite and aterm.
* Some more header file hygiene.
 2010-03-02 15:58:13 +00:00
Eelco Dolstra
24035b98b1 * Implement RemoteStore::queryValidPaths(). 2010-02-26 12:05:01 +00:00
Eelco Dolstra
e42401ee7b * Implement RemoteStore::queryDerivationOutputs(). 2010-02-25 15:52:22 +00:00
Eelco Dolstra
af565c348a * Support read-only access to the database. 2010-02-24 16:44:43 +00:00
Eelco Dolstra
e33f67ff0b * Refactor the upgrade / database initialisation logic a bit. 2010-02-24 16:30:20 +00:00
Eelco Dolstra
84a4dd5ff0 * Don't use fdatasync since it doesn't work on Snow Leopard. 
* Don't refer to config.h in util.hh, because config.h is not
  installed (http://hydra.nixos.org/build/303053).
 2010-02-24 15:46:06 +00:00
Eelco Dolstra
cfe742cfc5 * A function to query just the database id of a valid path. 2010-02-24 15:07:23 +00:00
Eelco Dolstra
6baa2a2f5e 2010-02-24 14:39:52 +00:00
Eelco Dolstra
9fd85c94de * Use `truncate' journal mode, which should be a bit faster. 2010-02-24 14:22:34 +00:00
Eelco Dolstra
fefd467539 * `helpText' is now zero-terminated. 2010-02-24 13:24:27 +00:00
Eelco Dolstra
a3c63d0d6c * Disable fsync() in SQLite if the fsync-metadata option is set to 
false.
* Change the default for `fsync-metadata' to true.
* Disable `fsync-metadata' in `make check'.
 2010-02-24 13:12:57 +00:00
Eelco Dolstra
90b6352d0a * Do registerValidPaths() in one transaction, which is much faster. 
E.g. it cuts the runtime of the referrers test from 50s to 23s.
 2010-02-24 12:48:00 +00:00
Eelco Dolstra
fae0427324 * ADDITIONAL_NETWORK_LIBS -> LIBS. 2010-02-24 12:25:48 +00:00
Eelco Dolstra
fa6a4fcb11 * Add ${sqlite_lib} everywhere. Just adding it in `libstore' doesn't 
work on x86_64 when sqlite is compiled statically.
 2010-02-24 12:18:48 +00:00
Eelco Dolstra
5954eadf67 * Remove the fdatasync check since it's no longer needed. 2010-02-24 12:16:50 +00:00
Eelco Dolstra
462bd50aef * Use normal (rather than full) synchronous mode, which I gather from 
the description at http://www.sqlite.org/atomiccommit.html should be
  safe enough.
 2010-02-24 10:57:57 +00:00
Eelco Dolstra
63b09c5e41 2010-02-23 22:31:38 +00:00
Eelco Dolstra
b4e6d98fc3 * configure: flag --with-sqlite. 2010-02-23 22:12:46 +00:00
Eelco Dolstra
2b20318b0e 2010-02-22 14:24:37 +00:00
Eelco Dolstra
9cda616949 * The database needs a trigger to get rid of self-references to 
prevent a foreign key constraint violation on the Refs table when
  deleting a path.
 2010-02-22 14:18:55 +00:00
Eelco Dolstra
c4d388add4 * Get derivation outputs from the database instead of the .drv file, 
which requires more I/O.
 2010-02-22 12:44:36 +00:00
Eelco Dolstra
103cfee056 * Revert r19650 (implement gc-keep-outputs by looking for derivations 
with the same name as the output) and instead use the
  DerivationOutputs table in the database, which is the correct way to
  to do things.
 2010-02-22 11:44:17 +00:00
Eelco Dolstra
299ff64812 * Put the derivation outputs in the database. This is useful for the 
garbage collector.
 2010-02-22 11:15:50 +00:00
Eelco Dolstra
1930570ad9 * Foreign key support in SQLite is not a persistent setting, so enable 
it at startup.
* Implement negative caching.  Now `make check' passes.
 2010-02-19 17:15:22 +00:00
Eelco Dolstra
9c9a88e9e2 * Implement more stuff. 2010-02-19 16:43:25 +00:00
Eelco Dolstra
762cee72cc * Implement registerValidPath(). 2010-02-19 16:04:51 +00:00
Eelco Dolstra
268f9aaf28 * Implemented queryValidPaths() and verifyStore(). 2010-02-18 16:51:27 +00:00
Eelco Dolstra
836e5b6f57 * Implemented queryReferrers(). 2010-02-18 16:21:59 +00:00
Eelco Dolstra
77cb9e3fb1 * Implement queryPathInfo(). 2010-02-18 15:52:57 +00:00
Eelco Dolstra
885e22b16e * Implement isValidPath(). 2010-02-18 15:11:08 +00:00
Eelco Dolstra
cfb09e0fad * Automatically abort transactions if they go out of scope without 
committing.
 2010-02-18 14:40:07 +00:00
Eelco Dolstra
e0305bb7a8 * Some wrapper objects to ensure that SQLite objects are properly 
destroyed.
 2010-02-18 14:30:42 +00:00
Eelco Dolstra
a053d2d8e5 * Add the deriver to the ValidPaths table. In principle we could now 
store all the derivers of a path efficiently.  But that opens a big
  can of worms with respect to garbage collector semantics.
 2010-02-18 13:48:18 +00:00
Eelco Dolstra
dbddac0fe9 * Assign an integer id to every row in the ValidPaths table in order 
to make the Refs table more space-efficient.  For instance, this
  reduces the size of the database on my laptop from 93 MiB to 18
  MiB.  (It was 72 MiB with the old schema on an ext3 disk with a 1
  KiB block size.)
 2010-02-18 13:40:46 +00:00