Commit graph

195 commits

Author SHA1 Message Date
Eelco Dolstra
18a48d80a0 Show function names in error messages
Functions in Nix are anonymous, but if they're assigned to a
variable/attribute, we can use the variable/attribute name in error
messages, e.g.

while evaluating `concatMapStrings' at `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/pkgs/lib/strings.nix:18:25':
...
2013-05-16 19:08:02 +02:00
Eelco Dolstra
1b3a03f161 Show which function argument was unexpected
Fixes #116.
2013-05-16 17:56:14 +02:00
Eelco Dolstra
229567293c Shut up a compiler warning 2013-05-16 17:48:19 +02:00
Eelco Dolstra
bdd4646338 Revert "Prevent config.h from being clobbered"
This reverts commit 28bba8c44f.
2013-03-08 01:24:59 +01:00
Eelco Dolstra
28bba8c44f Prevent config.h from being clobbered 2013-03-07 23:55:55 +01:00
Eelco Dolstra
5f18cd2e84 Make "${./path} ..." evaluate to a string, not a path
Wacky string coercion semantics caused expressions like

  exec = "${./my-script} params...";

to evaluate to a path (‘/path/my-script params’), because
anti-quotations are desuged to string concatenation:

  exec = ./my-script + " params...";

By constrast, adding a space at the start would yield a string as
expected:

  exec = " ${./my-script} params...";

Now the first example also evaluates to a string.
2013-02-08 20:04:04 +01:00
Eelco Dolstra
8eed07cda4 nix-env -q --out-path: Support multiple outputs
We now print all output paths of a package, e.g.

  openssl-1.0.0i  bin=/nix/store/gq2mvh0wb9l90djvsagln3aqywqmr6vl-openssl-1.0.0i-bin;man=/nix/store/7zwf5r5hsdarl3n86dasvb4chm2xzw9n-openssl-1.0.0i-man;/nix/store/cj7xvk7fjp9q887359j75pw3pzjfmqf1-openssl-1.0.0i

or (in XML mode)

  <item attrPath="openssl" name="openssl-1.0.0i" system="x86_64-linux">
    <output name="bin" path="/nix/store/gq2mvh0wb9l90djvsagln3aqywqmr6vl-openssl-1.0.0i-bin" />
    <output name="man" path="/nix/store/7zwf5r5hsdarl3n86dasvb4chm2xzw9n-openssl-1.0.0i-man" />
    <output name="out" path="/nix/store/cj7xvk7fjp9q887359j75pw3pzjfmqf1-openssl-1.0.0i" />
  </item>
2012-11-28 13:49:44 +01:00
Eelco Dolstra
6c98e6a5de Optionally ignore null-valued derivation attributes
This allows adding attributes like

  attr = if stdenv.system == "bla" then something else null;

without changing the resulting derivation on non-<bla> platforms.

We once considered adding a special "ignore" value for this purpose,
but using null seems more elegant.
2012-11-27 15:01:32 +01:00
Eelco Dolstra
408a7bfac1 nix-instantiate: Fix read-only evaluation 2012-11-26 17:41:10 +01:00
Eelco Dolstra
88164325fa Fix a segfault when auto-calling a "a@{...}" function
Since the called function can return its argument attribute set
(e.g. "a"), the latter should not be allocated on the stack.

Reported by Shea.
2012-11-09 15:09:31 +01:00
Eelco Dolstra
0a7084567f Add a ‘--repair’ flag to nix-instantiate
This allows repairing corrupted derivations and other source files.
2012-10-03 15:09:18 -04:00
Eelco Dolstra
76e88871b2 Templatise tokenizeString() 2012-09-19 15:43:23 -04:00
Eelco Dolstra
e94806d030 Merge branch 'master' into no-manifests 2012-08-27 11:09:07 -04:00
Eelco Dolstra
767101824a Avoid concatenating lists of one string 2012-08-13 15:10:29 -04:00
Eelco Dolstra
e5c589d271 Don't allocate empty lists
This saves about 4 MB when evaluating a NixOS system configuration.
2012-08-13 15:02:09 -04:00
Eelco Dolstra
3e89ef597c Optimise concatenating a list to an empty list
More precisely, in concatLists, if all lists except one are empty,
then just return the non-empty list.  This reduces the number of list
element allocations by 32% when evaluating a NixOS system
configuration.
2012-08-13 14:58:54 -04:00
Eelco Dolstra
198d0338be Add a primop ‘concatLists’
This can serve as a generic efficient list builder.  For instance, the
function ‘catAttrs’ in Nixpkgs can be rewritten from

  attr: l: fold (s: l: if hasAttr attr s then [(getAttr attr s)] ++ l else l) [] l

to

  attr: l: builtins.concatLists (map (s: if hasAttr attr s then [(getAttr attr s)] else []) l)

Statistics before:

  time elapsed: 1.08683
  size of a value: 24
  environments allocated: 1384376 (35809568 bytes)
  list elements: 6946783 (55574264 bytes)
  list concatenations: 37434
  values allocated: 1760440 (42250560 bytes)
  attribute sets allocated: 392040
  right-biased unions: 186334
  values copied in right-biased unions: 591137
  symbols in symbol table: 18273
  number of thunks: 1297673
  number of thunks avoided: 1380759
  number of attr lookups: 430802
  number of primop calls: 628912
  number of function calls: 1333544

Statistics after (including new catAttrs):

  time elapsed: 0.959854
  size of a value: 24
  environments allocated: 1010198 (26829296 bytes)
  list elements: 1984878 (15879024 bytes)
  list concatenations: 30488
  values allocated: 1589760 (38154240 bytes)
  attribute sets allocated: 392040
  right-biased unions: 186334
  values copied in right-biased unions: 591137
  symbols in symbol table: 18274
  number of thunks: 1040925
  number of thunks avoided: 1038428
  number of attr lookups: 438419
  number of primop calls: 474844
  number of function calls: 959366
2012-08-13 01:53:10 -04:00
Eelco Dolstra
4ccd48ce24 Add a "filter" primop
Evaluation of a NixOS configuration spends quite a lot of time in the
"filter" function in Nixpkgs.  As implemented in Nixpkgs, this is a
O(n^2) operation, so it's a good candidate for providing a more
efficient (i.e. primop) implementation.  Using it gives a ~10% speed
increase and a significant reduction in the number of evaluations.

Statistics before (on a NixOS system config):

  time elapsed: 1.3258
  size of a value: 24
  environments allocated: 1980939 (50127080 bytes)
  list elements: 14679308 (117434464 bytes)
  list concatenations: 50828
  values allocated: 2098938 (50374512 bytes)
  attribute sets allocated: 392040
  right-biased unions: 186334
  values copied in right-biased unions: 591137
  symbols in symbol table: 18271
  number of thunks: 1645752
  number of thunks avoided: 1921196
  number of attr lookups: 430798
  number of primop calls: 838807
  number of function calls: 1930107

Statistics after:

  time elapsed: 1.17982
  size of a value: 24
  environments allocated: 1543334 (39624560 bytes)
  list elements: 9612638 (76901104 bytes)
  list concatenations: 37434
  values allocated: 1854933 (44518392 bytes)
  attribute sets allocated: 392040
  right-biased unions: 186334
  values copied in right-biased unions: 591137
  symbols in symbol table: 18272
  number of thunks: 1392467
  number of thunks avoided: 1507311
  number of attr lookups: 430801
  number of primop calls: 691600
  number of function calls: 1492502
2012-08-13 00:28:08 -04:00
Eelco Dolstra
62f72eb9e1 Add some more evaluations stats 2012-08-12 23:41:48 -04:00
Eelco Dolstra
e82767910c Add some basic profiling support to the evaluator
Setting the environment variable NIX_COUNT_CALLS to 1 enables some
basic profiling in the evaluator.  It will count calls to functions
and primops as well as evaluations of attributes.

For example, to see where evaluation of a NixOS configuration spends
its time:

$ NIX_SHOW_STATS=1 NIX_COUNT_CALLS=1 ./src/nix-instantiate/nix-instantiate '<nixos>' -A system --readonly-mode
...
calls to 39 primops:
    239532 head
    233962 tail
    191252 hasAttr
...
calls to 1595 functions:
    224157 `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/pkgs/lib/lists.nix:17:19'
    221767 `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/pkgs/lib/lists.nix:17:14'
    221767 `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/pkgs/lib/lists.nix:17:10'
...
evaluations of 7088 attributes:
    167377 undefined position
    132459 `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/pkgs/lib/attrsets.nix:119:41'
     47322 `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs/pkgs/lib/attrsets.nix:13:21'
...
2012-08-12 23:29:28 -04:00
Eelco Dolstra
97421eb5ec Refactor settings processing
Put all Nix configuration flags in a Settings object.
2012-07-30 19:55:41 -04:00
Eelco Dolstra
1217204c81 Remove dead code 2012-07-17 14:07:52 -04:00
Eelco Dolstra
f491ae97d4 * Inline some functions and get rid of the indirection through
EvalState::eval().  This gives a 12% speedup on ‘nix-instantiate
  /etc/nixos/nixos/ -A system --readonly-mode’ (from 1.01s to 0.89s).
2012-02-04 13:50:25 +00:00
Eelco Dolstra
2bda12ef3b * Print elapsed evaluation time. 2012-02-04 13:27:11 +00:00
Eelco Dolstra
330df4b4db * Allow comparisons between derivations by comparing the outPath
attributes.
2012-01-19 23:08:47 +00:00
Eelco Dolstra
9fe24c5a0d * Don't create thunks for simple constants (integers, strings, paths)
and allocate them only once.
* Move Value and related functions into value.hh.
2012-01-07 17:26:33 +00:00
Eelco Dolstra
b52966e821 * Remove dead code. 2012-01-04 21:47:48 +00:00
Eelco Dolstra
35f2a6ba82 * Don't use dynamic_cast, it's very slow. "nix-instantiate
/etc/nixos/nixos -A system" spent about 10% of its time in
  dynamic_cast.
2012-01-04 21:24:11 +00:00
Eelco Dolstra
921111d197 * Move the implementation of the ‘derivation’ primop into a separate
file.
2012-01-03 14:01:47 +00:00
Shea Levy
24b65937e1 Remove the unused sCurrentOutput symbol 2011-11-06 06:28:34 +00:00
Shea Levy
2721e9f56f Merge from trunk 2011-11-06 00:13:09 +00:00
Eelco Dolstra
a12095d3be * In printValueAsXML, handle the case where a "type" attribute is not
a string.  This happens in the NixOS option system.
* Remove a bogus comparison of a unsigned integer with -1.
2011-10-27 19:06:23 +00:00
Shea Levy
ffa038f66d Add an sCurrentOutput member to EvalState 2011-09-16 11:30:44 +00:00
Eelco Dolstra
2d663b502d * Cache the result of file evaluation (i.e, memoize evalFile()). This
prevents files from being evaluated and stored as values multiple
  times.  For instance, evaluation of the ‘system’ attribute in NixOS
  causes ‘nixpkgs/pkgs/lib/lists.nix’ to be evaluated 2019 times.

  Caching gives a modest speedup and a decent memory footprint
  reduction (e.g., from 1.44s to 1.28s, and from 81 MiB to 59 MiB with
  GC_INITIAL_HEAP_SIZE=100000 on my system).
2011-08-06 19:45:43 +00:00
Eelco Dolstra
00a724ebc6 * Remove a debug statement. 2011-08-06 18:24:43 +00:00
Eelco Dolstra
07340b8be7 * Add the Nix corepkgs to the end of the search path. This makes it
possible for other Nix expressions to use corepkgs (mostly useful
  for the buildenv function).
2011-08-06 18:23:38 +00:00
Eelco Dolstra
1ecc97b6bd * Add a Nix expression search path feature. Paths between angle
brackets, e.g.

    import <nixpkgs/pkgs/lib>

  are resolved by looking them up relative to the elements listed in
  the search path.  This allows us to get rid of hacks like

    import "${builtins.getEnv "NIXPKGS_ALL"}/pkgs/lib"

  The search path can be specified through the ‘-I’ command-line flag
  and through the colon-separated ‘NIX_PATH’ environment variable,
  e.g.,

    $ nix-build -I /etc/nixos ...

  If a file is not found in the search path, an error message is
  lazily thrown.
2011-08-06 16:05:24 +00:00
Eelco Dolstra
54945a2950 * Refactoring: move parseExprFromFile() and parseExprFromString() into
the EvalState class.
2011-08-06 13:02:55 +00:00
Eelco Dolstra
0a623a10c7 * Allow a default value in attribute selection by writing
x.y.z or default

  (as originally proposed in
  https://mail.cs.uu.nl/pipermail/nix-dev/2009-September/002989.html).

  For instance, an expression like

    stdenv.lib.attrByPath ["features" "ckSched"] false args

  can now be written as

    args.features.ckSched or false
2011-07-13 12:19:57 +00:00
Eelco Dolstra
2b9e29b1c8 * Change the right-hand side of the ‘.’ operator from an attribute to
an attribute path.  This is a refactoring to support default values.
2011-07-06 12:28:57 +00:00
Eelco Dolstra
5637037802 * In the ‘?’ operator, allow attribute paths. For instance, you can
write ‘attrs ? a.b’ to test whether ‘attrs’ has an attribute ‘a’
  containing an attribute ‘b’.  This is more convenient than ‘attrs ?
  a && attrs.a ? b’.

  Slight change in the semantics: it's no longer an error if the
  left-hand side of ‘?’ is not an attribute set.  In that case it just
  returns false.  So, ‘null ? foo’ no longer throws an error.
2011-07-06 10:58:17 +00:00
Eelco Dolstra
538b7caab0 * Don't allocate a big initial GC address space on machines with
little RAM.  Even if the memory isn't actually used, it can cause
  problems with the overcommit heuristics in the kernel.  So use a VM
  space of 25% of RAM, up to 384 MB.
2011-02-10 14:31:04 +00:00
Eelco Dolstra
5a6b039802 * Don't call GC_expand_hp unless we're actually using the garbage
collector.
2011-02-09 22:59:50 +00:00
Eelco Dolstra
0c4828ea05 * new(UseGC) is inexplicably slower than GC_MALLOC, so prefer the
latter.
2010-10-28 12:50:01 +00:00
Eelco Dolstra
e11e6fb1c6 * Handle out of memory condition. 2010-10-28 12:29:40 +00:00
Eelco Dolstra
43535499f3 * When allocating an attribute set, reserve enough space for all
elements.  This prevents the vector from having to resize itself.
2010-10-24 20:09:37 +00:00
Eelco Dolstra
e0b7fb8f27 * Keep attribute sets in sorted order to speed up attribute lookups.
* Simplify the representation of attributes in the AST.
* Change the behaviour of listToAttrs() in case of duplicate names.
2010-10-24 19:52:33 +00:00
Eelco Dolstra
2dc6d50941 * Don't create thunks for variable lookups (if possible). This
significantly reduces the number of values allocated (e.g. from 8.7m
  to 4.9m for the Bittorrent test).
2010-10-24 14:20:02 +00:00
Eelco Dolstra
0b305c534f * Store attribute sets as a vector instead of a map (i.e. a red-black
tree).  This saves a lot of memory.  The vector should be sorted so
  that names can be looked up using binary search, but this is not the
  case yet.  (Surprisingly, looking up attributes using linear search
  doesn't have a big impact on performance.)

  Memory consumption for

    $ nix-instantiate /etc/nixos/nixos/tests -A bittorrent.test --readonly-mode

  on x86_64-linux with GC enabled is now 185 MiB (compared to 946
  MiB on the trunk).
2010-10-24 00:41:29 +00:00
Eelco Dolstra
a247d20604 * Fix compiling without Boehm.
* Fix the stats.
2010-10-23 22:58:24 +00:00