Commit graph

768 commits

Author SHA1 Message Date
Manuel Jacob
12c1776df9 Don't try to link libdl on FreeBSD.
There is no libdl on FreeBSD.  Instead the symbols are included in libc.
2015-10-06 22:28:30 +02:00
Eelco Dolstra
4ba6bc184c Shut up clang warnings 2015-09-18 01:22:35 +02:00
Iwan Aucamp
3db950aab7 Removed unnecessary included 2015-08-03 11:48:34 +02:00
Eelco Dolstra
2bac04c5ff Fix stack consumption 2015-07-31 20:28:25 +02:00
Iwan Aucamp
75837651f1 Output line number on infinite recursion 2015-07-31 20:26:44 +02:00
Eelco Dolstra
76cc8e97a2 Add sort primop 2015-07-28 18:39:39 +02:00
Eelco Dolstra
50807f3dd5 Add primop genList
This can be used to implement functions like ‘imap’ (or for that
matter, ‘map’) without the quadratic complexity incurred by calling
‘++’ repeatedly.
2015-07-28 17:28:35 +02:00
Eelco Dolstra
d6d5885c15 Add replaceStrings primop
This is a generalisation of replaceChars in Nixpkgs.
2015-07-24 15:32:24 +02:00
Eelco Dolstra
2e8fd4c5cd Add concatStringsSep as a primop
This fixes the quadratic behaviour of concatStrings/concatStringsSep
in Nixpkgs.
2015-07-24 02:38:09 +02:00
Eelco Dolstra
cb4320c1a0 Cleanup 2015-07-23 23:14:07 +02:00
Eelco Dolstra
c8bb2371eb Optimize empty sets
This reduces the number of Bindings allocations by about 10%.
2015-07-23 23:11:08 +02:00
Eelco Dolstra
19eddecc0f Merge branch 'attr-set-hh' of https://github.com/nbp/nix
Conflicts:
	src/libexpr/eval.cc
2015-07-23 22:16:01 +02:00
Eelco Dolstra
b83801f8b3 Optimize small lists
The value pointers of lists with 1 or 2 elements are now stored in the
list value itself. In particular, this makes the "concatMap (x: if
cond then [(f x)] else [])" idiom cheaper.
2015-07-23 22:05:09 +02:00
Eelco Dolstra
14be783676 Add primops all and any
These are used thousands of times during NixOS evaluation, so it's
useful to speed them up.
2015-07-23 19:23:11 +02:00
Shea Levy
39e27a04b8 Importing derivations: Add name attribute to make a valid drv 2015-07-23 17:04:07 +02:00
Shea Levy
1ed55234d9 Allow derivations-as-srcs in the context of builtins.toFile files 2015-07-23 17:03:47 +02:00
Eelco Dolstra
61af14a921 Add foldl' primop 2015-07-23 17:03:02 +02:00
Eelco Dolstra
0a2bee307b Make <nix/fetchurl.nix> a builtin builder
This ensures that 1) the derivation doesn't change when Nix changes;
2) the derivation closure doesn't contain Nix and its dependencies; 3)
we don't have to rely on ugly chroot hacks.
2015-07-20 04:38:46 +02:00
Eelco Dolstra
6bd2c7bb38 OCD: foreach -> C++11 ranged for 2015-07-17 20:13:56 +02:00
Eelco Dolstra
f39979c6d3 Make printValue() interruptible
Fixes #572.
2015-07-17 11:33:39 +02:00
Nicolas B. Pierron
db21cfa688 Move attribute set data structures into their own header file.
This modification moves Attr and Bindings structures into their own header
file which is dedicated to the attribute set representation. The goal of to
isolate pieces of code which are related to the attribute set
representation. Thus future modifications of the attribute set
representation will only have to modify these files, and not every other
file across the evaluator.
2015-07-14 19:23:17 +02:00
Guillaume Maudoux
467977f203 Fix the parsing of "$"'s in strings. 2015-07-03 14:09:58 +02:00
Guillaume Maudoux
65e4dcd69b Fix the hack that resets the scanner state. 2015-07-03 13:53:36 +02:00
Eelco Dolstra
65f17cd330 Support URLs in $NIX_PATH
This didn't work (despite claims in the manual), because the colon in
"http://" was parsed as a element separator. So handle "://"
specially.
2015-06-17 16:20:11 +02:00
Eelco Dolstra
bc51175dc0 Add tarball tests 2015-06-01 16:18:23 +02:00
Eelco Dolstra
da196ec68f Document tarball downloading 2015-06-01 15:14:44 +02:00
Eelco Dolstra
c1323b53e3 Fix fetchurl/fetchTarball 2015-06-01 15:08:09 +02:00
Eelco Dolstra
1c88e100e7 readFile: Check against nul bytes 2015-06-01 15:07:42 +02:00
Eelco Dolstra
920f5fd4dd Fix import-from-derivation in restricted eval mode
This relaxes restricted mode to allow access to anything in the
store. In the future, it would be better to allow access to only paths
that have been constructed in the current evaluation (so a hard-coded
/nix/store/blabla in a Nix expression would still be
rejected). However, note that reading /nix/store itself is still
rejected, so you can't use this so get access to things you don't know
about.
2015-05-22 12:18:23 +02:00
Eelco Dolstra
6519f06f39 nix-env/nix-instantiate/nix-build: Support URIs
For instance, you can install Firefox from a specific Nixpkgs revision
like this:

  $ nix-env -f 63def04891.tar.gz -iA firefox

Or build a package from the latest nixpkgs-unstable channel:

  $ nix-build https://nixos.org/channels/nixpkgs-unstable/nixexprs.tar.xz -A hello
2015-05-06 14:54:31 +02:00
Eelco Dolstra
9451ef3731 Allow URLs in the Nix search path
E.g. to install "hello" from the latest Nixpkgs:

  $ nix-build '<nixpkgs>' -A hello -I nixpkgs=https://nixos.org/channels/nixpkgs-unstable/nixexprs.tar.xz

Or to install a specific version of NixOS:

  $ nixos-rebuild switch -I nixpkgs=63def04891.tar.gz
2015-05-05 17:09:42 +02:00
Eelco Dolstra
35d30d67eb Make downloads interruptable 2015-05-05 14:39:48 +02:00
Eelco Dolstra
035aeb9547 Fix using restricted mode with chroots 2015-04-16 18:46:17 +02:00
Eelco Dolstra
4ed2187377 Use cached result if there is a network error 2015-04-09 12:49:13 +02:00
Eelco Dolstra
1fc905ad4c Move curl stuff into a separate file 2015-04-09 12:12:50 +02:00
Eelco Dolstra
c1f04fae35 Implement a TTL on cached fetchurl/fetchTarball results
This is because we don't want to do HTTP requests on every evaluation,
even though we can prevent a full redownload via the cached ETag. The
default is one hour.
2015-04-09 11:55:36 +02:00
Eelco Dolstra
60340ce3e2 Implement caching of fetchurl/fetchTarball results
ETags are used to prevent redownloading unchanged files.
2015-04-09 11:42:04 +02:00
Eelco Dolstra
000b5a000f Add fetchTarball builtin
This function downloads and unpacks the given URL at evaluation
time. This is primarily intended to make it easier to deal with Nix
expressions that have external dependencies. For instance, to fetch
Nixpkgs 14.12:

  with import (fetchTarball https://github.com/NixOS/nixpkgs-channels/archive/nixos-14.12.tar.gz) {};

Or to fetch a specific revision:

  with import (fetchTarball 2766a4b44e.tar.gz) {};

This patch also adds a ‘fetchurl’ builtin that downloads but doesn't
unpack its argument. Not sure if it's useful though.
2015-03-25 17:29:09 +01:00
Eelco Dolstra
7ea6ecf855 addToStore(): Take explicit name argument 2015-03-25 17:06:12 +01:00
Eelco Dolstra
6f0c6e20e0 Don't rely on __noChroot for corepkgs
This doesn't work anymore if the "strict" chroot mode is
enabled. Instead, add Nix's store path as a dependency. This ensures
that its closure is present in the chroot.
2015-03-24 11:15:45 +01:00
Eelco Dolstra
b005e63ccf Disable scanning for interior pointers
This may remove the "Repeated allocation of very large block"
warnings.
2015-03-19 20:10:08 +01:00
Eelco Dolstra
726f7f7fc9 Fix Boehm API violation
We were calling GC_INIT() after doing an allocation (in the baseEnv
construction), which is not allowed.
2015-03-19 20:02:37 +01:00
Eelco Dolstra
da6b704b19 Check return values from malloc/strdup 2015-03-19 14:11:35 +01:00
Eelco Dolstra
fa47279440 Print some Boehm GC stats 2015-03-18 16:24:54 +01:00
Eelco Dolstra
7a84143910 valueSize(): Take into account list/bindings/env size 2015-03-18 14:41:28 +01:00
Daniel Hahler
e659978ced Fix typos: s/the the/the/ 2015-03-06 16:43:22 +01:00
Eelco Dolstra
17c71334e1 forceValueDeep: Add to error prefix 2015-03-06 15:10:12 +01:00
Eelco Dolstra
5badc8f975 Improve error message 2015-03-06 14:24:08 +01:00
Eelco Dolstra
15d2d3c34e Add restricted evaluation mode
If ‘--option restrict-eval true’ is given, the evaluator will throw an
exception if an attempt is made to access any file outside of the Nix
search path. This is primarily intended for Hydra, where we don't want
people doing ‘builtins.readFile ~/.ssh/id_dsa’ or stuff like that.
2015-02-23 15:54:31 +01:00
Eelco Dolstra
c33244d7c1 Merge branch 'tilde-paths' of https://github.com/shlevy/nix 2015-02-19 14:55:06 +01:00
Shea Levy
c4653afbcd tilde paths: The rest of the string has to start with a slash anyway 2015-02-19 08:52:13 -05:00
Shea Levy
e3e38a048e tilde paths: construct the entire path at parse time 2015-02-19 08:51:21 -05:00
Shea Levy
4b7c9f834c tilde paths: get HOME at parse time 2015-02-19 08:49:10 -05:00
Eelco Dolstra
9bedd9b09b Remove obsolete reference to ~ operator 2015-02-19 14:41:22 +01:00
Shea Levy
4646e94610 ExprConcatStrings: canonicalize concatenated paths 2015-02-19 08:39:25 -05:00
Shea Levy
e0953d53de Allow the leading component of a path to be a ~ 2015-02-19 08:05:16 -05:00
Eelco Dolstra
2be7f79fd4 Remove tab 2015-02-05 17:21:30 +01:00
Shea Levy
73bf32ce94 Merge remote-tracking branch 'shlevy/baseNameOf-no-copy'
baseNameOf: Don't copy paths to the store first
2015-01-29 03:29:09 -05:00
Eelco Dolstra
c2a8b5c42d Fix assertion failure in nix-env
$ nix-env -f ~/Dev/nixops/ -iA foo
  nix-env: src/libexpr/eval.hh:57: void nix::Bindings::push_back(const nix::Attr&): Assertion `size_ < capacity' failed.
  Aborted
2015-01-15 12:15:22 +01:00
Eelco Dolstra
2a3b1df423 Fix builtins.readDir on XFS
The DT_UNKNOWN fallback code was getting the type of the wrong path,
causing readDir to report "directory" as the type of every file.

Reported by deepfire on IRC.
2015-01-09 14:56:25 +01:00
Eelco Dolstra
153a943de7 Show position info for failing <...> lookups 2015-01-07 13:43:55 +01:00
Eelco Dolstra
6fec43ccb3 Remove quotes around filenames in position info 2015-01-07 12:08:10 +01:00
Eelco Dolstra
4e0607369e Pedantry 2014-12-14 01:51:14 +01:00
Eelco Dolstra
8aedaf111e Remove canary stuff 2014-12-12 10:59:50 +01:00
Shea Levy
50c3352811 builtins.readFile: realise context associated with the path 2014-12-10 12:26:53 +01:00
Shea Levy
608110804c Make all ExternalValueBase functions const 2014-12-02 10:27:10 -05:00
Shea Levy
320659b0cd Allow external code using libnixexpr to add types
Code that links to libnixexpr (e.g. plugins loaded with importNative, or
nix-exec) may want to provide custom value types and operations on
values of those types. For example, nix-exec is currently using sets
where a custom IO value type would be more appropriate. This commit
provides a generic hook for such types in the form of tExternal and the
ExternalBase virtual class, which contains all functions necessary for
libnixexpr's type-polymorphic functions (e.g. `showType`) to be
implemented.
2014-12-02 10:27:04 -05:00
Eelco Dolstra
976df480c9 Add a primop for regular expression pattern matching
The function ‘builtins.match’ takes a POSIX extended regular
expression and an arbitrary string. It returns ‘null’ if the string
does not match the regular expression. Otherwise, it returns a list
containing substring matches corresponding to parenthesis groups in
the regex. The regex must match the entire string (i.e. there is an
implied "^<pat>$" around the regex).  For example:

  match "foo" "foobar" => null
  match "foo" "foo" => []
  match "f(o+)(.*)" "foooobar" => ["oooo" "bar"]
  match "(.*/)?([^/]*)" "/dir/file.nix" => ["/dir/" "file.nix"]
  match "(.*/)?([^/]*)" "file.nix" => [null "file.nix"]

The following example finds all regular files with extension .nix or
.patch underneath the current directory:

  let

    findFiles = pat: dir: concatLists (mapAttrsToList (name: type:
      if type == "directory" then
        findFiles pat (dir + "/" + name)
      else if type == "regular" && match pat name != null then
        [(dir + "/" + name)]
      else []) (readDir dir));

  in findFiles ".*\\.(nix|patch)" (toString ./.)
2014-11-25 11:47:06 +01:00
Eelco Dolstra
4e340a983f forceString(): Accept pos argument 2014-11-25 10:23:36 +01:00
Shea Levy
b0c5c2ac34 import derivation: cleanup
Before this there was a bug where a `find` was being called on a
not-yet-sorted set. The code was just a mess before anyway, so I cleaned
it up while fixing it.
2014-11-20 22:48:12 -05:00
Shea Levy
2719627bbe realiseContext: Handle all context types
Avoids an assertion
2014-11-15 21:43:51 -05:00
Shea Levy
997defa166 Add functors (callable attribute sets).
With this, attribute sets with a `__functor` attribute can be applied
just like normal functions. This can be used to attach arbitrary
metadata to a function without callers needing to treat it specially.
2014-11-15 16:12:05 -05:00
Eelco Dolstra
3a9b4a1467 Fix more warnings 2014-10-31 08:49:15 +01:00
Shea Levy
6062b12160 Fix build on gcc < 4.7 2014-10-20 12:15:50 -04:00
Eelco Dolstra
ecc2c8f464 Improve printing of ASTs 2014-10-20 09:13:21 +02:00
Shea Levy
0ee1ca628a baseNameOf: Don't copy paths to the store first 2014-10-18 20:28:28 -04:00
Shea Levy
d16e3c7f09 Export realiseContext in libnixexpr
Useful for importNative plugins
2014-10-17 22:15:09 -04:00
Eelco Dolstra
6bb4c0b712 mkList: Scrub better
Clearing v.app.right was not enough, because the length field of a
list only takes 32 bits, so the most significant 32 bits of v.app.left
(a.k.a. v.thunk.env) would remain. This could cause Boehm GC to
interpret it as a valid pointer.

This change reduces maximum RSS for evaluating the ‘tested’ job in
nixos/release-small.nix from 1.33 GiB to 0.80 GiB, and runtime by
about 8%.
2014-10-09 13:08:53 +02:00
Eelco Dolstra
986fbd6fab Typo 2014-10-09 11:34:48 +02:00
Eelco Dolstra
b6809608cc Get rid of some unnecessary ExprConcatStrings nodes in dynamic attrs
This gives a ~18% speedup in NixOS evaluation (after converting
most calls to hasAttr/getAttr to dynamic attrs).
2014-10-05 01:04:58 +02:00
Eelco Dolstra
1418806969 Show total allocations 2014-10-05 00:39:28 +02:00
Eelco Dolstra
c3f0a489f9 Add primop ‘catAttrs’ 2014-10-04 18:15:03 +02:00
Eelco Dolstra
d4fcbe1687 Add primop ‘attrValues’ 2014-10-04 16:41:24 +02:00
Eelco Dolstra
58d8a213b0 Tweak 2014-10-04 11:27:23 +02:00
Eelco Dolstra
3f8576a6ab Remove some duplicate code 2014-10-03 22:37:51 +02:00
Shea Levy
c08c802bf3 Add readDir primop 2014-10-03 22:32:11 +02:00
Eelco Dolstra
3b5fa8d50c Don't recompile the same regex over and over 2014-10-03 21:29:40 +02:00
Eelco Dolstra
104e55bb7f nix-env: Add regular expression support in selectors
So you can now do things like:

  $ nix-env -qa '.*zip.*'
  $ nix-env -qa '.*(firefox|chromium).*'
2014-10-03 21:29:40 +02:00
Eelco Dolstra
a17c23426e printValue(): Don't print <CYCLE> for repeated values 2014-10-01 15:54:40 +02:00
Eelco Dolstra
d61853430a Support control characters in JSON output 2014-09-30 00:41:18 +02:00
Eelco Dolstra
0ed1b924be Bindings: Remove copy constructor 2014-09-24 15:29:05 +02:00
Eelco Dolstra
ebb1dbb3e1 Add missing static 2014-09-23 15:08:27 +02:00
Eelco Dolstra
53b044c2f6 Don't evaluate inside a "throw"
Workaround for
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=41174. This caused
hydra-eval-jobs to ignore SIGINT.
2014-09-22 19:18:05 +02:00
Eelco Dolstra
0cd6596b0e Add ‘deepSeq’ primop
Note that unlike ‘lib.deepSeq’ in Nixpkgs, this handles cycles.
2014-09-22 16:05:00 +02:00
Eelco Dolstra
831fc8ea21 Make forceValueDeep work on values with cycles 2014-09-22 16:05:00 +02:00
Eelco Dolstra
1e0a799bef Rename strictForceValue -> forceValueDeep 2014-09-22 16:05:00 +02:00
Eelco Dolstra
022618c794 Handle cycles when printing a value
So this no longer crashes with a stack overflow:

  nix-instantiate -E --eval 'let as = { x = as; }; in as'

Instead it prints:

  { x = { x = <CYCLE>; }; }
2014-09-22 16:05:00 +02:00
Eelco Dolstra
a54c263402 Add ‘seq’ primop 2014-09-22 16:05:00 +02:00
Eelco Dolstra
eff120d1b9 Add a function ‘valueSize’
It returns the size of value, including all other values and
environments reachable from it. It is intended for debugging memory
consumption issues.
2014-09-22 16:05:00 +02:00
Eelco Dolstra
2d6cd8aafd attrNames: Don't allocate duplicates of the symbols 2014-09-19 18:11:46 +02:00