Commit graph

944 commits

Author SHA1 Message Date
Eelco Dolstra
8327a7a8fa
Merge branch 'dirOf-relative' of https://github.com/lheckemann/nix 2018-09-13 14:33:12 +02:00
Eelco Dolstra
91405986f4
Convert NIX_COUNT_CALLS to JSON too 2018-09-05 21:57:54 +02:00
Eelco Dolstra
0a2545f95c
Log stats to stderr
We shouldn't pollute stdout.
2018-09-05 21:35:58 +02:00
Michael Bishop
4b034f390c remove the old text format output 2018-09-02 18:25:23 -03:00
Michael Bishop
2fd1008c70 add JSON to NIX_SHOW_STATS 2018-09-01 20:05:06 -03:00
Michael Bishop
c29e5fbb13 improve the stats when profiling 2018-09-01 17:11:56 -03:00
Eelco Dolstra
475a0a54a9
fetchGit/fetchMercurial: Don't absolutize paths
This is already done by coerceToString(), provided that the argument
is a path (e.g. 'fetchGit ./bla'). It fixes the handling of URLs like
git@github.com:owner/repo.git. It breaks 'fetchGit "./bla"', but that
was never intended to work anyway and is inconsistent with other
builtin functions (e.g. 'readFile "./bla"' fails).
2018-09-01 00:19:49 +02:00
aszlig
0ad643ed5c
libexpr: Use int64_t for NixInt
Using a 64bit integer on 32bit systems will come with a bit of a
performance overhead, but given that Nix doesn't use a lot of integers
compared to other types, I think the overhead is negligible also
considering that 32bit systems are in decline.

The biggest advantage however is that when we use a consistent integer
size across all platforms it's less likely that we miss things that we
break due to that. One example would be:

https://github.com/NixOS/nixpkgs/pull/44233

On Hydra it will evaluate, because the evaluator runs on a 64bit
machine, but when evaluating the same on a 32bit machine it will fail,
so using 64bit integers should make that consistent.

While the change of the type in value.hh is rather easy to do, we have a
few more options available for doing the conversion in the lexer:

  * Via an #ifdef on the architecture and using strtol() or strtoll()
    accordingly depending on which architecture we are. For the #ifdef
    we would need another AX_COMPILE_CHECK_SIZEOF in configure.ac.
  * Using istringstream, which would involve copying the value.
  * As we're already using boost, lexical_cast might be a good idea.

Spoiler: I went for the latter, first of all because lexical_cast does
have an overload for const char* and second of all, because it doesn't
involve copying around the input string. Also, because istringstream
seems to come with a bigger overhead than boost::lexical_cast:

https://www.boost.org/doc/libs/release/doc/html/boost_lexical_cast/performance.html

The first method (still using strtol/strtoll) also wasn't something I
pursued further, because it is also locale-aware which I doubt is what
we want, given that the regex for int is [0-9]+.

Signed-off-by: aszlig <aszlig@nix.build>
Fixes: #2339
2018-08-29 01:05:52 +02:00
Eelco Dolstra
c651b7bdc9
Revert "Fix parser/lexer generation with parallel make"
This reverts commit d277442df5.

Make sucks.
2018-08-23 00:23:35 +02:00
Eelco Dolstra
9b1bdf2db8
FIx floating point evaluation
Fixes #2361.
2018-08-19 11:59:49 +02:00
Eelco Dolstra
d277442df5
Fix parser/lexer generation with parallel make
Fun fact: rules with multiple targets don't work properly with 'make
-j'. For example, a rule like

  a b: c
    touch a b

is equivalent to

  a: c
    touch a b

  b: c
    touch a b

so with 'make -j', the 'touch' command will be run twice. See
e.g. https://stackoverflow.com/questions/2973445/gnu-makefile-rule-generating-a-few-targets-from-a-single-source-file.
2018-08-17 12:59:23 +02:00
Linus Heckemann
d7402c9cd5 dirOf: allow use on non-absolute paths 2018-08-13 11:27:35 +02:00
Eelco Dolstra
bc65e02d96
Merge pull request #2326 from aszlig/fix-symlink-leak
Fix symlink leak in restricted eval mode
2018-08-03 17:01:34 +02:00
aszlig
43e28a1b75
Fix symlink leak in restricted eval mode
In EvalState::checkSourcePath, the path is checked against the list of
allowed paths first and later it's checked again *after* resolving
symlinks.

The resolving of the symlinks is done via canonPath, which also strips
out "../" and "./". However after the canonicalisation the error message
pointing out that the path is not allowed prints the symlink target in
the error message.

Even if we'd suppress the message, symlink targets could still be leaked
if the symlink target doesn't exist (in this case the error is thrown in
canonPath).

So instead, we now do canonPath() without symlink resolving first before
even checking against the list of allowed paths and then later do the
symlink resolving and checking the allowed paths again.

The first call to canonPath() should get rid of all the "../" and "./",
so in theory the only way to leak a symlink if the attacker is able to
put a symlink in one of the paths allowed by restricted evaluation mode.

For the latter I don't think this is part of the threat model, because
if the attacker can write to that path, the attack vector is even
larger.

Signed-off-by: aszlig <aszlig@nix.build>
2018-08-03 06:46:43 +02:00
Eelco Dolstra
45bcf5416a
Merge branch 'prim_mapAttr-fix' of https://github.com/volth/nix 2018-07-31 20:05:07 +02:00
Eelco Dolstra
a7fb7d3cde
Merge pull request #2303 from volth/patch-4
parser.y: fix assoc of -> and < > <= >=
2018-07-23 11:38:15 +02:00
volth
deaa6e9a34
parser.y: right-associativity of -> 2018-07-23 07:28:48 +00:00
volth
85fe4a819c
parser.y: fix assoc of -> and < > <= >=
The parser allowed senseless `a > b > c` but disallowed `a -> b -> c` which seems valid
It might be a typo
2018-07-21 15:24:51 +00:00
volth
e2b114cfe1
prim_foldlStrict: call forceValue() before value is copied
forceValue() were called after a value is copied effectively forcing only one of the copies keeping another copy not evaluated.
This resulted in its evaluation of the same lazy value more than once (the number of hits is not big though)
2018-07-21 06:44:42 +00:00
Eelco Dolstra
c2de2ff385
Fix build on 32-bit systems and macOS
Apparently, on macOS, 'long' != 'int64_t'.

https://hydra.nixos.org/build/77100756
2018-07-11 21:12:18 +02:00
Eelco Dolstra
efd04888ca
Shup up a warning 2018-07-11 21:05:09 +02:00
Eelco Dolstra
875cd9da2b
Remove unused function printStats2()
Closes #2282.
2018-07-11 20:29:18 +02:00
volth
c03d73c1cf
prim_map, prim_genList: no need to force function argument which will be stored in mkApp 2018-07-07 23:30:17 +00:00
volth
627e28ba33 prim_mapAttrs: `f' must be evaluated lazily to avoid infinite recursion 2018-07-06 21:52:54 +00:00
volth
1515c65616 prim_concatMap: no need to force value 2018-07-05 15:33:33 +00:00
volth
e6bf1a79d7 prim_mapAttrs: must be lazy to avoid infinite recursion 2018-07-05 15:33:12 +00:00
volth
841747b0e6
prim_concatMap: allocate intermediate list on stack 2018-07-05 12:37:37 +00:00
volth
ee218f99ca
primops.cc: fix comment 2018-07-05 11:58:15 +00:00
volth
403a76a18f lib.concatMap and lib.mapAttrs to be builtins 2018-07-05 02:54:09 +00:00
Eelco Dolstra
ea3c9dab5f
Include cpptoml for build simplicity 2018-07-03 18:39:36 +02:00
Eelco Dolstra
3b1f54cf06
Add a fromTOML primop
This is primarily useful for processing Cargo.lock files.
2018-07-03 18:39:36 +02:00
Eelco Dolstra
a92ed973e5
Store floating point numbers in double precision
Even on 32-bit systems, Value has enough space to hold a double.
2018-07-03 18:39:32 +02:00
Eelco Dolstra
c905d8b0a8
GC_malloc -> GC_MALLOC
This makes it possible to build with -DGC_DEBUG.
2018-06-12 17:49:55 +02:00
Eelco Dolstra
455d1f01d0
Don't scan for roots in dynamic libraries
This reduces the risk of object liveness misdetection. For example,
Glibc has an internal variable "mp_" that often points to a Boehm
object, keeping it alive unnecessarily. Since we don't store any
actual roots in global variables, we can just disable data segment
scanning.

With this, the max RSS doing 100 evaluations of
nixos.tests.firefox.x86_64-linux.drvPath went from 718 MiB to 455 MiB.
2018-06-12 17:49:55 +02:00
Eelco Dolstra
30964103dc
Add temporary stats 2018-06-12 17:49:55 +02:00
Eelco Dolstra
24c6806994
Cache parse trees
This prevents EvalState::resetFileCache() from parsing everything all
over again.
2018-06-12 17:49:55 +02:00
Eelco Dolstra
6ad0a2f749
Remove duplicate definition of allocBytes() 2018-06-12 17:49:51 +02:00
Eelco Dolstra
5d4a9d5677
Fix static assertion failure on 32-bit systems 2018-05-30 17:47:30 +02:00
Eelco Dolstra
1672bcd230
Move evaluator-specific settings out of libstore 2018-05-30 13:29:50 +02:00
Eelco Dolstra
bd56b5fe3f
Merge pull request #2187 from bgamari/stoi-exceptions
json-to-value: Use strtol instead of strtoi
2018-05-28 14:54:49 +02:00
Ben Gamari
7f560b81ea json-to-value: Use strtol instead of strtoi
NixInt is long, so strtoi is too restrictive.
2018-05-26 18:43:46 -04:00
Will Dietz
93ae90de0f eval.cc: add message to static_assert, message can be omitted w/c++17 2018-05-26 14:04:41 -05:00
Eelco Dolstra
743359bc8a
Merge pull request #2157 from volth/bitwise
add builtins: __bitAnd,  __bitOr,  __bitXor
2018-05-24 15:00:39 +02:00
Eelco Dolstra
4bb8741b98
Make Env self-describing
If the Env denotes a 'with', then values[0] may be an Expr* cast to a
Value*. For code that generically traverses Values/Envs, it's useful
to know this.
2018-05-22 16:02:32 +02:00
Eelco Dolstra
9fd7cf98db
Memoise checkSourcePath()
This prevents hydra-eval-jobs from statting the same files over and
over again.
2018-05-22 13:02:14 +02:00
volth
6cc28c0589
add `mod' and bitwise builtins: camel-case function names 2018-05-16 10:52:19 +00:00
volth
f3c090f91c
add mod' and bitwise builtins: remove mod' and shifts 2018-05-16 06:57:11 +00:00
volth
49b7cf1813
add `mod' and bitwise builtins: remove infix functions 2018-05-16 06:55:24 +00:00
volth
8a6a14e1f5 add `mod' and bitwise builtins 2018-05-12 16:50:39 +00:00
Eelco Dolstra
1ad19232c4
Don't return negative numbers from the flex tokenizer
Fixes #1374.
Closes #2129.
2018-05-11 12:05:12 +02:00