Commit graph

4837 commits

Author SHA1 Message Date
Eelco Dolstra
d05bf04444 Treat SQLITE_PROTOCOL as SQLITE_BUSY
In the Hydra build farm we fairly regularly get SQLITE_PROTOCOL errors
(e.g., "querying path in database: locking protocol").  The docs for
this error code say that it "is returned if some other process is
messing with file locks and has violated the file locking protocol
that SQLite uses on its rollback journal files."  However, the SQLite
source code reveals that this error can also occur under high load:

  if( cnt>5 ){
    int nDelay = 1;                      /* Pause time in microseconds */
    if( cnt>100 ){
      VVA_ONLY( pWal->lockError = 1; )
      return SQLITE_PROTOCOL;
    }
    if( cnt>=10 ) nDelay = (cnt-9)*238;  /* Max delay 21ms. Total delay 996ms */
    sqlite3OsSleep(pWal->pVfs, nDelay);
  }

i.e. if certain locks cannot be not acquired, SQLite will retry a
number of times before giving up and returing SQLITE_PROTOCOL.  The
comments say:

  Circumstances that cause a RETRY should only last for the briefest
  instances of time.  No I/O or other system calls are done while the
  locks are held, so the locks should not be held for very long. But
  if we are unlucky, another process that is holding a lock might get
  paged out or take a page-fault that is time-consuming to resolve,
  during the few nanoseconds that it is holding the lock.  In that case,
  it might take longer than normal for the lock to free.
  ...
  The total delay time before giving up is less than 1 second.

On a heavily loaded machine like lucifer (the main Hydra server),
which often has dozens of processes waiting for I/O, it seems to me
that a page fault could easily take more than a second to resolve.
So, let's treat SQLITE_PROTOCOL as SQLITE_BUSY and retry the
transaction.

Issue NixOS/hydra#14.
2013-10-16 14:19:59 +02:00
Eelco Dolstra
c1994fecf9 nix-shell: Fix bash completion
Nixpkgs's stdenv setup script sets the "nullglob" option, but doing so
breaks Bash completion on NixOS (when ‘programs.bash.enableCompletion’
is set) and on Ubuntu.  So clear that flag afterwards.  Of course,
this may break stdenv functions in subtle ways...
2013-10-14 15:28:43 +02:00
Eelco Dolstra
672c3acc71 Adjust to the NixOS/Nixpkgs merge 2013-10-11 10:57:23 +02:00
Eelco Dolstra
7bdb85453d printStats(): Print the size of the symbol table in bytes 2013-10-08 15:37:08 +02:00
Eelco Dolstra
9deb822180 Deduplicate filenames in Pos
This saves ~4 MiB of RAM for NixOS system instantiation, and ~18 MiB
for "nix-env -qa".
2013-10-08 15:36:10 +02:00
Eelco Dolstra
b1e3b1a4ac Treat undefined variable errors consistently
Previously, a undefined variable inside a "with" caused an EvalError
(which can be caught), while outside, it caused a ParseError (which
cannot be caught).  Now both cause an UndefinedVarError (which cannot
be caught).
2013-10-08 14:45:36 +02:00
Eelco Dolstra
6b47de580f Show the exact position of undefined variables
In particular, undefined variable errors in a "with" previously didn't
show *any* position information, so this should help a lot in those
cases.
2013-10-08 14:40:51 +02:00
Eelco Dolstra
a5e0f64db3 Remove some unused functions 2013-10-08 12:30:23 +00:00
Eelco Dolstra
221a2daf34 Merge VarRef into ExprVar 2013-10-08 14:24:53 +02:00
Eelco Dolstra
176c666f36 Don't show calls to primops in stack traces
Since they don't have location information, they just give you crap
like:

  while evaluating the builtin function `getAttr':
  while evaluating the builtin function `derivationStrict':
  ...
2013-10-07 18:02:32 +02:00
Eelco Dolstra
c945f015de Fix segfault in nix-repl / hydra-eval-jobs
If a "with" attribute set fails to evaluate, we have to make sure its
Env record remains unchanged.  Otherwise, repeated evaluation gives a
segfault:

  nix-repl> :a with 0; { a = x; b = x; }
  Added 2 variables.

  nix-repl> a
  error: value is an integer while an attribute set was expected

  nix-repl> b
  Segmentation fault
2013-10-02 15:24:45 +02:00
Eelco Dolstra
28e0742966 Report OOM errors better 2013-10-02 14:34:36 +02:00
Eelco Dolstra
a5fb4b5b7c Fix typo 2013-10-02 14:22:49 +02:00
Eelco Dolstra
faaae44f2e build-remote.pl: Don't use substituters on the remote
It's kinda pointless to check substituters on the remote side, since
we just checked them locally.
2013-09-18 14:04:03 +02:00
Eelco Dolstra
f53574ebd6 RestoreSink: Slightly reduce the number of concurrent FDs 2013-09-17 12:06:59 +00:00
Eelco Dolstra
d5529f5b85 Version was called 1.6, not 1.6.0 2013-09-10 17:48:11 +02:00
Eelco Dolstra
b072fc04a7 Bump version number 2013-09-10 17:41:49 +02:00
Eelco Dolstra
fecad91b67 Update release notes 2013-09-10 11:21:30 +02:00
Eelco Dolstra
0220da3e10 Remove stray debug line 2013-09-06 17:20:19 +02:00
Eelco Dolstra
936f9d45ba Don't apply the CPU affinity hack to nix-shell (and other Perl programs)
As discovered by Todd Veldhuizen, the shell started by nix-shell has
its affinity set to a single CPU.  This is because nix-shell connects
to the Nix daemon, which causes the affinity hack to be applied.  So
we turn this off for Perl programs.
2013-09-06 16:36:56 +02:00
Domen Kožar
4b83830d0c typo 2013-09-06 15:18:07 +02:00
Eelco Dolstra
5904262640 nix-shell: Support a .drv as argument
Fixes #161.
2013-09-06 14:58:05 +02:00
Eelco Dolstra
2c1ecf8e81 nix-env -i: Add a flag ‘--remove-all’ / ‘-r’
This is equivalent to running ‘nix-env -e '*'’ first, except that it
happens in a single transaction.  Thus, ‘nix-env -i pkgs...’ replaces
the profile with the specified set of packages.

The main motivation is to support declarative package management
(similar to environment.systemPackages in NixOS).  That is, if you
have a specification ‘profile.nix’ like this:

  with import <nixpkgs> {};
  [ thunderbird
    geeqie
    ...
  ]

then after any change to ‘profile.nix’, you can run:

  $ nix-env -f profile.nix -ir

to update the profile to match the specification.  (Without the ‘-r’
flag, if you remove a package from ‘profile.nix’, it won't be removed
from the actual profile.)

Suggested by @zefhemel.
2013-09-03 21:21:14 +02:00
Eelco Dolstra
88c07341a6 nix-env: Use wildcard match by default
That is, you don't need to pass '*' anymore, so

  nix-env -qa

is equivalent to

  nix-env -qa '*'
2013-09-03 16:35:49 +02:00
Eelco Dolstra
07a08bddf0 nix-env: Load files in ~/.nix-defexpr on demand
So if you do "nix-env -qa -A nixos", then other channels won't be
parsed/evaluated at all.
2013-09-03 15:45:32 +02:00
Eelco Dolstra
c57ed84e28 Check for name collisions in the input Nix expressions 2013-09-03 15:25:51 +02:00
Eelco Dolstra
ef4f5ba85e Work on Values instead of Exprs
This prevents some duplicate evaluation in nix-env and
nix-instantiate.

Also, when traversing ~/.nix-defexpr, only read regular files with the
extension .nix.  Previously it was reading files like
.../channels/binary-caches/<name>.  The only reason this didn't cause
problems is pure luck (namely, <name> shadows an actual Nix
expression, the binary-caches files happen to be syntactically valid
Nix expressions, and we iterate over the directory contents in just
the right order).
2013-09-03 13:17:51 +00:00
Eelco Dolstra
06bb2d95b4 Reformat 2013-09-03 11:04:21 +00:00
Eelco Dolstra
6f809194d7 Get rid of the parse tree cache
Since we already cache files in normal form (fileEvalCache), caching
parse trees is redundant.

Note that getting rid of this cache doesn't actually save much memory
at the moment, because parse trees are currently not freed / GC'ed.
2013-09-03 13:01:42 +02:00
Eelco Dolstra
57d18df7d0 Add some support code for nix-repl 2013-09-02 18:34:04 +02:00
Eelco Dolstra
92077b4547 Get rid of a signedness warning 2013-09-02 16:39:17 +02:00
Eelco Dolstra
33972629d7 Fix whitespace 2013-09-02 16:29:15 +02:00
Eelco Dolstra
ac1b754138 Lower xz compression level
Fixes #84.
2013-09-02 13:32:51 +02:00
Eelco Dolstra
c28dfc0305 Manual: Make '' a bit clearer
Issue #162.
2013-09-02 13:24:29 +02:00
Eelco Dolstra
e9b92169a5 Fix manual error
Reported by Matija Šuklje.

Fixes #163.
2013-09-02 13:19:34 +02:00
Eelco Dolstra
ecd830b3b9 Update the release notes 2013-09-02 13:11:36 +02:00
Eelco Dolstra
6ec8dab06a Adda test for build-max-log-size 2013-09-02 12:44:30 +02:00
Eelco Dolstra
b29d3f4aee Only show trace messages when tracing is enabled 2013-09-02 12:01:04 +02:00
Eelco Dolstra
efe4289464 Add an option to limit the log output of builders
This is mostly useful for Hydra to deal with builders that get stuck
in an infinite loop writing data to stdout/stderr.
2013-09-02 11:58:18 +02:00
Shea Levy
afc6c1bad6 Simplify inherited attribute handling
This reduces the difference between inherited and non-inherited
attribute handling to the choice of which env to use (in recs and lets)
by setting the AttrDef::e to a new ExprVar in the parser rather than
carrying a separate AttrDef::v VarRef member.

As an added bonus, this allows inherited attributes that inherit from a
with to delay forcing evaluation of the with's attributes.

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-08-26 11:31:56 +02:00
Ivan Kozik
6cd6ce5608 Fix nix-shell command 2013-08-26 11:15:22 +02:00
Ivan Kozik
34bb806f74 Fix typos, especially those that end up in the Nix manual 2013-08-26 11:15:22 +02:00
Gergely Risko
c6c024ca6f Fix personality switching from x86_64 to i686
On Linux, Nix can build i686 packages even on x86_64 systems.  It's not
enough to recognize this situation by settings.thisSystem, we also have
to consult uname().  E.g. we can be running on a i686 Debian with an
amd64 kernel.  In that situation settings.thisSystem is i686-linux, but
we still need to change personality to i686 to make builds consistent.
2013-08-26 11:12:35 +02:00
Eelco Dolstra
03eaef3d7a Manual: Don't use actual hashes of Nix dependencies
These cause an unnecessary runtime dependency :-)
2013-08-23 10:12:20 +02:00
Eelco Dolstra
b3110a15e9 Fix corrupt PNG
Libpng used to accept this, but no longer does.
2013-08-21 12:53:51 +02:00
Eelco Dolstra
25a108bb9c Hack to clean up tests/test-tmp 2013-08-21 12:53:46 +02:00
Eelco Dolstra
d308aeaf53 Store Nix integers as longs
So on 64-bit systems, integers are now 64-bit.

Fixes #158.
2013-08-19 12:35:03 +02:00
Eelco Dolstra
297b762513 Turn on -Wall 2013-08-19 11:41:15 +02:00
Eelco Dolstra
46222bbc43 Typo 2013-08-14 22:32:49 +02:00
Shea Levy
58204a3c39 corepkgs/nar.nix: Prefer local builds
nar.nix's builder depends on coreutils and nix itself being in $PATH.
Unfortunately, there's no good way to ensure that these packages exist
in the same place on the remote machine: The local machine may have nix
installed in /usr, and the remote machine in /usr/local, but the
generated nar.sh builder will refer to /usr and thus fail on the remote
machine. This ensures that nar.sh is run on the same machine that
instantiates it.

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-08-14 22:32:41 +02:00