max/nix-super
1
0
Fork 0
mirror of https://github.com/privatevoid-net/nix-super.git synced 2024-11-11 00:36:20 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 1
4496 commits 7 branches 0 tags 64 MiB
Commit graph

23 commits

Author SHA1 Message Date
Eelco Dolstra
581e1bc5b4 FdSource: track number of bytes read 2016-02-26 16:16:08 +01:00
Eelco Dolstra
5f862658c3 Remove bad daemon connections from the pool 
This is necessary for long-running processes like hydra-queue-runner:
if a nix-daemon worker is killed, we need to stop reusing that
connection.
 2016-02-24 11:39:56 +01:00
Eelco Dolstra
71a5161365 Implement buildDerivation() via the daemon 2015-09-03 12:56:59 +02:00
Eelco Dolstra
b3491c781c More cleanup 2015-07-20 01:16:16 +02:00
Eelco Dolstra
6bd2c7bb38 OCD: foreach -> C++11 ranged for 2015-07-17 20:13:56 +02:00
Eelco Dolstra
1511aa9f48 Allow remote builds without sending the derivation closure 
Previously, to build a derivation remotely, we had to copy the entire
closure of the .drv file to the remote machine, even though we only
need the top-level derivation. This is very wasteful: the closure can
contain thousands of store paths, and in some Hydra use cases, include
source paths that are very large (e.g. Git/Mercurial checkouts).

So now there is a new operation, StoreAPI::buildDerivation(), that
performs a build from an in-memory representation of a derivation
(BasicDerivation) rather than from a on-disk .drv file. The only files
that need to be in the Nix store are the sources of the derivation
(drv.inputSrcs), and the needed output paths of the dependencies (as
described by drv.inputDrvs). "nix-store --serve" exposes this
interface.

Note that this is a privileged operation, because you can construct a
derivation that builds any store path whatsoever. Fixing this will
require changing the hashing scheme (i.e., the output paths should be
computed from the other fields in BasicDerivation, allowing them to be
verified without access to other derivations). However, this would be
quite nice because it would allow .drv-free building (e.g. "nix-env
-i" wouldn't have to write any .drv files to disk).

Fixes #173.
 2015-07-17 17:57:40 +02:00
Eelco Dolstra
829af22759 Print a warning when loading a large path into memory 
I.e. if you have a derivation with

  src = ./huge-directory;

you'll get a warning that this is not a good idea.
 2014-06-10 13:30:09 +02:00
Eelco Dolstra
fb8d8f5428 Remove unnecessary null pointer checks 
Fixes #225.
 2014-03-12 14:42:25 +01:00
Eelco Dolstra
5959c591a0 Process stderr from substituters while doing have/info queries 2013-06-07 15:02:14 +02:00
Eelco Dolstra
d5a5a83ad4 Use data() instead of c_str() where appropriate 2012-02-09 18:27:45 +01:00
Eelco Dolstra
273b288a7e * importPath() -> importPaths(). Because of buffering of the input 
stream it's now necessary for the daemon to process the entire
  sequence of exported paths, rather than letting the client do it.
 2011-12-16 22:31:25 +00:00
Eelco Dolstra
8d3dfa2c17 * Avoid expensive conversions from char arrays to STL strings. 2011-12-16 21:29:46 +00:00
Eelco Dolstra
e0bd307802 * Make the import operation through the daemon much more efficient 
(way fewer roundtrips) by allowing the client to send data in bigger
  chunks.
* Some refactoring.
 2011-12-16 19:44:13 +00:00
Eelco Dolstra
78598d06f0 * Clean up exception handling. 2011-12-16 15:45:42 +00:00
Eelco Dolstra
5a1b9ed0aa * Refactoring: move sink/source buffering into separate classes. 
* Buffer the HashSink.  This speeds up hashing a bit because it
  prevents lots of calls to the hash update functions (e.g. nix-hash
  went from 9.3s to 8.7s of user time on the closure of my
  /var/run/current-system).
 2011-12-15 16:19:53 +00:00
Eelco Dolstra
a3e0656cbb * Buffer reads in FdSource. Together with write buffering, this 
significantly cuts down the number of syscalls (e.g., for "nix-store
  -qR /var/run/current-system" via the daemon, it reduced the number
  of syscalls in the client from 29134 to 4766 and in the daemon from
  44266 to 20666).
 2011-12-15 12:32:08 +00:00
Eelco Dolstra
3a48282b06 * Buffer writes in FdSink. This significantly reduces the number of 
system calls / context switches when dumping a NAR and in the worker
  protocol.
 2011-12-14 23:30:06 +00:00
Eelco Dolstra
77d272623f * NAR archives: handle files larger than 2^32 bytes. Previously it 
would just silently store only (fileSize % 2^32) bytes.
* Use posix_fallocate if available when unpacking archives.
* Provide a better error message when trying to unpack something that
  isn't a NAR archive.
 2009-03-22 17:36:43 +00:00
Eelco Dolstra
a72709afd8 * Some refactoring: put the GC options / results in separate structs. 
* The garbage collector now also prints the number of blocks freed.
 2008-06-18 09:34:17 +00:00
Eelco Dolstra
bd955e15e1 * GCC 4.3.0 (Fedora 9) compatibility fixes. Reported by Gour and 
Armijn Hemel.
 2008-05-21 11:17:31 +00:00
Eelco Dolstra
0130ef88ea * Daemon mode (`nix-worker --daemon'). Clients connect to the server 
via the Unix domain socket in /nix/var/nix/daemon.socket.  The
  server forks a worker process per connection.
* readString(): use the heap, not the stack.
* Some protocol fixes.
 2006-12-04 17:17:13 +00:00
Eelco Dolstra
0565b5f2b3 * More remote operations. 
* Added new operation hasSubstitutes(), which is more efficient than
  querySubstitutes().size() > 0.
 2006-11-30 22:43:55 +00:00
Eelco Dolstra
40b3f64b55 * Skeleton of the privileged worker program. 
* Some refactoring: put the NAR archive integer/string serialisation
  code in a separate file so it can be reused by the worker protocol
  implementation.
 2006-11-30 19:19:59 +00:00