Commit graph

336 commits

Author SHA1 Message Date
Eelco Dolstra
7899fc959d Define HashType's size to allow it to be forward-declared 2016-01-27 17:46:01 +01:00
Eelco Dolstra
739bab0be7 Trivia 2016-01-27 17:18:31 +01:00
Eelco Dolstra
d45ad8fcf5 Make hashLength32() a method of Hash 2016-01-27 17:18:20 +01:00
Eelco Dolstra
5b8c09c124 string2Int: Barf on negative numbers for unsigned types 2016-01-20 16:45:05 +01:00
Eelco Dolstra
9a4706eb19 Fix compilation 2016-01-05 14:05:11 +01:00
Eelco Dolstra
6298afc047 Merge pull request #685 from vizanto/master
POSIX compliant directory access (fixes build on Solaris)
2016-01-05 13:49:55 +01:00
FrankHB
607d91517e Fixed unresolved 'BUFSIZ'
`BUFSIZ` is defined in header `<cstdio>`.
2016-01-05 13:33:40 +01:00
Eelco Dolstra
8f67325a7c Build sandbox support etc. unconditionally on Linux
Also, use "#if __APPLE__" instead of "#if SANDBOX_ENABLED" to prevent
ambiguity.
2015-12-10 11:47:17 +01:00
Danny Wilson
69b9d8fdbd Use DT_UNKNOWN when dirent d_type extension is not supported.
edolstra:
“…since callers of readDirectory have to handle the possibility of
 DT_UNKNOWN anyway, and we don't want to do a stat call for every
 directory entry unless it's really needed.”
2015-11-25 21:57:19 +01:00
Shea Levy
1d3529e93a Default arguments belong at declaration, not definition 2015-11-16 05:55:55 -05:00
Shea Levy
9b4cd20752 Fix copy-paste error 2015-11-16 05:54:34 -05:00
Shea Levy
58d2fac91d AutoDelete: Add default constructor with deletion disabled 2015-11-16 05:53:10 -05:00
Eelco Dolstra
7759a56bed Fix namespace issue 2015-11-09 21:28:25 +01:00
Danny Wilson
cdb346c65e Fix build on Solaris
d_type is not part of the POSIX spec unfortunately.
2015-11-07 04:51:33 +01:00
Eelco Dolstra
6f1743b1a5 Support SHA-512 hashes
Fixes #679.

Note: on x86_64, SHA-512 is considerably faster than SHA-256 (198 MB/s
versus 131 MB/s).
2015-11-04 16:37:49 +01:00
Eelco Dolstra
a6ca68a70c Require OpenSSL 2015-11-04 16:37:49 +01:00
Eelco Dolstra
1f735a3440 <nix/fetchurl.nix>: Support xz-compressed NARs 2015-10-30 12:34:30 +01:00
Eelco Dolstra
5c28943e8f int2String() -> std::to_string() 2015-10-29 13:26:55 +01:00
Jude Taylor
af1218a86a revert libutil change 2015-10-21 14:40:35 -07:00
Jude Taylor
e770f941d6 make sandbox builds more permissive 2015-10-21 12:38:52 -07:00
Eelco Dolstra
bec3c31608 nix-prefetch-url: Rewrite in C++ 2015-10-01 16:47:43 +02:00
Eelco Dolstra
4ba6bc184c Shut up clang warnings 2015-09-18 01:22:35 +02:00
Eelco Dolstra
71a5161365 Implement buildDerivation() via the daemon 2015-09-03 12:56:59 +02:00
Kirill Elagin
3b0f60e5c2 baseNameOf: Enhance basename compatibility
* If the path ends with a slash, drop it.
* If the remaining path doesn’t contain slashes, just return it.

Fixes #574.
2015-08-07 03:35:02 +02:00
Eelco Dolstra
db55940d9e Support systemd log severity prefixes
This is mostly useful for hydra-queue-runner.
2015-07-20 01:39:48 +02:00
Eelco Dolstra
b3491c781c More cleanup 2015-07-20 01:16:16 +02:00
Eelco Dolstra
6bd2c7bb38 OCD: foreach -> C++11 ranged for 2015-07-17 20:13:56 +02:00
Eelco Dolstra
1511aa9f48 Allow remote builds without sending the derivation closure
Previously, to build a derivation remotely, we had to copy the entire
closure of the .drv file to the remote machine, even though we only
need the top-level derivation. This is very wasteful: the closure can
contain thousands of store paths, and in some Hydra use cases, include
source paths that are very large (e.g. Git/Mercurial checkouts).

So now there is a new operation, StoreAPI::buildDerivation(), that
performs a build from an in-memory representation of a derivation
(BasicDerivation) rather than from a on-disk .drv file. The only files
that need to be in the Nix store are the sources of the derivation
(drv.inputSrcs), and the needed output paths of the dependencies (as
described by drv.inputDrvs). "nix-store --serve" exposes this
interface.

Note that this is a privileged operation, because you can construct a
derivation that builds any store path whatsoever. Fixing this will
require changing the hashing scheme (i.e., the output paths should be
computed from the other fields in BasicDerivation, allowing them to be
verified without access to other derivations). However, this would be
quite nice because it would allow .drv-free building (e.g. "nix-env
-i" wouldn't have to write any .drv files to disk).

Fixes #173.
2015-07-17 17:57:40 +02:00
Eelco Dolstra
65f17cd330 Support URLs in $NIX_PATH
This didn't work (despite claims in the manual), because the colon in
"http://" was parsed as a element separator. So handle "://"
specially.
2015-06-17 16:20:11 +02:00
Eelco Dolstra
7c4501886d Use std::vector::data() 2015-06-09 10:54:46 +02:00
Rok Garbas
dad754843a cygwin: looks like stdout/stdin are reserved words 2015-05-13 09:37:56 +02:00
Eelco Dolstra
60340ce3e2 Implement caching of fetchurl/fetchTarball results
ETags are used to prevent redownloading unchanged files.
2015-04-09 11:42:04 +02:00
Eelco Dolstra
a596c525ad Add base64 encoder/decoder 2015-02-10 11:33:33 +01:00
Eelco Dolstra
e0def5bc4b Use libsodium instead of OpenSSL for binary cache signing
Sodium's Ed25519 signatures are much shorter than OpenSSL's RSA
signatures. Public keys are also much shorter, so they're now
specified directly in the nix.conf option ‘binary-cache-public-keys’.

The new command ‘nix-store --generate-binary-cache-key’ generates and
prints a public and secret key.
2015-02-04 17:10:31 +01:00
Eelco Dolstra
0d1dafa0c4 Simplify parseHash32 2015-02-03 18:56:47 +01:00
Eelco Dolstra
db2ec59903 Simplify printHash32 2015-02-03 18:35:24 +01:00
aszlig
8b88d25cda libutil: Limit readLink() error to only overflows.
Let's not just improve the error message itself, but also the behaviour
to actually work around the ntfs-3g symlink bug. If the readlink() call
returns a smaller size than the stat() call, this really isn't a problem
even if the symlink target really has changed between the calls.

So if stat() reports the size for the absolute path, it's most likely
that the relative path is smaller and thus it should also work for file
system bugs as mentioned in 93002d69fc58c2b71e2dfad202139230c630c53a.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
Tested-by: John Ericson <Ericson2314@Yahoo.com>
2015-01-02 12:53:42 +01:00
aszlig
bbd45ac80f libutil: Improve errmsg on readLink size mismatch.
A message like "error: reading symbolic link `...' : Success" really is
quite confusing, so let's not indicate "success" but rather point out
the real issue.

We could also limit the check of this to just check for non-negative
values, but this would introduce a race condition between stat() and
readlink() if the link target changes between those two calls, thus
leading to a buffer overflow vulnerability.

Reported by @Ericson2314 on IRC. Happened due to a possible ntfs-3g bug
where a relative symlink returned the absolute path (st_)size in stat()
while readlink() returned the relative size.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
Tested-by: John Ericson <Ericson2314@Yahoo.com>
2015-01-02 12:53:42 +01:00
Eelco Dolstra
4e0607369e Pedantry 2014-12-14 01:51:14 +01:00
Eelco Dolstra
8bdff8c100 Merge branch 'cygwin-master' of https://github.com/ternaris/nix 2014-12-14 01:49:14 +01:00
Eelco Dolstra
f52b6c944e Fix some memory leaks 2014-12-12 15:01:16 +01:00
Eelco Dolstra
28f22b4653 Ensure we're writing to stderr in the builder
http://hydra.nixos.org/build/17862041
2014-12-12 14:35:44 +01:00
Eelco Dolstra
ad790022fd Doh 2014-12-12 13:41:00 +01:00
Eelco Dolstra
851b47bd7d Don't do vfork in conjunction with setuid 2014-12-10 18:01:01 +01:00
Eelco Dolstra
0e8fc118b3 Use vfork 2014-12-10 17:25:39 +01:00
Eelco Dolstra
ad332e1718 Revert "Use posix_spawn to run the pager"
This reverts commit d34d2b2bbf.
2014-12-10 13:48:50 +01:00
Marko Durkovic
629e8da3aa Explicitly include required C headers 2014-12-09 13:00:59 +01:00
Eelco Dolstra
d34d2b2bbf Use posix_spawn to run the pager
In low memory environments, "nix-env -qa" failed because the fork to
run the pager hit the kernel's overcommit limits. Using posix_spawn
gets around this. (Actually, you have to use posix_spawn with the
undocumented POSIX_SPAWN_USEVFORK flag, otherwise it just uses
fork/exec...)
2014-12-05 20:34:41 +01:00
Eelco Dolstra
976df480c9 Add a primop for regular expression pattern matching
The function ‘builtins.match’ takes a POSIX extended regular
expression and an arbitrary string. It returns ‘null’ if the string
does not match the regular expression. Otherwise, it returns a list
containing substring matches corresponding to parenthesis groups in
the regex. The regex must match the entire string (i.e. there is an
implied "^<pat>$" around the regex).  For example:

  match "foo" "foobar" => null
  match "foo" "foo" => []
  match "f(o+)(.*)" "foooobar" => ["oooo" "bar"]
  match "(.*/)?([^/]*)" "/dir/file.nix" => ["/dir/" "file.nix"]
  match "(.*/)?([^/]*)" "file.nix" => [null "file.nix"]

The following example finds all regular files with extension .nix or
.patch underneath the current directory:

  let

    findFiles = pat: dir: concatLists (mapAttrsToList (name: type:
      if type == "directory" then
        findFiles pat (dir + "/" + name)
      else if type == "regular" && match pat name != null then
        [(dir + "/" + name)]
      else []) (readDir dir));

  in findFiles ".*\\.(nix|patch)" (toString ./.)
2014-11-25 11:47:06 +01:00
Eelco Dolstra
a3e5c99d66 nix-daemon: Call exit(), not _exit()
This was preventing destructors from running. In particular, it was
preventing the deletion of the temproot file for each worker
process. It may also have been responsible for the excessive WAL
growth on Hydra (due to the SQLite database not being closed
properly).

Apparently broken by accident in
8e9140cfde.
2014-11-19 17:09:27 +01:00