Merge remote-tracking branch 'origin/master' into lazy-trees

doc/manual/src/SUMMARY.md.in

@@ -33,8 +33,8 @@
     - [Arguments and Variables](expressions/arguments-variables.md)
     - [Building and Testing](expressions/simple-building-testing.md)
     - [Generic Builder Syntax](expressions/generic-builder.md)
-  - [Writing Nix Expressions](expressions/expression-language.md)
+  - [Nix Expression Language](expressions/expression-language.md)
-    - [Values](expressions/language-values.md)
+    - [Data Types](expressions/language-values.md)
     - [Language Constructs](expressions/language-constructs.md)
     - [Operators](expressions/language-operators.md)
     - [Derivations](expressions/derivations.md)

doc/manual/src/advanced-topics/distributed-builds.md

@@ -12,14 +12,14 @@ machine is accessible via SSH and that it has Nix installed. You can
 test whether connecting to the remote Nix instance works, e.g.
 
 ```console
-$ nix ping-store --store ssh://mac
+$ nix store ping --store ssh://mac
 ```
 
 will try to connect to the machine named `mac`. It is possible to
 specify an SSH identity file as part of the remote store URI, e.g.
 
 ```console
-$ nix ping-store --store ssh://mac?ssh-key=/home/alice/my-key
+$ nix store ping --store ssh://mac?ssh-key=/home/alice/my-key
 ```
 
 Since builds should be non-interactive, the key should not have a

doc/manual/src/expressions/language-values.md

@@ -1,156 +1,164 @@
-# Values
+# Data Types
 
-## Simple Values
+## Primitives
 
-Nix has the following basic data types:
+- <a id="type-string" href="#type-string">String</a>
 
-  - *Strings* can be written in three ways.
+  *Strings* can be written in three ways.
-    
-    The most common way is to enclose the string between double quotes,
-    e.g., `"foo bar"`. Strings can span multiple lines. The special
-    characters `"` and `\` and the character sequence `${` must be
-    escaped by prefixing them with a backslash (`\`). Newlines, carriage
-    returns and tabs can be written as `\n`, `\r` and `\t`,
-    respectively.
-    
-    You can include the result of an expression into a string by
-    enclosing it in `${...}`, a feature known as *antiquotation*. The
-    enclosed expression must evaluate to something that can be coerced
-    into a string (meaning that it must be a string, a path, or a
-    derivation). For instance, rather than writing
-    
-    ```nix
-    "--with-freetype2-library=" + freetype + "/lib"
-    ```
-    
-    (where `freetype` is a derivation), you can instead write the more
-    natural
-    
-    ```nix
-    "--with-freetype2-library=${freetype}/lib"
-    ```
-    
-    The latter is automatically translated to the former. A more
-    complicated example (from the Nix expression for
-    [Qt](http://www.trolltech.com/products/qt)):
-    
-    ```nix
-    configureFlags = "
-      -system-zlib -system-libpng -system-libjpeg
-      ${if openglSupport then "-dlopen-opengl
-        -L${mesa}/lib -I${mesa}/include
-        -L${libXmu}/lib -I${libXmu}/include" else ""}
-      ${if threadSupport then "-thread" else "-no-thread"}
-    ";
-    ```
-    
-    Note that Nix expressions and strings can be arbitrarily nested; in
-    this case the outer string contains various antiquotations that
-    themselves contain strings (e.g., `"-thread"`), some of which in
-    turn contain expressions (e.g., `${mesa}`).
-    
-    The second way to write string literals is as an *indented string*,
-    which is enclosed between pairs of *double single-quotes*, like so:
-    
-    ```nix
-    ''
-      This is the first line.
-      This is the second line.
-        This is the third line.
-    ''
-    ```
-    
-    This kind of string literal intelligently strips indentation from
-    the start of each line. To be precise, it strips from each line a
-    number of spaces equal to the minimal indentation of the string as a
-    whole (disregarding the indentation of empty lines). For instance,
-    the first and second line are indented two spaces, while the third
-    line is indented four spaces. Thus, two spaces are stripped from
-    each line, so the resulting string is
-    
-    ```nix
-    "This is the first line.\nThis is the second line.\n  This is the third line.\n"
-    ```
-    
-    Note that the whitespace and newline following the opening `''` is
-    ignored if there is no non-whitespace text on the initial line.
-    
-    Antiquotation (`${expr}`) is supported in indented strings.
-    
-    Since `${` and `''` have special meaning in indented strings, you
-    need a way to quote them. `$` can be escaped by prefixing it with
-    `''` (that is, two single quotes), i.e., `''$`. `''` can be escaped
-    by prefixing it with `'`, i.e., `'''`. `$` removes any special
-    meaning from the following `$`. Linefeed, carriage-return and tab
-    characters can be written as `''\n`, `''\r`, `''\t`, and `''\`
-    escapes any other character.
-    
-    Indented strings are primarily useful in that they allow multi-line
-    string literals to follow the indentation of the enclosing Nix
-    expression, and that less escaping is typically necessary for
-    strings representing languages such as shell scripts and
-    configuration files because `''` is much less common than `"`.
-    Example:
-    
-    ```nix
-    stdenv.mkDerivation {
-      ...
-      postInstall =
-        ''
-          mkdir $out/bin $out/etc
-          cp foo $out/bin
-          echo "Hello World" > $out/etc/foo.conf
-          ${if enableBar then "cp bar $out/bin" else ""}
-        '';
-      ...
-    }
-    ```
-    
-    Finally, as a convenience, *URIs* as defined in appendix B of
-    [RFC 2396](http://www.ietf.org/rfc/rfc2396.txt) can be written *as
-    is*, without quotes. For instance, the string
-    `"http://example.org/foo.tar.bz2"` can also be written as
-    `http://example.org/foo.tar.bz2`.
 
-  - Numbers, which can be *integers* (like `123`) or *floating point*
+  The most common way is to enclose the string between double quotes,
-    (like `123.43` or `.27e13`).
+  e.g., `"foo bar"`. Strings can span multiple lines. The special
-    
+  characters `"` and `\` and the character sequence `${` must be
-    Numbers are type-compatible: pure integer operations will always
+  escaped by prefixing them with a backslash (`\`). Newlines, carriage
-    return integers, whereas any operation involving at least one
+  returns and tabs can be written as `\n`, `\r` and `\t`,
-    floating point number will have a floating point number as a result.
+  respectively.
 
-  - *Paths*, e.g., `/bin/sh` or `./builder.sh`. A path must contain at
+  You can include the result of an expression into a string by
-    least one slash to be recognised as such. For instance, `builder.sh`
+  enclosing it in `${...}`, a feature known as *antiquotation*. The
-    is not a path: it's parsed as an expression that selects the
+  enclosed expression must evaluate to something that can be coerced
-    attribute `sh` from the variable `builder`. If the file name is
+  into a string (meaning that it must be a string, a path, or a
-    relative, i.e., if it does not begin with a slash, it is made
+  derivation). For instance, rather than writing
-    absolute at parse time relative to the directory of the Nix
-    expression that contained it. For instance, if a Nix expression in
-    `/foo/bar/bla.nix` refers to `../xyzzy/fnord.nix`, the absolute path
-    is `/foo/xyzzy/fnord.nix`.
-    
-    If the first component of a path is a `~`, it is interpreted as if
-    the rest of the path were relative to the user's home directory.
-    e.g. `~/foo` would be equivalent to `/home/edolstra/foo` for a user
-    whose home directory is `/home/edolstra`.
-    
-    Paths can also be specified between angle brackets, e.g.
-    `<nixpkgs>`. This means that the directories listed in the
-    environment variable `NIX_PATH` will be searched for the given file
-    or directory name.
 
-    Antiquotation is supported in any paths except those in angle brackets.
+  ```nix
-    `./${foo}-${bar}.nix` is a more convenient way of writing 
+  "--with-freetype2-library=" + freetype + "/lib"
-    `./. + "/" + foo + "-" + bar + ".nix"` or `./. + "/${foo}-${bar}.nix"`. At
+  ```
-    least one slash must appear *before* any antiquotations for this to be
-    recognized as a path. `a.${foo}/b.${bar}` is a syntactically valid division
-    operation. `./a.${foo}/b.${bar}` is a path.
 
-  - *Booleans* with values `true` and `false`.
+  (where `freetype` is a derivation), you can instead write the more
+  natural
 
-  - The null value, denoted as `null`.
+  ```nix
+  "--with-freetype2-library=${freetype}/lib"
+  ```
 
-## Lists
+  The latter is automatically translated to the former. A more
+  complicated example (from the Nix expression for
+  [Qt](http://www.trolltech.com/products/qt)):
+
+  ```nix
+  configureFlags = "
+    -system-zlib -system-libpng -system-libjpeg
+    ${if openglSupport then "-dlopen-opengl
+      -L${mesa}/lib -I${mesa}/include
+      -L${libXmu}/lib -I${libXmu}/include" else ""}
+    ${if threadSupport then "-thread" else "-no-thread"}
+  ";
+  ```
+
+  Note that Nix expressions and strings can be arbitrarily nested; in
+  this case the outer string contains various antiquotations that
+  themselves contain strings (e.g., `"-thread"`), some of which in
+  turn contain expressions (e.g., `${mesa}`).
+
+  The second way to write string literals is as an *indented string*,
+  which is enclosed between pairs of *double single-quotes*, like so:
+
+  ```nix
+  ''
+    This is the first line.
+    This is the second line.
+      This is the third line.
+  ''
+  ```
+
+  This kind of string literal intelligently strips indentation from
+  the start of each line. To be precise, it strips from each line a
+  number of spaces equal to the minimal indentation of the string as a
+  whole (disregarding the indentation of empty lines). For instance,
+  the first and second line are indented two spaces, while the third
+  line is indented four spaces. Thus, two spaces are stripped from
+  each line, so the resulting string is
+
+  ```nix
+  "This is the first line.\nThis is the second line.\n  This is the third line.\n"
+  ```
+
+  Note that the whitespace and newline following the opening `''` is
+  ignored if there is no non-whitespace text on the initial line.
+
+  Antiquotation (`${expr}`) is supported in indented strings.
+
+  Since `${` and `''` have special meaning in indented strings, you
+  need a way to quote them. `$` can be escaped by prefixing it with
+  `''` (that is, two single quotes), i.e., `''$`. `''` can be escaped
+  by prefixing it with `'`, i.e., `'''`. `$` removes any special
+  meaning from the following `$`. Linefeed, carriage-return and tab
+  characters can be written as `''\n`, `''\r`, `''\t`, and `''\`
+  escapes any other character.
+
+  Indented strings are primarily useful in that they allow multi-line
+  string literals to follow the indentation of the enclosing Nix
+  expression, and that less escaping is typically necessary for
+  strings representing languages such as shell scripts and
+  configuration files because `''` is much less common than `"`.
+  Example:
+
+  ```nix
+  stdenv.mkDerivation {
+    ...
+    postInstall =
+      ''
+        mkdir $out/bin $out/etc
+        cp foo $out/bin
+        echo "Hello World" > $out/etc/foo.conf
+        ${if enableBar then "cp bar $out/bin" else ""}
+      '';
+    ...
+  }
+  ```
+
+  Finally, as a convenience, *URIs* as defined in appendix B of
+  [RFC 2396](http://www.ietf.org/rfc/rfc2396.txt) can be written *as
+  is*, without quotes. For instance, the string
+  `"http://example.org/foo.tar.bz2"` can also be written as
+  `http://example.org/foo.tar.bz2`.
+
+- <a id="type-number" href="#type-number">Number</a>
+
+  Numbers, which can be *integers* (like `123`) or *floating point*
+  (like `123.43` or `.27e13`).
+
+  Numbers are type-compatible: pure integer operations will always
+  return integers, whereas any operation involving at least one
+  floating point number will have a floating point number as a result.
+
+- <a id="type-path" href="#type-path">Path</a>
+
+  *Paths*, e.g., `/bin/sh` or `./builder.sh`. A path must contain at
+  least one slash to be recognised as such. For instance, `builder.sh`
+  is not a path: it's parsed as an expression that selects the
+  attribute `sh` from the variable `builder`. If the file name is
+  relative, i.e., if it does not begin with a slash, it is made
+  absolute at parse time relative to the directory of the Nix
+  expression that contained it. For instance, if a Nix expression in
+  `/foo/bar/bla.nix` refers to `../xyzzy/fnord.nix`, the absolute path
+  is `/foo/xyzzy/fnord.nix`.
+
+  If the first component of a path is a `~`, it is interpreted as if
+  the rest of the path were relative to the user's home directory.
+  e.g. `~/foo` would be equivalent to `/home/edolstra/foo` for a user
+  whose home directory is `/home/edolstra`.
+
+  Paths can also be specified between angle brackets, e.g.
+  `<nixpkgs>`. This means that the directories listed in the
+  environment variable `NIX_PATH` will be searched for the given file
+  or directory name.
+
+  Antiquotation is supported in any paths except those in angle brackets.
+  `./${foo}-${bar}.nix` is a more convenient way of writing
+  `./. + "/" + foo + "-" + bar + ".nix"` or `./. + "/${foo}-${bar}.nix"`. At
+  least one slash must appear *before* any antiquotations for this to be
+  recognized as a path. `a.${foo}/b.${bar}` is a syntactically valid division
+  operation. `./a.${foo}/b.${bar}` is a path.
+
+- <a id="type-boolean" href="#type-boolean">Boolean</a>
+
+  *Booleans* with values `true` and `false`.
+
+- <a id="type-null" href="#type-null">Null</a>
+
+  The null value, denoted as `null`.
+
+## List
 
 Lists are formed by enclosing a whitespace-separated list of values
 between square brackets. For example,
@@ -172,25 +180,27 @@ function and the fifth being a set.
 
 Note that lists are only lazy in values, and they are strict in length.
 
-## Sets
+## Attribute Set
 
-Sets are really the core of the language, since ultimately the Nix
+An attribute set is a collection of name-value-pairs (called *attributes*) enclosed in curly brackets (`{ }`).
-language is all about creating derivations, which are really just sets
-of attributes to be passed to build scripts.
 
-Sets are just a list of name/value pairs (called *attributes*) enclosed
+Names and values are separated by an equal sign (`=`).
-in curly brackets, where each value is an arbitrary expression
+Each value is an arbitrary expression terminated by a semicolon (`;`).
-terminated by a semicolon. For example:
+
+Attributes can appear in any order.
+An attribute name may only occur once.
+
+Example:
 
 ```nix
-{ x = 123;
+{
+  x = 123;
   text = "Hello";
   y = f { bla = 456; };
 }
 ```
 
-This defines a set with attributes named `x`, `text`, `y`. The order of
+This defines a set with attributes named `x`, `text`, `y`.
-the attributes is irrelevant. An attribute name may only occur once.
 
 Attributes can be selected from a set using the `.` operator. For
 instance,

docker.nix

@@ -6,6 +6,7 @@
 , channelURL ? "https://nixos.org/channels/nixpkgs-unstable"
 , extraPkgs ? []
 , maxLayers ? 100
+, nixConf ? {}
 }:
 let
   defaultPkgs = with pkgs; [
@@ -123,12 +124,17 @@ let
       (lib.attrValues (lib.mapAttrs groupToGroup groups))
   );
 
-  nixConf = {
+  defaultNixConf = {
     sandbox = "false";
     build-users-group = "nixbld";
-    trusted-public-keys = "cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=";
+    trusted-public-keys = [ "cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=" ];
   };
-  nixConfContents = (lib.concatStringsSep "\n" (lib.mapAttrsFlatten (n: v: "${n} = ${v}") nixConf)) + "\n";
+
+  nixConfContents = (lib.concatStringsSep "\n" (lib.mapAttrsFlatten (n: v:
+    let
+      vStr = if builtins.isList v then lib.concatStringsSep " " v else v;
+    in
+      "${n} = ${vStr}") (defaultNixConf // nixConf))) + "\n";
 
   baseSystem =
     let

src/libexpr/eval.cc

@@ -2476,19 +2476,19 @@ void EvalState::printStats()
             }
             {
                 auto list = topObj.list("functions");
-                for (auto & i : functionCalls) {
+                for (auto & [fun, count] : functionCalls) {
                     auto obj = list.object();
-                    if (i.first->name)
+                    if (fun->name)
-                        obj.attr("name", (const std::string &) i.first->name);
+                        obj.attr("name", (std::string_view) symbols[fun->name]);
                     else
                         obj.attr("name", nullptr);
-                    if (auto pos = positions[i.first->pos]) {
+                    if (auto pos = positions[fun->pos]) {
-                        // FIXME
+                        if (auto path = std::get_if<SourcePath>(&pos.origin))
-                        //obj.attr("file", (const std::string &) pos.file);
+                            obj.attr("file", path->to_string());
                         obj.attr("line", pos.line);
                         obj.attr("column", pos.column);
                     }
-                    obj.attr("count", i.second);
+                    obj.attr("count", count);
                 }
             }
             {
@@ -2496,8 +2496,8 @@ void EvalState::printStats()
                 for (auto & i : attrSelects) {
                     auto obj = list.object();
                     if (auto pos = positions[i.first]) {
-                        // FIXME
+                        if (auto path = std::get_if<SourcePath>(&pos.origin))
-                        //obj.attr("file", (const std::string &) pos.file);
+                            obj.attr("file", path->to_string());
                         obj.attr("line", pos.line);
                         obj.attr("column", pos.column);
                     }

src/libutil/json.cc

@@ -6,7 +6,8 @@
 
 namespace nix {
 
-void toJSON(std::ostream & str, const char * start, const char * end)
+template<>
+void toJSON<std::string_view>(std::ostream & str, const std::string_view & s)
 {
     constexpr size_t BUF_SIZE = 4096;
     char buf[BUF_SIZE + 7]; // BUF_SIZE + largest single sequence of puts
@@ -21,7 +22,7 @@ void toJSON(std::ostream & str, const char * start, const char * end)
     };
 
     put('"');
-    for (auto i = start; i != end; i++) {
+    for (auto i = s.begin(); i != s.end(); i++) {
         if (bufPos >= BUF_SIZE) flush();
         if (*i == '\"' || *i == '\\') { put('\\'); put(*i); }
         else if (*i == '\n') { put('\\'); put('n'); }
@@ -44,7 +45,7 @@ void toJSON(std::ostream & str, const char * start, const char * end)
 
 void toJSON(std::ostream & str, const char * s)
 {
-    if (!s) str << "null"; else toJSON(str, s, s + strlen(s));
+    if (!s) str << "null"; else toJSON(str, std::string_view(s));
 }
 
 template<> void toJSON<int>(std::ostream & str, const int & n) { str << n; }
@@ -55,11 +56,7 @@ template<> void toJSON<long long>(std::ostream & str, const long long & n) { str
 template<> void toJSON<unsigned long long>(std::ostream & str, const unsigned long long & n) { str << n; }
 template<> void toJSON<float>(std::ostream & str, const float & n) { str << n; }
 template<> void toJSON<double>(std::ostream & str, const double & n) { str << n; }
-
+template<> void toJSON<std::string>(std::ostream & str, const std::string & s) { toJSON(str, (std::string_view) s); }
-template<> void toJSON<std::string>(std::ostream & str, const std::string & s)
-{
-    toJSON(str, s.c_str(), s.c_str() + s.size());
-}
 
 template<> void toJSON<bool>(std::ostream & str, const bool & b)
 {
@@ -154,7 +151,7 @@ JSONObject::~JSONObject()
     }
 }
 
-void JSONObject::attr(const std::string & s)
+void JSONObject::attr(std::string_view s)
 {
     comma();
     toJSON(state->str, s);
@@ -162,19 +159,19 @@ void JSONObject::attr(const std::string & s)
     if (state->indent) state->str << ' ';
 }
 
-JSONList JSONObject::list(const std::string & name)
+JSONList JSONObject::list(std::string_view name)
 {
     attr(name);
     return JSONList(state);
 }
 
-JSONObject JSONObject::object(const std::string & name)
+JSONObject JSONObject::object(std::string_view name)
 {
     attr(name);
     return JSONObject(state);
 }
 
-JSONPlaceholder JSONObject::placeholder(const std::string & name)
+JSONPlaceholder JSONObject::placeholder(std::string_view name)
 {
     attr(name);
     return JSONPlaceholder(state);

src/libutil/json.hh

@@ -6,7 +6,6 @@
 
 namespace nix {
 
-void toJSON(std::ostream & str, const char * start, const char * end);
 void toJSON(std::ostream & str, const char * s);
 
 template<typename T>
@@ -107,7 +106,7 @@ private:
         open();
     }
 
-    void attr(const std::string & s);
+    void attr(std::string_view s);
 
 public:
 
@@ -128,18 +127,18 @@ public:
     ~JSONObject();
 
     template<typename T>
-    JSONObject & attr(const std::string & name, const T & v)
+    JSONObject & attr(std::string_view name, const T & v)
     {
         attr(name);
         toJSON(state->str, v);
         return *this;
     }
 
-    JSONList list(const std::string & name);
+    JSONList list(std::string_view name);
 
-    JSONObject object(const std::string & name);
+    JSONObject object(std::string_view name);
 
-    JSONPlaceholder placeholder(const std::string & name);
+    JSONPlaceholder placeholder(std::string_view name);
 };
 
 class JSONPlaceholder : JSONWriter

src/libutil/tests/json.cc

@@ -102,8 +102,8 @@ namespace nix {
 
     TEST(toJSON, substringEscape) {
         std::stringstream out;
-        const char *s = "foo\t";
+        std::string_view s = "foo\t";
-        toJSON(out, s+3, s + strlen(s));
+        toJSON(out, s.substr(3));
 
         ASSERT_EQ(out.str(), "\"\\t\"");
     }