nix-super/mk/tests.mk

# Run program $1 as part of ‘make installcheck’.
define run-install-test

  installcheck: $1

  _installcheck-list += $1

endef

# Color code from https://unix.stackexchange.com/a/10065
installcheck:
	@total=0; failed=0; \
	red=""; \
	green=""; \
	yellow=""; \
	normal=""; \
	if [ -t 1 ]; then \
		red="[31;1m"; \
		green="[32;1m"; \
		yellow="[33;1m"; \
		normal="[m"; \
	fi; \
	for i in $(_installcheck-list); do \
	  total=$$((total + 1)); \
	  printf "running test $$i..."; \
	  log="$$(cd $$(dirname $$i) && $(tests-environment) $$(basename $$i) 2>&1)"; \
	  status=$$?; \
	  if [ $$status -eq 0 ]; then \
	    echo " [$${green}PASS$$normal]"; \
	  elif [ $$status -eq 99 ]; then \
	    echo " [$${yellow}SKIP$$normal]"; \
	  else \
	    echo " [$${red}FAIL$$normal]"; \
	    echo "$$log" | sed 's/^/    /'; \
	    failed=$$((failed + 1)); \
	  fi; \
	done; \
	if [ "$$failed" != 0 ]; then \
	  echo "$${red}$$failed out of $$total tests failed $$normal"; \
	  exit 1; \
	else \
		echo "$${green}All tests succeeded$$normal"; \
	fi

.PHONY: check installcheck
-												Revert "Get rid of unicode quotes (#1140)"

This reverts commit f78126bfd6b6c8477fcdbc09b2f98772dbe9a1e7. There
really is no need for such a massive change...

											
										
										
											2016-11-26 01:37:43 +02:00
+								# Run program $1 as part of ‘make installcheck’.
-												GNU Make 3.81 compatibility

3.81 doesn't understand the ‘define foo =’ syntax, which was added in
3.82.  So use ‘define foo’ instead.

											
										
										
											2014-02-04 12:02:49 +02:00
+								define run-install-test
-												Initial commit (imported from the Nix repo)

											
										
										
											2013-12-10 16:54:34 +02:00
 								  installcheck: $1
-												Make variable names more regular

											
										
										
											2014-02-01 13:20:06 +02:00
+								  _installcheck-list += $1
-												Initial commit (imported from the Nix repo)

											
										
										
											2013-12-10 16:54:34 +02:00
 								endef
-												More elegant test output

I got sick of trying to find the failures in the sea of debug output, so
we now:
- Hide test output unless it fails
- Sprinkle in some simple color
- Pad results for a more tabular look

If Nix is getting a more friendly user interface, we might as well get
a friendlier developer interface, right? :)
											
										
										
											2017-10-03 07:54:00 +03:00
+								# Color code from https://unix.stackexchange.com/a/10065
-												installcheck: Don't depend on install

This is a hack to prevent "installcheck" from clobbering files fixed
up by Nixpkgs' fixupPhase.

											
										
										
											2014-02-01 16:33:27 +02:00
+								installcheck:
-												More elegant test output

I got sick of trying to find the failures in the sea of debug output, so
we now:
- Hide test output unless it fails
- Sprinkle in some simple color
- Pad results for a more tabular look

If Nix is getting a more friendly user interface, we might as well get
a friendlier developer interface, right? :)
											
										
										
											2017-10-03 07:54:00 +03:00
+									@total=0; failed=0; \
 									red=""; \
 									green=""; \
-												Show when tests are skipped

Also, don't depend on tput (ncurses). It's really not needed since
ANSI escape sequences have been standardized for 35 years or so.

											
										
										
											2017-11-07 13:06:58 +02:00
+									yellow=""; \
-												More elegant test output

I got sick of trying to find the failures in the sea of debug output, so
we now:
- Hide test output unless it fails
- Sprinkle in some simple color
- Pad results for a more tabular look

If Nix is getting a more friendly user interface, we might as well get
a friendlier developer interface, right? :)
											
										
										
											2017-10-03 07:54:00 +03:00
+									normal=""; \
 									if [ -t 1 ]; then \
-												Show when tests are skipped

Also, don't depend on tput (ncurses). It's really not needed since
ANSI escape sequences have been standardized for 35 years or so.

											
										
										
											2017-11-07 13:06:58 +02:00
+										red="[31;1m"; \
 										green="[32;1m"; \
 										yellow="[33;1m"; \
 										normal="[m"; \
-												More elegant test output

I got sick of trying to find the failures in the sea of debug output, so
we now:
- Hide test output unless it fails
- Sprinkle in some simple color
- Pad results for a more tabular look

If Nix is getting a more friendly user interface, we might as well get
a friendlier developer interface, right? :)
											
										
										
											2017-10-03 07:54:00 +03:00
+									fi; \
 									for i in $(_installcheck-list); do \
-												Initial commit (imported from the Nix repo)

											
										
										
											2013-12-10 16:54:34 +02:00
+									  total=$$((total + 1)); \
-												fixing bashisms in test code

This fixed the build on ubuntu/debian, where dash is the sh.

											
										
										
											2017-10-06 14:04:12 +03:00
+									  printf "running test $$i..."; \
-												More elegant test output

I got sick of trying to find the failures in the sea of debug output, so
we now:
- Hide test output unless it fails
- Sprinkle in some simple color
- Pad results for a more tabular look

If Nix is getting a more friendly user interface, we might as well get
a friendlier developer interface, right? :)
											
										
										
											2017-10-03 07:54:00 +03:00
+									  log="$$(cd $$(dirname $$i) && $(tests-environment) $$(basename $$i) 2>&1)"; \
-												Show when tests are skipped

Also, don't depend on tput (ncurses). It's really not needed since
ANSI escape sequences have been standardized for 35 years or so.

											
										
										
											2017-11-07 13:06:58 +02:00
+									  status=$$?; \
 									  if [ $$status -eq 0 ]; then \
-												Whitespace

											
										
										
											2017-10-09 16:03:15 +03:00
+									    echo " [$${green}PASS$$normal]"; \
-												Show when tests are skipped

Also, don't depend on tput (ncurses). It's really not needed since
ANSI escape sequences have been standardized for 35 years or so.

											
										
										
											2017-11-07 13:06:58 +02:00
+									  elif [ $$status -eq 99 ]; then \
 									    echo " [$${yellow}SKIP$$normal]"; \
-												Initial commit (imported from the Nix repo)

											
										
										
											2013-12-10 16:54:34 +02:00
+									  else \
-												Whitespace

											
										
										
											2017-10-09 16:03:15 +03:00
+									    echo " [$${red}FAIL$$normal]"; \
-												More elegant test output

I got sick of trying to find the failures in the sea of debug output, so
we now:
- Hide test output unless it fails
- Sprinkle in some simple color
- Pad results for a more tabular look

If Nix is getting a more friendly user interface, we might as well get
a friendlier developer interface, right? :)
											
										
										
											2017-10-03 07:54:00 +03:00
+									    echo "$$log" | sed 's/^/    /'; \
-												Initial commit (imported from the Nix repo)

											
										
										
											2013-12-10 16:54:34 +02:00
+									    failed=$$((failed + 1)); \
 									  fi; \
 									done; \
 									if [ "$$failed" != 0 ]; then \
-												More elegant test output

I got sick of trying to find the failures in the sea of debug output, so
we now:
- Hide test output unless it fails
- Sprinkle in some simple color
- Pad results for a more tabular look

If Nix is getting a more friendly user interface, we might as well get
a friendlier developer interface, right? :)
											
										
										
											2017-10-03 07:54:00 +03:00
+									  echo "$${red}$$failed out of $$total tests failed $$normal"; \
-												Initial commit (imported from the Nix repo)

											
										
										
											2013-12-10 16:54:34 +02:00
+									  exit 1; \
-												More elegant test output

I got sick of trying to find the failures in the sea of debug output, so
we now:
- Hide test output unless it fails
- Sprinkle in some simple color
- Pad results for a more tabular look

If Nix is getting a more friendly user interface, we might as well get
a friendlier developer interface, right? :)
											
										
										
											2017-10-03 07:54:00 +03:00
+									else \
-												Add pure evaluation mode

In this mode, the following restrictions apply:

* The builtins currentTime, currentSystem and storePath throw an
  error.

* $NIX_PATH and -I are ignored.

* fetchGit and fetchMercurial require a revision hash.

* fetchurl and fetchTarball require a sha256 attribute.

* No file system access is allowed outside of the paths returned by
  fetch{Git,Mercurial,url,Tarball}. Thus 'nix build -f ./foo.nix' is
  not allowed.

Thus, the evaluation result is completely reproducible from the
command line arguments. E.g.

  nix build --pure-eval '(
    let
      nix = fetchGit { url = https://github.com/NixOS/nixpkgs.git; rev = "9c927de4b179a6dd210dd88d34bda8af4b575680"; };
      nixpkgs = fetchGit { url = https://github.com/NixOS/nixpkgs.git; ref = "release-17.09"; rev = "66b4de79e3841530e6d9c6baf98702aa1f7124e4"; };
    in (import (nix + "/release.nix") { inherit nix nixpkgs; }).build.x86_64-linux
  )'

The goal is to enable completely reproducible and traceable
evaluation. For example, a NixOS configuration could be fully
described by a single Git commit hash. 'nixos-rebuild' would do
something like

  nix build --pure-eval '(
    (import (fetchGit { url = file:///my-nixos-config; rev = "..."; })).system
  ')

where the Git repository /my-nixos-config would use further fetchGit
calls or Git externals to fetch Nixpkgs and whatever other
dependencies it has. Either way, the commit hash would uniquely
identify the NixOS configuration and allow it to reproduced.

											
										
										
											2018-01-16 19:50:38 +02:00
+										echo "$${green}All tests succeeded$$normal"; \
-												Initial commit (imported from the Nix repo)

											
										
										
											2013-12-10 16:54:34 +02:00
+									fi
 								.PHONY: check installcheck