restrict-eval does not allow access to git flake inputs

[lheckemann]

Describe the bug
(might be a dup of #7090, not sure)

Steps To Reproduce

nix build --restrict-eval github:lheckemann-dump/fetcher-issues-a/39ad58c862ba4b2a53dbe659a03df4c9a64a464f#test1

Expected behavior

Building just fine. I'm particularly surprised because the test3 and test4 inputs (github: instead of git) are allowed.

Actual behaviour

error: access to URI 'https://github.com/lheckemann-dump/fetcher-issues-b.git' is forbidden in restricted mode
(use '--show-trace' to show detailed location information)

nix-env --version output
nix-env (Nix) 2.12.0pre20220913_2a1c63c (lazy-trees branch)
Also tested on nix-env (Nix) 2.8.1.

flake.nix for reference: https://github.com/lheckemann-dump/fetcher-issues-a/blob/39ad58c862ba4b2a53dbe659a03df4c9a64a464f/flake.nix

[SuperSandro2000]

Do you have allowed-uris set?

[lheckemann]

No, but either I shouldn't need it (because they're locked declared flake inputs) or it should fail in the same way with github: inputs.

[Ma27]

Since I got bitten by NixOS/hydra#1257 as well, I decided to investigate.

First of all a bit of context: flake inputs are fetched using https://github.com/NixOS/nix/blob/2.12.0/src/libexpr/flake/call-flake.nix#L15, i.e. pure Nix code. That means that there's no difference between a builtins.fetchTree {} in my Nix code (that's prohibited by restrict-eval) and a flake input to be fetched.

Now, why are e.g. github/gitlab inputs working fine? The only reason is because they don't have a url-attribute that could be passed to EvalState::checkURI():

$ nix-instantiate --eval -E 'builtins.fetchTree { type = "gitlab"; owner = "Ma27"; repo = "coredump-exporter"; }' --experimental-features 'flakes' --option restrict-eval true
{ lastModified = 1655377483; lastModifiedDate = "20220616110443"; narHash = "sha256-wm1QUoj96UQzLdv98Khp33JQ/97dE28y9GteaOEE6CE="; outPath = "/nix/store/51yprxa1q08jxk16ss1qx3zzvq0if4yk-source"; rev = "d05ce558dd83406af7ff941f4686e5260c2536e8"; shortRev = "d05ce55"; }
$ nix-instantiate --eval -E 'builtins.fetchTree { type = "gitlab"; owner = "Ma27"; repo = "coredump-exporter"; url = "https://gitlab.com"; }' --experimental-features 'flakes' --option restrict-eval true
error: access to URI 'https://gitlab.com' is forbidden in restricted mode
$ nix-instantiate --eval -E 'builtins.fetchTree { type = "gitlab"; owner = "Ma27"; repo = "coredump-exporter"; url = "https://gitlab.com"; }' --experimental-features 'flakes'
error: unsupported input attribute 'url'

As you can see, the gitlab-fetcher doesn't support the URL attribute at all, however it breaks on restrict-eval because access to said url is prohibited. Also, restrict-eval should avoid access to non-allowed URLs in my Nix code which is effectively not the case for github/gitlab inputs fetched with fetchTree. In other words, it's sheer luck apparently that github/gitlab fetchers are working fine. Not sure how to go from here, my workaround for now will be check if allowed-uris works with my Hydra or I'll use Linus's Hydra workaround.

cc @edolstra @thufschmitt for opinions.

[thufschmitt]

I'm not familiar at all with restrict-eval, but my understanding is that it's amongst other things a security feature to prevent semi-untrusted code to try to access arbitrary stuff on the evaluator's filesystem or network. That makes it partially orthogonal to flake inputs. So my guess would be that we need to fix the git{hub,lab} input types to respect these.

Now, why are e.g. github/gitlab inputs working fine? The only reason is because they don't have a url-attribute that could be passed to EvalState::checkURI():

Indeed. Which is something of an issue since we can change their host to anything. We probably want to fix that

[puffnfresh]

The behaviour has changed. It is at least consistent now:

$ nix build --restrict-eval github:lheckemann-dump/fetcher-issues-a/39ad58c862ba4b2a53dbe659a03df4c9a64a464f#test1
error: access to URI 'github:nixos/nixpkgs/72783a2d0dbbf030bff1537873dd5b85b3fb332f' is forbidden in restricted mode

But this issue should probably be closed given #8295.

[onsails]

After upgrading to 23.11 am getting error:

access to URI 'github:hercules-ci/flake-parts/34fed993f1674c8d06d58b37ce1e0fe5eebcb9f5' is forbidden in restricted mod

for inputs like github:username/repo in hydra even with allowed-uris: github: or allowed-uris: https://github.com or allowed-uris: http:// https://. What am I doing wrong?

[onsails]

Looked at nix source, apparently each prefix in allowed-uris should end with /, so I ended up with listing all flake inputs like this:

allowed-uris = https:// github:NixOS/ github:nixos/ github:hercules-ci/ github:numtide/ github:cachix/ github:nix-community/

[MalteT]

The trailing slash requirements makes it really difficult to use restricted eval with multiple flakes and their inputs. The nix manual sounds like one should be able to simply specify all allowed schemes, i.e. https: github: 🤔

Since I'm the only user of my hydra instance and I've only worked with flakes so far, I'm using this patch until there's a better solution:

diff --git a/src/hydra-eval-jobs/hydra-eval-jobs.cc b/src/hydra-eval-jobs/hydra-eval-jobs.cc
index 2794cc62..bd6416e9 100644
--- a/src/hydra-eval-jobs/hydra-eval-jobs.cc
+++ b/src/hydra-eval-jobs/hydra-eval-jobs.cc
@@ -327,7 +327,7 @@ int main(int argc, char * * argv)
 
         /* Prevent access to paths outside of the Nix search path and
            to the environment. */
-        evalSettings.restrictEval = true;
+        evalSettings.restrictEval = config->getBoolOption("evaluator_restrict_eval", true);
 
         /* When building a flake, use pure evaluation (no access to
            'getEnv', 'currentSystem' etc. */

And inside hydra.conf:

evaluator_restrict_eval = false

In my NixOS configuration:

services.hydra = {
  // ...
  package = pkgs.hydra.overrideAttrs (old: {
    patches = (if old ? patches then old.patches else []) ++ [
      ./hydra.patch // With the above content
    ];
  });
  extraConfig = ''
    evaluator_restrict_eval = false
  '';
};

Hope this helps someone :)

[Ma27]

Building just fine. I'm particularly surprised because the test3 and test4 inputs (github: instead of git) are allowed.

I think that this isn't the case anymore, so the original issue is probably resolved: github: isn't blindly allowed whereas e.g. other URLs are rejected. @NixOS/nix-team

[Ericson2314]

Yeah I don't know of any trailing slash restriction; I think as of 2.19 (not 2.19.0 though) at least, github: is a valid entry and just works.

[MalteT]

Well, it really seems to be fixed on all tags newer than 2.19.2 (which I'm sitting at). I tried to bisect this to find the fix, but I wasn't able to. But updating should fix my issue, thanks for the feedback!