NixOS channels: copy also PXE boot files

[domenkozar]

Once NixOS/nixpkgs#14740 is merged, it would be nice to also copy nix-build -A netboot nixos/release.nix result to the channel so NixOS can be booted using (i)PXE.

Note that once this is implemented, netboot should be part of the tested job.

[domenkozar]

cc @edolstra

[bobvanderlinden]

Just for some arguments why to merge something like this:

• This is needed for letting people boot through iPXE without them needing to build and host the PXE files themselves.
• This will also allow booting through https://netboot.xyz/, which is a convenient menu to boot various distro images directly over the internet.
• Lastly this will allow people using various VPS providers to more easily install NixOS. These services often allow some way to boot iPXE, but do not allow you to upload ISOs. It could be an alternative in some cases for nixos-assimilate and nixos-in-place.

[bobvanderlinden]

I think this can be added using downloadFile("nixos.netboot.x86_64-linux") here:

nixos-channel-scripts/mirror-nixos-branch.pl

Line 118 in 852e1b6

downloadFile("nixos.iso_minimal.x86_64-linux");

However, hydra doesn't output any files, as can be seen here: https://hydra.nixos.org/build/35046788

I'm not up to snuff with Hydra, so what should be done to get netboot on http://nixos.org/releases/?

[nshalman]

Part of the problem may be that unlike building an ISO which is a single file, the netboot build using symlinkJoin is generating a directory with multiple symlinks in it. What we really want to fetch are probably the three files themselves. The code in nixos/release.nix may need to be changed to make outputs that make sense for hydra and for downloadFile but I'm still getting myself oriented with Nix so it might take someone with more experience to figure out a good way to do that.

[bobvanderlinden]

If it is impossible to download the individual files from the symlinkJoin, then we could indeed add individual attributes for the individual files. Maybe like this? https://github.com/bobvanderlinden/nixpkgs/blob/patch-1/nixos/release.nix Let me know if it is worthy of a PR. It'll expose the files as netboot.x86_64-linux.kernel, while also having the previous directory as netboot_bundle.x86_64-linux.

[domenkozar]

I've just pushed NixOS/nixpkgs@bab8a2e which should expose build products for nixos.netboot.x86_64-linux.

Now we just need to write a perl function downloadToFolder("nixos.netboot.x86_64-linux", "netboot") that would take all nixos.netboot.x86_64-linux Hydra build products and put them into netboot folder.

As seen in downloadFile, it parses the following output: curl -H "Accept: application/json" -L https://hydra.nixos.org/eval/1287660/job/nixos.iso_minimal.x86_64-linux | python -m json.tool

[domenkozar]

We get all the build products now:

$ curl -H "Accept: application/json" -L  http://hydra.nixos.org/build/38603078 | python -m json.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   998  100   998    0     0   3339      0 --:--:-- --:--:-- --:--:--  4106
{
    "buildmetrics": {},
    "buildoutputs": {
        "out": {
            "path": "/nix/store/vfnqjqszmm7as0xgb6f7dsyxqrf2g5v6-netboot"
        }
    },
    "buildproducts": {
        "1": {
            "defaultpath": "",
            "filesize": null,
            "name": "bzImage",
            "path": "/nix/store/vfnqjqszmm7as0xgb6f7dsyxqrf2g5v6-netboot/bzImage",
            "sha1hash": null,
            "sha256hash": null,
            "subtype": "bzImage",
            "type": "file"
        },
        "2": {
            "defaultpath": "",
            "filesize": null,
            "name": "initrd",
            "path": "/nix/store/vfnqjqszmm7as0xgb6f7dsyxqrf2g5v6-netboot/initrd",
            "sha1hash": null,
            "sha256hash": null,
            "subtype": "initrd",
            "type": "file"
        },
        "3": {
            "defaultpath": "",
            "filesize": null,
            "name": "netboot.ipxe",
            "path": "/nix/store/vfnqjqszmm7as0xgb6f7dsyxqrf2g5v6-netboot/netboot.ipxe",
            "sha1hash": null,
            "sha256hash": null,
            "subtype": "ipxe",
            "type": "file"
        }
    },
    "buildstatus": 0,
    "finished": 1,
    "id": 38603078,
    "job": "nixos.netboot.x86_64-linux",
    "jobset": "trunk-combined",
    "jobsetevals": [
        1287697
    ],
    "nixname": "netboot",
    "priority": 100,
    "project": "nixos",
    "releasename": null,
    "starttime": 1471367762,
    "stoptime": 1471367763,
    "system": "x86_64-linux",
    "timestamp": 1471362859
}

[domenkozar]

Note that this means additional 380MB (in initrd) per channel update.

[domenkozar]

@nshalman @bobvanderlinden Any ideas about netbootxyz/netboot.xyz#37 (comment)

[nshalman]

When doing some testing for Cerana, I ran into that issue and figured out a hack that gets around needing to pass in the path to the stage2 init, namely, in boot.initrd.postMountCommands (code executed in stage1) I added the following two lines at the end:

cd /mnt-root
ln -s nix/store/*system*/init init

Given that there should only be a single stage2 init in live media, this might be an acceptable solution.

[bobvanderlinden]

@nshalman That seems like a hack to me. It searches though all /nix/store paths that way, non-recursively but still searching.

@domenkozar Would there be some way to get config.system.build.toplevel into stage1 without relying on passing it through using kernel command-line?

[nshalman]

@bobvanderlinden agreed. I called it a "hack", but "gross hack" may have been more appropriate. All my attempts to extract the path to it within the Nix expressions failed, unfortunately. Hopefully @domenkozar has an idea or two that I didn't think to try.

[cleverca22]

another option is to use builtins.unsafeDiscardStringContext when you generate stage1, that lets you not depend on the entire system

oh i think i see the problem, the top-level depends on the initrd, and the initrd build depends on the path to top-level

not sure what the above built-in will do to the build-time dep-graph, time to experiment!

doesnt solve the eval-time deps, the hash of one derivation depends on the hash of the other, and creates a cycle

[matthewbauer]

Any progress on this? What remains to be done?

[zimbatm]

#21 has been closed. Does it mean that this has been fixed?