Can specialArgs be avoided?

[blaggacao]

drv-parts/lib.nix

Line 10 in 26eceeb

specialArgs = {inherit dependencySets;};

This line parts ways with a "simplified" module interface by introducing higher order arguments to the module evaluation.

These aren't free. I'm not experienced enough to come up with a complete list of bullet points of why that is, but it's evident that two interfaces add additional implications and concerns over just a single interface.

Especially when they break up the module system DSL and revert "back" to the functional style (for any particular reason), as evidenced by the additional {} argument to the evaluator.

This may turn out to be a significant cost in the future (thinking about things like the proposed simplified module system for packages). Systems may be devised that are otherwise compatible with drv-parts' API but deny secondary interfaces such as specialArgs for good reasons.

Is there no way to get along without specialArgs and confine the design space to the module system DSL and don't mix it with the functional style of injection (i.e. specialArgs)?

[roberth]

The only reason for specialArgs to exist is the imports phase, which is missing from minimod. This means that the handling of "imports" becomes the responsibility of the code that puts together the list of modules to pass to minimod.
Whether minimod is even suitable for this application is yet to be determined. drv-parts uses module system features that don't have an equivalent in minimod. Maybe one of them could be added, but otherwise, the two will probably result in different styles. Which is better may ultimately have to do with opinion and/or performance requirements due to the scale of the project. In this regard nixpkgs may be differ from a typical flake.

Don't try to infer too much from what I'm saying here. The facts are yet to determined, by giving minimod a serious try.

[DavHau]

@blaggacao I think you have a good point there. Ideally drv-modules should be portable, so you can copy them and use them in another context.

Logically the declaration of required deps and the fulfillment of deps should be separated. Basically we want a similar mechanism as flakes have. The module should be able to define defaults for its deps but at the same time the user should be able to discover the required deps without having to fulfill them first.

Currently the design doesn't fully comply with that.

What you can do already, is to override the deps from the outside via a simple deps = mkForce {hello=...}. The mkForce would override the whole value and specialArgs would not have to be evaluated anymore (didn't test that yet).

So basically the modules are portable already but there is no mechanism do discover the required dependencies without looking at the source code of the module.

In #10 I tried adding requiredDeps which solves the discoverability issue, but after discussing with @roberth I wasn't sure anymore if adding complexity to the UX was justified.

Generally, I feel that if drv-parts should become a portable spec that is useful in many different scenarios, increasing the api surface a bit to gain better guarantees might be worth it.

[DavHau]

One thought that just came up. Maybe we could copy the inputs schema from flakes?

Example:

{
  # declare and fullfill dep in one line
  deps.bash.follows = "pkgs/bash";

  # declare dep only but don't fulfill
  deps.hello = {};

  # declare dep by url (flake + attribute)
  deps.foo.url = "github:user/repo#packages.foo";

  # this mechanism is extendable. We could add features in the future.
  # declare dep via semver
  deps.foo.semver = "^1.0.0";
  deps.foo.url = ...;
}

We don't have a locking mechanism yet, but already being able to discover what is required could be valuable.

In the simple case of nixpkgs, where none of that is wanted, this simply is sufficient:

{
  deps.bash.follows = "pkgs/bash";
}

The import mechanism would be responsible for populating the deps by, for example setting:

{
  deps.bash.derivation = pkg.bash;
}

[roberth]

You can write your packages in a way that separates the fulfillment of deps from the rest of the package, by means of splitting the module.

Compared to the pkgs.callPackage fun args pattern,

• dependencySets is the more flexible equivalent of pkgs
• module composition is callPackage
• one module is fun
• another module is args
• deps is the intersection of pkgs and the fun parameters, // args

[roberth]

Maybe we could copy the inputs schema from flakes?

This could probably be done in a way that's independent of the modules that are currently in drv-parts, either a new module to be imported, or possibly more of a "substrate" offering submodules where you can import your package modules to make them part of the dependency resolution.

Actually copying the schema though, I'm not so sure, or maybe? Flakes isn't much of a real package manager yet anyway. It needs a notion of private / public deps, and some sort of versioning before I'll call it that.

[blaggacao]

I'm still chewing & understanding the problem:

Logically the declaration of required deps and the fulfillment of deps should be separated.

I think this is the anchor phrase.

Bisecting this, I naively arrive at:

declaration of required deps

An option tree (with defaults / bottom values).

fulfillment of deps

Either the option (declaration) default (bottom values) or the regular definition value with default priority and a potential supremum.

mkForce

The use of which indicates to me that somebody is consuming the wrong element of the (ordered) priority set. The top value should be of last resort if you can't consume the bottom value or an intermediate with a possible supremum.

Arriving at the same conclusion as you have: If it's use is indicated, we almost certainly have a design issue, somewhere.

[blaggacao]

I have a sense we could expect the provider of the bottom value for deps, that is the creator of the pkg module, to do do deliberate work and generate these options and their defaults:

drv-parts.lib.mkDepsOptions { inherit (nixpkgs) hello goodbye; }