Provide web documentation for each error

[philiptaron]

There are now 28 different errors that nixpkgs-check-by-name can emit.

Following shellcheck, we should give these errors stable names (NPVxxx?) and document them somewhere.

I propose the following scheme:

1. Errors are marked as NPV-xxx (for nixpkgs-vet, the final name of this tool 😉). Three digits seems more than enough for us. The prefix is chosen for search-ability on the major search engines.
2. Each error is marked with a specific number. I recommend starting at 100, so there aren't leading zeros.
3. A wiki section is added to this project and a page added for each error. The page contains rationale for the error, recommended ways to fix the error, and workarounds if the error cannot be fixed.

This scheme is inspired by Shellcheck specifically. See for instance their page on error SC2034. Note that many other tools do similarly: Rust itself has an extensive index of errors.

Implementing this allows longer-form rationale and explanation than what can be done in the terminal, allowing that text to be more terse while not skipping out on good documentation.

[infinisil]

This sounds great to me! I'd also be up for renaming the repository to nixpkgs-vet if somebody makes a PR to update the name everywhere :)

[willbush]

Sounds good to me. I'm a fan of https://rust-lang.github.io/mdBook/ and was wondering if there's any interest in using that over GH wiki?

[willbush]

if somebody makes a PR to update the name everywhere :)

I'm up for that!

[philiptaron]

I also love mdBook also but picked the wiki for ease of editing like Shellcheck had. Making the thing easy to update means there's the power of everyone, not just the small portion of myself, Silvan, and you that this tool gets. 😄

[philiptaron]

if somebody makes a PR to update the name everywhere :)

I'm up for that!

I've opened #100 for the related rename.

[philiptaron]

Silvan, could you enable the wiki for this repository? I'll get some initial stubs in there.

[infinisil]

Done, I could also allow anybody to edit pages instead of just us with push access?

[philiptaron]

Let's keep it push access for now then widen once we have a release that references the particular error IDs that we publish and decide on.

[willbush]

Wiki doesn't have a PR workflow right?

[philiptaron]

Wiki doesn't have a PR workflow right?

Underneath the covers, it's a Git repo (naturally). https://github.com/NixOS/nixpkgs-vet.wiki.git

But the editing is making commits sans a PR workflow mostly.

[philiptaron]

Here's my proposal of the various error numbers and rough names

npv_100_by_name_undefined_attribute
npv_101_by_name_non_derivation
npv_102_by_name_internal_call_package_used
npv_103_by_name_cannot_determine_attribute_location
npv_104_by_name_override_of_non_syntactic_call_package
npv_105_by_name_override_of_non_top_level_package
npv_106_by_name_override_contains_wrong_call_package_path
npv_107_by_name_override_contains_empty_argument
npv_108_by_name_override_contains_empty_path
npv_109_by_name_shard_is_not_directory
npv_110_by_name_shard_is_invalid
npv_111_by_name_shard_is_case_sensitive_duplicate
npv_120_nix_eval_error
npv_121_nix_file_path_interpolation_unsupported
npv_122_nix_file_search_path_expression_unsupported
npv_123_nix_file_path_outside_of_directory
npv_124_nix_file_contains_unresolvable_path
npv_140_package_directory_is_not_directory
npv_141_package_has_wrong_name
npv_142_package_has_wrong_shard
npv_143_package_does_not_contain_package_nix
npv_144_package_nix_is_not_a_file
npv_145_package_contains_symlink_pointing_outside
npv_146_package_contains_unresolvable_symlink
npv_160_top_level_package_moved_back_from_by_name
npv_161_top_level_package_moved_with_custom_arguments
npv_162_new_top_level_package_should_be_by_name
npv_163_new_top_level_package_with_custom_arguments

[philiptaron]

Still to do in order to complete this story:

1. Make a wiki page for each of the possible errors. It should contain the following:
   a. The error text
   b. A rationale for the error
   c. An example of where the error could show up
   d. What to do next when the error is encountered
2. Make each error in the code emit a link to this page.