More output options

[oschwald]

The most recent release added support for GitHub workflow commands. Would you be open to supporting other output formats? The Checkstyle or JUnit XML formats would be nice given that there is a lot of tooling that supports them already, but even a JSON format that is easily machine readable would be useful.

Another thought is that many linting tools already support various output formats. For instance, golangci-lint:

  --out-format string              Format of output: colored-line-number|line-number|json|tab|checkstyle|code-climate|html|junit-xml|github-actions (default "colored-line-number")

ESLint supports many of the same formats.

Precious could potentially support reading in one of these machine-readable formats to improve its own output and allow lints to be associated with a particular line or character.

[autarch]

Yes, these both sound like good ideas.

As far as parsing the output of commands that it runs, I'd be up for that too. I suspect the best one to implement would be (and I hate to say this) checkstyle XML parsing. I'd love to go with JSON but just looking at eslint & and golangci-lint I can see that they're just making up their own schemas, so that's not really viable. What I don't want to do is have a custom parser for each tool.

Looking at the checkstyle output, it has all the info that is needed to turn this into GitHub annotations.

Making precious output something else wouldn't be too hard. Right now it only prints non-logging output in a few spots, so changing that to use some sort configurable thing would be pretty easy. As far as formats, as much as I'd love to use JSON, but absent any standard I don't see much point. I think checkstyle is probably the best bet for this too.

[autarch]

It'd probably be good to do a survey of other tools as well just to see if they all support checkstyle too.

[oschwald]

For both of these, it might make sense to design the configuration and code so that additional formats could potentially be added fairly easily in the future.

In terms of supporting JSON output, I think the primary advantage of that is making it easy for the user to transform the output into whatever they want.

Of the other tools that I use:

• Clippy - seems to only support a custom JSON output
• rustfmt - supports Checkstyle and a JSON format
• mypy - supports JUnit XML and some more obscure formats of other Python tools. Does not seem to support Checkstyle.
• Pylint - custom JSON and a Visual Studio output format. There is at least one third party plugin for Checkstyle though.
• php-cs-fixter - supports JUnit, Checkstyle, and custom JSON and XML.
• PHPStan - Checkstyle, JUnit, GitHub, GitLab, TeamCity, JSON.
• RuboCop - JUnit, custom JSON. There is a third-party Checkstyle plugin.
• perlcritic - doesn't seem to support any machine-readable output formats.

[autarch]

I started working on this locally and I'm implementing it with a new OutputWriter trait that the main precious code will use. So that'd make it pretty easy to support any format.

As far as JSON vs something else, the checkstyle XML is very simple. Here's an XSD for it - https://github.com/linkedin/pygradle/blob/master/pygradle-plugin/src/test/resources/checkstyle/checkstyle.xsd

But when I was talking about checkstyle, I was mostly talking about implementing parsing in precious. It seems like parsing checkstyle and using that to either emit GH annotations directly or to bundle it all into one giant checkstyle output might be useful.

[autarch]

Also related - https://github.com/marketplace/actions/java-checkstyle-annotations-uploader

[autarch]

Oh, ignore that action. I think it's parsing checkstyle's non-XML output or something.

This one seems to be what I was hoping existed - https://github.com/marketplace/actions/push-checkstyle-report

[autarch]

After spending some time looking at checkstyle, I've concluded that it's not great. It only handles errors, so there's no way to indicate that a file was tidied, or that a file was lint-clean.

I guess custom JSON it is, then.