Merge pull request #743 from github/lcartey/produce-ql-packs

Produce qlpacks as a build artifact

.github/workflows/code-scanning-pack-gen.yml

@@ -8,7 +8,6 @@ on:
       - main
       - next
       - "rc/**"
-
   push:
     branches:
       - main
@@ -98,15 +97,36 @@ jobs:
           CODEQL_HOME: ${{ github.workspace }}/codeql_home
         run: |
           PATH=$PATH:$CODEQL_HOME/codeql
-
-          codeql query compile --precompile --threads 0 cpp
-          codeql query compile --precompile --threads 0 c
+          # Precompile all queries, and use a compilation cache larger than default
+          # to ensure we cache all the queries for later steps
+          codeql query compile --precompile --threads 0 --compilation-cache-size=1024 cpp c
 
           cd ..
           zip -r codeql-coding-standards/code-scanning-cpp-query-pack.zip codeql-coding-standards/c/ codeql-coding-standards/cpp/ codeql-coding-standards/.codeqlmanifest.json codeql-coding-standards/supported_codeql_configs.json codeql-coding-standards/scripts/configuration codeql-coding-standards/scripts/reports codeql-coding-standards/scripts/shared codeql-coding-standards/scripts/guideline_recategorization codeql-coding-standards/schemas
 
       - name: Upload GHAS Query Pack
-        uses: actions/upload-artifact@v3
+        uses: actions/upload-artifact@v4
         with:
           name: code-scanning-cpp-query-pack.zip
           path: code-scanning-cpp-query-pack.zip
+
+      - name: Create qlpack bundles
+        env:
+          CODEQL_HOME: ${{ github.workspace }}/codeql_home
+        run: |
+          PATH=$PATH:$CODEQL_HOME/codeql
+
+          codeql pack bundle --output=common-cpp-coding-standards.tgz cpp/common/src
+          codeql pack bundle --output=common-c-coding-standards.tgz c/common/src
+          codeql pack bundle --output=misra-c-coding-standards.tgz c/misra/src
+          codeql pack bundle --output=cert-c-coding-standards.tgz c/cert/src
+          codeql pack bundle --output=cert-cpp-coding-standards.tgz cpp/cert/src
+          codeql pack bundle --output=autosar-cpp-coding-standards.tgz cpp/autosar/src
+          codeql pack bundle --output=misra-cpp-coding-standards.tgz cpp/misra/src
+          codeql pack bundle --output=report-coding-standards.tgz cpp/report/src
+
+      - name: Upload qlpack bundles
+        uses: actions/upload-artifact@v4
+        with:
+          name: coding-standards-codeql-packs
+          path: '*-coding-standards.tgz'

change_notes/2024-10-22-update-release-artifacts.md

@@ -0,0 +1,4 @@
+ - Modifications to the release artifacts:
+   - New CodeQL pack release artifacts have been created. These release artifacts can be downloaded from the release, and will be published to the GitHub registry under the `codeql` org for ease of deployment.
+   - The user manual has been updated to describe how to use the CodeQL packs.
+ - We no longer require a separate download of the CodeQL Standard Library for C++ - all queries have been pre-compiled and linked with the appropriate standard library.

docs/user_manual.md

@@ -29,13 +29,15 @@
 | 0.21.0  | 2024-05-01 | Luke Cartey     | Add MISRA C++ 2023 as under development, and clarify MISRA C 2012 coverage.                                             |
 | 0.22.0  | 2024-10-02 | Luke Cartey     | Add MISRA C 2023 as under development, and clarify MISRA C 2012 coverage.                                               |
 | 0.23.0  | 2024-10-21 | Luke Cartey     | Add assembly as a hazard.                                                                                               |
+| 0.24.0  | 2024-10-22 | Luke Cartey     | Add CodeQL packs as a usable output, update release artifacts list.                                                                                               |
 
 ## Release information
 
 This user manual documents release `2.37.0-dev` of the coding standards located at [https://github.com/github/codeql-coding-standards](https://github.com/github/codeql-coding-standards).
 The release page documents the release notes and contains the following artifacts part of the release:
 
-- `code-scanning-cpp-query-pack-2.37.0-dev.zip`: coding standard queries and scripts to be used with GitHub Code Scanning or the CodeQL CLI as documented in the section _Operating manual_.
+- `coding-standards-codeql-packs-2.37.0-dev.zip`: CodeQL packs that can be used with GitHub Code Scanning or the CodeQL CLI as documented in the section _Operating manual_.
+- `code-scanning-cpp-query-pack-2.37.0-dev.zip`: Legacy packaging for the queries and scripts to be used with GitHub Code Scanning or the CodeQL CLI as documented in the section _Operating manual_.
 - `supported_rules_list_2.37.0-dev.csv`: A Comma Separated File (CSV) containing the supported rules per standard and the queries that implement the rule.
 - `supported_rules_list_2.37.0-dev.md`: A Markdown formatted file with a table containing the supported rules per standard and the queries that implement the rule.
 - `user_manual_2.37.0-dev.md`: This user manual.
@@ -158,22 +160,52 @@ This section describes how to operate the "CodeQL Coding Standards".
 
 #### Pre-requisite: downloading the CodeQL CLI
 
-You must download a compatible version of the CodeQL CLI and CodeQL Standard Library for C++.
+You must download a compatible version of the CodeQL CLI, as specified in the release notes for the release you are using.
 
-**Option 1:** Use the CodeQL CLI bundle, which includes both required components:
+**Option 1:** Use the CodeQL CLI bundle, which includes both the CodeQL CLI and GitHub's default security queries:
 
  1. Download the CodeQL CLI bundle from the [`github/codeql-action` releases page](https://github.com/github/codeql-action/releases).
  2. Expand the compressed archive to a specified location on your machine.
  3. [Optional] Add the CodeQL CLI to your user or system path.
 
-**Option 2:** Fetch the components separately:
+This approach is recommended if you wish to use the default queries provided by GitHub in addition to the Coding Standards queries.
+
+**Option 2:** Use the CodeQL CLI binary:
 
  1. Download the CodeQL CLI from the [`github/codeql-cli-binaries` releases page](https://github.com/github/codeql-cli-binaries/releases)
  2. Expand the compressed archive to a specified location on your machine.
- 3. Using `git`, clone the [`github/codeql`](https://github.com/github/codeql) repository to a sibling directory of the CodeQL CLI. The `github/codeql` repository contains the CodeQL Standard Library for C++.
- 4. [Optional] Add the CodeQL CLI to your user or system path.
+3. [Optional] Add the CodeQL CLI to your user or system path.
+
+#### Pre-requisite: downloading the Coding Standards queries
+
+The Coding Standards packs can be downloaded into the local CodeQL package cache using the following command:
+
+```bash
+codeql pack download codeql/<standard>-<language>-coding-standards@<version>
+```
+
+The supported standards and languages are:
+ * `codeql/misra-c-coding-standards` - a CodeQL query pack for reporting violations of MISRA C.
+ * `codeql/cert-c-coding-standards` - a CodeQL query pack for reporting violations of CERT C.
+ * `codeql/misra-cpp-coding-standards` - a CodeQL query pack for reporting violations of MISRA C++.
+ * `codeql/cert-cpp-coding-standards` - a CodeQL query pack for reporting violations of CERT C++.
+ * `codeql/autosar-cpp-coding-standards` - - a CodeQL query pack for reporting violations of AUTOSAR for C++.
+
+Ensure that the `@<version>` string matches the desired Coding Standards version.
+
+Alternatively, the packs can be downloaded directly from a release on the `github/codeql-coding-standards` repository by choosing the `coding-standards-codeql-packs.zip`, which contains the following files:
+
+  * `misra-c-coding-standards.tgz` - a CodeQL query pack for reporting violations of MISRA C.
+  * `cert-c-coding-standards.tgz` - a CodeQL query pack for reporting violations of CERT C.
+  * `cert-cpp-coding-standards.tgz`  - a CodeQL query pack for reporting violations of CERT C++.
+  * `autosar-cpp-coding-standards.tgz` - a CodeQL query pack for reporting violations of AUTOSAR for C++.
+  * `common-cpp-coding-standards.tgz` - a CodeQL library pack, used if you are writing your own C++ queries against Coding Standards.
+  * `common-c-coding-standards.tgz` - a CodeQL library pack, used if you are writing your own C queries against Coding Standards.
+  * `report-coding-standards.tgz` - a CodeQL query pack for running diagnostics on databases.
 
-The release notes for the "CodeQL Coding Standards" pack you are using will specify the appropriate versions to use.
+Each pack will need to be decompressed using the `tar` program, and placed in a known location.
+
+Finally, we provide a legacy single zip containing all the artifacts from a release, named `code-scanning-cpp-query-pack.zip`. This also contains the CodeQL packs listed above.
 
 #### Creating a CodeQL database
 
@@ -194,26 +226,65 @@ Reference: [CodeQL CLI: Creating a CodeQL database](https://codeql.github.com/do
 
 #### Running the default analysis for one or more Coding Standards
 
-Once you have a CodeQL database for your project, you can run the "default" query suite. This will run all the "automated" queries for each implemented rule in the specified Coding Standards.
+Once you have a CodeQL database for your project you can run the default analysis for a specified Coding Standard using the `codeql database analyze` command by specifying the names of the QL packs which you want to run as arguments, along with a version specifier:
+
+```bash
+codeql database analyze --format=sarifv2.1.0 --output=<name-of-results-file>.sarif path/to/<output_database_name> codeql/<standard>-<language>-coding-standard@version
+```
+
+For example, this command would run MISRA C and CERT C with the default query sets:
+
+```bash
+codeql database analyze --format=sarifv2.1.0 --output=results.sarif path/to/<output_database_name> codeql/misra-c-coding-standard@version codeql/cert-c-coding-standard@version
+```
+The output of this command will be a [SARIF file](https://sarifweb.azurewebsites.net/) called `<name-of-results-file>.sarif`.
+
+##### Locating the Coding Standards CodeQL packs
+
+If you have downloaded a release artifact containing the packs, you will need to provide the `--search-path` parameter, pointing to each of the uncompressed query packs.
+```
+--search-path path/to/pack1:path/to/pack2
+```
+
+Alternatively, the packs can be made available to CodeQL without specification on the comamnd line by placing them inside the distribution under the `qlpacks/codeql/` directory, or placed inside a directory adjacent to the folder containing the distribution.
+
+##### Alternative query sets
+
+Each supported standard includes a variety of query suites, which enable the running of different sets of queries based on specified properties. In addition, a custom query suite can be defined as specified by the CodeQL CLI documentation, in order to select any arbitrary sets of queries in this repository. To run
+
+```bash
+codeql database analyze --format=sarifv2.1.0 --output=<name-of-results-file>.sarif path/to/<output_database_name> codeql/<standard>-<language>-coding-standard@version:codeql-suites/<alternative-suite>.qls
+```
 
-The query suites can be run by using the `codeql database analyze` command:
+If modifying the query suite, ensure that all Rules you expect to be covered by CodeQL in your Guideline Enforcement Plan (or similar) are included in the query suite, by running:
 
 ```bash
-codeql database analyze --format=sarifv2.1.0 --output=<name-of-results-file>.sarif path/to/<output_database_name> path/to/codeql-coding-standards/cpp/<coding-standard>/src/codeql-suites/<coding-standard>-default.qls...
+codeql resolve queries codeql/<standard>-<language>-coding-standard@version:codeql-suites/<alternative-suite>.qls
 ```
 
-For each Coding Standard you want to run, add a trailing entry in the following format: `path/to/codeql-coding-standards/cpp/<coding-standard>/src/codeql-suites/<coding-standard>-default.qls`.
+##### Supported SARIF versions
 
 The only supported SARIF version for use in a functional safety environment is version 2.1.0.
 To select this SARIF version you **must** specify the flag `--format=sarifv2.1.0` when invoking the database analyze command `codeql database analyze ...` as shown in the above example.
 
-Running the default analysis for one or more Coding Standards may require further performance customizations for larger codebases.
-The following flags may be passed to the `database analyze` command to adjust the performance:
+##### Performance optimizations
 
-- `--ram` - to specify the maximum amount of RAM to use during the analysis as [documented](https://codeql.github.com/docs/codeql-cli/manual/database-analyze/#options-to-control-ram-usage) in the CodeQL CLI manual.
-- `--thread` - to specify number of threads to use while evaluating as [documented](https://codeql.github.com/docs/codeql-cli/manual/database-analyze/#cmdoption-codeql-database-analyze-j) in the CodeQL CLI manual.
+Running the default analysis for one or more Coding Standards may require further performance customizations for larger codebases. The following flags may be passed to the `database analyze` command to adjust the performance:
 
-The output of this command will be a [SARIF file](https://sarifweb.azurewebsites.net/) called `<name-of-results-file>.sarif`.
+- `--ram` - to specify the maximum amount of RAM to use during the analysis as [documented](https://docs.github.com/en/code-security/codeql-cli/codeql-cli-manual/database-analyze#options-to-control-ram-usage) in the CodeQL CLI manual.
+- `--thread` - to specify number of threads to use while evaluating as [documented](https://docs.github.com/en/code-security/codeql-cli/codeql-cli-manual/database-analyze#-j---threadsnum) in the CodeQL CLI manual.
+
+##### Legacy approach
+
+If you have downloaded the legacy release artifact `code-scanning-query-pack.zip`, you can run the default query suite using the `codeql database analyze` command as follows:
+
+```bash
+codeql database analyze --format=sarifv2.1.0 --output=<name-of-results-file>.sarif path/to/<output_database_name> path/to/codeql-coding-standards/<language>/<coding-standard>/src/codeql-suites/<coding-standard>-default.qls...
+```
+
+For each Coding Standard you want to run, add a trailing entry in the following format: `path/to/codeql-coding-standards/<language>/<coding-standard>/src/codeql-suites/<coding-standard>-default.qls`. Custom query suites can be run by specifying the appropriate paths.
+
+All other options discussed above are valid.
 
 #### Running the analysis for audit level queries
 
@@ -223,8 +294,6 @@ Optionally, you may want to run the "audit" level queries. These queries produce
 codeql database analyze --format=sarifv2.1.0 --output=<name-of-results-file>.sarif path/to/<output_database_name> path/to/codeql-coding-standards/cpp/<coding-standard>/src/codeql-suites/<coding-standard>-audit.qls...
 ```
 
-For each Coding Standard you want to run, add a trailing entry in the following format: `path/to/codeql-coding-standards/cpp/<coding-standard>/src/codeql-suites/<coding-standard>-default.qls`.
-
 #### Producing an analysis report
 
 In addition to producing a results file, an analysis report can be produced that summarizes:

scripts/release/bump-version.sh

@@ -15,6 +15,7 @@ find . -name 'qlpack.yml' | grep -v './codeql_modules' | grep -v './scripts' | x
 # update the documentation. 
 
 find docs -name 'user_manual.md' -print0 | xargs -0 sed -i "s/code-scanning-cpp-query-pack-.*\.zip\`/code-scanning-cpp-query-pack-${1}.zip\`/"
+find docs -name 'user_manual.md' -print0 | xargs -0 sed -i "s/coding-standard-codeql-pack-.*\.zip\`/coding-standard-codeql-pack-${1}.zip\`/"
 find docs -name 'user_manual.md' -print0 | xargs -0 sed -i "s/supported_rules_list_.*\.csv\`/supported_rules_list_${1}.csv\`/"
 find docs -name 'user_manual.md' -print0 | xargs -0 sed -i "s/supported_rules_list_.*\.md\`/supported_rules_list_${1}.md\`/"
 find docs -name 'user_manual.md' -print0 | xargs -0 sed -i "s/user_manual_.*\.md\`/user_manual_${1}.md\`/"

scripts/release/release-layout.yml

@@ -10,6 +10,10 @@ layout:
     - workflow-artifact:
         name: "Code Scanning Query Pack Generation"
         artifact: code-scanning-cpp-query-pack.zip
+  coding-standards-codeql-packs.zip:
+    - workflow-artifact:
+        name: "Code Scanning Query Pack Generation"
+        artifact: coding-standards-codeql-packs
   supported_rules_list.csv:
     - shell: |
         python ${{ coding-standards.root }}/scripts/release/create_supported_rules_list.py --csv > supported_rules_list.csv