Add CodeQL Workflow for Code Security Analysis

[b4yuan]

Summary

This pull request introduces a CodeQL workflow to enhance the security analysis of this repository.

What is CodeQL

CodeQL is a static analysis tool that helps identify and mitigate security vulnerabilities. It is primarily intra-function but does provide some support for inter-function analysis. By integrating CodeQL into a GitHub Actions workflow, it can proactively identify and address potential issues before they become security threats.

For more information on CodeQL and how to interpret its results, refer to the GitHub documentation and the CodeQL documentation (https://codeql.github.com/ and https://codeql.github.com/docs/).

What this PR does

We added a new CodeQL workflow file (.github/workflows/codeql.yml) that

• Runs on every pull request (functionality to run on every push to main branches is included as a comment for convenience).
• Runs daily.
• Excludes queries with a high false positive rate or low-severity findings.
• Does not display results for git submodules, focusing only on our own codebase.

Validation

To validate the functionality of this workflow, we have run several test scans on the codebase and reviewed the results. The workflow successfully compiles the project, identifies issues, and provides actionable insights while reducing noise by excluding certain queries and third-party code.

Using the workflow results

If this pull request is merged, the CodeQL workflow will be automatically run on every push to the main branch and on every pull request to the main branch. To view the results of these code scans, follow these steps:

1. Under the repository name, click on the Security tab.
2. In the left sidebar, click Code scanning alerts.

Is this a good idea?

We are researchers at Purdue University in the USA. We are studying the potential benefits and costs of using CodeQL on open-source repositories of embedded software.

We wrote up a report of our findings so far. The TL;DR is “CodeQL outperforms the other freely-available static analysis tools, with fairly low false positive rates and lots of real defects”. You can read about the report here: https://arxiv.org/abs/2310.00205

Review of engineering hazards

License: see the license at https://github.com/github/codeql-cli-binaries/blob/main/LICENSE.md:

Here's what you may also do with the Software, but only with an Open Source Codebase and subject to the License Restrictions provisions below:

Perform analysis on the Open Source Codebase.

If the Open Source Codebase is hosted and maintained on GitHub.com, generate CodeQL databases for or during automated analysis, CI, or CD.

False positives: We find that around 20% of errors are false positives, but that these FPs are polarized and only a few rules contribute to most FPs. We find that the top rules contributing to FPs are: cpp/uninitialized-local, cpp/missing-check-scanf, cpp/suspicious-pointer-scaling, cpp/unbounded-write, cpp/constant-comparison, and cpp/inconsistent-null-check. Adding a filter to filter out certain rules that contribute to a high FP rate can be done simply in the workflow file.

[jadonk]

Is this spam or is this someone real? It looks like spam.

[b4yuan]

@jadonk I'm real. We have forked over two hundred repositories to study the benefits of static analysis tools. We actually opened three PRs a few months ago #222 #223 and #224 that were bugs found with CodeQL.

After forking each repository, manually building and analyzing each one, and presenting the results in the paper, we decided to reach out to devs and provide them with a CodeQL configuration to their respective repositories. While the process in which PRs was raised was automated, I can assure you that our intents are genuine and our biggest concern is improving code quality. If you have any questions, please feel free to follow up.

[jadonk]

k. how to get it to run now?

[b4yuan]

@jadonk You can see the results of the analysis in the Security tab under Code Scanning alerts. We recognize that CodeQL may flag errors that are false positives, or even errors that you may choose not to fix (e.g. runtime considerations). For these errors, you can dismiss them with a note in the interface.

[jadonk]

nothing there. just says configure scanning tool.

[b4yuan]

@jadonk Do you mind sharing a screenshot? (this is what it looks like on our fork)

[jadonk]

[b4yuan]

And if you click 'Configure scanning tool'?

[b4yuan]

If I had to guess, it's because there haven't actually been any runs of this workflow yet. The workflow is currently set to run nightly and on a PR (notice the commented-out trigger for a push to the main branch).

[jadonk]

I simply am not sure what I'm turning on and worry it will incur charges, etc. I am clearly being overly trusting to enable this. This doesn't feel like "security" to click to turn on features you don't understand.

[b4yuan]

Yes, I have just forked this repository again, changed the CodeQL workflow to run on every push, and here is before the workflow gets run:

And after the workflow finishes running:

The workflow needs to be run at least once, first, before it will upload to Code Scanning. With the current setup, I would wait until tomorrow (24 hours) for the scheduled run to finish. (Unless you'd really like to see now, in which case you can just un-comment the trigger for the workflow to run on every push to master.)

The .yml itself is the scanning tool (CodeQL) configuration. In fact, if you were to set up a new CodeQL scanning tool, the default config that GitHub would give you is decently similar to the one merged in this PR. We've added some features that we believe would be helpful to devs.

[b4yuan]

@jadonk Any update?

[jadonk]

working.

[b4yuan]

Great!