Skip to content

Latest commit

 

History

History
86 lines (69 loc) · 5.29 KB

dependency-upgrades.adoc

File metadata and controls

86 lines (69 loc) · 5.29 KB

Dependency upgrades

Solr has lots of 3rd party dependencies, defined in gradle/libs.versions.toml. Keeping them up-to-date is crucial for a number of reasons:

  • minimizing the risk of critical CVE vulnerabilities by staying on a recent and supported version

  • avoiding "dependency hell", that can arise from falling too far behind

Read the help/dependencies.txt file for an in-depth explanation of how dependencies are managed.

Manual dependency upgrades

In order to upgrade a dependency, you need to run through a number of steps:

  1. Identify the available versions from e.g. Maven Central

  2. Update the version in gradle/libs.versions.toml file

  3. Run ./gradlew writeLocks to re-generate versions.lock. Note that this may cause a cascading effect where the locked version of other dependencies also change.

  4. In case of a conflict, resolve the conflict according to help/dependencies.txt

  5. Check if there are any constraints that are obsolete after the dependency update

  6. Update the license and notice files of the changed dependencies. See help/dependencies.txt for details.

  7. Run ./gradlew updateLicenses to re-generate SHA1 checksums of the new jar files.

  8. Once in a while, a new version of a dependency will transitively bring in brand-new dependencies. You’ll need to decide whether to keep or exclude them. See help/dependencies.txt for details.

Reviewing Constraints

The constraints are defined in gradle/validation/dependencies.gradle. There, if the updated dependency is listed, the constraint can be reviewed, updated or removed.

The constraints fall into two "groups". In the first group there are dependency constraints from dependencies that our project directly includes and require version alignment to sync the versions across all transitive dependencies. In the second group are dependencies that are only present as transitive dependencies. There, we try to follow the convention to provide additional information with "which dependencies use what version", so that the next person reviewing the constraint does not have to look it up. However, this is quite time-consuming to analyze the dependencies and therefore subject to change.

In order to review a constraint, you have to check if the updated dependency is mentioned in any of the constraints, either as a reason for another dependency constraint or as the constraint’s dependency. Removing temporarily a constraint, the task writeLocks will fail if the constraint is still required.

This process and the constraints of dependencies.gradle are not optimal, as it is quite time-consuming and not obvious by just looking at it. We just haven’t found yet a more efficient way to maintain these constraints.

Renovate bot Pull Requests

The renovate bot may be replaced in the future with dependabot and this section may only be relevant for older versions (<10.0). See https://lists.apache.org/thread/1sb9ttv3lp57z2yod1htx1fykp5sj73z for updates.

A member of the Solr community operates a Github bot running Renovate, which files Pull Requests to Solr with dependency upgrade proposals. The PRs are labeled dependencies and do include changes resulting from ./gradlew writeLocks and updateLicenses.

Community members and committers can then review, and if manual changes are needed, help bring the PR to completion. For many dependencies, a changelog is included in the PR text, which may help guide the upgrade decision.

The scans are run on a schedule. New PRs are filed every Sunday, and only dependency versions that are at least 5 days old are suggested, to guard against bad apples. If a new major version of a dependency is available, that will get its own separate Pull Request, so you can choose.

If an upgrade is decided, simply merge (and backport) the PR. To skip an upgrade, close the PR. If a PR is left open, it will be re-used and auto updated whenever a newer patch- or minor version gets available. Thus, one can reduce churn from frequently-updated dependencies by delaying merge until a few weeks before a new release. One can also choose to change to a less frequent schedule or disable the bot, by editing renovate.json

Configuring renovate.json

While the bot runs on a GitHub repo external to the project, the bot behavior can be tailored by editing .github/renovate.json in this project. See Renovatebot docs for available options.