Support mill init from an existing sbt project
#4586
+3,207
−137
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…sting SBT project from its corresponding Gradle support `SbtBuildGenMain.scala` and `BuildGenTests.scala` are not fully adapted yet, and the unadapted code is temporarily commented out. An initial SBT plugin `ProjectTreePlugin` is added but not fully implemented yet. Only a task that writes some content to a file is added to test whether this approach works. Exception in `./mill main.init.sbt.compile`: ```text [1486] java.io.IOException: Scala signature package has wrong version [1486] expected: 5.0 [1486] found: 5.2 in package.class ``` Corresponding code for Maven and Gradle is slightly improved BTW.
…s Scala 2.13 while SBT uses Scala 2.12 `./mill main.init.sbt.test` passes now.
…-project-tree` plugin to an SBT build and writing and reading an empty `ProjectTree` model `./mill main.init.sbt.test` and `./mill main.init.sbt.sbt-mill-init-generate-project-tree.compile` run successfully. Other than these 2, the changes are not fully tested yet.
…ee` sbt task in `SbtBuildGenMain`, create a "scala-seed-project" test project in test resources with `sbt new`, and update `ProjectTreePlugin`
`./mill main.init.sbt.test` fails.
In the sbt launched under the directory "main/init/sbt/test/resources/scala-seed-project" with `sbt -addPluginSbtFile=/.../mill-init.sbt` with the following content, the `tasks` sbt command output doesn't include an added `millInitGenerateProjectTree` task:
```sbt
addSbtPlugin("com.lihaoyi" % "mill-main-init-sbt-sbt-mill-init-generate-project-tree" % "0.1.0-SNAPSHOT" from "file:/.../mill/out/main/init/sbt/sbtPluginResources.dest/sbt-mill-init-generate-project-tree.jar")
```
The following content of "mill-init.sbt" yields the following result:
```sbt
import mill.main.sbt.ProjectTreePlugin
addSbtPlugin("com.lihaoyi" % "mill-main-init-sbt-sbt-mill-init-generate-project-tree" % "0.1.0-SNAPSHOT" from "file:/.../mill/out/main/init/sbt/sbtPluginResources.dest/sbt-mill-init-generate-project-tree.jar")
enablePlugins(ProjectTreePlugin)
lazy val root = (project in file("."))
.enablePlugins(ProjectTreePlugin)
```
```text
/Users/shreckye/Projects/mill/mill-init.sbt:1: error: not found: object mill
import mill.main.sbt.ProjectTreePlugin
^
/Users/shreckye/Projects/mill/mill-init.sbt:8: error: not found: value ProjectTreePlugin
.enablePlugins(ProjectTreePlugin)
^
sbt.compiler.EvalException: Type error in expression
```
Reviewing the contents of https://repo1.maven.org/maven2/com/eed3si9n/sbt-assembly_2.12_1.0/2.3.1/sbt-assembly_2.12_1.0-2.3.1.jar, I see there is an "sbt/sbt.autoplugins" file which has the content:
```
sbtassembly.AssemblyPlugin
```
This probably means that such a file has to be included for an sbt plugin and a plain Mill module doesn't support this.
…ala-seed-project" and run `./mill main.init.sbt.test` The exception: ```text java.nio.file.NoSuchFileException: /Users/shreckye/Projects/mill/out/main/init/sbt/test/test.dest/sandbox/scala-seed-project/target/mill-init-project-tree.json ```
…n `sbt new sbt/sbt-autoplugin.g8` in "main/init/sbt"
…t to the testing frameworks BTW in the Scaladoc of `SbtBuildGenMain`
…gic that jumps between Mill and sbt The issues in commit 3c9f155 are resolved and `./mill main.init.sbt.test` now passes.
…project "sbt-mill-init-export-build", and generate build files for "scala-seed-project" in the test resources The generated "scala-seed-project" "build.mill" file is not completely correct yet. Changes: * All kinds of necessary model case classes are added in "Models.scala" in the "models" module. * A `MillPublishCrossScalaModule` is extracted in the Mill build files. * The sbt plugin project "sbt-mill-init-generate-project-tree" is renamed to "sbt-mill-init-export-build" as the plugin doesn't just export a tree of projects. * An `sbtPluginProjectSource` `Source` task is added in the `sbt` module's Mill build configuration to react to changes in the sbt Plugin project. * `BuildGenBase` is updated with a new type parameter `I` for `input`, as the `IrBaseInfo` is not retrieved from a module in sbt but from `ThisBuild`, and thus `BuildExport.defaultBuildInfo` needs to be passed. * A `BaseInfoFromSubproject` subtype trait is added. * A `BasicConfig` is extracted from `Config` for sbt. * `MavenBuildGenMain` and `GradleBuildGenMain` are updated accordingly and slightly improved. * Changes in `BuildGenUtil`: * `scalacOptions` is added similar to `javacOptions`. * More Scala test frameworks are added in `testModulesByGroup`.
…evious commit, adjust the format of some code, and inline an sbt task
…`ExportBuildPlugin` because its members are always needed when there is no `baseModule` * Add a `takeIrPomIfNeeded` function to also set a module's `PomSettings` when it's different from a base module's `PomSettings`. * Regenerate and update "test/resources/expected/scala-seed-project/build.mill".
…ix the tests `./mill main.init.maven.test` and `./mill main.init.gradle.test` which were broken
…t to the test resources Both `./mill main.init.sbt.test` and running `sbt` in the project directory fail with: ``` java.lang.UnsupportedOperationException: The Security Manager is deprecated and will be removed in a future release ``` This is likely because I am using JDK 21.
…mprove 1. Bump the sbt version in "sbt-multi-project-example" to v1.10.7 1. Print an info message to remind the user when the sbt command to run the `millInitExportBuild` sbt task fails. 1. Drop dependencies of unknown configurations. 1. Generate the expected snapshots for "sbt-multi-project-example" and review. 1. Comment the Local Maven Repository as it's machine-dependent.
…the config args, fixing some bugs
…y are in the `BaseModule` in the generated build files
…t the conversion, regenerate the snapshots, and fix a bug that `organizationHomepage` is passed as `IrPom.url` instead of `homepage` and regenerate again Code related to `organizationHomepage` is commented out as it's actually unnecessary.
…un them, review their results, and fix some bugs revealed - Fix compiling issues in `MavenBuildGenMain.scala` and `GradleBuildGenMain.scala` caused by `scalaVersion` introduced. - Copy the sbt plugin "sbt-mill-init-export-build-assembly.jar" to a temp directory to be consumed so it works in integration tests when Mill is packaged. - Improve a message in `ExportBuildPlugin`. - Make the generated `package` object always extends the `BaseModule` or `SbtModule` trait. - Fix a bug in `renderIvy` that the double colon `::` is placed at the wrong place. - Update the expected build file snapshots accordingly in test resources. - Update docs.
…ts after manually running and tweaking them
|
The CI seems to fail because there is no sbt set up. (See "main.init.sbt.sbtPluginJarResources failed" in https://github.com/ShreckYe/mill/actions/runs/13398010494/job/37429413361.) Shall I update the CI to set it up? Or shall I pull and add a script from sbt-extras to use instead? |
|
@ShreckYe yes please update the CI scripts |
An `java.io.IOException: Cannot run program "chmod"` exception thrown when the tests run in PowerShell is fixed BTW.
…d of `SbtTests` and update the unit test expected snapshots accordingly
…36Tests` without success, and add a comment about it
…ildGenMain`, and update the tests (unit tests and integration tests) accordingly
…ate the expected snapshots in test resources accordingly
…dule's `def`s in converted builds as required in com-lihaoyi#4586 (comment) - Refactor some types to use Scala 3 explicit nulls BTW. - Fix a bug that `pomSettings` in modules different from that in the base module is lost in builds converted from Maven, BTW. - Update unit test code to test the changes and the expected snapshots in test resources correspondingly.
…Tests` in the integration tests
…ResolvedTasks` in the meantime
- Reformat `ExportBuildPlugin.scala` BTW. - The existing unit tests and integration tests pass as tested on my device.
…endencies - Fix a bug that the custom `RuntimeException` is not thrown "main/init/package.mill" after the code is refactored to use OS-Lib. - Update unit test code to test the changes and the expected snapshots in test resources correspondingly. - The existing unit tests and integration tests pass as tested on my device. - Remove a redundant import `scala.language.higherKinds` in `ExportBuildPlugin.scala`.
…erted builds in the integration tests This completes commit 88c31b1. Add `millInitExportBuild / aggregate := false` in `ExportBuildPlugin` to prevent the `millInitExportBuild` sbt task run in every child project, which seems to have sped up the tests.
… in `MillInitSbtTests.scala`
|
Both |
Closed
|
Hi @lihaoyi I have finished the changes requested and replied to your code review comments with updates. I think it's ready for another round of review. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves #3450.
Feature summary
The conversion
The conversion does not support:
Implementation summary
To resolve the issue, I use sbt's addPluginSbtFile command to add a plugin jar assembled from the
sbt-mill-init-export-buildsbt project, which adds an sbt taskmillInitExportBuildto extract the settings for a build and serialize it with uPickle, and then import the build with the rest of the code inSbtBuildGenMain. Thesbt-mill-init-export-buildsbt plugin project is built with sbt, so it's wired up with Mill's build logic, with some cross build setups for the common models as sbt depends on Scala 2.12.Some questions and next steps
ThisBuild. However, not all projects seem to follow this convention, which can lead to very cumbersome build files with long duplicate settings, such asjavacOptions. I can add the similarbaseProjectconfig arg from Gradle's implementation to resolve this if this is necessary.