cabal-install: Be less eager to configure external programs

[mpickering]

In configureCompiler the call to configureAllKnownPrograms was too eager. When called it selected the version of tools from PATH (such as alex), and then when a package was configured these tools were passed using --with-alex options to configure, which meant that the build-tool-depends versions were not used. (See #10692)

Why was this call introduced in the first place? Because configureCompiler would a different result depending on whether:

• It is run for the first time, the ProgramDb will contain unconfigured programs.
• It is run subsequently, ProgramDb is read from disk, it does not contain unconfigured programs.

A surgical way to fix this is to avoid configuring the programs, and manually adding back the builtinPrograms to the ProgramDb, so the ProgramDb returned by configureCompiler always contains all the unconfigured programs.

The testcase is not so easy to write because

• The bug only surfaces when the build-tool you are depending on is known (ie alex, happy etc)
• But then it is tricky to write a test, as we can't depend on the known tools or bundle the source for them.
• So we create a fake "alex", which cabal then invokes on a fake ".x" file. This is maybe a bit fragile if the way cabal invokes alex changes in future, but then the test can be modified as well.

Also see #2241 and #9840

Fixes #10692

Please read Github PR Conventions and then fill in one of these two templates.

---

Template Α: This PR modifies behaviour or interface

Include the following checklist in your PR:

• Patches conform to the coding conventions.
• Any changes that could be relevant to users have been recorded in the changelog.
  • Is the change significant? If so, remember to add significance: significant in the changelog file.
• The documentation has been updated, if necessary.
• Manual QA notes have been included.
• Tests have been added. (Ask for help if you don’t know how to write them! Ask for an exemption if tests are too complex for too little coverage!)

[ulysses4ever]

Hey @mpickering! Any chance you could take a brief look at #2015? I believe it's closely related to the issue here. In particular, cabal repl runs with $PATH effective at the time of configuring, not at the time of running cabal repl. So, if you run cabal build on clean checkout, then change $PATH, then run cabal repl, inside the repl you won't see the change. This keeps blowing people's heads...

I tried to look into it, and it seems to stem, as you say, from storing the compiler's configuration on disk, and configuration includes $PATH. But I don't know what'd be the least intrusive way to fix it. E.g. should we just reconfigure from scratch inside cabal repl? Or should we treat $PATH in a special way, and maybe try to invalidate only it on every cabal repl run.

The code for cabal repl had a big update from your multi-repl patch. That's another reason I thought you may be a great candidate for guiding or, indeed, solving #2015.

[mpickering]

@ulysses4ever I've just been assigned to look at these two regressions, but I will comment on the issue about how I would begin with that problem.

[mpickering]

@sheaf and I discussed this and for the backport, the have gone with a revert of the original patch.

I have added two new tests which check for regressions introduced in the original patch.

In the future we will work on a proper fix the original issue the reverted patch attempted to fix.

[sheaf]

This should be ready now if anyone wants to take a look? @ulysses4ever @jasagredo perhaps?

[mpickering]

Is there anyone available to provide the second review on this patch?

[jasagredo]

It is unfortunate that we have to revert back to the wasteful re-computation of the step, but at least this fixes my issue. Thanks!