Avoid getting HTTP 429 through honoring repositories defined in settings.xml

[ppalaga]

New feature, improvement proposal

We recently started getting HTTP 429 from Maven Central when performing builds on a company internal Jenkins:

./mvnw -B -e -s path/to/settings.xml ...
Exception in thread "main" java.io.IOException: Server returned HTTP response code: 429 for URL: https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.9.11/apache-maven-3.9.11-bin.zip
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:2037)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1626)
	at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:224)
	at org.apache.maven.wrapper.DefaultDownloader.downloadInternal(DefaultDownloader.java:100)
	at org.apache.maven.wrapper.DefaultDownloader.download(DefaultDownloader.java:86)
	at org.apache.maven.wrapper.Installer.createDist(Installer.java:84)
	at org.apache.maven.wrapper.WrapperExecutor.execute(WrapperExecutor.java:160)
	at org.apache.maven.wrapper.MavenWrapperMain.main(MavenWrapperMain.java:73)

I know there is a possibility to change distributionUrl in .mvn/wrapper/maven-wrapper.properties
Unfortunately, it is not feasible in this case, because it is a public git repository that can be built by anybody, so we cannot set a company internal distributionUrl.

I wonder whether it would be possible to implement some basic parsing of maven mirrors and an repositories present in settings.xml and respect those when downloading the distro?

[cstamas]

@ppalaga @maxandersen @gsmet How is today best to produce small (native or not) CLI binary using Quarkus?

I am thinking about a "wrapper" CLI (Quarkus CLI) using static MIMA, minimized (to the bare minimum needed for task, tree shaking), made native (if possible)... as if this is doable, it could be new type of wrapper, that "understand" Maven env/config, as Peter wants, and could use proxy or whatever from user/custom settings.xml.

This could solve my pet peeve issue as well, like using GAV and not URL for wrapped Maven, and actual source (URL) from where resolution happens could/should be configurable.

This could be nice try for new "tree shaking" with static runtime to not have Sisu in play?

[maxandersen]

start with quarkus create app -x picocli that gives quarkus command line app. Make it do what you want and I can help or find help to strip it down.

note, we got work actively ongoing that uses aeash to greatly speedup command line parsing compared to picocli but when we have that it should be easy upgrade.

[ppalaga]

Sounds interesting, @cstamas.

Going native may have the disadvantage of having to produce many binaries for different OSes and CPU architectures. In addition to that, GraalVM still does not support all OS x Arch combos where Java is supported. So you most probably end up with some hybrid set of native binaries plus fat jar for non-common platforms. Anyway, how would you distribute those binaries/jar? Not sure embedding them in the end user project's git would be practical. Won't dowloading them from a remote URL re-create a variant of the current problem?

[cstamas]

Due this native issue, let's stick for start with "self sufficient JAR". As then, similar as today, (Maven and Gradle wrapper) JAR is present and checked in:

[cstamas@angeleyes jbang (main)]$ tree gradle/
gradle/
└── wrapper
    ├── gradle-wrapper.jar
    └── gradle-wrapper.properties

2 directories, 2 files
[cstamas@angeleyes jbang (main)]$ 

But, this could be in fact some mini-toolbox-mima-whatever CLI?

[cstamas]

@ppalaga but aside of "drømmer om", you are aware of following:

• use bin wrapper
• set env var MVNW_REPOURL to something like https://repo.company.com/central-proxy

Basically without this env var all goes as usual (today), while your CI may target your proxy.

[ppalaga]

set env var MVNW_REPOURL to something like https://repo.company.com/central-proxy

That sounds great! Thanks for the tip! I think that's a fast and relatively easy solution.