HDDS-16019. Publish httpfs gateway metrics to Prometheus via /prom endpoint

[arunk-kumar]

What changes were proposed in this pull request?

HttpFSServerWebServer builds an HttpServer2 directly rather than extending BaseHttpServer, so it had no /prom endpoint and could not be scraped by Prometheus. This meant httpfs metrics were invisible to any Prometheus-based monitoring, and httpfs was absent from Grafana dashboards that group by component.

Changes:

1. Extract the Prometheus servlet wiring from BaseHttpServer into a new public static helper method addPrometheusEndpoint(HttpServer2, ConfigurationSource, String). This lives in the same package as BaseHttpServer so it retains access to the package-private PROMETHEUS_SINK constant and HttpServer2.getWebAppContext().

2. Refactor BaseHttpServer to call the helper — no behaviour change for existing services.

3. Wire the helper into HttpFSServerWebServer: call addPrometheusEndpoint in the constructor (after HttpServer2 is built), register the returned sink with DefaultMetricsSystem in start(), and unregister in stop(). Gated by hdds.prometheus.endpoint.enabled (default: true), matching the behaviour of other services.

The security model is preserved: if hdds.prometheus.endpoint.token is set, /prom is added as an internal servlet (token-based auth, SPNEGO bypassed); otherwise it is a regular servlet protected by the server's auth filter.

This is a prerequisite for HDDS-15858 (add httpfs to the ZDU Rolling Upgrade Grafana dashboard), which will add the httpfs scrape target and build-info metrics on top.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-16019

Prereq for: HDDS-15858
Epic: HDDS-14496 (Zero Downtime Upgrade)

How was this patch tested?

Local verification against upstream/HDDS-14496-zdu:

• mvn -pl :ozone-httpfsgateway,:hdds-server-framework -am install -DskipTests -DskipShade -DskipRecon -DskipDocs — builds clean.
• mvn -pl :ozone-httpfsgateway,:hdds-server-framework checkstyle:check — 0 violations on both modules.
• mvn -pl :ozone-httpfsgateway,:hdds-server-framework apache-rat:check — 0 unapproved licences.
• mvn -pl :ozone-httpfsgateway,:hdds-server-framework test — 573 tests, 0 failures (1 pre-existing skip in TestNetworkTopologyImpl).

TestBaseHttpServer and TestPrometheusServletAuthorization in hdds-server-framework
pass without modification, confirming the refactor does not change existing behaviour.

[yandrey321]

did you check that Prometheus actually collect metrics from httpfs instances?

[errose28]

+1 to the above. Let's at least manually verify that prometheus is showing existing httpfs metrics in the docker environment (Add monitoring.yaml to your docker compose configuration). If possible it would be good to add a test case to the acceptance tests as well that curls from prometheus and reads httpfs metrics.

Also, can you update the target branch to master since this change isn't ZDU specific? After this is merged I can do a reverse merge to incorporate it into the ZDU branch so you can continue work on HDDS-15858.

[arunk-kumar]

Closing to reopen from the correct master-based branch (HDDS-16019-httpfs-prom-endpoint-master) as per the discussion. New PR coming shortly.

[arunk-kumar]

Updated PR: #10945