Don't escape UTF-8 names by default for OpenMetrics 2.0 #2517

When a scraper negotiates OpenMetrics 2.0 (Accept: application/openmetrics-text;version=2.0.0) without an escaping parameter, PrometheusScrapeHandler escapes UTF-8 metric and label names with EscapingScheme.DEFAULT (underscores). OpenMetrics 2.0 supports UTF-8 names natively and doesn't define an escaping parameter at all, so we could assume that scraper asking for 2.0.0 already supports UTF-8 names. In that case we shouldn't escape by default.

Prometheus main doesn't send escaping for OM 2.0 (https://github.com/prometheus/prometheus/blob/961c9ba40923ca4d7adf4a7a9167e56d0406df83/scrape/scrape.go#L794-L800), so UTF-8 names are currently escaped to underscores when scraping a Java target with OM 2.0. There's a Prometheus-side workaround in prometheus/prometheus@main...bwplotka/om2-accept-escaping, but the default here should be fixed regardless.

The escaping scheme is chosen without looking at the selected format:

• EscapingScheme.fromAcceptHeader returns DEFAULT when there's no escaping term:

  client_java/prometheus-metrics-config/src/main/java/io/prometheus/metrics/config/EscapingScheme.java

  Lines 29 to 67 in 9e9deb6

  	/** Default escaping scheme for names when not specified. */
  	public static final EscapingScheme DEFAULT = UNDERSCORE_ESCAPING;
  	public final String getValue() {
  	return value;
  	}
  	private final String value;
  	EscapingScheme(String value) {
  	this.value = value;
  	}
  	/**
  	* fromAcceptHeader returns an EscapingScheme depending on the Accept header. Iff the header
  	* contains an escaping=allow-utf-8 term, it will select NO_ESCAPING. If a valid "escaping" term
  	* exists, that will be used. Otherwise, the global default will be returned.
  	*/
  	public static EscapingScheme fromAcceptHeader(@Nullable String acceptHeader) {
  	if (acceptHeader != null) {
  	for (String p : acceptHeader.split(";")) {
  	String[] toks = p.split("=");
  	if (toks.length != 2) {
  	continue;
  	}
  	String key = toks[0].trim();
  	String value = toks[1].trim();
  	if (key.equals(ESCAPING_KEY)) {
  	try {
  	return EscapingScheme.forString(value);
  	} catch (IllegalArgumentException e) {
  	// If the escaping parameter is unknown, ignore it.
  	return DEFAULT;
  	}
  	}
  	}
  	}
  	return DEFAULT;
  	}

  .
• PrometheusScrapeHandler computes it before, and separately from, findWriter:

  client_java/prometheus-metrics-exporter-common/src/main/java/io/prometheus/metrics/exporter/common/PrometheusScrapeHandler.java

  Lines 73 to 91 in 9e9deb6

  	String acceptHeader = request.getHeader("Accept");
  	EscapingScheme escapingScheme = EscapingScheme.fromAcceptHeader(acceptHeader);
  	if (writeDebugResponse(snapshots, escapingScheme, debugParam, exchange)) {
  	return;
  	}
  	ExpositionFormatWriter writer = expositionFormats.findWriter(acceptHeader);
  	PrometheusHttpResponse response = exchange.getResponse();
  	response.setHeader("Content-Type", writer.getContentType());
  	if (shouldUseCompression(request)) {
  	response.setHeader("Content-Encoding", "gzip");
  	try (GZIPOutputStream gzipOutputStream =
  	new GZIPOutputStream(response.sendHeadersAndGetBody(200, 0))) {
  	writer.write(gzipOutputStream, snapshots, escapingScheme);
  	}
  	} else {
  	ByteArrayOutputStream responseBuffer =
  	new ByteArrayOutputStream(lastResponseSize.get() + 1024);
  	writer.write(responseBuffer, snapshots, escapingScheme);

  .
• ExpositionFormats.findWriter picks the OM 2.0 writer:

  client_java/prometheus-metrics-exposition-textformats/src/main/java/io/prometheus/metrics/expositionformats/ExpositionFormats.java

  Lines 63 to 86 in 9e9deb6

  	public ExpositionFormatWriter findWriter(@Nullable String acceptHeader) {
  	if (prometheusProtobufWriter.accepts(acceptHeader)) {
  	return prometheusProtobufWriter;
  	}
  	if (isOpenMetrics2Enabled() && openMetrics2TextFormatWriter.accepts(acceptHeader)) {
  	if (openMetrics2TextFormatWriter.getOpenMetrics2Properties().getContentNegotiation()) {
  	String version = parseOpenMetricsVersion(acceptHeader);
  	if ("2.0.0".equals(version)) {
  	return openMetrics2TextFormatWriter;
  	}
  	// version=1.0.0 or no version: fall through to OM1.
  	} else {
  	// contentNegotiation=false: OM2 handles all OpenMetrics requests.
  	return openMetrics2TextFormatWriter;
  	}
  	}
  	if (openMetricsTextFormatWriter.accepts(acceptHeader)) {
  	return openMetricsTextFormatWriter;
  	}
  	return prometheusTextFormatWriter;
  	}

  .

Proposed change (pseudo code):

// PrometheusScrapeHandler
String acceptHeader = request.getHeader("Accept");
ExpositionFormatWriter writer = expositionFormats.findWriter(acceptHeader);
EscapingScheme escapingScheme = EscapingScheme.fromAcceptHeader(acceptHeader, defaultFor(acceptHeader));

// Only default to UTF-8 when 2.0.0 was explicitly requested. With
// contentNegotiation=false the OM 2.0 writer also serves OM 1.0 / unversioned
// requests, e.g. from pre-3.0 Prometheus that can't parse UTF-8 names.
EscapingScheme defaultFor(String acceptHeader) {
  return "2.0.0".equals(parseOpenMetricsVersion(acceptHeader))
      ? EscapingScheme.ALLOW_UTF8
      : EscapingScheme.DEFAULT;
}

// EscapingScheme
public static EscapingScheme fromAcceptHeader(@Nullable String acceptHeader, EscapingScheme fallback) {
  // Same parsing as today. An explicit escaping term always wins, also for OM 2.0
  // (e.g. a Prometheus with legacy name validation asking for underscores).
  // Return fallback instead of DEFAULT when the term is missing or unknown.
}

Existing fromAcceptHeader(String) can keep delegating with DEFAULT to stay API compatible. Ideally the version and escaping term should come from the same Accept clause that findWriter matched. Today both helpers scan the whole header, so a parameter from another clause can leak in.

Tests to add (e.g. ExpositionFormatsTest, plus a scrape handler test):

• version=2.0.0 without escaping writes unescaped UTF-8 names.
• version=2.0.0;escaping=underscores still escapes.
• version=1.0.0, unversioned OpenMetrics and text/plain without escaping keep the underscores default, including with contentNegotiation=false.

The escaping schemes doc should get the same OM 2.0 exception, since its "Default Behavior" section says to use underscores whenever escaping is missing: https://github.com/prometheus/docs/blob/605cf81fefc2e8e91f8ba89bb1555ae52a43a318/docs/instrumenting/escaping_schemes.md#default-behavior.

Matching client_golang issue: prometheus/client_golang#2149