Skip to content

OTLP collector endpoint is defined in 9 places #745

Closed

Description

@git286

OTEL_EXPORTER_OTLP_ENDPOINT is hardcoded in the four workload manifests, then re-patched five more times in the kind overlays:

manifests/ate-install/ate-api-server.yaml:123
manifests/ate-install/ate-controller.yaml:95
manifests/ate-install/atelet.yaml:108
manifests/ate-install/atenet-router.yaml:157
manifests/ate-install/kind/kustomization.yaml:60,80,98,116
manifests/ate-install/kind/atelet/kustomization.yaml:42

Pointing a cluster at a different collector means editing nine spots across six files, and knowing which install path renders which.

Two of the targeted redeploy flags apply the base manifests raw, with no
kustomize, even in kind mode:

  • deploy_ate_apiserver → run_ko apply -f manifests/ate-install/ate-api-server.yaml
  • deploy_atenet → run_ko resolve -f manifests/ate-install/atenet-router.yaml

So on a kind cluster:

hack/install-ate-kind.sh --deploy-ate-system     # endpoint = otel-system       (right)
hack/install-ate-kind.sh --deploy-ate-apiserver  # endpoint = gke-managed-otel  (wrong)

The second command silently repoints ateapi at a Service that doesn't exist in kind. No error, no failed pod — traces and metrics just stop showing up. You find out when a dashboard is empty.

deploy_atelet doesn't have the bug because it branches on kind and renders the overlay. That inconsistency is the tell.

Suggested fix

One checked-in ConfigMap per environment (ate-otel-config), pulled in by every workload via envFrom. Two definitions instead of nine, and the targeted redeploys just apply the right ConfigMap for the environment.

configMapGenerator won't work here: the GKE path applies manifests/ate-install/ as a raw directory and the targeted paths apply single files, so the name has to stay stable — no hash suffix.

There's precedent already: ate-api-server.yaml pulls in the optional ate-api-server-envvars ConfigMap exactly this way.

Activity

  1. added this to the M2 milestone on Aug 4, 2026
  2. added a commit that references this issue on Aug 5, 2026
    7a9bb4b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions