Repository navigation
Description
The router's Envoy writes request secrets to its container log. On GKE this goes to Cloud Logging and stays there for 30 days. Anyone with log read access on the project can see it.
Two ways it leaks:
-
Debug logging is on in the base manifest.
manifests/ate-install/atenet-router.yamlstarts Envoy with--component-log-level upstream:debug,router:debug,ext_proc:debug. At this level Envoy dumps every request header for every request, soAuthorizationandCookiego into the log as-is. This is not a kind-only setting. It ships in the base install. -
The access log prints the query string. The router's HCMs use
StdoutAccessLog{}with no format (cmd/atenet/internal/router/xds.go). Envoy's default format logs the full path, including the query string. So?api_key=...and similar tokens are logged even with debug off.
Repro on kind with main:
kubectl -n ate-system port-forward svc/atenet-router 18099:80 &
curl -X POST "http://localhost:18099/?api_key=QUERYSECRET" \
-H "ate-target-actor: <atespace>/<running actor>" \
-H "Authorization: Bearer FAKE-TOKEN" \
-H "Cookie: session=FAKECOOKIE" -d '{}'
kubectl -n ate-system logs deploy/atenet-router -c envoy | grep -E 'FAKE-TOKEN|FAKECOOKIE|QUERYSECRET'
All three show up in the log.
The debug flag also makes the log noisy. On an almost idle GKE cluster the router's Envoy writes more than 1000 debug lines an hour.
Nothing in the repo reads these debug lines. The flag came in with #711 and looks like it was left over from local debugging.
Proposed fix:
- Drop
--component-log-levelfrom the router manifest. Envoy goes back toinfo. If someone needs debug, they can turn it on for a while through the admin port withPOST /logging?router=debug. No restart needed. - Set an explicit access log format: Envoy's default, but with
%PATH(NQ:ORIG_OR_PATH)%so the query is never logged. - Add a test that every router HCM uses that format.
Activity
Router no longer logs request credentials or query strings (agent-sub…