Language SDK task state writes skip the max_value_storage_bytes warning #74944

Description

[state_store] max_value_storage_bytes caps the size of a task state value. The public REST API rejects an oversized value with a 422, and a Python task logs a warning and still writes the value (task-sdk/src/airflow/sdk/execution_time/context.py), so a running task is never failed over the size.

Tasks run by the Go, TypeScript and Java SDKs get neither, because:

• the coordinator does not pass max_value_storage_bytes to the language SDK
  runtime (_build_runtime_env() in
  task-sdk/src/airflow/sdk/coordinators/_subprocess.py passes only
  default_retention_days from [state_store]), and
• neither the supervisor nor the Execution API checks the size on the write path.

Proposed change:

1. Have the coordinator pass the setting as AIRFLOW__STATE_STORE__MAX_VALUE_STORAGE_BYTES, as it already does for
   AIRFLOW__STATE_STORE__DEFAULT_RETENTION_DAYS.
2. In each SDK's task state Set, measure the value the way Python does (len(json.dumps(value)), which is also the length of the string the Execution API stores) and log a warning when it is over a positive limit. The write still proceeds.
3. When the variable is not set, skip the check rather than fail the write. This covers an older coordinator that does not pass it.

Use case/motivation

An operator sets max_value_storage_bytes to steer task state toward small coordination values. A Python task that writes more gets a warning in its log; a Go, TypeScript or Java task writing the same value gets nothing. On Postgres and SQLite the value column has no practical size limit, so the value is stored without anyone noticing. On MySQL (MEDIUMTEXT) the write fails at around 16 MB.

Related issues

Each SDK already has a TODO for this, added during review of the task state store PRs: Go in #73420, Java in #73464 and TypeScript in #73618. #74400 tracks a related language SDK state store gap ([workers] state_store_backend).

I have the coordinator and Go SDK change ready; TypeScript and Java would follow in separate PRs.

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct