Document Go cross-language argument binding - #71871
Conversation
e89a40e to
360cde8
Compare
|
+1, LGTM |
|
I wonder if we really need to have such different paths to get a literal vs XCom. Since the SDK process for execution is short-lived anyway, maybe it’s better to simply have a |
There was a problem hiding this comment.
I wonder if we really need to have such different paths to get a literal vs XCom. Since the SDK process for execution is short-lived anyway, maybe it’s better to simply have a GetLiteralArg Execution API endpoint that mirrors GetXCom? This way we don’t need to store any state between steps; the Go task gets everything it needs on Task.Execute.
Could you elaborate on what you mean by “store any state between steps,” and which steps you have in mind? Do you mean the binding spec persisted between Dag serialization and task execution, or the binding.Plan stored between Go task registration and Task.Execute? Thanks.
Literal arguments are already stored in the serialized Dag as part of #69757. At ti_run endpoint, Airflow reads that binding spec and includes it in TIRunContext.arg_bindings ( https://github.com/apache/airflow/blob/main/airflow-core/src/airflow/api_fastapi/execution_api/services/task_instances.py#L47-L61 )
Each binding contains either the literal value itself or the upstream task reference needed to retrieve the XCom.
Therefore, by the time Task.Execute is called, the Go runtime already has everything needed to resolve and bind the arguments. Only the actual XCom payload requires an additional lookup.
A GetLiteralArg endpoint would make literal and XCom retrieval look more similar, but it would move the literal lookup behind an additional Execution API request rather than eliminate the underlying state. The server would still need to read the literal from the serialized Dag.
Regardless of how the raw value is retrieved, the Go SDK must still analyze the function signature (which is what binding.Analyze does) and bind and decode each value into the corresponding Go parameter.
What
Was generative AI tooling used to co-author this PR?