Status of testing Providers that were prepared on October 14, 2026 #74741

Activity

1. potiuk

   added

   kind:metaHigh-level information important to the communityHigh-level information important to the community

   testing statusStatus of testing releasesStatus of testing releases

   on Oct 14, 2026
2. med9110 commented on Oct 15, 2026

   

   med9110

   on Oct 15, 2026

   More actions

   Hi @potiuk, I'd like to independently verify #74505 against Google provider 22.7.0rc2, focusing on compatibility with older Kubernetes providers. I'll report the installed versions, import results, and exception behavior.

   🎉React with 🎉1Reacted by Jarek Potiuk
3. atharvaajmera commented on Oct 15, 2026

   

   atharvaajmera

   on Oct 15, 2026

   Contributor

   More actions

   Verified #74192 with apache-airflow-providers-google==22.7.0rc2 from PyPI (Airflow 3.3.2, Python 3.12):

   • the three Dataproc triggers (DataprocSubmitTrigger, DataprocSubmitJobDirectTrigger, DataprocClusterTrigger) re-raise CancelledError after their cleanup — checked the code in the installed package, the bare raise is there in all three.
   • ran the updated trigger tests from main against the installed RC: 52 passed, 5 skipped (the skips are the on_kill() tests that defer to the Airflow 3.3+ cancellation path, as expected).

   😄React with 😄1Reacted by Jarek Potiuk
4. FrankYang0529 commented on Oct 16, 2026

   

   FrankYang0529

   on Oct 16, 2026

   Member

   More actions

   Tested following providers and all work as expected.

   • amazon 9.38.0rc2
     • Fix AWS auth manager login when the IdP is on another site #74529: A login started from Airflow works again over HTTPS.
   • dbt.cloud 4.11.0rc2
     • Fix deferred dbt Cloud tasks failing with tenacity retry_args #74489: A deferrable DbtCloudRunJobOperator with tenacity objects in hook_params["retry_args"] defers and succeeds. The worker task log now shows one warning for the dropped retry_args, and the trigger retries with retry_limit and retry_delay.
   • common.ai 1.0.0rc1
     • Fix agent HITL review for list, marker and function output types #74537: Output functions, TypeAliasType and NewType now push a dict, so a downstream xcom_pull no longer raises ImportError.str and [str, int] answers stay strings. dict and list[str] outputs now come back as a dict and a list instead of a Python repr string.
     • Let SandboxToolset be deep-copied again #74525: A deep-copied SandboxToolset can work again.

   🎉React with 🎉1Reacted by Jarek Potiuk
5. zozo123 commented on Oct 16, 2026

   

   zozo123

   on Oct 16, 2026

   Contributor

   More actions

   Verified the basic live backend flows for #71672 (Islo) and #71725 (Boat) with apache-airflow-providers-common-ai[boat,islo]==1.0.0rc1 installed from PyPI, Airflow 3.3.2, Python 3.12.12 (islo==0.3.20, boat-sdk==1.9.9).

   Both passed sandbox creation, command execution with preserved newlines, file write/read round-trip, directory listing, nonzero exit-code handling, and sandbox destruction. Boat used one sandbox for all checks with SandboxSpec(block_network=False); Islo used the default SandboxSpec().

   This was a direct backend smoke test, not an AgentOperator/Dag or full system-test run; timeout and network enforcement were not exercised. #73961 (OpenShell) and #74504 (OpenSandbox dns+nft enforcement) remain unverified by this run. #71676 is already checked in the issue.

   ❤️React with ❤️1Reacted by Jarek Potiuk
6. zozo123 commented on Oct 16, 2026

   

   zozo123

   on Oct 16, 2026

   Contributor

   More actions

   Follow-up for #73961 (OpenShell) and #74504 (OpenSandbox dns+nft enforcement): ran the upstream test_openshell.py and test_opensandbox.py unit suites from main against the installed PyPI apache-airflow-providers-common-ai==1.0.0rc1 package (Airflow 3.3.2, Python 3.12.12, macOS; openshell==0.1.4, opensandbox==1.1.1).

   • OpenShell: 143 passed; 18 Linux-only guest-wrapper cases skipped because they require /proc and setsid.
   • OpenSandbox: all 85 passed, including enforced-policy read-back, rejecting unenforced/address-layer policies with sandbox destruction, and owned-transport behavior with proxy environment variables.
   • Total: 228 passed, 18 skipped, no failures (5.75 s).

   These are unit results with mocked services, not live OpenShell/OpenSandbox service verification. Together with the live Islo/Boat smoke checks above, this reports the available verification for all five @zozo123 entries; #71676 was already checked. Leaving release checklist acceptance to the release manager.

   🚀React with 🚀1Reacted by Jarek Potiuk
7. kaxil commented on Oct 17, 2026

   

   kaxil

   on Oct 17, 2026

   Last edited by kaxil

   Member

   More actions

   Tested common.ai 1.0.0rc1 from the PyPI wheel on Airflow 3.3.2 (Python 3.12), with live runs against Claude Sonnet 5.5, TypeSafe Jev and Modal sandboxes. Also ran the Airflow 2.11.2 path, and Airflow 3.4.0b1 for the 3.4-only features.

   Works as described in the PRs:

   • Add AirflowDurability capability for durable agent replay across task retries #74385, Graduate durable agent execution to stable in Common AI #74704, Fix Common AI durable retries not replaying tools from a capability without an id #74314, Name the tools a durable agent retry will run again #73873, Fix durable agent retries replaying the model's rejected answers instead of asking again #74720, Fix Common AI durable retries and tool-approval resumes on pydantic-ai 2.50+ #74313 (durable execution): a retry replays completed model steps and tool calls instead of running them again (a side-effecting tool ran once across two tries). A tool whose result couldn't be recorded is named and runs again. Answers the output validator rejected are asked again, not replayed. Usage with audio_seconds loads on every resume.
   • Bound LLM operator usage_limits across task retries #74396, Account for AgentOperator spend on failed runs and across retries #73706 (usage and spend across retries): attempt 2 hits UsageLimitExceeded early because attempt 1's requests count, and the failed attempt's usage XCom holds its own usage.
   • Add include_traceback to model-backed retry policies #74308, Graduate ClassifierRetryPolicy to stable in Common AI #74707 (retry policies): with include_traceback=True the chained traceback reaches the classifier; ClassifierRetryPolicy on Jev picks retry or fail per category and escalates a below-bar answer to LLMRetryPolicy.
   • Cache repeated agent prompts by default in AgentOperator #73994 (prompt caching): a repeated task reads from the cache, and cache_prompt=False turns it off.
   • Add Anthropic tool runner adapter for common.ai toolsets #73962, Rework common.ai Strands support into a plugin and add Google ADK #73898, Add Strands Agents support for common.ai toolsets #73587, Count and trace tool calls made by native agent frameworks #73901, Resume Strands agents from their last checkpoint when a task retries #74404 (framework adapters): the Anthropic tool runner, Strands and Google ADK examples run unmodified, with masked tool results and common_ai.tool_calls counted per framework. A Strands task resumes from its checkpoint on retry and re-runs only the failed tool.
   • Add a vendor-neutral managed-agent hook contract to Common AI #73532 (managed-agent contract): exercised with a stub implementation of the hook contract, including failover and refusals. The Bedrock and Vertex hooks themselves were not exercised.
   • Add human approval for agent tool calls #73586, Let an agent ask for tool approval in more than one step #74686, Return an approved agent answer as output_type after human review #73904, Reject unsupported output_type and bound HITL XCom pull retries #74450, Let the HITL review plugin address one loop pass or mapped slot #74351, Expose the resolved LLM model name on a dedicated XCom key #74272, Log capability tool calls in AgentOperator #72202 (tool approval and HITL), under a real scheduler and API server: approve and reject both behave, multi-step approval works on 3.4.0b1 (and fails cleanly on 3.3 as documented), the approved typed answer reaches a downstream task, an unreadable review XCom fails after 10 reads, and review by map index and by loop pass touches only that slot.
   • Add ObjectStorageToolset for reading files on object storage #73899, Tell the model how many rows a Parquet or Avro file holds #73925, Decode base64 for bytes-typed HookToolset parameters #70433, Let HookToolset pin arguments the model must not choose #73900, Tell the model a pinned hook argument is fixed for every allowed method #74380, Rename HookToolset tool_name_prefix to tool_prefix #74447, Export DataFusionToolset, AgentSkillsToolset and LoggingToolset… #74449, Let Common AI toolsets use the agent's tool retry budget #73957, Make SQLToolset check_query follow allow_writes like query #74448, Mask secrets in common.ai tool results before they reach the model #73897, Mask tool output from every capability in Common AI agents #73993, Make execute_tool the public method AirflowToolset subclasses implement #73938, Add capabilities to Common AI AgentOperator #73984, Remove code_mode from AgentOperator in favor of the CodeMode capability #74312, Add embedding kwargs to common AI hooks and operators #72002, Validate the connection type in PydanticAIHook.get_hook #74626 (toolsets): checked live with an agent where the model is involved. Secrets in tool output reach the model as *** from toolsets, capabilities and retry messages alike. Stop rejecting valid non-ASCII Anthropic batches as too large #74662 was checked offline (the batch submission itself was stubbed).
   • Add Modal provider with ModalHook and connection type #73418, Let the Modal sandbox backend read credentials from a modal connection #74302, Export files an agent built in a sandbox to object storage #73990, Let the sandbox toolset attach to a sandbox another task provisioned #73559 (Modal): the modal connection authenticates with no ambient credentials, and an agent ran commands, exported a file to object storage, and attached to a sandbox another task provisioned. (The Modal provider was 0.1.0rc1 in this environment.)
   • Move Common AI provider from incubation to production lifecycle #74705, Mark experimental common.ai features ahead of 1.0 #73896, Graduate Common AI features to stable ahead of 1.0 #74679 (lifecycle): production lifecycle, and no experimental warnings on the graduated features. The old usage in each 1.0.0 migration note fails as described and the new usage works; two notes overstate details (below).

   Found, with fixes in open PRs for 1.0.1. None of them blocks this release:

   • Add max_retries to AgentSkillsToolset #74381: AgentSkillsToolset itself works, but a fresh [skills] install resolves pydantic-ai-skills 2.1.0, which removed SkillsToolset, so every run fails. 0.10.0 is affected the same way. Cap pydantic-ai-skills below 2 for the Common AI skills extra #74899 caps it below 2.
   • Add a Model tab to the task-instance page for common.ai #74334: the Model tab never loads on 3.4.0b1, because it scopes on operator_names and core expects operator_name. Only affects 3.4, which is in beta. Fix Common AI Model tab not loading on Airflow 3.4 #74898.
   • durable=True with duplicate toolset ids, or a tool that defers, fails with pydantic-ai errors instead of the documented ones. Reject duplicate durable toolset ids and fail deferred tools with UnsupportedToolDeferralError #74902, which also makes the changelog's "unique id=" note true.
   • The changelog's get_schema note says to check truncated first, but a full result has no truncated key. Fix stale Common AI docs and add a guide to writing a toolset #74904.
   • Stale rows on the supported services page (Add a generated supported services page for the common.ai provider #72939) and other docs fixes: Fix stale Common AI docs and add a guide to writing a toolset #74904. Example Dags that never reach their feature against a real model: Make Common AI example Dags reach the behaviour they show #74901. The pydantic-ai first-run banner in task logs when Airflow is started from a coding agent: Keep the pydantic-ai first-run banner out of Common AI task logs #74900.

   🚀React with 🚀1Reacted by Shahar Epstein
8. justinpakzad commented on Oct 17, 2026

   

   justinpakzad

   on Oct 17, 2026

   Contributor

   More actions

   #74000 has been tested and is working as expected.

   🚀React with 🚀1Reacted by Jarek Potiuk
9. SEPURI-SAI-KRISHNA commented on Oct 17, 2026

   

   SEPURI-SAI-KRISHNA

   on Oct 17, 2026

   Contributor

   More actions

   Verified dbt.cloud 4.11.0rc2 for #74199 on Airflow 3.4.0, Python 3.12.3, tenacity 9.1.4.

   One script run unchanged against three provider versions, with DbtCloudRunJobOperator(deferrable=True, hook_params={"retry_limit": 7, "retry_args": {"stop": stop_after_attempt(3)}}) and the hook mocked, so this covers the serialization boundary rather than a live dbt Cloud run. In all three the task defers and the job is recorded as started before it does.

   • 4.10.0: the trigger receives no hook_params, so the hook in the trigger is built with none of them. Serializing the trigger arguments succeeds.
   • 4.11.0rc1: the trigger receives retry_args and retry_limit, and putting the trigger arguments through airflow.sdk.serde.serialize, which is what Trigger.encrypt_kwargs uses to store them, raises TypeError: Cannot serialize object of type <class 'tenacity.stop.stop_after_attempt'>. The job is already started at that point, which is the orphaned run that kept dbt.cloud out of the previous wave.
   • 4.11.0rc2: the trigger keeps retry_limit, logs a warning naming retry_args as dropped, and the trigger arguments serialize. DbtCloudHook builds from what survived.

   Only the values that fail the JSON check are dropped. Given retry_limit, retry_delay and a tenacity retry_args together, rc2 keeps retry_limit and retry_delay and drops only retry_args. The operator half keeps all three, since the operator builds its own hook from the unfiltered mapping. The trigger docstring describes this.

   operators/dbt.py and triggers/dbt.py in the rc2 wheel are byte identical by sha256 to the same files at current main 20ccfcd7457a, and they are the only source files #74199 and #74489 changed between them.

   🎉React with 🎉1Reacted by Jarek Potiuk
10. takayoshi-makabe commented on Oct 17, 2026

    

    takayoshi-makabe

    on Oct 17, 2026

    Contributor

    More actions

    Verified #69739 with apache-airflow-providers-dbt-cloud==4.11.0rc2 (Airflow 3.3.2, Python 3.12).

    Ran the same script against 4.10.0 and 4.11.0rc2, calling execute_complete directly on DbtCloudRunJobOperator(deferrable=True) and DbtCloudJobRunSensor(deferrable=True) with the hook mocked:

    • event=None: TypeError: 'NoneType' object is not subscriptable on 4.10.0 -> DbtCloudTriggerEventException on rc2 (operator and sensor)
    • unknown status ("foo"): marked success on 4.10.0 -> raises DbtCloudTriggerEventException on rc2 (operator and sensor)
    • sensor status="timeout": marked success on 4.10.0 -> DbtCloudJobRunException with the trigger's message on rc2
    • sensor error/cancelled: empty AirflowException on 4.10.0 -> DbtCloudJobRunException with the trigger's message on rc2
    • status="success" still succeeds, and the operator's timeout/error/cancelled handling is unchanged

    Works as expected.

    ❤️React with ❤️1Reacted by Jarek Potiuk
11. AlejandroMorgante commented on Oct 17, 2026

    

    AlejandroMorgante

    on Oct 17, 2026

    Contributor

    More actions

    Validated the following changes included in the current provider release candidates:

    • apache-airflow-providers-google==22.7.0rc2 — Fix GKE operator imports with older Kubernetes providers #74505: verified compatibility with Kubernetes provider 10.21.0: existing GKE operators remain usable, while GKEPodExecOperator raises the expected dependency-version error with the original import error preserved. Also verified with Kubernetes 10.24.0; focused regression tests passed (13 passed in 6.63s).
    • apache-airflow-providers-google==22.7.0rc2 — Add GKEPodExecOperator for existing Pods #72577: example_kubernetes_engine.py passed against real GKE Autopilot (1 passed in 1073.04s), including command execution in an existing Pod, XCom output verification, and Pod/cluster cleanup. An independent cluster listing confirmed cleanup. The first attempt hit a Pod scheduling timeout; the unchanged system test passed on rerun without increasing timeouts.

    Environment: Breeze, Airflow 3.3.2, Python 3.11.16; Google provider installed from the PyPI RC wheel, Kubernetes 10.21.0 for compatibility checks and 10.24.0 for regression/system tests.

    👍React with 👍1Reacted by Jarek Potiuk
12. X33834 commented on Oct 17, 2026

    

    X33834

    on Oct 17, 2026

    Contributor

    More actions

    Unable to verify #73099 at this time — I don't have the Airflow + provider rc environment available to run the test. Please uncheck the box for #73099; I'll revisit if I can set up the test environment later. Apologies for the delay.
13. potiuk commented on Oct 17, 2026

    

    potiuk

    on Oct 17, 2026

    MemberAuthor

    More actions

    The providers prepared on 2026-10-14 are released: amazon 9.38.0, common.compat 1.22.0, dbt.cloud 4.11.0 and google 22.7.0. common.ai 1.0.0 was excluded and will follow as 1.0.0rc2, with the pydantic-ai-skills cap from #74899, in a separate ad-hoc wave with an accelerated vote. Thanks to everyone who tested the release candidates!

    ---

    Drafted-by: Claude Code (Opus 5); reviewed by @potiuk before posting
14. potiuk

    closed this as completedon Oct 17, 2026
15. potiuk

    added a commit that references this issue on Oct 17, 2026

    Fix stale Common AI docs and add a guide to writing a toolset (#74904)

    ...

    70363ba