Skip to content

Handles flow exceptions on EcsRunTaskOperator using provided exit codes - #47256

Closed
mariotaddeucci wants to merge 7 commits into
apache:mainfrom
mariotaddeucci:feature/airflow-aws-ecs-exceptions-by-exit-code
Closed

Handles flow exceptions on EcsRunTaskOperator using provided exit codes#47256
mariotaddeucci wants to merge 7 commits into
apache:mainfrom
mariotaddeucci:feature/airflow-aws-ecs-exceptions-by-exit-code

Conversation

@mariotaddeucci

@mariotaddeucci mariotaddeucci commented Mar 1, 2025

Copy link
Copy Markdown
Contributor

Description:
This PR introduces the ability to raise AirflowFailException or AirflowSkipException in EcsRunTaskOperator based on the exit code of the ECS task.

Motivation

Currently, when an ECS task fails with specific exit codes, such as 137 (Out of Memory), the task may be retried unnecessarily. This can lead to resource wastage and inefficient execution.

With this change, users can define exit codes that should trigger an AirflowFailException, immediately failing the task and preventing retries. Additionally, they can specify exit codes that should raise an AirflowSkipException, allowing the DAG to skip the task and avoid consuming unnecessary resources.

Changes

  • Added support for mapping exit codes to AirflowFailException or AirflowSkipException in EcsRunTaskOperator.
  • Introduced new parameters:
    • fail_on_exit_codes: List of exit codes that trigger an AirflowFailException.
    • skip_on_exit_codes: List of exit codes that trigger an AirflowSkipException.
  • Updated documentation and tests to cover the new functionality.

Example Usage

EcsRunTaskOperator(
    task_id="ecs_task",
    cluster="my-cluster",
    task_definition="my-task-def",
    overrides={},
    launch_type="FARGATE",
    fail_on_exit_codes=[137],
    skip_on_exit_codes=[2]
)

@boring-cyborg boring-cyborg Bot added the provider:amazon AWS/Amazon - related issues label Mar 1, 2025

@o-nikolas o-nikolas left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you done some manual testing in real DAGs that show this is working as you expect?

Comment on lines +396 to +398
:param skip_on_exit_codes: List of exit codes which will make the task to be marked as skipped with AirflowSkipException.
:param fail_on_exit_codes: List of exit codes which will make the task to be immediately marked as failed with AirflowFailException.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
:param skip_on_exit_codes: List of exit codes which will make the task to be marked as skipped with AirflowSkipException.
:param fail_on_exit_codes: List of exit codes which will make the task to be immediately marked as failed with AirflowFailException.
:param skip_on_exit_codes: List of exit codes which will cause the task to be marked as skipped with AirflowSkipException.
:param fail_on_exit_codes: List of exit codes which will cause the task to be immediately marked as failed with AirflowFailException.

@mariotaddeucci

Copy link
Copy Markdown
Contributor Author

Have you done some manual testing in real DAGs that show this is working as you expect?

Yes It Works, in fact Im running this in production. It follows the same aproach of BashOperator but instead of using only one exit code, this passes a list of exist code

@eladkal

eladkal commented Apr 16, 2025

Copy link
Copy Markdown
Contributor

@mariotaddeucci can you rebase and resolve conflicts?

@github-actions

github-actions Bot commented Jun 1, 2025

Copy link
Copy Markdown
Contributor

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.

@github-actions github-actions Bot added the stale Stale PRs per the .github/workflows/stale.yml policy file label Jun 1, 2025
@github-actions github-actions Bot closed this Jun 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

provider:amazon AWS/Amazon - related issues stale Stale PRs per the .github/workflows/stale.yml policy file

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants