Support elasticsearch 9

[eladkal]

The elasticsearch provider limits the library version

airflow/providers/elasticsearch/pyproject.toml

Line 65 in be8252e

"elasticsearch>=8.10,<9",

The Task:

1. Test the provider with elasticsearch 9
2. Accommodate for changes/fixes if needed and verify logs appear correctly in elasticsearch as well in Airflow UI (including success and failure cases)
3. Ideally we should support elasticsearch>=8.10 but if it's too much trouble to support both 8 and 9 we can go elasticsearch>=9.0
4. Make modifications where needed to the elasticsearch systemtest in airflow-e2e-tests/tests/airflow_e2e_tests/remote_log_elasticsearch_tests/test_remote_logging_elasticsearch.py
5. Provide proof that the fix works by presenting UI images from both elastic and Airflow UI.
6. Verify the hooks in airflow/providers/elasticsearch/hooks/elasticsearch.py works as expected.

Note to contributors: This task requires knowledge in elasticsearch and ability to test and verify against a real ealsticsearch environment.

[eladkal]

cc @Owen-CH-Leung @Subham-KRLX in case one of you will be interested in fixing this

[laurenhall1526-netizen]

Thanks for the details! I'll test with Elasticsearch 9, adjust for any required changes, and verify logs appear correctly in both Elasticsearch and the Airflow UI. If supporting both 8 and 9 is too complex, I'll stick with elasticsearch>=9.0. I'll also update the test_remote_logging_elasticsearch.py and provide UI screenshots as proof.

…

On Sun, Mar 22, 2026, 10:14 AM Elad Kalif ***@***.***> wrote: *eladkal* created an issue (apache/airflow#64064) <#64064> The elasticsearch provider limits the library version https://github.com/apache/airflow/blob/be8252e6287b550919379cf15c159d82972a6c91/providers/elasticsearch/pyproject.toml#L65 *The Task:* 1. Test the provider with elasticsearch 9 2. Accommodate for changes/fixes if needed and verify logs appear correctly in elasticsearch as well in Airflow UI (including success and failure cases) 3. Ideally we should support elasticsearch>=8.10 but if it's too much trouble to support both 8 and 9 we can go elasticsearch>=9.0 4. Make modifications where needed to the elasticsearch systemtest in airflow-e2e-tests/tests/airflow_e2e_tests/remote_log_elasticsearch_tests/test_remote_logging_elasticsearch.py 5. Provide proof that the fix works by presenting UI images from both elastic and Airflow UI. *Note to contributors:* This task requires knowledge in elasticsearch and ability to test and verify against a real ealsticsearch environment. — Reply to this email directly, view it on GitHub <#64064?email_source=notifications&email_token=B7IHA46HKSAO63RQ4OQYU234R77OJA5CNFSL4Z3JMQ5C6L3HNF2C22DVMIXUS43TOVSS6NBRGE3DGMBRGIYDLJTSMVQXG33OVJZXKYTTMNZGSYTFMSSWK5TFNZ2KYZTPN52GK4S7MNWGSY3L>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/B7IHA4YXPNYMHL6TJWKQ2QT4R77OJAVCNFSM6AAAAACW2YPQ4SVHI2DSMVQWIX3LMV43ASLTON2WKOZUGEYTMMZQGEZDANI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[Subham-KRLX]

cc @Owen-CH-Leung @Subham-KRLX in case one of you will be interested in fixing this

This looks like an interesting task I am currently a bit occupied but I will definitely take a look at this issue when I have some time and see how I can help.

[eladkal]

Noting this may be a small task. I don't know how much changed between 8->9 (in what relevant for Airflow)