Update the Logstash output doc with guidance on using a single ES output

[vishaangelova]

Summary

This PR adds a Use a single Elasticsearch output section documenting an alternative to the dual-output example in the Fleet Logstash output doc.

Closes #7854

Verification

Used Claude to check against logstash-plugins/logstash-output-elasticsearch at HEAD:

• data_stream_event_action_tuple calls the same common_event_params, which reads [@metadata][_ingest_document], so this works with data_stream => true.
• The behavior landed in Elasticsearch output plugin 11.21.0; Logstash 8.12.0 is the first release that bundles it.
• resolve_document_id prefers an explicit document_id, so the two must not be combined. Documented as a caveat.
• The elastic_integration filter replaces the whole [@metadata][_ingest_document] map via event.setField, so the mutate must come after it. Also documented.

Reviewer note

[@metadata][_ingest_document][id] is documented as an output of the elastic_integration filter, but the Elasticsearch output plugin's docs never mention that it consumes the field. Documenting it here makes an internal contract semi-public, so I'd like a Logstash maintainer (cc @robbavey, per the issue) to confirm this is acceptable before merge.

A follow-up PR will apply the same treatment to docs/reference/filebeat/filebeat-deduplication.md in elastic/beats, which carries the identical dual-output pattern.

Generative AI disclosure

1. Did you use a generative AI (GenAI) tool to assist in creating this contribution?

• Yes
• No

Tool(s) and model(s) used: Cursor (Claude Opus 5)

[github-actions]

Elastic Docs AI PR menu

Check the box to run an AI review for this pull request.

• Review docs changes (docs-review). Status: not started.

Powered by GitHub Agentic Workflows and docs-actions. For more information, reach out to the docs team.

[github-actions]

🔍 Preview links for changed docs

• reference/fleet/logstash-output.md

[github-actions]

Elastic Docs Style Checker (Vale)

Summary: 10 warnings, 1 suggestion found

⚠️ Warnings (10): Fix when the suggestion improves clarity or correctness.

File	Line	Rule	Message
reference/fleet/logstash-output.md	96	Elastic.MenuArrows	Use '→' to separate menu items, not '' or '='. Example: Select Manage index → Add lifecycle policy.
reference/fleet/logstash-output.md	96	Elastic.MenuArrows	Use '→' to separate menu items, not '' or '='. Example: Select Manage index → Add lifecycle policy.
reference/fleet/logstash-output.md	103	Elastic.MenuArrows	Use '→' to separate menu items, not '' or '='. Example: Select Manage index → Add lifecycle policy.
reference/fleet/logstash-output.md	103	Elastic.QuotesPunctuation	Place punctuation inside closing quotation marks.
reference/fleet/logstash-output.md	104	Elastic.MenuArrows	Use '→' to separate menu items, not '' or '='. Example: Select Manage index → Add lifecycle policy.
reference/fleet/logstash-output.md	105	Elastic.MenuArrows	Use '→' to separate menu items, not '' or '='. Example: Select Manage index → Add lifecycle policy.
reference/fleet/logstash-output.md	106	Elastic.MenuArrows	Use '→' to separate menu items, not '' or '='. Example: Select Manage index → Add lifecycle policy.
reference/fleet/logstash-output.md	107	Elastic.MenuArrows	Use '→' to separate menu items, not '' or '='. Example: Select Manage index → Add lifecycle policy.
reference/fleet/logstash-output.md	108	Elastic.MenuArrows	Use '→' to separate menu items, not '' or '='. Example: Select Manage index → Add lifecycle policy.
reference/fleet/logstash-output.md	109	Elastic.MenuArrows	Use '→' to separate menu items, not '' or '='. Example: Select Manage index → Add lifecycle policy.

💡 Suggestions (1): Optional style improvements. Apply when helpful.

File	Line	Rule	Message
reference/fleet/logstash-output.md	104	Elastic.Ellipses	Use ellipses sparingly. Remove the ellipsis unless it appears in UI text.

---

The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale.

[alexandra5000]

LGTM

[vishaangelova]

@robbavey, could you or someone from your team have a look at the changes in this PR? Thanks!

[lucabelluccini]

I think a Logstash developer should review this. We are relying on [@metadata][_ingest_document][id] which is not really well publicly documented.

[lucabelluccini]

I do not know if we should protect this copy by if [@metadata][_id] { ...

[vishaangelova]

Updated in 3996c3b.

[lucabelluccini]

If one uses the elastic_integration for all the events, the mutate is un-necessary.

If one uses the elastic_integration only on a part of the data, then we would need something like this (pseudocode):

if [... criteria to use elastic_integration] {
  elastic_integration {
    ...
  }
} else {
  // Here we have the events which are not processed by elastic_integration
  mutate {
    copy => { "[@metadata][_id]" => "[@metadata][_ingest_document][id]" }
  }
}

[vishaangelova]

I used Claude to research this a bit. It checked logstash-filter-elastic_integration and logstash-output-elasticsearch at HEAD, and flagged the following:

The filter does not copy Agent [@metadata][_id] into [@metadata][_ingest_document][id]. It writes that field only when the ingest pipeline sets _id, and it replaces the whole _ingest_document map, so a mutate before the filter is discarded. The Elasticsearch output reads _ingest_document[id], not [@metadata][_id].

So omitting mutate when every event goes through the filter would drop Agent IDs unless the ingest pipeline also set _id. An unconditional copy after the filter would overwrite an ID the pipeline did set.

Updated in 3996c3b: the section now uses one condition that covers both cases.

[robbavey]

@vishaangelova @lucabelluccini Let's chat about this