Skip to content

fix(#9627): cap every read of the AI Assistant's portal and gateway answers - #9754

Merged
robfrank merged 4 commits into
mainfrom
fix/9627-ai-assistant-unbounded-portal-reads
Oct 11, 2026
Merged

robfrank merged 4 commits into
mainfrom
fix/9627-ai-assistant-unbounded-portal-reads

Conversation

@robfrank

@robfrank robfrank commented Oct 11, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #9627

The AI Assistant read every answer from the customer portal and the AI gateway with no byte cap. Stream lines (the portal's NDJSON turn and the gateway's SSE chat) went through BufferedReader.readLine(), which accumulates a line until it sees a newline, and the silence bound does not end a peer that keeps sending. The streamed chat's two refusal bodies used readAllBytes(), and the four buffered gateway calls used BodyHandlers.ofString(), bounded in time, not in bytes. This PR applies the bounds the support module already has:

  • Stream lines: SupportLineReader gets a strict variant, SupportLineReader.failingOver(reader, maxLine), which throws LineTooLongException as soon as a line goes past the cap, without reading the rest of it (the lenient log-reader mode is unchanged). AiPortalChat.readTurn and AiChatHandler.handleStreamingRequest read through it with AiPortalChat.MAX_EVENT_CHARS (4M chars). On the portal path an over-long line ends the turn as SupportPortalException("portal_error"). On the gateway path it ends the SSE stream in band, the same way any other mid-stream failure does.
  • Refusal bodies of the streamed chat (401/403 and non-200): readNBytes(64 KB), the same as SupportPortalClient.openStream.
  • Buffered gateway calls (callGateway of chat and analyze-profiler, /api/activate, the tool-result POST): SupportPortalClient.LimitedStringBody with the 16 MB SupportPortalClient.MAX_RESPONSE_BYTES. The class is now public and takes a peer name, so its error says "The AI gateway answered with more than ... KB".

Test plan

  • SupportLineReaderTest (new, 4): strict reader stops within one chunk of the cap on an endless line; lines up to the cap read unchanged; the cap applies across chunk boundaries; the lenient mode still cuts and carries on.
  • AiPortalChatTest (+2): a 32 MB newline-free turn line ends the turn as portal_error, and the fake portal could deliver less than half of it. A line under the cap reads as before.
  • Issue9627AiGatewayUnboundedReadTest (new, 12; 8 at first push, 4 added in review): for each gateway read, a gateway that floods 48 MB is answered correctly and gets cut off before delivering half of it. The 8th test checks that an answer under the cap is unchanged.
  • All 8 regression tests fail when the read sites are reverted to BufferedReader / readAllBytes / ofString (verified locally), and pass with the fix.
  • All server/.../ai/*Test and server/.../support/*Test classes plus HardcodedTestServerPortsTest: 355 run, all pass except AiSchemaDigestTest.aSchemaOfFiveHundredTypesStaysSmallAndFast. That test fails locally with Too many open files (macOS maxfiles soft limit 256) while creating 500 types, and it touches no changed class.

Coverage

Invariant: no read of an answer from the AI portal or gateway buffers more than a fixed cap on the heap.

Sweep: grep -rnE "BodyHandlers\.of(String|InputStream|ByteArray)\(\)|readAllBytes\(\)|readLine\(\)" server/src/main/java/com/arcadedb/server/ai server/src/main/java/com/arcadedb/server/support

Entry point Read Status
AiPortalChat.readTurn (portal NDJSON) readLine() fixed, AiPortalChatTest
AiChatHandler.handleStreamingRequest SSE loop readLine() fixed, aStreamedChatLineLongerThanTheCapEndsTheStream
AiChatHandler.handleStreamingRequest 401/403 body readAllBytes() fixed, aStreamedChatUnauthorizedWithAnEndlessBodyReadsOnlyItsStart
AiChatHandler.handleStreamingRequest non-200 body readAllBytes() fixed, aStreamedChatRefusedWithAnEndlessBodyReadsOnlyItsStart
AiChatHandler.postToolResult ofString() fixed, aToolResultAnsweredWithAnEndlessBodyIsCapped
AiChatHandler.callGateway ofString() fixed, aBufferedChatAnswerLargerThanTheCapIsRefused
AiAnalyzeProfilerHandler.callGateway ofString() fixed, aProfilerAnalysisAnswerLargerThanTheCapIsRefused
AiActivateHandler /api/activate ofString() fixed, anActivationAnswerLargerThanTheCapIsRefused
AiPortalClient.send/stop/usage SupportPortalClient.call already capped (LimitedStringBody, 16 MB)
SupportPortalClient.openStream refusal body readNBytes(64 KB) already capped
AiConfiguration, ChatStorage Files.readAllBytes local files this server wrote, not peer text, so not in scope

Same-shape reads outside the AI module (ha-raft peer queries, PostBatchHandler, LeaderCommandForwarder, RemoteDatabase) read from authenticated cluster peers or, client-side, from an ArcadeDB server. They are not a third-party vendor stream, so this issue does not cover them.

Known gaps: None.

Adversarial pass

No independent subagent ran: this session had no Task/Agent tool. The author did a self-check instead:

  • InputStreamReader.read(char[]) blocking: like BufferedReader.fill, it returns once nothing more is available(), so heartbeats are still relayed as they arrive (the existing Issue8473/Issue8642 streaming tests pass).
  • Memory per stream is now one 64K-char chunk plus at most MAX_EVENT_CHARS. Before, an 8K-char buffer plus an unbounded line.
  • The non-200 exception message still carries up to 64 KB of the body (it is logged, never returned to Studio). It is bounded, so it is left as is.

Review cycles

Cycle Head Changes Bot outcome
1 f677e2d93e initial fix claude: approve with 6 minor suggestions. Applied #1 (a cut or non-JSON 401/403 body now falls back to token_invalid via refusalBody) and #2 (error bodies go into exception messages and logs only as a 1 KB excerpt, which also covers the buffered response.body() messages and the tool-result log line)
2 f5750cf782 refusalBody, excerpt + tests claude: LGTM once #1 is considered. Applied #1: the activation error echoed to the user is excerpted
3 30df31b49e activation excerpt + test claude: 6 suggestions. Applied #4 (catch only JSONException). This also showed that JSONException's message embeds the whole input, so parseAnswer now reports a non-JSON 200 answer by its excerpt only
4 b22385d51e parseAnswer, narrowed catch + test claude: "No blocking issues found", only minor notes. No changes needed

Deferred / skipped items

  • "AiChatHandler.limitedBody() ... could live in a small shared helper or on SupportPortalClient" (cycle 1): cosmetic. It stays next to the other AI gateway bounds (gatewayTimeoutMs and friends).
  • "Mutable statics ... keep an eye on parallel test execution" (cycles 1-4): these follow the existing gatewayTimeoutMs / streamSilenceMs seams, which are documented as JVM-wide and restored in @AfterEach / finally. The server module does not run test classes in parallel.
  • "Cap units: 4M chars can be 4 to 8 MB" (cycle 1): bounded either way, so this is a note only.
  • "Gateway path test / confirm coverage of the four call sites" (cycles 1, 3): already covered by Issue9627AiGatewayUnboundedReadTest, one test per call site plus the SSE line.
  • "Comment or shared constant for maxEventChars reuse" (cycles 2, 3): the MAX_EVENT_CHARS javadoc already names both streams ("one NDJSON envelope, one SSE line"), and the SSE read site carries a comment.
  • "@Tag(\"slow\")" (cycles 2, 4): the whole class runs in about 1.5 s, because the readers cut each flood off early.
  • "Keep non-needed members package-private" (cycle 3): already so. The 1-arg LimitedStringBody(long) and the ResponseTooLargeException constructor stay package-private.
  • "Per-event cap versus total" (cycle 3): nothing accumulates across lines. Each delta, SSE event or heartbeat is relayed to the browser as it is read, and only the single result/done event is kept, so the heap per stream is one line at most. Total duration stays under the silence bound and MAX_ROUNDS by design ("a conversation that keeps producing events runs as long as it needs to").
  • "Port from allocateFreePorts" (cycle 4): HardcodedTestServerPortsTest's javadoc explicitly exempts a held-bound new ServerSocket(0, backlog, address) for "the stub gateways of the AI tests". That scan passes with this class included.
  • "GlobalConfiguration entry for the caps" (cycle 4): the inbound SSE lines are tool calls and the final answer. Tool results (such as a large get_schema) go outbound in a POST, so they are not subject to the 4M-char line cap.

Final state

clean-approval after 4 cycles. Merge is the developer's responsibility.

🤖 Generated with Claude Code

…nswers

Stream lines (portal NDJSON turn, gateway SSE) are read through a strict
SupportLineReader that fails at 4M chars instead of BufferedReader.readLine();
refusal bodies of the streamed chat are read with readNBytes(64 KB); the four
buffered gateway calls use LimitedStringBody with the 16 MB support cap.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 8 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 4324c8fb-8e0a-4ea2-8966-16e8df169a60

📥 Commits

Reviewing files that changed from the base of the PR and between 367be85 and b22385d.


📒 Files selected for processing (10)
  • server/src/main/java/com/arcadedb/server/ai/AiActivateHandler.java
  • server/src/main/java/com/arcadedb/server/ai/AiAnalyzeProfilerHandler.java
  • server/src/main/java/com/arcadedb/server/ai/AiChatHandler.java
  • server/src/main/java/com/arcadedb/server/ai/AiPortalChat.java
  • server/src/main/java/com/arcadedb/server/support/SupportLineReader.java
  • server/src/main/java/com/arcadedb/server/support/SupportPortalClient.java
  • server/src/test/java/com/arcadedb/server/ai/AiPortalChatTest.java
  • server/src/test/java/com/arcadedb/server/ai/FakeAiPortal.java
  • server/src/test/java/com/arcadedb/server/ai/Issue9627AiGatewayUnboundedReadTest.java
  • server/src/test/java/com/arcadedb/server/support/SupportLineReaderTest.java

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@mergify

mergify Bot commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

This pull request does not currently match the merge queue conditions, so it cannot be queued from here. The box comes back if it matches again.

@codacy-production

codacy-production Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

Not up to standards ⛔

🔴 Issues 2 medium

Alerts:
⚠ 2 issues (≤ 0 issues of at least minor severity)

Results:
2 new issues

Category Results
BestPractice 2 medium

View in Codacy

🟢 Metrics 14 complexity

Metric Results
Complexity 14

View in Codacy

🟢 Coverage 90.74% diff coverage

Metric Results
Coverage variation Report missing for 367be851
Diff coverage ✅ 90.74% diff coverage

View coverage diff in Codacy

Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (367be85) Report Missing Report Missing Report Missing
Head commit (b22385d) 262080 211469 80.69%

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#9754) 54 49 90.74%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

1 Codacy didn't receive coverage data for the commit, or there was an error processing the received data. Check your integration for errors and validate that your coverage setup is correct.

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@claude

claude Bot commented Oct 11, 2026

Copy link
Copy Markdown

Review

Overall this is a solid, well-scoped fix for #9627. It reuses the existing SupportLineReader and LimitedStringBody instead of adding new machinery, and the tests exercise the real failure mode (an endless newline-free line, with a check that the stream was closed early).

What looks good

  • The strict mode throws before appending, so the reader holds at most maxLine chars plus one chunk. The length > room check is correct at the boundary: a line of exactly the cap passes, and one more char fails.
  • The lenient log-reader behavior is unchanged, and the tests cover that.
  • The new LimitedStringBody(max, peer) constructor keeps the old package-private one, so the existing call sites are untouched.
  • The tests follow the project style (assertThat, no raw wall-clock assertions). The static test knobs are restored in finally.

Suggestions (minor)

  1. Truncated refusal JSON: the 401/403 path now does readNBytes(64 KB) and then new JSONObject(bodyStr). If a body is cut mid-JSON, the parse throws and the token_invalid/token_expired code is lost. Real refusals are tiny, so this is unlikely. A fallback to the default code when the parse fails would be more robust. Check that the existing catch already does this.
  2. Non-200 error text: the "Gateway returned status N: " + bodyStr message can now carry up to 64 KB of peer-controlled text into the exception, and probably into the client response and logs. Consider truncating it to roughly 1 to 2 KB for display.
  3. Visibility: AiChatHandler.limitedBody() is called from AiActivateHandler and AiAnalyzeProfilerHandler, so it is in effect shared. It could live in a small shared helper, or on SupportPortalClient, rather than on a handler class. This is cosmetic.
  4. Mutable statics: maxResponseBytes and maxEventChars are JVM-wide volatile statics that tests mutate. This is already documented and restored, so it is fine. Keep an eye on parallel test execution, because tests that mutate them cannot run concurrently with others that read them.
  5. Cap units: MAX_EVENT_CHARS is 4M chars, but a StringBuilder of 4M chars can be 4 to 8 MB (Latin-1 or UTF-16). With concurrent streams this is still bounded, so I see no problem, only a note.
  6. Gateway path test: the PR description says the gateway SSE stream ends in band on an over-long line. Please confirm that Issue9627AiGatewayUnboundedReadTest covers both the stream path and the 16 MB buffered path for each of the four call sites (chat, profiler, activate, tool-result).

No security or performance regressions found. The change is bounded and adds no new dependencies. Looks good to merge once the points above are considered.

…ror bodies in messages

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@claude

claude Bot commented Oct 11, 2026

Copy link
Copy Markdown

Review

Solid, well-scoped fix for #9627. It reuses the existing support-module bounds (SupportLineReader, LimitedStringBody) instead of new machinery, and the tests assert the right thing: the peer could not deliver most of the flood.

Good

  • Strict failingOver throws before the rest of the line is read (at most maxLine plus one chunk); the lenient log-reader mode is untouched.
  • refusalBody/excerpt keep the 401/403 -> token_invalid fallback working on cut or non-JSON bodies, and keep peer text out of messages and logs.
  • Regression tests use finite floods with shortened caps and assert on bytes sent, not wall-clock time, matching CLAUDE.md.

Minor, non-blocking

  1. AiActivateHandler still echoes errBody.getString("error", ...) to the client unbounded (up to the 16 MB cap) and builds new JSONObject(response.body()) directly. Using AiChatHandler.excerpt(...) and refusalBody would be consistent with the other call sites.
  2. The gateway SSE path takes its cap from AiPortalChat.maxEventChars. Fine, but a short comment or shared constant would show it is deliberate.
  3. The static volatile test knobs are JVM-wide. They are restored in finally/AfterEach, but would need isolation if these classes ever ran in parallel.
  4. Issue9627AiGatewayUnboundedReadTest floods 48 MB in each of about 9 tests. Early close should keep it fast, but consider @Tag("slow") if CI time grows.

No security or performance concerns beyond the above. LGTM once (1) is considered.

…user

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@claude

claude Bot commented Oct 11, 2026

Copy link
Copy Markdown

Review

Overall this is a solid, well-scoped fix for #9627. It reuses the existing LimitedStringBody and SupportLineReader instead of adding new machinery. The strict mode fails fast, without reading the rest of the line. I read the diff only and did not run the build or tests.

What looks good

  • The strict failingOver reader holds at most maxLine plus one chunk. The check runs before the append, so a line of exactly maxLine characters still reads fine.
  • The refusal bodies are now capped (readNBytes(64 KB)), and error text is cut with excerpt() before it reaches exception messages or logs. That also stops a hostile gateway from flooding the logs.
  • refusalBody() falls back to the default code and message when a body is cut or isn't JSON. Before, a truncated 401/403 body would have turned into a parse failure.
  • The tests cover the endless-line case, the boundary at the cap, cap handling across chunk boundaries, and that the lenient mode is unchanged.

Suggestions

  1. Gateway SSE path test. The new tests I could see cover AiPortalChat.readTurn and SupportLineReader. I did not see a test for AiChatHandler.handleStreamingRequest ending the SSE stream in band on an over-long line, or one for the limitedBody() rejection on the buffered calls. Both are claimed in the description. A regression test for each, using the package-private maxEventChars and maxResponseBytes hooks you added, would cover them.
  2. Reuse AiPortalChat.maxEventChars from AiChatHandler. The gateway handler reads the portal-chat class's mutable static. That works, but the shared cap now lives in a class that is only conceptually related. A small shared constant, or a comment saying the two are intentionally tied, would help.
  3. Widened visibility. SupportLineReader, LimitedStringBody and ResponseTooLargeException are now public. That is needed for cross-package use. Keep the constructors and methods that don't need to be public package-private, e.g. the 1-arg LimitedStringBody(long), which is only used internally.
  4. refusalBody catches RuntimeException. It is fine here, but consider catching only the JSON parse exception so unrelated bugs aren't hidden.
  5. Mutable static volatile test hooks. They are JVM-wide, and your try/finally restore handles that. Ideally the fixtures would inject these values per instance so parallel test classes can't interfere. Not a blocker.
  6. Per-event cap versus total. Each line is capped at 4M chars, but a peer can still send an unbounded number of valid lines. The silence bound and the MAX_ROUNDS limit probably cover this, but the description doesn't say what bounds the total bytes of a stream. It may be worth stating that explicitly or adding a total budget.

No security concerns beyond the above, and no new dependencies. The code follows the CLAUDE.md conventions I checked (final usage, no fully qualified names, test syntax).

…answer by its start

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@claude

claude Bot commented Oct 11, 2026

Copy link
Copy Markdown

Review

Solid, well-scoped fix for #9627. It reuses the existing LimitedStringBody and SupportLineReader instead of adding new machinery, and every unbounded read I could find in server/.../ai/ is now bounded (a grep for ofString() handlers and readAllBytes finds only the request publishers and local config files).

What looks good

  • The strict failingOver mode throws before appending, so the heap holds at most maxLine plus one chunk. The lenient log-reader behavior is unchanged.
  • refusalBody / parseAnswer / excerpt keep a truncated or non-JSON body from failing on the parse or flooding logs and user-visible errors. The 401 path still falls back to the default token_invalid code.
  • The tests assert both the server's answer and that the peer could not deliver most of the flood, so the fix is proven rather than just the status code. The cap overrides are restored in finally / @AfterEach.

Minor suggestions (non-blocking)

  1. LineTooLongException in the gateway SSE path: confirm the in-band error message sent to the client does not echo exception text beyond the generic message. The text is fixed here, so it is probably fine.
  2. AiChatHandler.maxResponseBytes and AiPortalChat.maxEventChars are JVM-wide mutable statics used only as test seams. That works, but anything else that runs in parallel in the same JVM would see the shortened caps. Check that these test classes do not run concurrently with other AI tests.
  3. Issue9627AiGatewayUnboundedReadTest streams up to 48 MB per test through a raw ServerSocket. Check that its port comes from allocateFreePorts, as CLAUDE.md requires. If it binds an ephemeral port via new ServerSocket(0), that goes against the guidance. Consider @Tag("slow") if the class takes noticeable time.
  4. The 4M-char event cap and the 16 MB buffered cap are hardcoded constants. That is fine for now, but a GlobalConfiguration entry would make them tunable for unusually large tool results. For example, a large get_schema payload inside a single SSE line must stay under 4M chars.

No blocking issues found. LGTM once point 3 is confirmed.

@codecov

codecov Bot commented Oct 11, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 90.74074% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 77.73%. Comparing base (8c7c999) to head (b22385d).
⚠️ Report is 11 commits behind head on main.

Files with missing lines Patch % Lines
...m/arcadedb/server/ai/AiAnalyzeProfilerHandler.java 25.00% 3 Missing ⚠️
...ain/java/com/arcadedb/server/ai/AiChatHandler.java 91.30% 2 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #9754      +/-   ##
============================================
+ Coverage     77.72%   77.73%   +0.01%     
  Complexity     4438     4438              
============================================
  Files          2195     2195              
  Lines        229358   229463     +105     
  Branches      49632    49655      +23     
============================================
+ Hits         178267   178378     +111     
+ Misses        32203    32187      -16     
- Partials      18888    18898      +10     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant