HADOOP-19993. Fix SpotBugs SE_BAD_FIELD in McpHttpServlet by jojochuang · Pull Request #8753 · apache/hadoop

jojochuang · 2026-09-25T03:29:46Z

Summary

Mark only requestHandler in McpHttpServlet as transient to fix SE_BAD_FIELD. Keep objectMapper non-transient because ObjectMapper implements Serializable, avoiding SE_TRANSIENT_FIELD_NOT_RESTORED and reducing the SpotBugs warning count to zero.
Unblocks Yetus spotbugs-strict-precheck for PRs that touch modules depending on hadoop-common (e.g. HDFS-17981 / HDFS-17981. Make the bind address for HDFS NFS Gateway configurable #8744).

Context

JIRA: HADOOP-19993. Fix SpotBugs SE_BAD_FIELD in McpHttpServlet

Follow-up to YARN-11977 (MCP HTTP server in hadoop-common). Trunk currently reports one extant SpotBugs warning:

McpHttpServlet defines non-transient non-serializable instance field requestHandler.

HttpServlet implements Serializable, but these servlet instances are not serialized in normal deployment; transient matches common servlet practice.

Test plan

./mvnw -pl hadoop-common-project/hadoop-common -DskipTests test-compile spotbugs:spotbugs
Yetus trunk spotbugs precheck on PR

https://issues.apache.org/jira/browse/HADOOP-19993

Made with Cursor

Mark MCP servlet dependency fields transient so trunk hadoop-common passes SpotBugs strict precheck (YARN-11977 follow-up). Co-authored-by: Cursor <cursoragent@cursor.com>

slfan1989

LGTM

hadoop-yetus · 2026-09-25T07:34:05Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 58s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	50m 0s		trunk passed
+1 💚	compile	18m 7s		trunk passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu
+1 💚	compile	18m 9s		trunk passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu
+1 💚	checkstyle	1m 28s		trunk passed
+1 💚	mvnsite	2m 0s		trunk passed
+1 💚	javadoc	1m 26s		trunk passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu
+1 💚	javadoc	1m 24s		trunk passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu
-1 ❌	spotbugs	3m 16s	/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html	hadoop-common-project/hadoop-common in trunk has 1 extant spotbugs warnings.
+1 💚	shadedclient	37m 12s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	37m 47s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 15s		the patch passed
+1 💚	compile	16m 57s		the patch passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu
+1 💚	javac	16m 57s		the patch passed
+1 💚	compile	18m 6s		the patch passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu
+1 💚	javac	18m 6s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	1m 26s		the patch passed
+1 💚	mvnsite	1m 59s		the patch passed
+1 💚	javadoc	1m 20s		the patch passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu
+1 💚	javadoc	1m 21s		the patch passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu
-1 ❌	spotbugs	3m 24s	/new-spotbugs-hadoop-common-project_hadoop-common.html	hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 1 fixed = 1 total (was 1)
+1 💚	shadedclient	36m 51s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	24m 4s		hadoop-common in the patch passed.
+1 💚	asflicense	1m 10s		The patch does not generate ASF License warnings.
		243m 2s

Reason	Tests
SpotBugs	module:hadoop-common-project/hadoop-common
	The field org.apache.hadoop.mcp.McpHttpServlet.objectMapper is transient but isn't set by deserialization In McpHttpServlet.java:but isn't set by deserialization In McpHttpServlet.java

Subsystem	Report/Notes
Docker	ClientAPI=1.56 ServerAPI=1.56 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8753/1/artifact/out/Dockerfile
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 5132640f047a 5.15.0-190-generic #200-Ubuntu SMP Fri Aug 7 15:06:04 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `90f0d1d`
Default Java	Ubuntu-17.0.20.1+1-1-24.04-Ubuntu
Multi-JDK versions	/usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.20.1+1-1-24.04-Ubuntu
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8753/1/testReport/
Max. process+thread count	1297 (vs. ulimit of 10000)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8753/1/console
versions	git=2.43.0 maven=3.9.15 spotbugs=4.9.7
Powered by	Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

joseluisll · 2026-09-25T08:38:11Z

Thanks for picking this up. I reproduced the warning locally and I think the patch is right in approach but over-applied by one field: as written it swaps SE_BAD_FIELD for SE_TRANSIENT_FIELD_NOT_RESTORED, so hadoop-common still reports one extant warning and trunk precommit stays red.

Three runs on a clean branch off trunk at 90f0d1da37c, same command each time, reading hadoop-common-project/hadoop-common/target/spotbugsXml.xml (not just the exit code):

./mvnw -pl hadoop-common-project/hadoop-common -DskipTests -P'!native-win' test-compile spotbugs:spotbugs

1. Baseline, unmodified trunk — total_bugs = 1

type='SE_BAD_FIELD' priority='2' rank='16' category='BAD_PRACTICE'
Class: org.apache.hadoop.mcp.McpHttpServlet
Field: name='requestHandler' signature='Lorg/apache/hadoop/mcp/McpRequestHandler;'
Class org.apache.hadoop.mcp.McpHttpServlet defines non-transient non-serializable instance field requestHandler

2. This PR's diff (both fields transient) — total_bugs = 1

type='SE_TRANSIENT_FIELD_NOT_RESTORED' priority='2' rank='16' category='BAD_PRACTICE'
Class: org.apache.hadoop.mcp.McpHttpServlet
Field: name='objectMapper' signature='Lcom/fasterxml/jackson/databind/ObjectMapper;'
The field org.apache.hadoop.mcp.McpHttpServlet.objectMapper is transient but isn't set by deserialization

SE_BAD_FIELD is indeed gone, but the new warning lands on objectMapper.

3. Only requestHandler marked transient — total_bugs = 0

Clean. (total_classes='2648' on all three runs; McpHttpServlet is in the analyzed set each time.)

The reason for the asymmetry: Jackson's ObjectMapper is itself Serializable, so it never tripped SE_BAD_FIELD — which is why trunk reports one warning rather than two. Marking it transient is what introduces SE_TRANSIENT_FIELD_NOT_RESTORED. McpRequestHandler is not Serializable, so transient there is the sanctioned fix and draws no complaint. Both fields being final turned out not to matter to either detector.

So the suggestion is just to drop the objectMapper hunk and keep:

  private final ObjectMapper objectMapper;
  private final transient McpRequestHandler requestHandler;

That also matches what other hadoop-common servlets do — ProfileServlet.process and JMXJsonServlet.mBeanServer / jsonFactory use transient for exactly the non-serializable collaborators and leave the rest alone.

Happy to be wrong if your run shows something different — worth confirming, since the Jenkins run on this PR only checks that the patch is clean, not that the module reaches zero.

ayushtkn

LGTM

joseluisll

@jojochuang SpotBugs on hadoop-common, with this PR as-is and with the suggestion below applied (WSL2 Ubuntu 24.04, OpenJDK 17.0.20, Maven 3.9.16, spotbugs-maven-plugin 4.9.7.0, 2666 classes analyzed in both runs):

Eliminate the "transient" from objectMapper, and thats it.

K0K0V0K · 2026-10-06T09:34:37Z

Hi all!

First of all sorry to blocking so many PR with this previous change.
I did not realised this can cause other PRs spot bug failures.

During the development i realised this "loop" problem and found a solution for this
#8665 (comment)

And found some workaround of the problem in WebAppProxyServlet.java

hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/WebAppProxyServlet.java

Line 727 in dccedc2

* YARN isn't currently serializing this class, but findbugs

Hope this can help.

…adoop/mcp/McpHttpServlet.java ObjectMapper implements Serializable and does not need to be transient. Remove the modifier to avoid SE_TRANSIENT_FIELD_NOT_RESTORED, while keeping requestHandler transient to fix SE_BAD_FIELD. Co-authored-by: Jose Luis López López <joseluisll@gmail.com>

hadoop-yetus · 2026-10-07T05:14:40Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 56s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	51m 52s		trunk passed
+1 💚	compile	17m 50s		trunk passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu
+1 💚	compile	18m 20s		trunk passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu
+1 💚	checkstyle	1m 32s		trunk passed
+1 💚	mvnsite	2m 2s		trunk passed
+1 💚	javadoc	1m 27s		trunk passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu
+1 💚	javadoc	1m 22s		trunk passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu
-1 ❌	spotbugs	3m 16s	/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html	hadoop-common-project/hadoop-common in trunk has 1 extant spotbugs warnings.
+1 💚	shadedclient	36m 40s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 21s		the patch passed
+1 💚	compile	16m 59s		the patch passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu
+1 💚	javac	16m 59s		the patch passed
+1 💚	compile	18m 15s		the patch passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu
+1 💚	javac	18m 15s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	1m 27s		the patch passed
+1 💚	mvnsite	2m 1s		the patch passed
+1 💚	javadoc	1m 23s		the patch passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu
+1 💚	javadoc	1m 23s		the patch passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu
+1 💚	spotbugs	3m 24s		hadoop-common-project/hadoop-common generated 0 new + 0 unchanged - 1 fixed = 0 total (was 1)
+1 💚	shadedclient	37m 2s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	23m 59s		hadoop-common in the patch passed.
+1 💚	asflicense	1m 14s		The patch does not generate ASF License warnings.
		245m 12s

Subsystem	Report/Notes
Docker	ClientAPI=1.56 ServerAPI=1.56 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8753/2/artifact/out/Dockerfile
GITHUB PR	#8753
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 3984e8aaa7aa 5.15.0-190-generic #200-Ubuntu SMP Fri Aug 7 15:06:04 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `7dae3b7`
Default Java	Ubuntu-17.0.20.1+1-1-24.04-Ubuntu
Multi-JDK versions	/usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.20.1+1-1-24.04-Ubuntu
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8753/2/testReport/
Max. process+thread count	1297 (vs. ulimit of 10000)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8753/2/console
versions	git=2.43.0 maven=3.9.15 spotbugs=4.9.7
Powered by	Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

slfan1989 · 2026-10-07T08:28:30Z

@jojochuang Thanks for the contribution! Yetus confirms that the existing SpotBugs warning is fixed with no new warnings, and both the existing unit tests and GitHub Actions Build passed. @ayushtkn @joseluisll @K0K0V0K Thanks for the review!

HADOOP-19993. Fix SpotBugs SE_BAD_FIELD in McpHttpServlet.

14bf7db

Mark MCP servlet dependency fields transient so trunk hadoop-common passes SpotBugs strict precheck (YARN-11977 follow-up). Co-authored-by: Cursor <cursoragent@cursor.com>

github-actions Bot added trunk Common labels Sep 25, 2026

slfan1989 approved these changes Sep 25, 2026

View reviewed changes

deepujain mentioned this pull request Sep 27, 2026

HADOOP-19815. Document Path trailing slash and URI resolution semantics #8307

Merged

5 of 6 tasks

ayushtkn approved these changes Sep 27, 2026

View reviewed changes

joseluisll mentioned this pull request Sep 27, 2026

HADOOP-19979. Fix four flaky tests that race with work on other threads. #8717

Closed

3 of 6 tasks

joseluisll reviewed Sep 27, 2026

View reviewed changes

Comment thread hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/mcp/McpHttpServlet.java Outdated

joseluisll mentioned this pull request Oct 4, 2026

YARN-12005. Fix flaky TestRouterWebServicesREST#testUpdateNodeResource #8788

Merged

3 of 6 tasks

slfan1989 merged commit 1f1b215 into apache:trunk Oct 7, 2026
3 of 6 checks passed

Navigation Menu

Repository navigation

HADOOP-19993. Fix SpotBugs SE_BAD_FIELD in McpHttpServlet - #8753

HADOOP-19993. Fix SpotBugs SE_BAD_FIELD in McpHttpServlet#8753
slfan1989 merged 2 commits into
apache:trunkfrom
jojochuang:HADOOP-19993-spotbugs-mcp

jojochuang commented Sep 25, 2026 •

edited by slfan1989

Loading

Uh oh!

slfan1989 left a comment

Uh oh!

hadoop-yetus commented Sep 25, 2026

Uh oh!

joseluisll commented Sep 25, 2026

Uh oh!

ayushtkn left a comment

Uh oh!

joseluisll left a comment •

edited

Loading

Uh oh!

Uh oh!

K0K0V0K commented Oct 6, 2026

Uh oh!

hadoop-yetus commented Oct 7, 2026

Uh oh!

Uh oh!

slfan1989 commented Oct 7, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

jojochuang commented Sep 25, 2026 • edited by slfan1989 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Context

Test plan

Uh oh!

slfan1989 left a comment

Choose a reason for hiding this comment

Uh oh!

hadoop-yetus commented Sep 25, 2026

Uh oh!

joseluisll commented Sep 25, 2026

Uh oh!

ayushtkn left a comment

Choose a reason for hiding this comment

Uh oh!

joseluisll left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

K0K0V0K commented Oct 6, 2026

Uh oh!

hadoop-yetus commented Oct 7, 2026

Uh oh!

Uh oh!

slfan1989 commented Oct 7, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

jojochuang commented Sep 25, 2026 •

edited by slfan1989

Loading

joseluisll left a comment •

edited

Loading