Skip to content

[CELEBORN-2239] Support Spark 4.1#3571

Open
SteNicholas wants to merge 1 commit intoapache:mainfrom
SteNicholas:CELEBORN-2239
Open

[CELEBORN-2239] Support Spark 4.1#3571
SteNicholas wants to merge 1 commit intoapache:mainfrom
SteNicholas:CELEBORN-2239

Conversation

@SteNicholas
Copy link
Member

@SteNicholas SteNicholas commented Dec 23, 2025

What changes were proposed in this pull request?

Support Spark 4.1.

Why are the changes needed?

Spark 4.1.1 has already released, which release notes refer to Spark Release 4.1.1.

Does this PR resolve a correctness bug?

No.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

CI.

@SteNicholas
Copy link
Member Author

Ping @pan3793, @turboFei.

@github-actions
Copy link

This PR is stale because it has been open 20 days with no activity. Remove stale label or comment or this will be closed in 10 days.

@github-actions github-actions bot added stale and removed stale labels Jan 14, 2026
@SteNicholas
Copy link
Member Author

@pan3793, I have bumped version to 4.1.1 and excluded netty dependencies from spark-sql dependency. PTAL.

@pan3793
Copy link
Member

pan3793 commented Feb 4, 2026

@SteNicholas AFAIK, Spark 4.1 uses Netty 4.2 exclusive classes, so integration test should fail (suppose you also exclude Netty 4.2 from test scope), in this case, I'm afraid the right path is - upgrading Netty 4.2 for Celeborn first, then try integration with Spark 4.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants