Skip to content

fix: spark-k8s graceful shutdown on SIGTERM#1564

Merged
Maleware merged 7 commits into
mainfrom
fix/spark-graceful-shutdown
Jul 3, 2026
Merged

fix: spark-k8s graceful shutdown on SIGTERM#1564
Maleware merged 7 commits into
mainfrom
fix/spark-graceful-shutdown

Conversation

@Maleware

@Maleware Maleware commented Jul 3, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Description

Spark executer didn't receive SIGTERM when driver received. Resulted in waiting for graceperiod and then the executer received SIGKILL from k8s.

This fixes the issue by:

  1. wrapping Tini around the run-spark.sh script.
  2. Trapping SIGTERM signal and explicitly passing it to child process.

@Maleware

Maleware commented Jul 3, 2026

Copy link
Copy Markdown
Member Author 
    harness.go:403: run tests finished
    harness.go:510: cleaning up
    harness.go:567: removing temp folder: ""
--- PASS: kuttl (403.35s)
    --- PASS: kuttl/harness (0.00s)
        --- PASS: kuttl/harness/smoke_openshift-false_spark-4.1.2_s3-use-tls-true (403.33s)
PASS

@Maleware Maleware marked this pull request as ready for review July 3, 2026 12:45
@Maleware Maleware requested a review from razvan July 3, 2026 12:45
@Maleware Maleware self-assigned this Jul 3, 2026
@Maleware Maleware moved this to Development: Waiting for Review in Stackable Engineering Jul 3, 2026
razvan
razvan reviewed Jul 3, 2026
View reviewed changes

@razvan razvan left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changelog entry is missing. otherwise it lgtm

@NickLarsenNZ NickLarsenNZ moved this from Development: Waiting for Review to Development: In Review in Stackable Engineering Jul 3, 2026
@Maleware

Maleware commented Jul 3, 2026

Copy link
Copy Markdown
Member Author

@razvan done: fc41d15

razvan
razvan approved these changes Jul 3, 2026
View reviewed changes

@razvan razvan left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks

@Maleware Maleware added this pull request to the merge queue Jul 3, 2026
Merged via the queue into main with commit 94eb429 Jul 3, 2026
3 checks passed
@Maleware Maleware deleted the fix/spark-graceful-shutdown branch July 3, 2026 12:59
@NickLarsenNZ NickLarsenNZ added release/26.7.0 release-note Denotes a PR that will be considered when it comes time to generate release notes. labels Jul 3, 2026
@Maleware

Maleware commented Jul 3, 2026

Copy link
Copy Markdown
Member Author

Release-Note:

Spark: Fixing SIGTERM propagation from driver to executor to allow graceful executor shutdown.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@razvan razvan razvan approved these changes

Assignees

@Maleware Maleware

Labels

release/26.7.0 release-note Denotes a PR that will be considered when it comes time to generate release notes.

Projects

Stackable Engineering
Status: Development: In Review

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Maleware @razvan @NickLarsenNZ