Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-50985][SS] Classify Kafka Timestamp Offsets mismatch error instead of assert and throw error for missing server in KafkaTokenProvider #49662

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

[SPARK-50985][SS] Classify Kafka Timestamp Offsets mismatch error instead of assert and throw error for missing server in KafkaTokenProvider #49662

wants to merge 2 commits into from

Conversation

micheal-o
Copy link
Contributor

What changes were proposed in this pull request?

  1. In Kafka offset reader, for timestamp offsets, we assert that the assigned partitions is equal to specified partitions. Removing the assert and throwing a classified error for better error presentation
  2. In Kafka config updater, when we setAuthenticationConfigIfNeeded, we don't handle the case where user didn't specify the bootsrap server, hence leading to NoSuchElementException. Now throwing a properly classified error.

Why are the changes needed?

Error classification for better user experience

Does this PR introduce any user-facing change?

Yes, error classification

How was this patch tested?

Added new tests

Was this patch authored or co-authored using generative AI tooling?

No

@HyukjinKwon HyukjinKwon changed the title [SPARK-50985][SS] - Classify Kafka Timestamp Offsets mismatch error instead of assert and throw error for missing server in KafkaTokenProvider [SPARK-50985][SS] Classify Kafka Timestamp Offsets mismatch error instead of assert and throw error for missing server in KafkaTokenProvider Jan 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
DSTREAM SQL STRUCTURED STREAMING
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@micheal-o