[receiver/kafkareceiver] autocommit set false does not take effect when exporter failed

[ChrisYe2015]

Component(s)

receiver/kafka

What happened?

Description

I used kafka receiver and Jaeger exporter，and turned off autocommit in otelcol-config.yml.
When Jaeger is unavailable, I expect kafka messages not be consumed and the lag increase，but not working。
May I ask if my configuration is incorrect or if the current feature does not support Exporter failure and prevent submission of offset

Steps to Reproduce

1.Stop Jaeger
2.disable autocommit and restart otel-collector
2.Send message to kafka

Expected Result

Lag increase and continue to consume when Jaeger is ready

Actual Result

Lag = 0 and offset commited

Collector version

v0.116.0

Environment information

Environment

linux docker

OpenTelemetry Collector configuration

receivers:
  kafka/traces:
    brokers: ["kafka:9092"]
    topic: otlp_spans
    autocommit:
      enable: false
    message_marking: 
      after: true
      on_error: false

exporters:
  debug:
  #verbosity: detailed
  otlp:
    endpoint: "jaeger:8027"
    tls:
      insecure: true

service:
  telemetry:
    logs:
      level: debug
  pipelines:
    traces:
      receivers: [kafka/traces]
      processors: [batch]
      exporters: [otlp, debug]

Log output

2025-01-10T06:08:39.495Z        warn    [email protected]/clientconn.go:1384 [core] [Channel #1 SubChannel #2]grpc: addrConn.createTransport failed to connect to {Addr: "xxx.xx.xx.xx:8027", ServerName: "xxx.xx.xx.xx:8027", }. Err: connection error: desc = "transport: Error while dialing: dial tcp xxx.xx.xx.xx:8027: connect: connection refused"     {"grpc_log": true}
2025-01-10T06:08:39.495Z        info    [email protected]/clientconn.go:1204 [core] [Channel #1 SubChannel #2]Subchannel Connectivity change to TRANSIENT_FAILURE, last error: connection error: desc = "transport: Error while dialing: dial tcp xxx.xx.xx.xx:8027: connect: connection refused"       {"grpc_log": true}
2025-01-10T06:08:39.495Z        info    pickfirst/pickfirst.go:184      [pick-first-lb] [pick-first-lb 0xc001a13a70] Received SubConn state update: 0xc001a13b00, {ConnectivityState:TRANSIENT_FAILURE ConnectionError:connection error: desc = "transport: Error while dialing: dial tcp xxx.xx.xx.xx:8027: connect: connection refused" connectedAddress:{Addr: ServerName: Attributes:<nil> BalancerAttributes:<nil> Metadata:<nil>}}   {"grpc_log": true}
2025-01-10T06:08:50.430Z        debug   [email protected]/kafka_receiver.go:550    Kafka message claimed   {"kind": "receiver", "name": "kafka/traces", "data_type": "traces", "value": ..."

Additional context

No response

[github-actions]

Pinging code owners:

• receiver/kafka: @pavolloffay @MovieStoreGuy

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[apolzek]

@ChrisYe2015 could you show the details of how your messages are published in Kafka ? Since you haven’t set an encoding, I believe you are using the default, which is the otlp_proto. If you could help me reproduce the issue, I’m interested in your case. I believe it might be more related to the Kafka configurations themselves rather than the connector. For example, have you tried retention configurations like KAFKA_LOG_RETENTION_HOURS ?