如何排查消费者无法连接到Kafka问题

在开发和测试过程中，我们可能会遇到无法连接 Kafka 的情况，本文使用 kafka-console-consumer，来模拟几类常见的连接报错

• 密码类型选择 Scram

• 使用 SASL_SSL接入点，公网连接，客户端配置文件如下：

sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="rudonx" password="xxxxxx";        
sasl.mechanism=SCRAM-SHA-256
security.protocol=SASL_SSL

terminated during authentication

完整报错如下：

[2024-02-06 16:17:30,053] WARN [Consumer clientId=consumer-1, groupId=beijingtai] Connection to node -1 (kafka-xxxxxx.kafka.volces.com/xxx.xxx.xx.xx:9492) terminated during authentication. This may happen due to any of the following reasons: (1) Authentication failed due to invalid credentials with brokers older than 1.0.0, (2) Firewall blocking Kafka TLS traffic (eg it may only allow HTTPS traffic), (3) Transient network issue. (org.apache.kafka.clients.NetworkClient)

出现此报错，建议检查您的客户端 IP 是否在白名单中，同时比较容易忽略的点为，IP 地址已经在白名单中存在，但是未绑定到实例上。 参考文档：

• https://www.volcengine.com/docs/6439/115071
• https://www.volcengine.com/docs/6439/115070

Authentication failed

完整报错如下：

[2024-02-06 17:01:15,191] ERROR Error processing message, terminating consumer process:  (kafka.tools.ConsoleConsumer$)
org.apache.kafka.common.errors.SaslAuthenticationException: Authentication failed during authentication due to invalid credentials with SASL mechanism SCRAM-SHA-256
Processed a total of 0 messages

出现此报错，我们建议您检查客户端配置文件中的用户名和密码部分

sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="rudonx" password="xxxxxx"; 

如果密码忘记，您可以在控制台上手动修改密码，参考文档如下：

• https://www.volcengine.com/docs/6439/72044
• https://www.volcengine.com/docs/6439/72047

Not authorized to access topics/ group

完整报错如下：没有权限访问指定 Topic

[2024-02-06 17:08:25,404] ERROR Error processing message, terminating consumer process:  (kafka.tools.ConsoleConsumer$)
org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized to access topics: [top1]
Processed a total of 0 messages

没有权限访问指定 Group

[2024-02-06 17:12:04,375] ERROR Error processing message, terminating consumer process:  (kafka.tools.ConsoleConsumer$)
org.apache.kafka.common.errors.GroupAuthorizationException: Not authorized to access group: beijingtai
Processed a total of 0 messages

出现此类报错，您应该检查用户的 Topic 和 Group 相关权限，可以在实例的 ACL 管理中进行查看和修改 参考文档：

• https://www.volcengine.com/docs/6439/72046

如果您有其他问题，欢迎您联系火山引擎技术支持服务