SoFunction
Updated on 2025-03-10

Problems encountered in Docker deployment of kafka and solutions

[root@iZwz9geewjuo2sogidqn50Z ~]# docker logs 6ae1e49888b3
[Configuring] '' in '/opt/kafka/config/'
[Configuring] 'port' in '/opt/kafka/config/'
Excluding KAFKA_HOME from broker config
[Configuring] '' in '/opt/kafka/config/'
[Configuring] 'listeners' in '/opt/kafka/config/'
Excluding KAFKA_VERSION from broker config
[Configuring] '' in '/opt/kafka/config/'
[Configuring] '' in '/opt/kafka/config/'
[2022-05-13 03:41:21,371] INFO Registered kafka:type=kafka.Log4jController MBean (.Log4jControllerRegistration$)
[2022-05-13 03:41:22,119] INFO Setting -D =true to disable client-initiated TLS renegotiation (.X509Util)
[2022-05-13 03:41:22,334] INFO Registered signal handlers for TERM, INT, HUP ()
[2022-05-13 03:41:22,342] INFO starting ()
[2022-05-13 03:41:22,342] INFO Connecting to zookeeper on 39.108.99.163:2181 ()
[2022-05-13 03:41:22,376] INFO [ZooKeeperClient Kafka server] Initializing a new session to 39.108.99.163:2181. ()
[2022-05-13 03:41:22,384] INFO Client environment:=3.5.9-83df9301aa5c2a5d284a9940177808c01bc35cef, built on 01/06/2021 20:03 GMT ()
[2022-05-13 03:41:22,384] INFO Client environment:=6ae1e49888b3 ()
[2022-05-13 03:41:22,384] INFO Client environment:=11.0.15 ()
[2022-05-13 03:41:22,385] INFO Client environment:=Oracle Corporation ()
[2022-05-13 03:41:22,385] INFO Client environment:=/usr/local/openjdk-11 ()
[2022-05-13 03:41:22,385] INFO Client environment:=/opt/kafka/bin/../libs/activation-1.1.:/opt/kafka/bin/../libs/aopalliance-repackaged-2.6.:/opt/kafka/bin/../libs/argparse4j-0.7.:/opt/kafka/bin/../libs/audience-annotations-0.5.:/opt/kafka/bin/../libs/commons-cli-1.:/opt/kafka/bin/../libs/commons-lang3-3.8.:/opt/kafka/bin/../libs/connect-api-2.8.:/opt/kafka/bin/../libs/connect-basic-auth-extension-2.8.:/opt/kafka/bin/../libs/connect-file-2.8.:/opt/kafka/bin/../libs/connect-json-2.8.:/opt/kafka/bin/../libs/connect-mirror-2.8.:/opt/kafka/bin/../libs/connect-mirror-client-2.8.:/opt/kafka/bin/../libs/connect-runtime-2.8.:/opt/kafka/bin/../libs/connect-transforms-2.8.:/opt/kafka/bin/../libs/hk2-api-2.6.:/opt/kafka/bin/../libs/hk2-locator-2.6.:/opt/kafka/bin/../libs/hk2-utils-2.6.:/opt/kafka/bin/../libs/jackson-annotations-2.10.:/opt/kafka/bin/../libs/jackson-core-2.10.:/opt/kafka/bin/../libs/jackson-databind-2.10.5.:/opt/kafka/bin/../libs/jackson-dataformat-csv-2.10.:/opt/kafka/bin/../libs/jackson-datatype-jdk8-2.10.:/opt/kafka/bin/../libs/jackson-jaxrs-base-2.10.:/opt/kafka/bin/../libs/jackson-jaxrs-json-provider-2.10.:/opt/kafka/bin/../libs/jackson-module-jaxb-annotations-2.10.:/opt/kafka/bin/../libs/jackson-module-paranamer-2.10.:/opt/kafka/bin/../libs/jackson-module-scala_2.13-2.10.:/opt/kafka/bin/../libs/-api-1.2.:/opt/kafka/bin/../libs/-api-1.3.:/opt/kafka/bin/../libs/-2.6.:/opt/kafka/bin/../libs/-api-2.0.:/opt/kafka/bin/../libs/-api-2.1.:/opt/kafka/bin/../libs/-api-2.3.:/opt/kafka/bin/../libs/javassist-3.27.:/opt/kafka/bin/../libs/-api-3.1.:/opt/kafka/bin/../libs/-api-2.1.:/opt/kafka/bin/../libs/jaxb-api-2.3.:/opt/kafka/bin/../libs/jersey-client-2.:/opt/kafka/bin/../libs/jersey-common-2.:/opt/kafka/bin/../libs/jersey-container-servlet-2.:/opt/kafka/bin/../libs/jersey-container-servlet-core-2.:/opt/kafka/bin/../libs/jersey-hk2-2.:/opt/kafka/bin/../libs/jersey-server-2.:/opt/kafka/bin/../libs/jetty-client-9.4.43.:/opt/kafka/bin/../libs/jetty-continuation-9.4.43.:/opt/kafka/bin/../libs/jetty-http-9.4.43.:/opt/kafka/bin/../libs/jetty-io-9.4.43.:/opt/kafka/bin/../libs/jetty-security-9.4.43.:/opt/kafka/bin/../libs/jetty-server-9.4.43.:/opt/kafka/bin/../libs/jetty-servlet-9.4.43.:/opt/kafka/bin/../libs/jetty-servlets-9.4.43.:/opt/kafka/bin/../libs/jetty-util-9.4.43.:/opt/kafka/bin/../libs/jetty-util-ajax-9.4.43.:/opt/kafka/bin/../libs/jline-3.12.:/opt/kafka/bin/../libs/jopt-simple-5.0.:/opt/kafka/bin/../libs/kafka-clients-2.8.:/opt/kafka/bin/../libs/kafka-log4j-appender-2.8.:/opt/kafka/bin/../libs/kafka-metadata-2.8.:/opt/kafka/bin/../libs/kafka-raft-2.8.:/opt/kafka/bin/../libs/kafka-shell-2.8.:/opt/kafka/bin/../libs/kafka-streams-2.8.:/opt/kafka/bin/../libs/kafka-streams-examples-2.8.:/opt/kafka/bin/../libs/kafka-streams-scala_2.13-2.8.:/opt/kafka/bin/../libs/kafka-streams-test-utils-2.8.:/opt/kafka/bin/../libs/kafka-tools-2.8.:/opt/kafka/bin/../libs/kafka_2.13-2.8.:/opt/kafka/bin/../libs/kafka_2.13-2.8.:/opt/kafka/bin/../libs/log4j-1.2.:/opt/kafka/bin/../libs/lz4-java-1.7.:/opt/kafka/bin/../libs/maven-artifact-3.8.:/opt/kafka/bin/../libs/metrics-core-2.2.:/opt/kafka/bin/../libs/netty-buffer-4.1.:/opt/kafka/bin/../libs/netty-codec-4.1.:/opt/kafka/bin/../libs/netty-common-4.1.:/opt/kafka/bin/../libs/netty-handler-4.1.:/opt/kafka/bin/../libs/netty-resolver-4.1.:/opt/kafka/bin/../libs/netty-transport-4.1.:/opt/kafka/bin/../libs/netty-transport-native-epoll-4.1.:/opt/kafka/bin/../libs/netty-transport-native-unix-common-4.1.:/opt/kafka/bin/../libs/osgi-resource-locator-1.0.:/opt/kafka/bin/../libs/paranamer-2.:/opt/kafka/bin/../libs/plexus-utils-3.2.:/opt/kafka/bin/../libs/reflections-0.9.:/opt/kafka/bin/../libs/rocksdbjni-5.18.:/opt/kafka/bin/../libs/scala-collection-compat_2.13-2.3.:/opt/kafka/bin/../libs/scala-java8-compat_2.13-0.9.:/opt/kafka/bin/../libs/scala-library-2.13.:/opt/kafka/bin/../libs/scala-logging_2.13-3.9.:/opt/kafka/bin/../libs/scala-reflect-2.13.:/opt/kafka/bin/../libs/slf4j-api-1.7.:/opt/kafka/bin/../libs/slf4j-log4j12-1.7.:/opt/kafka/bin/../libs/snappy-java-1.1.8.:/opt/kafka/bin/../libs/zookeeper-3.5.:/opt/kafka/bin/../libs/zookeeper-jute-3.5.:/opt/kafka/bin/../libs/zstd-jni-1.4. ()
[2022-05-13 03:41:22,386] INFO Client environment:=/usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib ()
[2022-05-13 03:41:22,386] INFO Client environment:=/tmp ()
[2022-05-13 03:41:22,386] INFO Client environment:=<NA> ()
[2022-05-13 03:41:22,386] INFO Client environment:=Linux ()
[2022-05-13 03:41:22,386] INFO Client environment:=amd64 ()
[2022-05-13 03:41:22,386] INFO Client environment:=3.10.0-862.14.4.el7.x86_64 ()
[2022-05-13 03:41:22,386] INFO Client environment:=root ()
[2022-05-13 03:41:22,386] INFO Client environment:=/root ()
[2022-05-13 03:41:22,386] INFO Client environment:=/ ()
[2022-05-13 03:41:22,386] INFO Client environment:=978MB ()
[2022-05-13 03:41:22,386] INFO Client environment:=1024MB ()
[2022-05-13 03:41:22,386] INFO Client environment:=1024MB ()
[2022-05-13 03:41:22,395] INFO Initiating client connection, connectString=39.108.99.163:2181 sessionTimeout=18000 watcher=$ZooKeeperClientWatcher$@205d38da ()
[2022-05-13 03:41:22,405] INFO value is 4194304 Bytes ()
[2022-05-13 03:41:22,414] INFO value is 0. feature enabled= ()
[2022-05-13 03:41:22,417] INFO [ZooKeeperClient Kafka server] Waiting until connected. ()
[2022-05-13 03:41:22,459] INFO Opening socket connection to server 39.108.99.163/39.108.99.163:2181. Will not attempt to authenticate using SASL (unknown error) ()
[2022-05-13 03:41:22,470] INFO Socket connection established, initiating session, client: /172.17.0.4:33848, server: 39.108.99.163/39.108.99.163:2181 ()
[2022-05-13 03:41:22,522] INFO Session establishment complete on server 39.108.99.163/39.108.99.163:2181, sessionid = 0x1004f2b9d3a0000, negotiated timeout = 18000 ()
[2022-05-13 03:41:22,527] INFO [ZooKeeperClient Kafka server] Connected. ()
[2022-05-13 03:41:22,760] INFO [feature-zk-node-event-process-thread]: Starting ($ChangeNotificationProcessorThread)
[2022-05-13 03:41:22,779] INFO Feature ZK node at path: /feature does not exist ()
[2022-05-13 03:41:22,780] INFO Cleared cache ()
[2022-05-13 03:41:23,158] INFO Cluster ID = fSpFkgCsTqSS5HeP0e355g ()
[2022-05-13 03:41:23,163] WARN No file under dir /kafka/kafka-logs-6ae1e49888b3/ ()
[2022-05-13 03:41:23,246] INFO KafkaConfig values: 
        = null
        = PLAINTEXT://39.108.99.163:9092
        = null
        = null
        = 11
        = 1
        = 
        = true
        = true
        = 10
        = 2000
        = 0
        = true
        = null
        = 9000
        = null
        = producer
        = 100
        = 600000
        = 0
        = null
        = true
        = 3
        = 5000
        = null
        = 25
        = 1000
        = 1000
        = 2000
        = 2000
        = 20
        = []
        = 11
        = 1
        = 30000
        = null
        = 1
        = 3600000
        = 86400000
        = null
        = 604800000
        = null
        = 1
        = true
        = 57671680
        = 1000
        = 0
        = 1800000
        = 2147483647
        = 6000
        = 
        = 60000
        = null
        = 2.8-IV1
        = 10
        = []
        = 300
        = 10
        = PLAINTEXT:PLAINTEXT,SSL:SSL,SASL_PLAINTEXT:SASL_PLAINTEXT,SASL_SSL:SASL_SSL
        listeners = PLAINTEXT://0.0.0.0:9092
        = 15000
        = 134217728
        = 86400000
        = true
        = 0.9
        = 524288
        = 1.7976931348623157E308
        = 9223372036854775807
        = 0.5
        = 0
        = 1
        = [delete]
        = /tmp/kafka-logs
        = /kafka/kafka-logs-6ae1e49888b3
        = 9223372036854775807
        = null
        = 60000
        = 9223372036854775807
        = 60000
        = 4096
        = 10485760
        = true
        = 2.8-IV1
        = 9223372036854775807
        = CreateTime
        = false
        = -1
        = 300000
        = 168
        = null
        = null
        = 168
        = 0
        = null
        = null
        = 1073741824
        = 60000
        = 2147483647
        = 2147483647
        = 2147483647
        = 
        = 1000
        = 1048588
        = null
        = []
        = 2
        = INFO
        = 30000
        = 1
        = -1
        = 8
        = 3
        = 1
        = 1
        = null
        = 1
        = 4096
        = -1
        = 5000
        = 5242880
        = 600000
        = 10080
        = 0
        = 50
        = 1
        = 104857600
        = AES/CBC/PKCS5Padding
        = 4096
        = 128
        = null
        = null
        = null
        port = 9092
        = null
        = []
        = 1000
        = -1
        = 500
        = 9223372036854775807
        = 9223372036854775807
        = 11
        = 1
        = 1000
        = 1048576
        = 1
        = 10485760
        = 500
        = 5000
        = 30000
        = null
        = 65536
        = 30000
        = 11
        = 1
        = 30000
        = 1000
        = null
        = [GSSAPI]
        = null
        = /usr/bin/kinit
        = 60000
        = [DEFAULT]
        = null
        = 0.05
        = 0.8
        = null
        = null
        = 300
        = 60
        = 0.8
        = 0.05
        = GSSAPI
        = GSSAPI
        = null
        = PLAINTEXT
        = null
        = 30000
        = 10000
        = 102400
        = 104857600
        = 102400
        = []
        = none
        = [TLSv1.2, TLSv1.3]
        = https
        = null
        = null
        = SunX509
        = null
        = null
        = null
        = null
        = JKS
        = DEFAULT
        = TLSv1.3
        = null
        = null
        = PKIX
        = null
        = null
        = null
        = JKS
        = 10000
        = 900000
        = 3600000
        = 5242880
        = 1
        = 50
        = 1
        = 104857600
        = 604800000
        = false
        = null
        = 39.108.99.163:2181
        = 18000
        = 10
        = 18000
        = false
        = null
        = false
        = false
        = null
        = HTTPS
        = null
        = null
        = null
        = false
        = TLSv1.2
        = null
        = null
        = null
        = 2000
 ()
[2022-05-13 03:41:23,269] INFO KafkaConfig values: 
        = null
        = PLAINTEXT://39.108.99.163:9092
        = null
        = null
        = 11
        = 1
        = 
        = true
        = true
        = 10
        = 2000
        = 0
        = true
        = null
        = 9000
        = null
        = producer
        = 100
        = 600000
        = 0
        = null
        = true
        = 3
        = 5000
        = null
        = 25
        = 1000
        = 1000
        = 2000
        = 2000
        = 20
        = []
        = 11
        = 1
        = 30000
        = null
        = 1
        = 3600000
        = 86400000
        = null
        = 604800000
        = null
        = 1
        = true
        = 57671680
        = 1000
        = 0
        = 1800000
        = 2147483647
        = 6000
        = 
        = 60000
        = null
        = 2.8-IV1
        = 10
        = []
        = 300
        = 10
        = PLAINTEXT:PLAINTEXT,SSL:SSL,SASL_PLAINTEXT:SASL_PLAINTEXT,SASL_SSL:SASL_SSL
        listeners = PLAINTEXT://0.0.0.0:9092
        = 15000
        = 134217728
        = 86400000
        = true
        = 0.9
        = 524288
        = 1.7976931348623157E308
        = 9223372036854775807
        = 0.5
        = 0
        = 1
        = [delete]
        = /tmp/kafka-logs
        = /kafka/kafka-logs-6ae1e49888b3
        = 9223372036854775807
        = null
        = 60000
        = 9223372036854775807
        = 60000
        = 4096
        = 10485760
        = true
        = 2.8-IV1
        = 9223372036854775807
        = CreateTime
        = false
        = -1
        = 300000
        = 168
        = null
        = null
        = 168
        = 0
        = null
        = null
        = 1073741824
        = 60000
        = 2147483647
        = 2147483647
        = 2147483647
        = 
        = 1000
        = 1048588
        = null
        = []
        = 2
        = INFO
        = 30000
        = 1
        = -1
        = 8
        = 3
        = 1
        = 1
        = null
        = 1
        = 4096
        = -1
        = 5000
        = 5242880
        = 600000
        = 10080
        = 0
        = 50
        = 1
        = 104857600
        = AES/CBC/PKCS5Padding
        = 4096
        = 128
        = null
        = null
        = null
        port = 9092
        = null
        = []
        = 1000
        = -1
        = 500
        = 9223372036854775807
        = 9223372036854775807
        = 11
        = 1
        = 1000
        = 1048576
        = 1
        = 10485760
        = 500
        = 5000
        = 30000
        = null
        = 65536
        = 30000
        = 11
        = 1
        = 30000
        = 1000
        = null
        = [GSSAPI]
        = null
        = /usr/bin/kinit
        = 60000
        = [DEFAULT]
        = null
        = 0.05
        = 0.8
        = null
        = null
        = 300
        = 60
        = 0.8
        = 0.05
        = GSSAPI
        = GSSAPI
        = null
        = PLAINTEXT
        = null
        = 30000
        = 10000
        = 102400
        = 104857600
        = 102400
        = []
        = none
        = [TLSv1.2, TLSv1.3]
        = https
        = null
        = null
        = SunX509
        = null
        = null
        = null
        = null
        = JKS
        = DEFAULT
        = TLSv1.3
        = null
        = null
        = PKIX
        = null
        = null
        = null
        = JKS
        = 10000
        = 900000
        = 3600000
        = 5242880
        = 1
        = 50
        = 1
        = 104857600
        = 604800000
        = false
        = null
        = 39.108.99.163:2181
        = 18000
        = 10
        = 18000
        = false
        = null
        = false
        = false
        = null
        = HTTPS
        = null
        = null
        = null
        = false
        = TLSv1.2
        = null
        = null
        = null
        = 2000
 ()
[2022-05-13 03:41:23,371] INFO [ThrottledChannelReaper-Fetch]: Starting ($ThrottledChannelReaper)
[2022-05-13 03:41:23,371] INFO [ThrottledChannelReaper-Request]: Starting ($ThrottledChannelReaper)
[2022-05-13 03:41:23,371] INFO [ThrottledChannelReaper-Produce]: Starting ($ThrottledChannelReaper)
[2022-05-13 03:41:23,372] INFO [ThrottledChannelReaper-ControllerMutation]: Starting ($ThrottledChannelReaper)
[2022-05-13 03:41:23,412] INFO Log directory /kafka/kafka-logs-6ae1e49888b3 not found, creating it. ()
[2022-05-13 03:41:23,463] INFO Loading logs from log dirs ArraySeq(/kafka/kafka-logs-6ae1e49888b3) ()
[2022-05-13 03:41:23,466] INFO Attempting recovery for all logs in /kafka/kafka-logs-6ae1e49888b3 since no clean shutdown file was found ()
[2022-05-13 03:41:23,515] INFO Loaded 0 logs in 52ms. ()
[2022-05-13 03:41:23,520] INFO Starting log cleanup with a period of 300000 ms. ()
[2022-05-13 03:41:23,551] INFO Starting log flusher with a default period of 9223372036854775807 ms. ()
[2022-05-13 03:41:24,767] INFO Updated connection-accept-rate max connection creation rate to 2147483647 ()
[2022-05-13 03:41:24,786] INFO Awaiting socket connections on 0.0.0.0:9092. ()
[2022-05-13 03:41:24,962] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Created data-plane acceptor and processors for endpoint : ListenerName(PLAINTEXT) ()
[2022-05-13 03:41:25,125] INFO [broker-0-to-controller-send-thread]: Starting ()
[2022-05-13 03:41:25,240] INFO [ExpirationReaper-0-Fetch]: Starting ($ExpiredOperationReaper)
[2022-05-13 03:41:25,240] INFO [ExpirationReaper-0-DeleteRecords]: Starting ($ExpiredOperationReaper)
[2022-05-13 03:41:25,242] INFO [ExpirationReaper-0-Produce]: Starting ($ExpiredOperationReaper)
[2022-05-13 03:41:25,249] INFO [ExpirationReaper-0-ElectLeader]: Starting ($ExpiredOperationReaper)
[2022-05-13 03:41:25,312] INFO [LogDirFailureHandler]: Starting ($LogDirFailureHandler)
[2022-05-13 03:41:25,423] INFO Creating /brokers/ids/0 (is it secure? false) ()
[2022-05-13 03:41:25,512] INFO Stat of the created znode at /brokers/ids/0 is: 25,25,1652413285479,1652413285479,1,0,0,72144642777939968,210,0,25
 ()
[2022-05-13 03:41:25,514] INFO Registered broker 0 at path /brokers/ids/0 with addresses: PLAINTEXT://39.108.99.163:9092, czxid (broker epoch): 25 ()
[2022-05-13 03:41:25,802] INFO [ExpirationReaper-0-topic]: Starting ($ExpiredOperationReaper)
[2022-05-13 03:41:25,836] INFO [ExpirationReaper-0-Heartbeat]: Starting ($ExpiredOperationReaper)
[2022-05-13 03:41:25,862] INFO Successfully created /controller_epoch with initial epoch 0 ()
[2022-05-13 03:41:25,864] INFO [ExpirationReaper-0-Rebalance]: Starting ($ExpiredOperationReaper)
[2022-05-13 03:41:25,894] INFO [GroupCoordinator 0]: Starting up. ()
[2022-05-13 03:41:25,926] INFO [GroupCoordinator 0]: Startup complete. ()
[2022-05-13 03:41:25,938] INFO Feature ZK node created at path: /feature ()
[2022-05-13 03:41:26,031] INFO Updated cache from existing <empty> to latest FinalizedFeaturesAndEpoch(features=Features{}, epoch=0). ()
[2022-05-13 03:41:26,051] INFO [ProducerId Manager 0]: Acquired new producerId block (brokerId:0,blockStartProducerId:0,blockEndProducerId:999) by writing to Zk with path version 1 ()
[2022-05-13 03:41:26,052] INFO [TransactionCoordinator id=0] Starting up. ()
[2022-05-13 03:41:26,066] INFO [TransactionCoordinator id=0] Startup complete. ()
[2022-05-13 03:41:26,107] INFO [Transaction Marker Channel Manager 0]: Starting ()
[2022-05-13 03:41:26,336] INFO [ExpirationReaper-0-AlterAcls]: Starting ($ExpiredOperationReaper)
[2022-05-13 03:41:26,373] INFO [/config/changes-event-process-thread]: Starting ($ChangeEventProcessThread)
[2022-05-13 03:41:26,434] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Starting socket server acceptors and processors ()
[2022-05-13 03:41:26,507] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Started data-plane acceptor and processor(s) for endpoint : ListenerName(PLAINTEXT) ()
[2022-05-13 03:41:26,507] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Started socket server acceptors and processors ()
[2022-05-13 03:41:26,519] INFO Kafka version: 2.8.1 ()
[2022-05-13 03:41:26,519] INFO Kafka commitId: 839b886f9b732b15 ()
[2022-05-13 03:41:26,519] INFO Kafka startTimeMs: 1652413286507 ()
[2022-05-13 03:41:26,527] INFO [KafkaServer id=0] started ()
[2022-05-13 03:41:26,631] INFO [broker-0-to-controller-send-thread]: Recorded new controller, from now on will use broker 39.108.99.163:9092 (id: 0 rack: null) ()