Please follow the below steps when you get the above error.

I got this error in KAFKA and everytime its getting down due to this.

In Kafka, every topic is (optionally) split into many partitions. For each partition some files are maintained by brokers (for index and actual data).

kafka-topics –zookeeper localhost:2181 –describe –topic topic_name

The above command will give you the number of partitions for topic topic_name. The default number of partitions per topic num.partitions is defined under /etc/kafka/server.properties

The total number of open files could be very huge if the broker hosts many partitions and a particular partition has many log segment files.

You can see the current file descriptor limit by running

ulimit -n 

To solve the issue you either need to change the limit of open file descriptors:

ulimit -n <noOfFiles>

or somehow reduce the number of open files (for example, reduce number of partitions per topic).

Post Views: 336

Leave a Reply