Getting spark to read kafka data
Setup kafka bin /zookeeper-server-start .sh config /zookeeper .properties bin /kafka-server-start .sh config /server .properties Create a topic bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test To write something to your topic bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test Working with spark You can get the script from here . To submit spark python code spark-submit --packages org.apache.spark:spark-sql-kafka-0-10_2.11:2.4.1 kafka.py localhost subscribe test