Apache Kafka is a distributed publish-subscribe messaging system which can scale out to handle millions of messages per second. In this talk, we will cover the basics of this powerful system, including general architectural and design principles. We will also build a .NET project which uses Kafka to produce and consume messages quickly and efficiently.


The slides are available in HTML 5 format. All modern browsers (including tablets and phones) should be able to navigate the slides successfully.

The slides are licensed under Creative Commons Attribution-ShareAlike.

Demo Code

The demonstration code is available on my GitHub repository. This includes all of the console applications and Kafka scripts to work with topics.

To get the flight data we loaded and the airport data we used to enrich our data sets, check out the Data expo '09 contest's set of flight data, including the 2008 flight data and airports CSV file.

The source code is licensed under the terms offered by the GPL. The slides are licensed under Creative Commons Attribution-ShareAlike.

Links And Further Information

Kafka Links

Kafka .NET Providers

At this point, I highly recommend using the Confluent .NET client for Kafka due to its official support. Nevertheless, I think it's valuable to have an idea of other Kafka libraries which are available, as until late 2016, there were several viable alternatives.

F# Links