How-to for real-time alerting, analytics and reporting at scale with Apache Kafka and Apache Ignite

How-to for real-time alerting, analytics and reporting at scale with Apache Kafka and Apache Ignite

Are you tasked to build a system or upgrade an existing architecture to a solution capable of handling unbound streams of data, do real-time alerting, storing always growing terabytes and petabytes of data and, finally, act on the data within milliseconds SLAs. Sounds like a challenging but rather a common task? Yes! Should you as an engineer or architect build it up off the ground? Unlikely!

What if we tell you that by integrating Apache Kafka with Apache Ignite you’ll solve all of the requirements faster and easier. A battle-tested recipe is simple - take Kafka Connect and have your data stream through Kafka pipelines, add a pinch of KSQL to act on the streams with SQL in real-time with zero delays, rinse and flush the preprocessed data in Ignite as in-memory databases and get further insights by analyzing your hot and cold datasets.

In this talk, Denis and Viktor will demonstrate how to implement the solution in practice, will explain architectural reasoning and the benefits of real-time integration, share common usage patterns. As always, we'll build a streaming data pipeline using nothing but our bare hands, Apache Ignite, Kafka Connect, and KSQL.



Edward 1-4


Product Manager
GridGain Systems
Denis Magda runs Product Management at GridGain Systems, and is the Vice President of the Apache Ignite PMC. He is an expert in distributed systems and platforms who actively contributes to Apache Ignite and helps companies and individuals deploy it for mission-critical applications. You can be sure to come across Denis at conferences, workshop and other events sharing his knowledge about use case, best practices, and implementation tips and tricks on how to build efficient applications with in-memory data grids, distributed databases and in-memory computing platforms including Apache Ignite and GridGain.

Before joining GridGain and becoming a part of Apache Ignite community, Denis worked for Oracle where he led the Java ME Embedded Porting Team -- helping bring Java to IoT.
Developer Advocate
Viktor Gamov is a Developer Advocate at Confluent, the company that makes a streaming platform based on Apache Kafka.
Working in the field, Viktor Gamov developed comprehensive expertise in building enterprise application architectures using open source technologies. He enjoys helping different organizations design and develop low latency, scalable and highly available distributed systems.
Back in his consultancy days, he co-authored O’Reilly’s «Enterprise Web Development.»
He is a professional conference speaker on distributed systems, Java, and JavaScript topics, and is regular on events including JavaOne, Devoxx, OSCON, QCon, and others ( He blogs at and produces the podcasts Razbor Poletov (in Russian) and co-hosts
Follow Viktor on Twitter @gamussa, where he posts there about gym life, food, open source, and, of course, Kafka and Confluent!

Slides & Recordings

   Download Slides