Exactly Once Guaranteed- Using In-Memory Stream Processing- How to do it

Distributed systems typically offer several levels of guaranteed message delivery, including at-least once processing, at-most once processing, and exactly once processing. Most streaming architects often deploy a distributed message platform with an at least-once guarantee, which guarantees that the message will be sent, but with the caveat that it’s possible some messages may be duplicated. These cases where you can get the same message twice is a very typical distributed system problems when networks are unreliable. When you publish a message and you don’t hear back, you either lost it or you didn’t, and so you try to send it again and that’s when you introduce a duplicate. But, In today's mission critical streaming applications, you can't miss a transaction that might end a life, wreck a car or ruin your companies reputation!  Because performance matters when dealing with streaming data- in-memory processing and in-memory data is key. Keeping data in motion is important for the application performance. Organizations building real-time stream processing systems need to use an in-memory paradigm and be able to use any message broker and trust the platform to deliver each message exactly once. Come to this session to learn how to build in-memory applications on streaming data that do exactly one guaranteed processing without you having to code the needed guarantees into the applications. 

Speakers
Niranjan Dhomse
Niranjan
Dhomse
Sr Enterprise Architect
at
Neeve Research

Niranjan is a Bay Area Software technology leader and architect with 15+ years of global software development experience in large companies as well as hands-on experience in smaller start-ups. Niranjan is a highly capable, hands-on engineer and architect of complex enterprise solutions as well as a solid manager and leader. with an extensive background in the development, deployment, and support of high-speed trading, e-commerce, personalized pricing, and retail systems.

.

Slides