Harnessing the power of Spark for enterprise data engineering and analytics

Harnessing the power of Spark for enterprise data engineering and analytics

In our work across clients, we have built several large scale Spark based data pipelines and analytics solutions, learning first-hand how to bring scalable in-memory computing to business users. We have had to build extreme-engineering solutions to crunch the SLAs for enterprise data processing and fine tune Spark jobs while also helping users transition from old school technologies to Spark based analytics workbenches. In this talk, we will share our some of the typical problem statements for enterprise data processing and analytics enablement our clients bring to us, our brief perspective on the solution, and more importantly our experiences and learning on the technical implementations. This will cover areas such as key performance pitfalls in Spark for typical data management jobs, helping users learn the dos and donts when using Spark for analytics, integrating data management and AI algorithms in pipelines, and experiences from implementing some interesting frameworks for analytics such as Spark Modular View.

Schedule:

Room:

Albert 2-3
Speakers
Vickye
Jain
Associate Principal
at
ZS Associates
Vickye has spent the past decade helping Lifesciences clients in North America develop cutting edge technology solutions to transform analytics. He is a techno-functional expert with deep expertise in several business domains associated with Lifesciences companies as well as technical architectures for Big Data & Cloud computing. He is a part of ZS' Big Data CoE and in the recent years his efforts have been focused on helping develop a number of Spark based data management solutions for multiple clients.
Shiv
Singh
at
ZS Associates

Slides & Recordings

   Download Slides