Supercharging Performance Of Apache Spark Applications In The Cloud Data Platforms 2018

What You’ll Learn:

1. Challenges of Running Spark on Cloud:
   • Understand the unique challenges faced when deploying Spark in cloud environments, including compute optimization, object store reads, and writes.
2. Optimal Resource Utilization:
   • Discover how Sparklens, our open-source Spark tuning tool, can identify performance bottlenecks and recommend the ideal number of executors for your workload.
3. Efficient Data Caching with Rubix:
   • Learn about Rubix, our innovative in-memory data caching system, designed to significantly reduce read times from cloud object stores and enhance overall application performance.
4. Direct Writes Optimization:
   • Gain insights into our advancements in optimizing Spark’s interaction with cloud storage, ensuring more efficient data writes and minimizing operation costs.
5. Empirical Performance Benchmarks:
   • Witness the tangible benefits of our optimizations through rigorous performance benchmarking, highlighting substantial speed improvements in Spark operations.