Auto Tuning Twitter Hadoop Jobs (Or: Don’t Touch That Analytics Dial!) Data Platforms 2018

Curious about how to optimize your Hadoop jobs for better performance and cost savings? Join us as we delve into the world of auto-tuning Twitter’s Hadoop jobs and learn how this innovative approach can revolutionize your analytics pipelines.

What You’ll Discover:

Scalability and Efficiency:
- Gain insights into Twitter’s massive-scale analytics infrastructure, processing over 150,000 Hadoop jobs daily across 58,000 nodes.
- Understand the challenges of manual parameter tuning and its impact on cluster resources.
The Auto-Tuning Solution:
- Explore how auto-tuning eliminates the guesswork by dynamically adjusting job parameters based on SLA requirements.
- Learn how the auto-tuning service estimates parameters like mappers, reducers, and memory to optimize job runtime.
Future Innovations:
- Get a glimpse into upcoming enhancements, including support for Spark jobs and open-sourcing the auto-tuning service.
- Discover opportunities to contribute to this cutting-edge project and join the team at Twitter.

Please fill in the form to watch the webinar

Note: By filling and submitting this form you understand and agree that the use of Qubole’s website is subject to the General Website Terms of Use. Additional details regarding Qubole’s collection and use of your personal information, including information about access, retention, rectification, deletion, security, cross-border transfers and other topics, is available in the Privacy Policy. If you have any questions regarding the webform language, please contact [email protected].