QDSIn 2017, Qubole saved customers $140M in cloud costs by smartly—and automatically— leveraging all the cloud resources available to business. With a new tool from Qubole, EMR customers can now find out if they’re poised to get a slice of that savings in 2018.
The promise of the cloud—scalability, agility, and cost savings—can only be fully realized if resources are optimally managed. Public clouds like AWS offer different types of cloud resources for a reason. Cheap AWS S3, more expensive EC2, spare-capacity Spot instances that can be purchased at a discount, and more. While this portfolio of resources offers the possibility of rapid scaling, agility, and cost savings, actually putting these principles into practice is not straightforward. Many companies are missing out on massive cost savings by not using these resources as efficiently as possible.
Qubole’s Qubole Data Service (QDS) is designed to maximize those potential savings by allocating exactly the right cloud resources for every job. We’ve found that the QDS formula typically offers customers a 50% reduction in the total cost of ownership and as much as 80% in some cases.
The QDS Formula
Qubole’s cost-saving approach involves three elements:
- Separating compute and storage. QDS automatically allocates cheaper resources like AWS S3 for persistent storage and brings faster, more expensive resources like AWS EC2 online only when they’re needed for data processing. QDS Cluster Lifecycle management automatically starts and stops these clusters as they’re needed, reducing the cost of running idle clusters.
- Optimizing compute node allocation. Even while clusters are running, QDS intelligently and predictively adds and subtracts nodes to provide maximum efficiency. The common industry practice of provisioning a cluster for peak demand is inherently inefficient, but avoidable with QDS.
- Optimizing the cost of nodes. QDS automatically shops the market for discounted Spot nodes and integrates them with other node types in heterogeneous clusters. With this approach, cost, performance, and availability can be maximized by employing exactly the right mix of machines for every workload—even if they’re not of the same type.
Project Your Cost Saving Potential
Theory and statistics are one thing, but you need to be sure a cost-saving approach will work for your unique use case. To give businesses an idea of what the QDS formula could bring to their cost equation, we’ve developed an online tool that enables AWS EMR customers to analyze EMR log data and estimate the cost savings Qubole could bring to their operations.
Try the TCO Calculator to find out how much extra efficiency might be waiting for your data processing with Qubole.
