At Qubole, we’ve made significant progress on our adoption of Spark on Qubole with new features and scalability. And to accommodate growing demands and leverage technological advancements made by the Apache Spark community, we at Qubole continue to support new versions as soon as possible while still supporting a trailing list of stable versions for production workloads. By focusing on the latest major versions, we can concentrate our operational efforts into testing and quality to ensure the best production experience possible.
In an effort to align our pace of development with the project, we did a careful analysis of which versions of Spark were being used by our customers. Based on our conclusions, we have made some difficult decisions to discontinue support for Spark versions older than 1.6.2. This will not affect you immediately, and we are putting a timeline in place for the deprecation of these versions.
Most of these changes will happen over the next few months, but today we are giving you advance notice. As always, we will strive to keep changes to a minimum and announce them in a timely manner.
Spark 1.x — Supported Versions Schedule
We are introducing two new stages to Spark version support—Deprecation and Sunset. In the first stage, “Deprecation”, the version will be marked as “Deprecated” in the cluster settings when selecting a Spark version. In the second stage, “Sunset”, which happens 90 days from the “Deprecation” date, the Spark version will be removed altogether from the cluster settings, and the version will no longer be supported by the Qubole support team. We will exercise due diligence in ensuring we do not break such unsupported versions. However, a feature in a new release might be backward compatible with an older revision, thus rendering it unusable. While this is not our intent, we must be realistic about the future of our development.
The deprecation policy for Spark 1.x versions will take effect on April 19, 2017, with the first set of versions being deprecated on July 31, 2017 — which will include: 1.3.0, 1.3.1, 1.4.0, 1.4.1, 1.5.0, 1.5.1, 1.6.0, and 1.6.1.
Our Commitment To Customers
Our change in policy is designed to bring you the best in technology as you’ve come to expect from Qubole. Our team remains committed to advancing the leading big data platform in the cloud that meets your needs and enables you to become a successful data-driven organization.
If you’re interested in trying out Qubole, sign up today for a free trial! To get an overview, click here to register for a live demo.
