We’re excited to announce support for Identity and Access Management (IAM) Roles for delegating permissions and access to Qubole. IAM Roles is a security best practice on AWS. Customers no longer need to provide access and secret keys to Qubole, making access control more secure.
Here’s some background on why Qubole requires access to our customers’ resources. Qubole is a web service that runs on infrastructure provided by the three major public clouds: Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform. For Big Data analysis and processing, we orchestrate storage and compute resources owned in the customer’s account. In order to do this, our customers delegate the necessary permissions to Qubole.
This model of keeping resources within the customer’s account provides a lot of benefits. For one, the data remains under the ownership of the customer. Another benefit is with price discounted compute resources – customers can use the specific instances that they purchased through programs such as reserved instances in AWS and commitment plans on Azure. Finally, customers can run their compute instances in a private subnet, such as AWS’s Virtual Private Cloud (VPC). This allows for greater security by having finer-grained control for security groups and IP address ranges within a logically isolated network.
With IAM Roles, customers can delegate the necessary access to Qubole without exchanging any confidential access keys. Qubole becomes an IAM user and can assume the IAM Role to obtain temporary security credentials that are used to make AWS API calls. For Qubole, this means we can start clusters and nodes using the role without exchanging any keys.
QDS provides a very easy way to use IAM Roles in our web UI, as shown in the screenshot below.
Here’s a diagram that shows how all the entities and resources work with each other once IAM Roles are used.
Thank you to Vishal Gupta, Yogesh Garg, Malay Majithia, Aman Goel, Abhishek Srivastava, Prasanna Santhanam, and our intern last summer Karandeep Johar for shipping this important feature for our customers!
Starting August 20th, all new Qubole accounts have this feature enabled. If you would like to use IAM Roles in an account created prior to that date, you can contact us at [email protected].
You can get started with IAM roles by following this guide in our documentation. If you’re new to Qubole, check out our 15-day free trial of Qubole for AWS. We’d love to hear your feedback on Qubole as well as our support for IAM Roles!
