Welcome to the Qubole Spark Notebooks Workshop!
This hands-on workshop is designed to give a full understanding of the Data Science workflow using Qubole Spark Notebooks to show we can do a wide variety of use cases (ETL, ML, and Dashboarding Analytics) all on the same data with automated big data clusters. Organizations using the same platform for processing all types of big data workloads enables them to manage growth and complexity, react faster to customer needs, and improve collaboration–all at the same time. And seamless integration with disparate systems is important when evaluating any new big data infrastructure. Especially when dealing with structured and unstructured datasets.

Spark Getting Started Guide

This notebook is a part of a Spark Guide aimed to get you up to speed with the basics of Apache Spark quickly for different use cases. This notebook is great for if you are new to Spark, or just getting started with doing Data Science with a big data engine. The Guide also serves as a useful resource for advanced Spark user including Spark improvements, advanced tutorials, and best practices:
+ Spark Training Courses
+ PySpark Cheatsheet
+ Spark Engineering Blog

Workshop Overview

In this workshop, you will use appropriate tools, engines, and interfaces all within QDS to build an end-to-end solution to address business scenarios common in retail and eCommerce use cases in understanding what products other consumers are buying in order to make real-time online recommendations, or determine product marketing and buying decisions. Business and marketing teams also could find it helpful know which products are most commonly purchased together, with use cases such as:
• Make optimizations in marketing campaigns
• Position components or products together that will generate a stronger engagement rate
• Understand interaction and product correlation data
• Help sales for the lesser viewed products.
• Recover revenue for a product that is getting a lot of engagement, but not conversions

This workshop walks through various stages of a typical Data Science batch processing workflow with Qubole Spark Notebooks:
- Section 1. Ingest and explore structured and unstructured data from external sources
- Section 2. Explore & visualize data with SparkSQL
- Section 3. Correlate structured and unstructured datasets to uncover business questions and validate claim
- Section 4. Use SparkMLlib’s advanced data mining library to group frequently purchased products for recommendations
- Section 5. Analyze newly enriched and grouped data

Qubole Spark Notebooks are great for creating Data Science applications using Scala, Python, and R to develop and test Machine Learning jobs. Notebooks are also a helpful utility with running ETL jobs in Apache Spark, as well as visualizing the results of SQL in a single, collaborative environment.

Qubole Architecture

alt text

This will download a JSON file of the Notebook that can be imported into your Qubole account to test yourself.

Don't have a Qubole account? Click here for a free Qubole Test Drive