-
Day 1: Spark Overview 13
-
A brief history of SparkCapitol1.1
-
Where Spark fits in the big data landscapeCapitol1.2
-
Apache Spark vs. Apache MapReduce: An overall architecture comparisonCapitol1.3
-
Cluster Architecture: cluster manager, workers, executors; Spark Context; Cluster Manager Types; Deployment scenariosCapitol1.4
-
How Spark schedules and executes jobs and tasksCapitol1.5
-
Resilient Distributed Datasets : Fundamentals & hands on exercisesCapitol1.6
-
Ways to create an RDD: Parallelize Collection; Read from external data source; From existing RDDCapitol1.7
-
Introduction to Transformations and ActionsCapitol1.8
-
CachingCapitol1.9
-
RDD TypesCapitol1.10
-
How transformations lazily build up a Directed Acyclic Graph (DAG)Capitol1.11
-
ShufflingCapitol1.12
-
Hands on: using Spark for ETLCapitol1.13
-
-
Day 2: SparkSQL and DataFrames/Datasets : Fundamentals and hands on exercises 5
-
What are DataFrames/Datasets vs RDD’sCapitol2.1
-
The DataFrames/Datasets APICapitol2.2
-
Spark SQLCapitol2.3
-
Creating and running DataFrame operationsCapitol2.4
-
Reading from multiple data sources and hands on exercises: HDFS; noSQL; HiveCapitol2.5
-
-
Day 3: Spark Streaming 6
-
When to use Spark StreamingCapitol3.1
-
DStreamCapitol3.2
-
Structured streaming: Building a Spark streams out of Kafka topics; Windowing & Aggregation; Register a Spark DF streamCapitol3.3
-
Spark’s MLlib and MLlib Pipeline API (Spark.ml) for Machine LearningCapitol3.4
-
Spark MLlib and Spark.mlCapitol3.5
-
Machine Learning Examples: Collaborative filtering: Alternating Least Squares; Classification and regressionCapitol3.6
-
This content is protected, please login and enroll course to view this content!