• Cursuri

    CATEGORII CURSURI

    • Big Data
    • Database
    • Front-End
    • Java
    • Linux & Infrastructure
    • Software Security
    • Vezi alte cursuri

    Despre cursuri

    • Cursuri
    • Instructorii noştri
    • Cum devii instructor

  • Evenimente
  • Portofoliu
  • Despre noi
  • Contact
Contact
0753.029.187
academy@esolutions.ro
academy.esolutions.roacademy.esolutions.ro
  • Cursuri

    CATEGORII CURSURI

    • Big Data
    • Database
    • Front-End
    • Java
    • Linux & Infrastructure
    • Software Security
    • Vezi alte cursuri

    Despre cursuri

    • Cursuri
    • Instructorii noştri
    • Cum devii instructor

  • Evenimente
  • Portofoliu
  • Despre noi
  • Contact

Big Data

  • Home
  • Toate cursurile
  • Big Data
  • Streams processing in Big Data architectures

Streams processing in Big Data architectures

mm
Valentina Crisan
Big Data
Gratuit
distrib
  • Prezentare generală
  • Plan de curs
  • Instructor

Stream Processing refers to the real-time processing of data-in-motion, meaning near-real time / real time computations of unbounded streams of records (usually small sizes), dynamic data continuously generated by multiple data sources. Continuous streams of data sources could be sensor events, website users activity, credit card transactions,.. etc.

Stream Processing frameworks are distributed architectures able to handle/process instantly continuous streams of data, handling the state of the operations and also being able to recover in case one of the machines in the cluster fails. This course goes in detail into 2 existing streams processing solutions : Apache Kafka KSQL & Kafka Streams API and Apache Spark Structured Streaming but as well into the basics of streams – using Apache Kafka (understand partitioning, how data is consumed from a stream,…).

In this course you will understand:

– Streams concepts
– Stream processing vs batch processing
– Relevant use cases and the architectures used
– Which are the most important components/features of a stream processing
solution/architectures
– Which are the most known stream processing solutions and how they
differentiate
– Kafka Streaming (KSQL and Kafka Streams API)
– Spark Structured Streaming
– How to create a solution based on KSQL and Spark Structured Streaming

This course is designed for solution architects, data engineers, machine learning engineers, software engineers and data scientists with a basic knowledge of scalable data processing techniques such as Hadoop, MapReduce, etc.

This course is introductory level.

Pre-requisites: a good understanding of IT distributed systems, understanding of Hadoop concepts: mainly HDFS, Apache Spark knowledge a plus, Scala basic level

Caracteristici curs

  • Capitole 24
  • Quizzes 0
  • Durata 3 Zile
  • Nivel cunostinte Orice nivel
  • Limba Romana
  • Cursanti 12
  • Assessments Da
CursuriBig DataStreams processing in Big Data architectures
  • Ziua 1 12

    • Capitol1.1
      Fundamentals of streams with Apache Kafka & KSQL
    • Capitol1.2
      Learn the fundamentals of streams of data, including how to work with the Apache Kafka ecosystem, data schemas, ApacheAvro, Kafka Connect and REST proxy, KSQL
    • Capitol1.3
      Understanding the way streams are stored in distributed systems. Storing the stream in a distributed manner will help the stream computation to be scaled out to a distributed cluster for filtering and transforming the streams
    • Capitol1.4
      Partitioning strategies for streaming topologies
    • Capitol1.5
      Understand concept of order and time in streams
    • Capitol1.6
      How data can be consumed from a stream – what means parallel processing of streams data
    • Capitol1.7
      Streams processing using KSQL
    • Capitol1.8
      Working with streams & tables
    • Capitol1.9
      Understanding time concept in stream processing
    • Capitol1.10
      Late events processing
    • Capitol1.11
      Windows/aggregates
    • Capitol1.12
      State handling – how the KSQL architecture handles state and failures
  • Ziua 2 11

    • Capitol2.1
      Streaming architectures/solutions overview (with use cases) – Kafka Streams, Flink, Spark Streaming, Storm, Samza
    • Capitol2.2
      Spark Structured Streams Processing (using Kafka)
    • Capitol2.3
      Intro Apache Spark: understand architecture and concepts: RDDs vs Dataframes/Datasets
    • Capitol2.4
      Concepts of Structured Streaming
    • Capitol2.5
      Stream DF’s
    • Capitol2.6
      Queries on streams
    • Capitol2.7
      Triggers
    • Capitol2.8
      Output modes: Console, File, Memory, Kafka
    • Capitol2.9
      Checkpointing – understand how Spark saves the state of the persistent queries and how handles failure
    • Capitol2.10
      Window aggregations
    • Capitol2.11
      Handling of late data – watermarking
  • Ziua 3 1

    • Capitol3.1
      End to end use case: Meetup.com RSVPs processing with Kafka & Spark
mm
Valentina Crisan
Executive MBA ASEBUSS & Kennesaw State University - Michael J. Coles College of Business Atlanta
  • Prezentare generală
  • Plan de curs
  • Instructor
Gratuit
  • Share:

Recomandari

Neo4j Detalii
mm
Calin Constantinov

Neo4j

12
Free
Introduction to noSQL Detalii
mm
Valentina Crisan

Introduction to noSQL

12
Free
Machine Learning Crash Course using Python Detalii
mm
Tatiana Petrache

Machine Learning Crash Course using Python

12
Free
Exploratory Data Analysis Detalii
mm
Tatiana Petrache

Exploratory Data Analysis

12
Free
Introduction in Apache Spark Detalii
mm
Valentina Crisan

Introduction in Apache Spark

12
Free

Categorii cursuri

  • Big Data
  • Database
  • Front-End
  • Java
  • Linux & Infrastructure
  • Software Security
  • Vezi alte cursuri

Cere detalii

Nume *

Email *

Telefon

Curs

Mesaj

Făcând clic pe butonul „Trimite” de mai jos, înțelegeți și sunteți de acord că utilizarea prezentului website se supune termenilor si condițiilor de utilizare


Cursuri recente

Neo4j

Neo4j

Streams processing in Big Data architectures

Streams processing in Big Data architectures

Introduction to noSQL

Introduction to noSQL

Vezi toate cursurile

logo-eduma-the-best-lms-wordpress-theme

0753.029.187

academy@esolutions.ro

Companie

  • Despre noi
  • Cursuri
  • Contact

Suport

  • Întrebări frecvente
  • Catalog cursuri

Recomandări

  • eSolutions Grup
  • Different Angle Cluster

Privacy

  • Terms

a service of eSolutions.

VREI SĂ DEVII INSTRUCTOR?

Alătură-te echipei noastre!

Aplică acum