Apache Spark – PySpark

سرفصل های دوره

PySpark

1. THE FUNDAMENTALS

1. Data VS Information

2. Data Storage and Processing

3. Data Sources

4. Big Data Introduction

2. THE FOUNDATIONS OF BIG DATA

1. Emergence of Big Data

2. Basic Terminologies

3. Central theme of Big Data

4. Requirements of Programming Model

5. Understand Distributed Processing through a Story

3. ENVIRONMENT AND INSTALLATION

1. Oracle VirtualMachine_Installation

2. How to install Ubuntu operating system on Virtual Box

3. How to install PySpark on Ubuntu with Java and Python 3

4. How to configure Pyspark with Pycharm with_Installation

5. Google Cloud Platform Setup

4. HADOOP ECOSYSTEM

1. Introduction to Hadoop Ecosystem

5. PYTHON FOR PYSPARK

1. INTRODUCTION TO PROGRAMMING

2. Introduction to Python

3. Environment for Python

4. Executing Python Code

5. Syntax, Indentation and Comments

6. Syntax, Indentation and Comments - Practical

7. Variables

8. Variable Practicals

9. Python Datatypes

10. Python Datatypes Practicals

11. Python Operator Concepts

12. Python Operator Practicals

13. Control Flows in Python

14. Control Flows - IF ELSE Concepts

15. If Else Practical

16. Loops Theory

17. Loops Practical

18. Python Function Concepts

19. Python Function Hands-on

6. APACHE SPARK

1. Why Spark

2. Advantages of Spark

3. What is Spark

4. Components of Spark

5. History of Spark

7. OVERVIEW OF SPARK

1. Architecture of Spark

2. Spark Session

3. Spark Session Terminal And Jupyter notebook Hands-On

4. Spark Language API

5. Dataframes and Partitions

6. Spark Transformations

7. Spark Actions

8. STRUCTURED API OVERVIEW

1. Structured APIs - Dataframes and Datasets

2. Schema Definition

3. Spark Types

4. Structured API Execution

9. OPERATIONS ON DATAFRAMES

1. Dataframe Columns

2. Columns as Expression

3. Dataframe Rows

4. Ways of Creating Dataframe

5. Methods to Manipulate Columns

6. DataFrame Transformations

7. Dataframe Transformation - Columns

8. Dataframe Transformations - Rows Part1

9. Dataframe Transformation - Rows Part2

10. WORKING WITH DIFFERENT TYPES OF DATABASE

1. Introduction to working with Different Types of Data

2. Working with Booleans

3. Working with Strings

4. Working with Strings Practical1

5. Working with Strings Practical2

6. Working with Date and Time Stamps

7. Working with Null Concepts

8. Working with Nulls Practicals

9. Working with Complex Types

10. Working with Complex types practical

11. User Defined Functions - Concepts

12. Working with Complex types practical

11. CREATING DATAFRAMES FROM DIFFERENT SOURCES

1. Data Sources Introduction

2. Read-API- Data Sources

3. Read-API-Practical

4. Write-API-Data Sources

5. Write-API-Practical

6. Reading from CSV Files

7. Writing into CSV Files

8. Reading from JSON Files and Writing into JSON

9. Reading from Parquet and writing into Parquet

10. Reading from ORC and writing into ORC

11. Unstructured Data - Text File - Reading and Writing

12. Introduction to reading data from structured sources

13. Reading data from structured sources - Database - Concepts

14. Reading data from structured sources - Database - Practicals

15. Query Pushdown Concepts

16. Query Pushdown Praticals

17. Writing into structured sources - Database - Concepts

18. Writing into structured sources - Database - Practicals

12. AGGREGATIONS

1. Introduction to Aggregations

2. Aggregataion Concepts - Count

3. Aggregation Practical-1-Count

4. Aggregation Concepts - First, Sum and Average

5. Aggregation - Practical 2 - First Last Average

6. Aggregation-Practical-3-StatisticalFunctions

7. Aggregation Concepts - Grouping

8. Aggregation-Practical-4-GroupBy

9. Aggregation Concepts - Window Functions

10. Aggregation-Practical-5-WindowFunctions

11. Aggregation Concepts - RollUp and Cube

12. Aggregation-Practical-6-RollupandCube

13. SPARK JOINS

1. Spark Joins Theory-1-Introduction

2. Spark Joins Theory-2-How Joins Work

3. Spark Joins-Theory-3-Inner Joins

4. Spark Joins -Practical -1-Innerjoins

5. Saprk Joins - Theory-4 - Outer Joins

6. Spark Joins -Practical - Outer Joins

7. Spark Joins -Theory - 5-Left Semi And Anti Joins

8. Spark Joins - Practical - Left Semi And Anti Joins

9. Spark Joins -Theory -6-CrossJoin

10. Spark Joins - Practical- Cross Joins

11. Spark Joins -Theory -7-Challenges In Joins

12. Spark Joins-5-Practical-Tackling the Challenges in Joins

13. Spark Joins -Theory -8-Communication Strategies

14. RESILIENT DISTRIBUTED DATASETS- RDDs

1. What is an RDD

2. Introduction to Low Level APIs

3. Properties Of RDD

4. When to use RDDs

5. Creating RDDs

6. RDD Practical-1-Creating RDDs

7. RDD Lineage

8. RDD Transformations

9. RDD - Transformations Practical

10. RDD Actions

11. RDD Actions - Practical

12. RDDT Saving To File

13. RDD Saving to a File - Practical

15. DISTRIBUTED VARIABLES

1. Distributed Variables - Introduction

2. Broadcast Variables

3. Broadcast Variables - Practical

4. Accumulators

5. Accumulators - Practical

16. HOW SPARK WORKS ON A CLUSTER

1. Introduction

2. How Spark runs on a Cluster - Cluster Manager

3. How Spark runs on a Cluster - Execution Modes

4. Life Cycle a Spark Application - Outside Spark

5. Life Cycle of a Spark Application - Inside Spark

53,700 تومان

خرید اشتراک افزودن به سبد خرید

خرید دانلودی فوری

در این روش نیاز به افزودن محصول به سبد خرید و تکمیل اطلاعات نیست و شما پس از وارد کردن ایمیل خود و طی کردن مراحل پرداخت لینک های دریافت محصولات را در ایمیل خود دریافت خواهید کرد.

تولید کننده: Udemy-Training