The Big Data Developer Course

سرفصل های دوره

Become big Data Engineer with skills like Hadoop, Sqoop, Hive, Spark, Scala, Cassandra, HBase, NIFI, Kafka and much more

1. Introduction to the course

1. What is this course about

2. How to make best use of this course

2. Introduction to the Big Data World

1. Introduction to Hadoop

2. How MapReduce works

3. What is Big Data

4. [Notes] What is Big Data.html

5. Hadoop 1.0 Architecture

6. Hadoop 2.0 Architecture

7. Hadoop 3.0 Architecture

3. Setting up Cluster and doing hands on with Hadoop

1. Cloudera Software Installation

2. [Notes] Cloudera Software Installation.html

3. Hadoop Commands

4. [Notes] Hadoop Commands.html

5. Row Storage vs Column Storage

6. Serialized File Formats

7. [Notes] Serialized File Formats.html

4. Sqoop

1. Sqoop Introduction

2. Sqoop Import

3. [Notes] sqoop import.html

4. Sqoop Multiple Mappers

5. [Notes] Sqoop Multiple Mappers.html

6. import portion of data

7. [Notes] import portion of data.html

8. Sqoop eval and change the file delimiter

9. [Notes] Sqoop eval and change the file delimiter.html

10. incremental import

11. [Notes] incremental import.html

12. Password Protection

13. [Notes] Password Protection.html

14. Using Last Modified

15. [Notes] Using Last Modified.html

16. Import multiple File Formats

17. [Notes] Import multiple File Formats.html

18. Import multiple Tables

19. [Notes] Import multiple Tables.html

20. Handling Null during Import

21. [Notes] Handling Null during Import.html

22. Sqoop export

23. [Notes] Sqoop export.html

24. Sqoop Performance Tuning

25. [Notes] Sqoop Performance Tuning.html

5. Hive

1. Hive-Data Preparation

2.1 america.zip

2.2 countries.zip

2.3 india.zip

2. [Notes] Hive-Data Preparation.html

3. What is Hive

4. [Notes] What is Hive.html

5. Create and load a table in Hive

6. [Notes] Create and load a table in Hive.html

7. Hive Table Types

8. [Notes] Hive Table Types.html

9.1 america.zip

9.2 countries.zip

9.3 india.zip

9. Hive Partitions

10.1 america.zip

10.2 countries.zip

10.3 india.zip

10. [Notes] Hive Partitions.html

11. Hive Use Case

12. [Notes] Hive Use Case.html

13. Hive Buckets

14. [Notes] Hive Buckets.html

15. Schema Evolution in Hive

16. [Notes] Schema Evolution in Hive.html

17. Execute hive queries using a script

18. [Notes] Execute hive queries using a script.html

19. Working with Dates in Hive

20. [Notes] Working with Dates in Hive.html

21. Joins in Hive

22. [Notes] Joins in Hive.html

23. MSCK Repair

24. [Notes] MSCK Repair.html

25. Performance Tuning in Hive

26. [Notes] Performance Tuning in Hive.html

27. Hive vs SQL

28. [Notes] Hive vs SQL.html

29.1 hive notes.zip

29.2 hive queries.zip

29.3 hivebuiltinfunctions.zip

29. Hive Additional Resources

6. Installation for Spark and Scala

1. Installing and setting up Spark and Scala

2. [Notes] Installing and setting up Spark and Scala - Download links.html

7. Lets learn Scala

1. Introduction to Scala

2. Executing our First Scala Program

3. Scala Basics

4. Conditional Statements

5. Loops in Scala

6. Functions in Scala

7. Scala Class

8. Scala Inheritance Introduction

9. Single Inheritance

10. Multilevel Inheritance

11. Hierarchical Inheritance

12. Scala Traits - for Mutliple Inheritance

13. Hybrid Inheritance

14. Method overriding and Method Overloading

15. Singleton and Companion Object

16. Case Class

17. Abstraction and Final

18. Higher Order Functions and Lambda Expressions

19. What is Partially Applied Function

20. What is Currying

21. What is Option Type

22. Pattern Matching in Scala

23. Exception Handling in Scala

24. Scala Collections

25. [Notes] Scala Collections.html

26. Collection Methods

27. [Notes] Collection Methods.html

28. Group By vs Grouped

29. Variable Arguments - What is it and how is it useful

30. Working with Files

8. Introduction to Spark

1. What is Spark

2. Why is Spark Faster than MapReduce

9. Spark RDDs

1. RDD Basics - Reading and Writing a File

2. [Notes] RDD Basics - Reading and Writing a File.html

3. Deploying code to Cluster

4. Use Case - Analyze the Log Data

5. [Notes] Use Case - Analyze the Log Data.html

6. Common RDD Transformations and Actions

7.1 sales.zip

7. What is Pair RDD

8.1 words.zip

8. Use Case - The word count example

9. Using Schema RDD

10. Using Row RDD

10. Spark DataFrames

1. What is Spark DataFrame

2. Creating DataFrames from RDD

3. Spark Seamless Dataframe- Reading and Writing

4. [Notes] Spark Seamless Dataframe- Reading and Writing.html

5.1 spark-avro_2.11-4.0.0 (1).zip

5. Reading and Writing AVRO Data

6.1 commons-io-2.8.0 (1).zip

6.2 spark-xml_2.11-0.11.0 (1).zip

6.3 txw2-2.3.3 (1).zip

6.4 xmlschema-core-2.2.5 (1).zip

6. Reading and Writing XML Data

7. [Notes] Reading and Writing XML Data.html

8.1 random_user.zip

8.2 user.zip

8. Reading Multi Lines Json

9. [Notes] Reading Multi Lines Json.html

10. Write Modes in Spark

11. Passing schema to a file

12. Applying Transformations using tempView and DSL

11. Spark Advance

1. Lets explore more transformations

2. [Notes] Lets explore more transformations.html

3. How to remove duplicates

4. Sorting the Data

5.1 account.zip

5. Handling Nulls in Spark

6. Working with String Functions

7. Working with Dates

8. Applying aggregation

9. Spark Windowing Functions

10. Pivoting in Spark

11. Passing List of columns to Dataframe

12.1 department.csv

12.2 employees.csv

12. Joins in Spark

13. Use case - Bank Transaction Data

14. Reading Current days file

15.1 dml.zip

15.2 input.zip

15. Working with Fixed Width File

16. [Notes] Working with Fixed Width File.html

12. Productionalizing your Code

1. How does your code run in Production

2. Deploy in prod - parameterizing the filenames

3.1 application.zip

3.2 application.zip

3.3 config-1.3.4 (1).zip

3. Parameterize using Config File

4. [Notes] Parameterize using Config File.html

5. Spark Hive Integration

6. Memory Tuning in Spark

13. Complex Data Processing

1.1 file.zip

1. Working with Json

2.1 file1.zip

2.2 file2.zip

2. Working with MultiLine Json

3.1 address.zip

3. Working with Nested Json

4.1 cake.zip

4. Working with Nested Json - Struct and Array

5. Reading Json from a web URL and flattening it

6. [Notes] Reading Json from a web URL and flattening it.html

7. Flattening data by creating a Function

8. [Notes] Flattening data by creating a Function.html

9.1 address_flat.zip

9.2 address.zip

9. Complex Data Generation

10.1 product.zip

10. Flatten XML File

14. NOSQL Databases

1. What is a NOSQL Database

2. Working with HBase

3. [Notes] Working with HBase.html

4.1 hbase-client-1.1.2.2.6.2.0-205 (1).zip

4.2 hbase-common-1.1.2.2.6.2.0-205 (1).zip

4.3 hbase-protocol-1.1.2.2.6.2.0-205 (1).zip

4.4 hbase-server-1.1.2.2.6.2.0-205 (1).zip

4.5 shc-core-1.1.1-2.1-s_2.11 (1).zip

4. Spark HBase Integration

5. [Notes] Spark HBase Integration.html

6. Cassandra Introduction

7. Cassandra setup and working with Cassandra

8.1 apache-cassandra-3.11.13-bin.tar.zip

8.2 datastax-community-64bit_2.2.3.zip

8. [Notes] Cassandra setup and working with Cassandra.html

9. Cassandra Spark Integration

10.1 jsr166e-1.1.0 (1).zip

10.2 spark-cassandra-connector_2.11-2.3.1 (1).zip

10. [Notes] Cassandra Spark Integration.html

11. Cassandra Query Limitations

15. Apache NIFI

1. Apache NIFI Introduction

2. Apache NIFI Installation

3. [Notes] Apache NIFI Installation.html

4. Lets work with Apache NIFI Tool

5. Streaming from a web URL

6. Back Pressure and Connection Queue

7. Promoting to Another Environment

16. Working with Streaming Data

1. Spark DStream

2. Kafka Introduction

3. kafka installation and topic creation

4. [Notes] kafka installation and topic creation.html

5. Kafka NIFI Integration

6. How Kafka Works

7. Spark Kafka Integration

8.1 kafka-clients-0.10.0.1.zip

8.2 lz4-1.3.0.zip

8.3 scala-library-2.11.8.zip

8.4 slf4j-api-1.7.21.zip

8.5 snappy-java-1.1.2.6.zip

8.6 spark-sql-kafka-0-10_2.11-2.3.0.zip

8.7 spark-streaming-kafka-0-10_2.11-2.3.0.zip

8.8 spark-tags_2.11-2.3.0.zip

8.9 unused-1.0.0.zip

8. [Notes] Spark Kafka Integration.html

9. Using Kafka Offset Explorer tool

10. Kafka Delivery Guarantee Feature

11.1 file1.zip

11.2 file2.zip

11.3 file3.zip

11.4 file4.zip

11.5 file5.zip

11. Spark Structured Streaming

12. [Notes] Spark Structured Streaming.html

13. Spark Kafka Integration

14. [Notes] Spark Kafka Integration.html

17. Extra

1. Performance Tuning Tips

139,000 تومان

خرید اشتراک افزودن به سبد خرید

خرید دانلودی فوری

در این روش نیاز به افزودن محصول به سبد خرید و تکمیل اطلاعات نیست و شما پس از وارد کردن ایمیل خود و طی کردن مراحل پرداخت لینک های دریافت محصولات را در ایمیل خود دریافت خواهید کرد.

تولید کننده: Udemy-Training