Databricks – Master Azure Databricks for Data Engineers

سرفصل های دوره

Learn Azure Databricks for professional data engineers using PySpark and Spark SQL

1. Before you start

1. Course Prerequisites

2. About the Course

3. How to access Course Material and Resources

4.1 Capstone Project.zip

4.2 Notebooks.zip

4.3 SampleData.zip

4. Note for Students - Before Start

2. Introduction

1. Introduction to Data Engineering

2. Apache Spark to Data Engineering Platform

3.1 02-Introduction.pdf

3. Introduction to Databricks Platform

3. Getting Started

1. What will you learn in this section

2. Creating Azure Cloud Account

3. Azure Portal Overview

4. Creating Databricks Workspace Service

5. Introduction to Databricks Workspace

6.1 03-Getting Started.pdf

6. Azure Databricks Platform Architecture

4. Working in Databricks Workspace

1. What will you learn in this section

2. How to create Spark Cluster

3. Working with Databricks Notebook

4. Notebook Magic Commands

5.1 04-Working in Databricks Workspace.pdf

5.2 ch4-working in databricks workspace.zip

5. Databricks Utilities Package

5. Working with Databricks File System - DBFS

1. What will you learn in this section

2. Introduction to DBFS

3. Working with DBFS Root

4.1 05-Working with Databricks File System - DBFS.pdf

4.2 ch5-working with databricks file system - dbfs.zip

4.3 people.zip

4. Mounting ADLS to DBFS

6. Working with Unity Catalog

1. What will you learn in this section

2. Introduction to Unity Catalog

3. Setup Unity Catalog

4. Unity Catalog User Provisioning

5.1 06-Setting up Unity Catalog.pdf

5.2 ch6-working with unity catalog.zip

5. Working with Securable Objects

7. Working with Delta Lake and Delta Tables

1. What will you learn in this section

2. Introduction to Delta Lake

3. Creating Delta Table

4. Sharing data for External Delta Table

5. Reading Delta Table

6. Delta Table Operations

7. Delta Table Time Travel

8. Convert Parquet to Delta

9. Delta Table Schema Validation

10. Delta Table Schema Evolution

11. Look Inside Delta Table

12.1 07-Working with Delta Lake and Delta Tables.pdf

12.2 CH7-Data Files.zip

12.3 ch7-working with delta lake and delta tables.zip

12. Delta Table Utilities and Optimization

8. Working with Databricks Incremental Ingestion Tools

1. What will you learn in this section

2. Architecture and Need for Incremental Ingestion

3. Using Copy Into with Manual Schema Evolution

4. Using Copy Into with Automatic Schema Evolution

5. Streaming Ingestion with Manual Schema Evolution

6. Streaming Ingestion with Automatic Schema Evolution

7. Introduction to Databricks Autoloader

8.1 08-Working with Databricks Incremental Ingestion Tools.pdf

8.2 CH8-Data Files.zip

8.3 ch8-working with databricks incremental ingestion tools.zip

8. Autoloader with Automatic Schema Evolution

9. Working with Databricks Delta Live Tables (DLT)

1. What will you learn in this section

2. Introduction to Databricks DLT

3. Understand DLT Use Case Scenario

4. Setup DLT Scenario Dataset

5. Creating DLT Workload in SQL

6. Creating DLT Pipeline for your Workload

7.1 09-Working with Databricks Delta Live Tables (DLT).pdf

7.2 CH9-Data Files.zip

7.3 ch9-working with databricks delta live tables (dlt).zip

7. Creating DLT Workload in Python

10. Databricks Project and Automation Features

1. What will you learn in this section

2. Working with Databricks Repos

3. Working with Databricks Workflows

4. Working with Databricks Rest API

5.1 10-Databricks Project and Automation Features.pdf

5.2 CH10-Data Files.zip

5.3 ch10-databricks project and automation features.zip

5.4 Deploy.zip

5. Working with Databricks CLI

11. Capstone Project

1. Project Scope and Background

2. Taking out the operational requirement

3. Storage Design

4. Implement Data Security

5. Implement Resource Policies

6. Decouple Data Ingestion

7. Design Bronze Layer

8. Design Silver and Gold Layer

9. Setup your environment

10. Create a workspace

11. Create and Storage Layer

12. Setup Unity Catalog

13. Create Metadata Catalog and External Locations

14. Setup your source control

15. Start Coding

16. Test your code

17. Load historical data

18. Ingest into bronze layer

19. Process the silver layer

20. Handling multiple updates

21. Implementing Gold Layer

22. Creating a run script

23. Preparing for Integration testing

24. Creating Test Data Producer

25. Creating Integration Test for Batch mode

26. Creating Integration Test for Stream mode

27. Implementing CI CD Pipeline

28. Develop Build Pipeline

29. Develop Release Pipeline

30.1 Data Set.zip

30.2 Notebooks.zip

30.3 Other Code.zip

30. Creating Databricks CLI Script

12. Final Word

1. Congratulations

2. Keep Learning and Keep Growing.html

139,000 تومان

خرید اشتراک افزودن به سبد خرید

خرید دانلودی فوری

در این روش نیاز به افزودن محصول به سبد خرید و تکمیل اطلاعات نیست و شما پس از وارد کردن ایمیل خود و طی کردن مراحل پرداخت لینک های دریافت محصولات را در ایمیل خود دریافت خواهید کرد.

تولید کننده: Udemy