Ultimate Azure Data Factory: Cloud Data Engineering

سرفصل های دوره

Real world Modern Data Warehouse project for Data Engineers using Azure Data Factory, Sql, Data Lake, Databricks [DP203]

1. Overview

1. Welcome

2. What you will learn

3. Goal of this course

4. Commitment

5.1 ADFTutorial.zip

5. Course Materials

6.1 01-Chapter1 - Course Overview.pdf

6.2 02-Chapter2 - Introduction to Azure Data Factory.pdf

6.3 03-Chapter3 - Project Overview.pdf

6.4 04-Chapter4 - Environment.pdf

6.5 05-Chapter5 - Building a Data Pipeline.pdf

6.6 06-Chapter6 - Pipeline Activities and Parameters.pdf

6.7 07-Chapter7 - Mapping Data Flows.pdf

6.8 08-Chapter8 - Implementing Flowlets.pdf

6.9 09-Chapter9 - Controlling Pipeline Flow.pdf

6.10 10-Chapter10 - Building the Data Warehouse - Part 1.pdf

6.11 11-Chapter11 - Building the Data Warehouse - Part2.pdf

6.12 12-Chapter12 - Building the Delta Lake.pdf

6.13 13-Chapter13 - Presentation Layer.pdf

6.14 14-Chapter14 - Overview of Triggers.pdf

6.15 15-Chapter15 - Monitoring.pdf

6.16 16-Chapter16 - Conclusion.pdf

6. Course Slides.html

2. Introduction

1.1 02-Chapter2 - Introduction to Azure Data Factory.pdf

1. Introduction to Azure Data Factory

2. Why Azure Data Factory

3. What is Azure Data Factory

4. Benefits of Azure Data Factory

5. Azure Account

6. User Interface Azure Portal

7. Module Summary

3. Project Overview

1.1 03-Chapter3 - Project Overview.pdf

1. Hands-On Project Overview

2. Business Case for the Project

3. Solution Requirements

4. Architectural Patterns

5. Modern Data Warehouse Architecture

6. Hands-On Project Architecture

7. Repositories

8. Module Summary

4. Environment

1.1 04-Chapter4 - Environment.pdf

1. Module Overview

2. Software Tools

3. Software Tools Setup

4. Azure Resources

5. Setup Azure Resources

6. Setup Azure Resources in Azure Portal

7. Setup Azure Resource Group

8. Setup Azure Data Lake Storage

9. Setup Azure Data Factory Resource

10. Setup Azure Sql DB Resource

11. Review Azure Resources

12. Setup Azure Data Studio

13. Setup Azure Storage Explorer

14. Module Summary

5. Building a Data Pipeline

1.1 05-Chapter5 - Building a Data Pipeline.pdf

1. Module Overview

2. Building Blocks of Azure Data Factory - Main Components

3. Building Blocks of Azure Data Factory - Pipelines and Activities

4. Building Blocks of Azure Data Factory - How they Tie Together

5. Azure Data Factory User Interface - Main Page

6. Azure Data Factory User Interface - Authoring Canvas

7. Data Sources

8. Data Sources - Data Ingestion

9. Data Sources - Data Organization

10. Building the Data Pipeline

11. Building the Data Pipeline - Creating the Containers

12. Building the Data Pipeline - Creating the Pipeline

13. Building the Data Pipeline - Review and Organize

14. Importing Semi-Structured Data

15. Importing Semi-Structured Data - Building the Pipeline

16. Importing Semi-Structured Data - Organizing the Pipeline

17. Importing Semi-Structured Data - Recap of the Lesson

18. Naming Conventions

19. Module Summary

6. Pipeline Activities and Parameters

1.1 06-Chapter6 - Pipeline Activities and Parameters.pdf

1. Module Overview

2. Activities

3. Activity Dependencies

4. Activity Dependencies - Examples

5. Copy Activity

6. Copy Activity Concepts - Examples

7. Expressions and Variables

8. Expressions and Variables - Examples

9. Parameters

10. Parameters - Examples

11. Azure Key Vault - Overview

12. Azure Key Vault - Setup

13. Azure Key Vault - Create Linked Service

14. Importing Semi-Structured Data

15. Module Summary

7. Mapping Data Flows

1.1 07-Chapter7 - Mapping Data Flows.pdf

1. Module Overview

2. Introduction to Mapping Data Flows

3. Scenarios for Mapping Data Flows

4. User Interface of Mapping Data Flows

5. User Interface of Mapping Data Flows - Debug Feature

6. Implementing a Mapping Data Flow - Overview

7. Implementing a Mapping Data Flow - Pipeline and Data Sources

8. Implementing a Mapping Data Flow - Adding Transformations

9. Implementing a Mapping Data Flow - Pipeline Execution

10. Mapping Data Flow - Concepts

11. Mapping Data Flow - Concepts Example

12. Performance of Mapping Data Flows - Integration Runtime

13. Performance of Mapping Data Flows

14. Module Summary

8. Implementing Flowlets

1.1 08-Chapter8 - Implementing Flowlets.pdf

1. Module Overview

2. Introduction to Flowlets

3. Scenarios for Flowlets

4. User Interface of Flowlets - Overview

5. User Interface of Flowlets - Create a Demo Flowlet

6. Implementing a Flowlet - Create Flowlet

7. Implementing a Flowlet - Use the Flowlet

8. Module Summary

9. Controlling Pipeline Flow

1.1 09-Chapter9 - Controlling Pipeline Flow.pdf

1. Module Overview

2. Asserts

3. Implementing Asserts - Assert Expect True

4. Implementing Asserts - Identifying Error Rows

5. Implementing Asserts - Processing Error Rows

6. Error Handling Overview

7. Implementing Error Handling - Fail Activity

8. Implementing Error Handling - Capturing Errors

9. Implementing Error Handling - Logging Errors

10. Implementing Error Handling - Review of Error Pipeline

11. Integrating Data Quality and Error Handling

12. Building Pipelines using Pre-Built Templates

13. Module Summary

10. Building the Data Warehouse - Part 1

1.1 10-Chapter10 - Building the Data Warehouse - Part 1.pdf

1. Module Overview

2. Data Warehouse Overview

3. Data Warehouse Models

4. Data Warehouse Vino World

5. Data Process

6. Building the Azure Sql Database - Create the Stage Tables

7. Building the Azure Sql Database - Create the DW Tables

8. Building the Staging Layer Master Data - Master data

9. Building the Staging Layer Master Data - Product data

10. Building the Staging Layer Master Data - Metadata approach

11. Building the Staging Layer Master Data - Create Parameter Datasets

12. Building the Staging Layer Master Data - Create Metadata Pipeline

13. Building the Staging Layer Master Data - Pipeline execution

14. Building the Staging Layer Transaction Data

15. Building the Staging Layer Product Data - Combine Product Data

16. Building the Staging Layer Transaction Data - Combine Sales Data

17. Module Summary

11. Building the Data Warehouse - Part 2

1.1 11-Chapter11 - Building the Data Warehouse - Part2.pdf

1. Module Overview

2. Dimensions - Overview of Dimensions

3. Dimensions - Slowly Changing Dimensions

4. Dimensions - Master Dimensions and SCD Type

5. Building Type1 Dimensions - Using Data Flows

6. Building Type 1 Dimensions - Pipeline Review

7. Building Type 1 Dimensions - Using Stored Procedures

8. Dimensions - Overview of Type2 Dimensions

9. Building Type 2 Dimensions - Product Dimension

10. Building Type 2 Dimensions - Using Data Flows - Step1

11. Building Type 2 Dimensions - Using Data Flows - Step2

12. Building Type 2 Dimensions - Pipeline Review

13. Building Type 2 Dimensions - Using Stored Procedures

14. Building Dimensions - Build remaining dimensions

15. Facts - Overview

16. Building Facts

17. Data Warehouse Review and Data Analysis

18. Module Summary

12. Building the Delta Lake

1. Module Overview

2. Recap of what we implemented

3. What we will implement

4. Azure Databricks

5. What is Azure Databricks

6. Core Artifacts of Azure Databricks

7. Setup Azure Databricks

8. Setup Databricks Resource

9. Databricks UI Overview

10. Databricks Cluster Overview

11. Create Databricks Cluster

12. Azure Service Principal and Access to Data Lake Storage

13. Mount Azure Data Lake Storage

14. Overview of Delta Lake Implementation

15. What is a Delta Lake

16. Create Data Source for the Delta Table

17. Create Delta Table

18. Load Delta Table

19. Update Delta Table

20. Delta Table Concepts

21. Create Linked Service to Databricks from Data Factory

22. Executing Databricks Notebook from Data Factory

23. Module Summary

13. Presentation Layer

1.1 13-Chapter13 - Presentation Layer.pdf

1. Module Overview

2. Overview - Modern Data Warehouse

3. Overview - What we implemented

4. Overview - What we will implement

5. PowerBI - Installation

6. PowerBI - Overview

7. PowerBI - Connecting to the Data Warehouse

8. PowerBI - Building the Tabular Model

9. PowerBI - Building the Report

10. PowerBI - Report Requirements

11. PowerBI - Report Review

12. Module Summary

14. Triggers

1.1 14-Chapter14 - Overview of Triggers.pdf

1. Module Overview

2. Overview of Triggers

3. Approach to Pipeline Execution

4. Implementing a Master Pipeline

5. Executing the Master Pipeline

6. Implementing Event-based triggers

7. Executing Event-based Triggers

8. Scheduling Pipelines

9. Creating a Tumbling Window Trigger

10. Module Summary

15. Monitoring

1.1 15-Chapter15 - Monitoring.pdf

1. Module Overview

2. Executing Event-based triggers

3. Overview of Data Factory Monitoring

4. What do we monitor in Azure Data Factory

5. Visual Monitoring in Azure Data Factory

6. Pipeline Recovery

7. Setup Alerts

8. Validate the Alert

9. Metrics

10. Module Summary

16. Section 16 Conclusion

1.1 16-Chapter16 - Conclusion.pdf

1. Summary

189,000 تومان

افزودن به سبد خرید

خرید دانلودی فوری

در این روش نیاز به افزودن محصول به سبد خرید و تکمیل اطلاعات نیست و شما پس از وارد کردن ایمیل خود و طی کردن مراحل پرداخت لینک های دریافت محصولات را در ایمیل خود دریافت خواهید کرد.

تولید کننده: Udemy