Masterclass on Azure Data Factory – Data Engineer for 2024

سرفصل های دوره

Learn ETL & Big Data Processing - Azure Data Factory(ADF), SQL Server, Blob, Datalake, REST API, PySpark, Databricks

01 - Initial Concepts

001 Data Factory creation

002 Resource Group

003 Top Level Concepts in ADF

004 Linked Service and Datasets

005 Blob Storage

006 Variables in ADF

007 Parameters

008 JSON Structure

009 Integration Runtime

02 - Pipeline Activities

001 Copy Activity

002 Append Variable Activity

003 Delete Activity

004 Execute Pipeline Activity

005 Fail Activity

006 Get Metadata Activity

007 Lookup Activity

008 Set Variable Activity

009 Wait Activity

03 - Conditional and Loop Activities

001 Switch Activity

002 Filter Activity

003 ForEach Activity

004 Until Activity

005 If Condition

04 - Data Flow Transformations

001 Conditional Split Transformation

002 Exists Transformation

003 Union Transformation

004 Join Transformation

005 Aggregate Transformation

006 Surrogate Key Transformation

007 Select Transformation

008 Lookup Transformation

009 Derived Column Transformation

010 Pivot Transformation

011 Unpivot Transformation

012 Rank Transformation

013 Window Transformation

014 Flatten Transformation

015 Filter Transformation

05 - Real Time Scenarios

001 Part 1 - Copy files dynamically from one storage location to other

002 Part 2 - Copy files dynamically from one storage location to other

003 Set Variable self assignment handling

004 Fetch file names from folder and assign to variable

005 Copying specific file pattern files from Source to Sink

006 ADF pipeline logging

007 Process CSV file and split single column into multiple columns

008 Copy Missed files from Source to Destination

009 Run ADF pipeline manaully-Call pipeline from other ADF pipeline (with REST API)

06 - Email Notification with ADF and Logic App

001 Email notification theory

002 Logic App Creation in Azure

003 Logic App and ADF Configuration for Email notification

004 Email notification configuration before and after pipeline with Stored procedure

07 - Data Load for Multiple Tables and multiple approaches

001 Full Load and Incemental load Theory

002 Copying multiple tables from database to database using ADF array

08 - Slowly Changing Dimensions

001 Fact and Dimension tables introduction

002 Slowly Chaning Dimension - Type 1

003 Slowly Changing Dimension - Type 2

004 Slowly Changing Dimension - Type 6

09 - Logs Maintenece in ADF

001 Logs Table Creation

002 Starting Logs Stored Procedure Creation

10 - Azure Key Vault

001 Azure Key Vault - Creation

11 - Azure Databricks - PySpark

001 Azure Databricks Service -Creation

002 JDBC Connection -How to connect to SQL server & read table from Azure Databricks

003 Mount to ADLS or Blob from Azure Databricks using PySpark

004 Read CSV file from DBFS with schema and header

005 withColumn() Transformation Function

12 - Azure Databricks - Delta Lake & Delta Table

001 Delta lake introduction

002 Datawarehouse Vs Data Lake Vs Delta Lake

003 Delta Table Architecture

004 Delta Table creation and observing its structure

139,000 تومان

خرید اشتراک افزودن به سبد خرید

خرید دانلودی فوری

در این روش نیاز به افزودن محصول به سبد خرید و تکمیل اطلاعات نیست و شما پس از وارد کردن ایمیل خود و طی کردن مراحل پرداخت لینک های دریافت محصولات را در ایمیل خود دریافت خواهید کرد.

تولید کننده: Udemy