Master Azure Databricks

سرفصل های دوره

Learn Databricks concepts, PySpark, Spark Structure Streaming, Delta lake, Databricks SQL Analytics, REST API & CLI

1. Introduction

1. Course Structure

2. Azure DataBricks

1. What is Data Pipeline

2. Azure databricks

3. Azure Databricks Architecture

4. Azure Account Setup

5. WorkSpace Setup

3. Workspace, Notebook, Job, DBFS

1. Agenda

2. Navigate the Workspace

3. Databricks Runtimes

4. Clusters Part 1

5. Cluster Part 2

6. Notebooks

7. Libraries

8. Repos for Git integration

9. Databricks File System (DBFS)

10.1 dbutils.pdf

10. DBUTILS

11. Widgets

12. Workflows

13. Metastore - Setup external Metastore

4. RDD, Database, Tables

1. Agenda

2. RDD Introduction

3.1 create_rdd.pdf

3. Lab Creating RDD

4.1 rdd_transformation.pdf

4. Lab RDD transformations

5.1 rdd_actions.pdf

5. Lab RDD Actions

6. Lab RDD Joins

7.1 pair_rdd.pdf

7. Lab Pair RDD

8.1 broadcast.pdf

8. Lab Broadcast Variables

9.1 accumulator.pdf

9. Lab Accumulators

10. Lab Convert RDD to DataFrame

11. LabImport & read data

12. Lab Create a table using the UI

13. Lab Create a table in a Notebook

5. DataFrame

1. Agenda

2. What is DataFrame

3.1 CREATE_DF.pdf

3. Lab Create DataFrames

4.1 schemas.pdf

4. Lab Define Schema

5.1 functions.pdf

5. Lab Functions

6.1 casting.pdf

6. Lab Casting Operations

7.1 filters.pdf

7. Lab Filter Transformation

8.1 update.pdf

8. Lab Update, Update ALL & UpdateByName

9.1 order_by.pdf

9. Lab OrderBy & SortBY

10.1 groupby.pdf

10. Lab GroupBy

11. Lab Remove Duplicates

12.1 window.pdf

12. Lab Window Functions

13.1 date&time.pdf

13. Lab Date and Timestamp Functions

14. Lab UDF (User Defined Function)

15.1 JOIN.pdf

15. Lab JOIN

16.1 handle_bad_data.pdf

16. Lab Handle corrupt records using the badRecordsPath

17.1 filemeta.pdf

17. LabFile metadata column

6. Data Source

1. Agenda

2.1 parquet.pdf

2. Lab Read and Write Parquet File

3. Lab Read and Write CSV Files

4. Lab Read and Write JSON Files

5.1 xml.pdf

5. Lab Read and Write XML Files

6.1 excel.pdf

6. Lab Read and Write Excel file

7. Lab Read and Write SQL databases using JDBC

8.1 azure_blob.pdf

8. Lab Read and Write Azure blob storage

7. Spark Structure Streaming

1. Agenda

2. What is Spark Structure Streaming

3. Data Source & Sink

4.1 input_source.pdf

4. Lab Rate & File Source

5.1 kafka_setup.pdf

5. Lab Kafka Source

6. Lab Sink Console, Memory, File & Custom

7. Lab Build Streaming ETL

8. Lab Setup Event Hub

9. Lab Event Hub Producer

10. Lab Integrate Event Hubs with Data Bricks

11. Lab Transformation

12. Streaming ETL 5 Ingest into Azure Data storage

13. Twitter Sentiment Analysis - Introduction

14. Setup Twitter Developer Account

15. Twitter Sentiment Analysis - II

16. Twitter Sentiment Analysis - III

8. DataBricks SQL

1. Agenda

2. Components in Databricks SQL

3. Lab Configuring a SQL Endpoint

4. Lab Creating a Table from a CSV File

5. Lab Create Queries

6. Lab Parameterized Query

7. Lab Query Profile

8. Lab Building Visualization (Table, BAR & PIE )

9. Lab Building Line Chart & Counter Chart

10. Lab Adding Charts to Dashboards

11. Lab Defining a Query Alert

12. Lab Access Control on Databricks SQL Objects

13. Lab Data Object Access Control

14. Lab Transfer Ownership

15. Lab Access SQL Endpoint from Python

16. Lab Databricks SQL CLI

9. Delta Lake

1. Agenda

2. Data Lakehouse Architecture

3. Medallion Lakehouse architecture

4. Delta Lake

5. Lab Create Delta Table (SQL & Python)

6. Lab Read & Write Delta Table

7. Lab Update Delete Merge

8. Lab Schema Validation

9. Lab Time Travel

10. Lab Convert a Parquet table to a Delta table

11. Lab Generated Columns

12.1 etl_pipeline.pdf

12. Lab Incremental ETL load

13. Lab Incremental ETL load (@version property)

10. Administration

1. Agenda

2. Manage User & Group

3. Lab Add User into Azure Active Directory

4. Lab Create Group

5. Lab Table Access Control

6. Lab Workspace, Cluster, Job Access

11. DataBricks Security

1. Agenda

2. DataBricks Security

3. Lab Secret management

4. Lab Part I - Column level Security

5. Lab Part II - Column level Security

6. Lab Row level Security

12. DataBricks CLI & Rest API

1. Agenda

2. DataBricks CLI

3. Setting up Databricks CLI

4. Lab Workspace CLIS

5. Lab Cluster CLI

6. Lab DBFS CLI

7. Lab Jobs CLI

8. Databricks CLI on Windows

9. REST API

10.1 invoke_rest.pdf

10. Lab Invoke REST API

11.1 job_rest_api.pdf

11. Lab Job REST API

12.1 token_rest_api.pdf

12. Lab Token Rest API

13.1 group_rest_api.pdf

13. Lab Group API

13. BI Tools

1. Agenda

2. What is BI

3. Power BI

4. Lab Setup BI Environments

5. Lab Connecting Power BI to DataBricks with JDBC

6. Lab Connecting Power BI to DataBricks with Partner Connect

7. What is Tableau & Download Tableau

8. Lab Connecting Tableau to DataBricks

9. Lab Connecting Tableau to DataBricks with Partner Connect

14. CICD Devops

1. Agenda

2. Lab Setup Azure Devops

3. Lab Integrate Databricks with Azure Devops

4. Lab Create Build pipeline

5. Lab Create Release Pipeline

139,000 تومان

خرید اشتراک افزودن به سبد خرید

خرید دانلودی فوری

در این روش نیاز به افزودن محصول به سبد خرید و تکمیل اطلاعات نیست و شما پس از وارد کردن ایمیل خود و طی کردن مراحل پرداخت لینک های دریافت محصولات را در ایمیل خود دریافت خواهید کرد.

تولید کننده: Udemy-Training