Building Batch Data Pipelines on Google Cloud

سرفصل های دوره

Data pipelines typically fall under one of the Extra-Load, Extract-Load-Transform or Extract-Transform-Load paradigms. This course describes which paradigm should be used and when for batch data.

1. Introduction

1. Course Introduction

2. Introduction to Building Batch Data Pipelines

1. Module introduction

2. EL, ELT, ETL

3. Quality considerations

4. How to carry out operations in BigQuery

5. Shortcomings

6. ETL to solve data quality issues

03. Executing Spark on Dataproc

01. Module introduction

02. The Hadoop ecosystem

03. Running Hadoop on Dataproc

04. Cloud Storage instead of HDFS

05. Optimizing Dataproc

06. Optimizing Dataproc storage

07. Optimizing Dataproc templates and autoscaling

08. Optimizing Dataproc monitoring

09. Lab Intro - Running Apache Spark jobs on Dataproc

11. Summary

04. Serverless Data Processing with Dataflow

01. Module introduction

02. Introduction to Dataflow

03. Why customers value Dataflow

04. Building Dataflow pipelines in code

05. Key considerations with designing pipelines

06. Transforming data with PTransforms

07. Lab Intro - Building a Simple Dataflow Pipeline

09. Lab - Serverless Data Analysis with Dataflow - A Simple Dataflow Pipeline (

10. Aggregate with GroupByKey and Combine

11. Lab Intro - MapReduce in Beam

13. Lab - Serverless Data Analysis with Beam - MapReduce in Beam (Java)

14. Side inputs and windows of data

15. Lab Intro - Serverless Data Analysis with Dataflow - Side Inputs

17. Lab - Serverless Data Analysis with Dataflow - Side Inputs (Java)

18. Creating and re-using pipeline templates

19. Summary

05. Manage Data Pipelines with Cloud Data Fusion and Cloud Composer

01. Module introduction

02. Introduction to Cloud Data Fusion

03. Components of Cloud Data Fusion

04. Cloud Data Fusion UI

05. Build a pipeline

06. Explore data using wrangler

07. Lab Intro - Building and executing a pipeline graph

09. Orchestrate work between Google Cloud services with

10. Apache Airflow environment

11. DAGs and Operators

12. Workflow scheduling

13. Monitoring and Logging

14. Lab Intro - An Introduction to Cloud Composer

6. Course Summary

1. Course Summary

235,000 تومان

افزودن به سبد خرید

خرید دانلودی فوری

در این روش نیاز به افزودن محصول به سبد خرید و تکمیل اطلاعات نیست و شما پس از وارد کردن ایمیل خود و طی کردن مراحل پرداخت لینک های دریافت محصولات را در ایمیل خود دریافت خواهید کرد.

تولید کننده: PluralSight