00:00

Learning Apache Airflow

سرفصل های دوره

Get an introduction to Apache Airflow with Google cloud architect and data engineer Janani Ravi. With Apache Airflow—an open-source platform for developing, scheduling, and monitoring batch oriented workflows— you can schedule individual tasks that you want to run as a part of your processing pipeline, specify dependencies between these tasks, all programmatically in Python. Airflow manages these dependencies and ensures that tasks are executed in the right order. Join Janani as she shows you how to run workflows in Airflow, define tasks and dependencies, and use Python and SQLite operators. Learn about conditional branching, and how catch up and backfill work in Apache Airflow.

01 - Introduction

01 - An overview of Apache Airflow

02 - 1. Introducing Apache Airflow

01 - Apache Airflow concepts and architecture

02 - Initial setup macOS and Windows

03 - Understanding Airflow configuration

04 - Running Airflow

03 - 2. Running Workflows on Airflow

01 - Navigating and exploring the Airflow UI

02 - Creating your first DAG

03 - Running your first DAG

04 - Creating a DAG using the with keyword

04 - 3. Defining Tasks and Dependencies

01 - Specifying task dependencies

02 - Using the Bitshift operators for dependency specification

03 - Using bash scripts in the DAG

05 - 4. Using the Python Operator and XCom

01 - Introducing the Python operator

02 - Python operators and dependencies

03 - Passing parameters to Python callables

04 - Introducing XCom

05 - Using XCom to pass values

06 - Implementing a data transformation DAG pipeline, part 1

07 - Implementing a data transformation DAG pipeline, part 2

06 - 5. Using the SQLite Operator for SQL Operations

01 - Setting up the SQLite DB and connection

02 - Introducing the SQLite operator

03 - Implementing a SQL-based DAG pipeline, part 1

04 - Implementing a SQL-based DAG pipeline, part 2

07 - 6. Conditional Branching and TaskGroups

01 - Introducing branching

02 - Conditional branching with variables

03 - Task groups and edge labels

08 - 7. Catch Up, Backfill, and CRON Expressions

01 - Catch up with all previously scheduled runs

02 - Backfill scheduled runs

09 - Conclusion

01 - Summary and next steps

139,000 تومان

خرید اشتراک افزودن به سبد خرید

خرید دانلودی فوری

در این روش نیاز به افزودن محصول به سبد خرید و تکمیل اطلاعات نیست و شما پس از وارد کردن ایمیل خود و طی کردن مراحل پرداخت لینک های دریافت محصولات را در ایمیل خود دریافت خواهید کرد.

تولید کننده: LinkedIn Learning (Lynda)