وب سایت تخصصی شرکت فرین
دسته بندی دوره ها

Snowflake cloud database with ELT(Airflow+Python+Talend)

سرفصل های دوره

Learn to integrate ETL tools with Snowflake and leverage Airflow for ELT with snowflake.Process 250+ GB data.ELT flow.


1. Module 1
  • 1. Module 1 Intro

  • 2. prerequisites
  • 1.1 course ddl stmts.zip
  • 1. Setup snowflake environment
  • 2.1 Setting up virtual machine.docx
  • 2.2 Virtual machine download alternate link 2.html
  • 2.3 Virtual box download.html
  • 2.4 Virtual machine download.html
  • 2.5 Virtual machine download alternate link.html
  • 2. Virtual machine download
  • 3.1 airflow.zip
  • 3. Mount local folder to vm and start vm
  • 4.1 ssh pwd.zip
  • 4.2 Visual studio code download.html
  • 4. Setup Vs code for VM
  • 5. Configure connection details
  • 6. Github link.html

  • 3. Airflow intro
  • 1. Airflow introduction
  • 2. Airflow architecture
  • 3. First Dag
  • 4.1 Postgress airflow metadata db connection details.docx
  • 4.2 postgress airflow metadata db connection details.zip
  • 4. Introduction to metadata DB
  • 5. Airflow scheduler
  • 6. Scheduling in airflow
  • 7. Executors in airflow
  • 8. Airflow webserver
  • 9. Handle failures

  • 4. Snowflake airflow ELT
  • 1. Introduction
  • 2. Load to snowflake example 1
  • 3. [Demo] Load to snowflake example 1
  • 4. Load to snowflake example 2
  • 5. Load to snowflake example 3

  • 5. Snowflake python api intro -- Basics
  • 1.1 Python API.html
  • 1.2 python api.zip
  • 1.3 Python API.xlsx
  • 1. Introduction
  • 2. Connection test 1
  • 3. Connection test 2
  • 4. execute method
  • 5. cursor attributes
  • 6. execute many
  • 7. execute async part 1
  • 8. execute async part 2
  • 9. execute async part 3
  • 10. Formatting pyformat
  • 11. Formatting qmark
  • 12. Result metadata
  • 13. Type code

  • 6. Snowflake python api intro -- Advanced
  • 1. Fetch pandas arrow
  • 2. Context manager
  • 3. Result batch
  • 4. Get result batch
  • 5. Fetch arrow batches
  • 6. Fetch pandas batches
  • 7. To pandas arrow
  • 8. Execute stream
  • 9. Execute string
  • 10. Cursor dict

  • 7. Snowflake python ELT
  • 1. Copy data s3 to snowflake
  • 2. Copy data s3 to snowflake ( parallel )
  • 3. Execute ELT query Python way
  • 4. Schedule airflow job For python code

  • 8. Talend introduction
  • 1. Talend as open source ETL tool.
  • 2.1 Installing Talend.docx
  • 2. Download and install talend.
  • 3. Talend workspace introduction.
  • 4.1 Installing Talend.docx
  • 4.2 Section 2 lec 4 emp data.zip
  • 4.3 section 2 lec 4 emp ddl.zip
  • 4.4 Snowflake project.zip
  • 4.5 TOS DI-20181026 1147-V7.1.1.zip
  • 4. Build first job and configure talend
  • 5. Build sample job Level 1
  • 6. Build sample job Level 2
  • 7. Build sample job Level 3
  • 8. Building sample job Level4
  • 9. Build sample job Level5
  • 10. Build sample job Schedule using airflow
  • 11. Summary

  • 9. Module 2
  • 1. Module 2 intro

  • 10. Create generic components
  • 1. Analyst requirement.
  • 2.1 S3snowcp.zip
  • 2. S3 to snowflake copy app demo
  • 3. Requirement to process data.
  • 4.1 Eltjobdemo.zip
  • 4. Elt app demo
  • 5. Scheduling job.
  • 6. Writing generic dag code part1
  • 7. Writing generic dag code part2
  • 8. Writing generic dag code part3

  • 11. Building workflow
  • 1. Introduction
  • 2.1 Data for this section.html
  • 2. High level design
  • 3.1 Data for this section.html
  • 3. Copy data from s3
  • 4. Parsing data
  • 5. Audit src records
  • 6. Archive and create view part 1
  • 7. Archive and create view part 2
  • 8. Score records part 1
  • 9. Score records part 2
  • 10. prepare dag code
  • 11. Execute airflow job
  • 12. Improve the approach
  • 13. Parameter registry
  • 14. Summary

  • 12. Nyc traffic data demo
  • 1. Introduction.
  • 2.1 Nyc website.html
  • 2.2 Nyc data volume.xlsx
  • 2.3 Nyc data.7z
  • 2. Nyc traffic data overview
  • 3. Load sample data
  • 4. Nyc full data load
  • 5.1 Queries.zip
  • 5. Snow sight Dashboard

  • 13. Bonus sections starts
  • 1. Bonus section.html

  • 14. Generic jar logic
  • 1.1 create stage obj audit tbls.zip
  • 1.2 Creating integration object.docx
  • 1. Building talend generic copy job.
  • 2. Evaluating audit tables.
  • 3. ELT transform job demo.
  • 4.1 elt audit tbls ddl.zip
  • 4. ELT Generic job code demo.

  • 15. Bonus section Exporting talend jobs as jar apps
  • 1. Introduction.
  • 2. Managers requirement.
  • 3. Solution step 1 preparing query.
  • 4. Solution step 2 create talend job.
  • 5. Solution step 3 parameterise talend job.
  • 6.1 manager requirement1.zip
  • 6. Solution step 4 create jar application.
  • 7. Managers requirement 2.
  • 8. Solution step 1 creating talend job.
  • 9.1 managers requirement2.zip
  • 9. Solution step 2 using context groups.
  • 10. Developer problem
  • 11.1 cost warning.zip
  • 11. Cost warning app
  • 139,000 تومان
    بیش از یک محصول به صورت دانلودی میخواهید؟ محصول را به سبد خرید اضافه کنید.
    خرید دانلودی فوری

    در این روش نیاز به افزودن محصول به سبد خرید و تکمیل اطلاعات نیست و شما پس از وارد کردن ایمیل خود و طی کردن مراحل پرداخت لینک های دریافت محصولات را در ایمیل خود دریافت خواهید کرد.

    ایمیل شما:
    تولید کننده:
    مدرس:
    شناسه: 41230
    حجم: 5738 مگابایت
    مدت زمان: 530 دقیقه
    تاریخ انتشار: 15 آبان 1403
    دسته بندی محصول
    طراحی سایت و خدمات سئو

    139,000 تومان
    افزودن به سبد خرید