Data Pipeline Orchestration and Lineage Management

Build in-demand skills with a full year of Coursera Plus, now $239. Save now.

Data Pipeline Orchestration and Lineage Management

This course is part of Data Management, Governance & Optimization Specialization

Instructor: TSV Learning

Included with Learn more

2 modules

Gain insight into a topic and learn the fundamentals.

4 hours to complete

Flexible schedule

Learn at your own pace

2 modules

Gain insight into a topic and learn the fundamentals.

4 hours to complete

Flexible schedule

Learn at your own pace

What you'll learn

Compare Apache Airflow, Prefect, and Dagster to evaluate workflow orchestration, DAG dependencies, and pipeline execution strategies.
Apply OpenLineage, column-level lineage, idempotency, atomicity, and data contracts to improve pipeline reliability and traceability.
Evaluate cross-cloud orchestration, monitoring, data governance, and CI/CD practices to improve pipeline scalability and reliability.

Skills you'll gain

Tools you'll learn

Details to know

Shareable certificate

Add to your LinkedIn profile

See how employees at top companies are mastering in-demand skills

Learn more about Coursera for Business

logos of Petrobras, TATA, Danone, Capgemini, P&G and L'Oreal

Build your subject-matter expertise

This course is part of the Data Management, Governance & Optimization Specialization

When you enroll in this course, you'll also be enrolled in this Specialization.

Learn new concepts from industry experts
Gain a foundational understanding of a subject or tool
Develop job-relevant skills with hands-on projects
Earn a shareable career certificate

There are 2 modules in this course

Build reliable, scalable, and traceable data pipelines with modern workflow orchestration and data lineage management. This course explores how to improve pipeline reliability, transparency, and governance using Apache Airflow, OpenLineage, and data observability.

Designed for data professionals with basic Python, SQL, and pipeline knowledge, you'll compare Apache Airflow, Prefect, and Dagster while exploring DAG dependencies, workflow execution, and column-level data lineage. Learn how idempotency, atomicity, and data contracts strengthen pipeline reliability. Examine cross-cloud orchestration, state management, concurrency, scaling, and monitoring strategies to troubleshoot failures and optimize workflows. Discover how data catalogs support discovery, privacy, ownership, and cost attribution. Evaluate CI/CD practices, circuit breakers, and blue-green deployments to manage pipeline changes safely and protect downstream consumers. Through e-commerce, financial reporting, and real-time advertising case studies, you'll analyze orchestration strategies, assess lineage tools, and make informed decisions about pipeline reliability and auditability. What makes this course distinctive is its integrated approach to workflow orchestration, data lineage, operational reliability, and governance, helping you build trust in data-driven operations.

Compare orchestration architectures and analyze Airflow components, OpenLineage, and column-level dependencies. Apply reliability principles and evaluate cross-cloud execution, scaling, and monitoring strategies.

What's included

6 videos4 assignments

6 videosTotal 39 minutes

Workflow Orchestration and Lineage9 minutes
Apache Airflow Core Components6 minutes
Column-Level and Open Lineage5 minutes
Atomic and Reliable Data Pipelines5 minutes
Cross-Cloud Orchestration and Lineage6 minutes
Scaling and Monitoring Orchestrators7 minutes

4 assignmentsTotal 60 minutes

Building Reliable, Traceable Workflows30 minutes
Orchestration Foundations with Airflow10 minutes
Tracing Data and Ensuring Reliability10 minutes
Orchestrating Across Clouds at Scale10 minutes

Evaluate data catalogs, cost observability, and automated delivery practices. Assess safe deployment patterns, lineage tools, and industry case studies to improve accountability, troubleshooting, and data trust.

What's included

5 videos3 assignments

5 videosTotal 30 minutes

Data Catalog and Cost Observability7 minutes
CI-CD for Data Pipelines5 minutes
Circuit Breakers and Safe Deployments6 minutes
Modern Data Lineage Tools6 minutes
Data Lineage Case Studies6 minutes

3 assignmentsTotal 50 minutes

Operating and Evolving Data Pipelines30 minutes
Pipeline Visibility and Automated Delivery10 minutes
Safe Deployments and Lineage in Practice10 minutes

Earn a career certificate

Add this credential to your LinkedIn profile, resume, or CV. Share it on social media and in your performance review.

Instructor

TSV Learning

The Skill Valley

83 Courses544 learners

Offered by

The Skill Valley

Explore more from Support and Operations

Coursera
Orchestrate & Recover Real-Time Data Pipelines
Course
Edureka
Data Engineering Workflow Orchestration with Airflow
Course
Packt
Orchestrating Data Pipelines and Advanced Data Strategies
Course
Packt
Data Pipelines and Cloud Data Lakes: Spark, AWS, and Airflow
Course

Why people choose Coursera for their career

Felipe M.

Learner since 2018

"To be able to take courses at my own pace and rhythm has been an amazing experience. I can learn whenever it fits my schedule and mood."

Jennifer J.

Learner since 2020

"I directly applied the concepts and skills I learned from my courses to an exciting new project at work."

Larry W.

Learner since 2021

"When I need courses on topics that my university doesn't offer, Coursera is one of the best places to go."

Chaitanya A.

"Learning isn't just about being better at your job: it's so much more than that. Coursera allows me to learn without limits."

What brings you to Coursera today?

Start my career

Change my career

Grow in my current role

Explore topics outside of work

Give yourself skills that last, now $239
Save now
Gift your team new skills
Save 40% now

Frequently asked questions

To access course materials, assignments, and earn a Certificate, you'll need to purchase the Certificate experience when you enroll in a course. Eligible learners may also have the option to start with a Free Trial. Some courses may also offer a Full Course, No Certificate option. This lets you access course materials, submit required assessments, and receive a final grade, but you won't be able to earn or purchase a Certificate.

When you enroll in the course, you get access to all of the courses in the Specialization, and you earn a certificate when you complete the work. Your electronic Certificate will be added to your Accomplishments page - from there, you can print your Certificate or add it to your LinkedIn profile.

Yes. In select learning programs, you can apply for financial aid or a scholarship if you can’t afford the enrollment fee. If fin aid or scholarship is available for your learning program selection, you’ll find a link to apply on the description page.