ml-pipeline-workflow

ML Pipeline Workflow

Complete end-to-end MLOps pipeline orchestration from data preparation through model deployment.

Do not use this skill when

• The task is unrelated to ml pipeline workflow
• You need a different domain or tool outside this scope

Instructions

• Clarify goals, constraints, and required inputs.
• Apply relevant best practices and validate outcomes.
• Provide actionable steps and verification.
• If detailed examples are required, open resources/implementation-playbook.md.

Overview

This skill provides comprehensive guidance for building production ML pipelines that handle the full lifecycle: data ingestion → preparation → training → validation → deployment → monitoring.

Use this skill when

• Building new ML pipelines from scratch
• Designing workflow orchestration for ML systems
• Implementing data → model → deployment automation
• Setting up reproducible training workflows
• Creating DAG-based ML orchestration
• Integrating ML components into production systems

What This Skill Provides

Core Capabilities

1. Pipeline Architecture

• End-to-end workflow design
• DAG orchestration patterns (Airflow, Dagster, Kubeflow)
• Component dependencies and data flow
• Error handling and retry strategies

1. Data Preparation

• Data validation and quality checks
• Feature engineering pipelines
• Data versioning an