When building a new data pipeline or diagnosing why an existing one is unreliable or too slow.
You are a senior {{role}} brought in to help a developer or tech professional complete a {{use_case}} task. # Context - Pack: Developers & Tech Professionals - Category: System Design & Architecture - Use case: Data Pipeline Architecture - Source task: - Design a data pipeline for {{describe_the_use_case_analytics_ml_training_reporting_etl}}. Data sources: {{list_sources}}. Data volume: {{volume_frequency}}. Target: {{data_warehouse_data_lake_ml_feature_store_reporting_db}}. Recommend: - 1. pipeline architecture (Lambda / Kappa / Streaming-first) - 2. technology stack for ingestion, processing, and storage - 3. data quality checks at each stage - 4. failure handling and retry strategy - 5. monitoring and alerting approach for pipeline health # Goal Architecture recommendation with stack, stage-by-stage data quality checks, failure handling, and pipeline monitoring design. # Constraints - Produce a complete, usable first draft in one response. - Avoid generic filler, vague advice, and unsupported claims. - Make the output specific, practical, and ready to use. # Output Architecture recommendation with stack, stage-by-stage data quality checks, failure handling, and pipeline monitoring design.
{{double-curly}} with your real context.When building a new data pipeline or diagnosing why an existing one is unreliable or too slow.
Design for late-arriving data from day one — real-world data pipelines always receive out-of-order events, and retrofit solutions are expensive.
Write a complete, SEO-optimised blog post on the given topic. Include a compelling headline, an engaging introduction, 4-5 subheadings with detailed body paragraphs, and a strong conclusion with a cal
Write a complete email newsletter including subject line, preview text, opening hook, main body content (3 short sections), and a clear call to action.
Write a complete YouTube video script including a strong hook (first 30 seconds), structured main content with transitions, and a closing that encourages likes, comments, and subscriptions.
Write a complete LinkedIn article that establishes professional authority, shares a genuine insight, and encourages professional discussion.