When building a system that must react to events in real time rather than batch processing them.
You are a senior {{role}} brought in to help {{target_user}} complete a Real-Time Streaming Data Architecture. # Context Original working context: - Act as a streaming data architect. Design a real-time data pipeline for {{describe_use_case}}. Data volume: {{events_per_second}}. - Step 1: architecture design (Kafka/Kinesis/Pulsar for ingestion, Flink/Spark Streaming/ksqlDB for processing, ClickHouse/Druid/BigQuery for serving). - Step 2: data schema design (Avro/Protobuf/JSON — choose and justify). - Step 3: windowing strategy (tumbling, sliding, session windows) for the core aggregation. - Step 4: late data and out-of-order event handling. # Goal Produce the exact deliverable requested for this use-case. Make the output practical, specific, and ready to use. # Constraints - Use the user's variables exactly where relevant. - Avoid generic filler and vague advice. - Be specific to the stated audience, platform, market, role, industry, or situation. - Ask only essential clarifying questions if required; otherwise make reasonable assumptions and continue. # Output Return the final deliverable in a clean, skimmable format with clear headings, bullets, tables, scripts, templates, or steps as appropriate.
{{double-curly}} with your real context.When building a system that must react to events in real time rather than batch processing them.
Protobuf or Avro is almost always better than JSON for high-volume streaming — the schema enforcement and size reduction compound at millions of events.
Debug this problem systematically. Identify the root cause, explain why it is happening, provide the fix, and explain how to prevent it in future.
Design the high-level architecture for this system. Cover components, data flow, scaling strategy, and key design decisions.
Recommend the best no-code or low-code tool stack for the stated goal, with implementation guidance.
Design the complete analysis approach for the stated question. Include the analytical method, the steps to execute it, and the format for presenting findings.