Difficulty: 8/10log ingestion, stream processing, storage & analyticsDistributed architecture with a message queue, worker nodes, and a distributed database, handling 100 GB of log data per dayScenario Questions0-2 years experience1How would you build a simple pipeline to collect logs from a few servers and make them searchable?2If you need to parse each log line to extract a timestamp and severity before storing, what steps would you take?3What would you do if a log producer sent malformed JSON into your pipeline?2-5 years experience1Follow-up QuestionsWhat metrics would you put in place to detect pipeline bottlenecks?How would you enforce data retention and compliance requirements?Can you estimate the cost impact of your storage choices at petabyte scale?Suggest Improvement
