Thinking about building your own data pipeline? You're going to need more than band-aids and bubblegum to hold it together, but don’t just take our word for it.
Join us as Matt Snyder and Nate Piquette from Omnissa’s Security team share their hard-earned lessons while constructing and scaling a home-grown data pipeline. Spoiler alert: it wasn’t always smooth sailing, especially when growing SIEM ingestion to multiple terabytes per day.
This session will cover:
- Exploring the trade-offs between developing your own solution and purchasing one, and why the decisions you make today can ripple into long-term consequences.
- The ups and downs of growing a data pipeline from small to massive, and how to keep your sanity intact as you scale from a few logs to terabytes per day.
- Practical strategies for managing unruly data, ensuring reliability, and building a scalable pipeline.
Whether you're a seasoned engineer or new to pipeline construction, this talk will give you practical insights to help you navigate your own data pipeline journey.