Speaker: Manos Karpathiotakis

Meta's hardware infrastructure comprises millions of machines, all of which generate data. The overall traffic generated reaches several petabytes every hour, while its processing typically takes place somewhere other than where it was originally generated. The task of collecting, aggregating, and delivering this traffic volume (with low latency and high throughput) requires a systematic approach. This talk will go through the architectural evolution of Scribe, a data transport service that encapsulates all the complexity behind moving data from point A to point B in streaming fashion.