Streaming
Presentations
Resilient Real-Time Data Streaming Across the Edge and Hybrid Cloud
Hybrid cloud architectures are the new black for most companies. A cloud-first strategy is evident for many new enterprise architectures, but some use cases require resiliency across edge sites and multiple cloud regions. Data streaming with the Apache Kafka ecosystem is a perfect technology for...
Past Presentations
Extreme Programming Meets Realtime Data
At Unruly, we're fast and furious both in terms of the volumes we're handling and in our development process. Scaling incrementally has raised some interesting challenges in how to deal with growing data volumes, and by trying to deliver value in small steps, we've ended up with some...
Microservices At The Heart of BBC iPlayer
The BBC iPlayer API used to be a monolithic Java application. Two years ago we switched to Node.js microservices and we have been iterating on it ever since. We have learnt many things along the way including how to keep distributed systems decoupled, how to monitor the health of our services in...
Power of the Log:LSM & Append Only Data Structures
This talk is about the beauty of sequential access and append only data structures. We'll do this in the context of a little known paper entitled “Log Structured Merge Trees”. LSM describes a surprisingly counterintuitive approach to storing and accessing data in a sequential fashion. It came...
Interviews
Streaming a Million likes/second: Real-time Interactions on Live Video
What is the work you're doing today?
I'm the Tech Lead for LinkedIn Messaging and LinkedIn’s Real-time Distribution Platform. This is a platform that we use to deploy server-to-client streaming technology to power many dynamic experiences on LinkedIn. This includes instant distribution of likes, comments and concurrent viewer counts on live videos, instant...
Read Full InterviewFrom Batch to Streaming to Both
What is the work you're doing today?
I am a Principal Software Engineer at Skyscanner working on the data platform. This is the central data platform that powers all the Skyscanners' events, metrics and logs. My primary role there is making sure that the 2 million or so events we receive every second arrives safely and securely in long term storage, which is in S3,...
Read Full Interview