SESSION + Live Q&A
Causal Consistency For Large Neo4j Clusters
In this talk we'll explore the new Causal clustering architecture for Neo4j. We'll see how Neo4j uses the Raft protocol for a robust underlay for intensive write operations, and how the asynchronous new scale-out mechanism provides enormous capacity for very demanding graph workloads.
We'll discuss the cluster architecture's new causal consistency model. Causal consistency is a big leap forward compared to the commonplace eventual consistency which makes it convenient to write applications that use the full capacity of the cluster. In particular we'll show how despite the mixture of concensus protocols and asynchronous replication, that Neo4j allows users to read their own writes straightforwardly and discuss why this is such a difficult achievement in distributed systems.
For the application developer, we'll show how Neo4j's Causal Clustering optimised drivers makes it easyto write applications that scale smoothly from a single server to a large, distributed cluster: a practical motivation for the distributed systems enthusiast.
Speaker
Jim Webber
Chief Scientist @Neo4j
Dr. Jim Webber is Chief Scientist with Neo Technology the company behind the popular open source graph database Neo4j, where he where he works on R&D for highly scalable graph databases and writes open source software. Jim has written two books on integration and distributed systems:...
Read moreFind Jim Webber at:
Location
Fleming, 3rd flr.
Track
Modern Distributed Architectures
Topics
NoSQLAP SystemCP SystemInterview AvailableShare
From the same track
Straggler Free Data Processing in Cloud Dataflow
One of the main causes of performance problems in distributed data processing systems (from the original MapReduce to modern Spark and Flink) is "stragglers." Stragglers are parts of the input that take an unexpectedly long time to process, delaying the completion of the whole job, and wasting...
Eugene Kirpichov
Cloud Dataflow Sr SE @Google
Spotify's Reliable Event Delivery System
Spotify’s event delivery system is one of the foundational pieces of Spotify’s data infrastructure. It has a key requirement to reliably deliver complete data with a predictable latency and make it available to Spotify developers via well-defined interface. Delivered data is than used to...
Igor Maravic
Software Engineer @Spotify
Realtime & Personalized Notifications @Twitter
Twitter Notifications Infrastructure enables hundreds of millions of users to stay informed about what’s going on in their Twitter world. Our systems process large volumes of data (aka the Twitter firehose) and deliver realtime and personalized notifications to all kinds of users, ranging from...
Gary Lam
Tech Lead Notifications, Staff Software Engineer @ Twitter
Saurabh Pathak
Leads Notifications Team @Twitter
Distributed Systems Theory for Practical Engineers
Distributed Systems are a complex topic. There's abundant research about it but sometimes it is hard for a beginner to know where to start. I would like to outline the main concepts of distributed systems, so the interested person can have a clear path on how to start their own research as well....
Alvaro Videla
Distributed Systems Engineer