Speaker: Glen Mailer

After working with Honeycomb for a little while and starting to instrument our existing code with events, I’d become enamoured with the level of observability possible with that sort of telemetry. In particular, how easy it became to interactively and visually explore how my systems were behaving in production.

With these experiences fresh in my mind, I found myself in an unusual position of beginning development of a new service written in Go, when Go was a new language for services at the company (previously it had only been used for CLIs). This meant that we couldn’t use any existing internal libraries, and had a bit of freedom to imagine something different.

So I proposed an experiment to my team:
What if we never included a logging library in the new service, and only used event tracing?

This is the story of how that went.

Twenty years ago Kent Beck coined the term “test-driven development”: write tests first, develop the code later. Today, even if not practising true TDD, the idea of writing code without tests is an immediate warning sign to any developer. Yet, most teams still continue shipping code without adequate instrumentation to observe real system behaviour in production. Is it time we move to observability-driven development: “instrument first, develop later”?