Speaker: Ismaël Mejía

The data engineering field has evolved at a tremendous pace in the last decade, new systems that enable the processing of huge amounts of data generated enormous opportunities, as well as challenges for software practitioners. All these new tools and methodologies created a new set of requirements for data engineers who tend to live in a constant state of catching up.  

In this talk, we review the current data landscape and discuss both technical and organizational ideas to avoid being overwhelmed by the current lack of consolidation of the data engineering world. We will discuss ideas from adopting open source APIs and open standards to more recent data management methodologies around operations, data sharing, and data products that enable us to create and maintain resilient and reliable data architectures.

The complexity of tools, distributed systems, and the CAP theorem introduce tradeoffs that practitioners cannot avoid or ignore as they embrace the world of modern data pipelines. What strategies can you employ? This is where data products come into play. Understanding the business objectives of data products helps us make informed decisions about tools, architecture, and services. Join this panel to learn from data thought leaders!