I recently had the pleasure of hosting a webinar with the team over at Rudderstack. It focused on Data Engineering and managing dynamic schemas at scale. More specifically, Rudderstack-generated, dynamic schemas at scale. This blog will discuss the tools, software, and methods to do just that.

In the daily life of a Site Reliability Engineer, the main goal is to reduce all the work we call toil. But what is toil? Toil is the kind of work tied to running a production service that tends to be manual, repetitive, automatable, tactical, devoid of enduring value, and scales linearly as a service grows. This blog post describes our journey to automate our nodes rotation process when we have a new AMI release and the open source tools we built on this.