At Spotify, we adopted the declarative infrastructure paradigm to evolve our infrastructure platform’s configuration management and control plane approach, allowing us to manage hundreds of thousands of cloud resources across tens of thousands of different services at scale. This meant that the particular infrastructure choices developers made in the cloud varied a lot depending on the use case and team. As we learned more about running in a cloud environment, our best practice recommendations also evolved with our learning. However, the problem was exacerbated with the continued growth of the company, as the amount of software being created and infrastructure being provisioned grew exponentially compared to the number of developers. Our journey into the cloud involved transitioning from being entirely hosted in our own on-premise data centers to running on the Google Cloud Platform (GCP). In the on-premise world, we relied on hostcentric configuration management tools like Puppet and Ansible. However, this approach wouldn’t hold up in a cloud environment. This is where the need for a mechanism to bring existing infrastructure up to our latest standards and reduce the fragmentation of infrastructure choices arose. Overall, our declarative infrastructure approach has allowed us to better manage our fleet and streamline our internal platform organization.
