Data is a big deal at IFTTT. Our business development and marketing teams rely on data to make critical business decisions. The product team relies on data to run tests, learn about product usage, and make product decisions. The data team builds products like the Recipe recommendation system and tools for spam detection. Our partners rely on data for insights and real-time performance alerts. Our data infrastructure is highly scalable, available, and flexible enough to support rapid product iteration. This post provides an overview of our data infrastructure and architecture, including data sources like MySQL, Kafka, and partner APIs. We use Kafka as our data transport layer, allowing easy addition of new data consumers. The data on S3 is ingested to AWS Redshift using Cranium, our in-house ETL platform. Cranium enables ad-hoc reporting with Ruby and D3, while Chartio supports most data visualization. This comprehensive data infrastructure empowers IFTTT in making data-driven decisions and delivering a seamless user experience.
