Our client, an enterprise software platform company, has more than 40,000 installations of their log management software and needed to better understand global customer software usage.
We built automated data pipelines to extract, transform and load data from multiple sources including web, social media, and application servers, creating a realtime unified store of data in S3 ready for analytics. We designed an architecture including S3 as the data lake, Athena as the query engine, cloud formation to orchestrate nightly data aggregation, and realtime visualizations on AWS Quicksight.