Site Impact - Infrastructure & Data Modernization

Role in Project

Site Impact needed a way to modernize and scale-out their data management platform. We were tasked with architecting a push button, built to scale infrastructure to service their business operations.

‍

The Challenge

Site Impact reached out to us to discuss the ability of tackling uptime and scale problems faced with their current data management platform. Their current database and web platform would consistently go offline each month, causing $100,000’s in lost revenue every quarter.

‍

The Solution

After assessing the project during a Discovery phase, we were able to extract an MVP in order to produce what was desired. We focused heavily on resiliency for their infrastructure and modernization for their application stack. The output of this was migrating from VM’s to GKE for their frontend application & moving from MSSQL to BigQuery for their data anaylitics. The data management side would be handled by Composer, Dataproc and GCS.

‍

The Results

Key Technologies:

• ‍GKE – Autoscaling capabilities to allow for infinite growth and prevention of single point of failures for their frontend application.
• BigQuery – Ability to execute efficient SQL queries on tables 400GB large with hundreds of millions of rows of data, some tables spanning 600+ columns.
• Composer – Airflow provides a dependency-driven ETL pipeline which runs all needed manipulations and automatically presents the data up to BigQuery.
• Dataproc – Pyspark code utilizing dataproc’s compute, built to handle PB’s of data.

‍

We focused heavily on resiliency for their infrastructure and modernization for their application stack. The output of this was migrating from VM’s to GKE for their frontend application.

‍