Work

Case Studies & Engagement Log

Data engineering, streaming, and AI work across source systems, stacks, and cloud providers. Filter by what you are actually looking for.

This system was built by our founders in senior engineering roles, before Data Sharks was founded. The architecture and outcomes are described as delivered.

2 entries

Streaming & CDC5M/day CDC Records Processed in 1h Batches

SAP ECC to Databricks Lakehouse: Enterprise CDC Architecture on Azure

Engagement: Global Industrial Manufacturer

Architected an enterprise Lakehouse solution migrating 5M daily CDC records from SAP ECC (Oracle) to Azure Databricks. Features ODP/OData ingestion, automated PySpark schema evolution & drift alerting, multi-tier data quality quarantine, and Serverless compute optimization.

SparkPythonSQL
Streaming & CDC~60% Compute Cost Saved

Near Real-Time Data Ingestion

Engagement: Global E-Commerce Platform

Architected a high-throughput near real-time streaming ingestion pipeline syncing NoSQL document stores to a cloud analytical warehouse via event-driven triggers.

Cloud RunEventarcPub/SubTerraform+1