Stripe · stripe.com · checked today
Senior Staff Engineer, Big Data Infrastructure
Stripe is a financial infrastructure platform for businesses. Millions of companies - from the world’s largest enterprises to the most ambitious startups - use Stripe to accept payments, grow their revenue, and accelerate new business opportunities. Our mission is to increase the GDP of the internet, and we have a staggering amount of work ahead.
Skills, with evidence
- Airflow / orchestration
Drive the architecture and delivery of critical modernization bets, including YARN to Kubernetes migrations, native vectorization (Gluten/Velox), and Airflow and Data Catalog modernization
must have - Spark
Experience with big data infrastructure at scale, such as Spark, Iceberg, S3, Airflow, YARN, and Celeborn
must have - AWS
Serve as the technical leader for the Big Data Infrastructure organization, overseeing distributed systems that manage 20% of Stripe's total AWS spend
must have · not practised here - Kubernetes
Drive the architecture and delivery of critical modernization bets, including YARN to Kubernetes migrations, native vectorization (Gluten/Velox), and Airflow and Data Catalog modernization
must have · not practised here - Data quality
Experience developing, maintaining, and debugging distributed systems while optimizing their reliability and security
must have - Data modelling
Experience driving a culture of operational excellence, with a deep understanding of metrics, alerts, and dashboards
- Azure
Data Lake (Iceberg, Hive, S3, Azure Blob Storage)
not practised here - Warehousing
Data Lake (Iceberg, Hive, S3, Azure Blob Storage)
- Governance & security
Drive the architecture and delivery of critical modernization bets, including YARN to Kubernetes migrations, native vectorization (Gluten/Velox), and Airflow and Data Catalog modernization
not practised here - SQL
Data Lake (Iceberg, Hive, S3, Azure Blob Storage)
- Dashboards & BI
Experience driving a culture of operational excellence, with a deep understanding of metrics, alerts, and dashboards
not practised here
Your plan
- The SQL screen: correct, then fast≈ 2 h
Data quality · SQL
- Airbnb: converting on a day with no published rateAdvanced
- Amazon: net revenue across two fan-outsAdvanced
- Build a percentile tableAdvanced
- Duplicate snapshot keysAdvanced
- Flag outlier ordersAdvanced
- Data modelling: the round most people fail≈ 4 h
Data modelling · Warehousing
- Pipeline design: safe to run twice≈ 4 h
Airflow / orchestration
- Fourteen months of wrong numbersAdvanced
- You fixed it. Who already used it?Advanced
- The payload changed overnightAdvanced
- Creator payout runAdvanced
- Delete me from everywhereAdvanced
- Spark: read the plan Spark actually ran≈ 2 h
Spark
- applyInPandas per country: what Spark ships to Python, and the native rewriteAdvanced
- Filters you wrote in the wrong place: where Catalyst moves themAdvanced
- snappy, gzip or zstd: measure the Parquet codec trade-off yourselfAdvanced
- A correlated COUNT(*) subquery: the join Spark runs, and the count bug it avoidsAdvanced
- Find the four suspects in a slow pipeline (one of them is innocent)Advanced
- Say it out loud≈ 1 h
20 drills · Advanced + Intermediate≈ 13 hours
Not covered by the plan: AWS, Kubernetes, Azure, Governance & security, Dashboards & BI.
Readiness
Counted from drills you have completed anywhere on D8LooP.
leaves in 11 dremoved the moment Stripe closes it