Mastercard · mastercard.wd1.myworkdayjobs.com · checked today
Principal Data Engineer (Data Platforms)
Mastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we’re helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible.
Skills, with evidence
- Data modelling
Strong foundations in modern data architectures such as lakehouse and medallion, data lifecycle management, data modeling, database design, distributed systems, and performance optimization.
must have - Data quality
Build governance, privacy, quality, lineage, cataloging, retention, and access management into the platform by design.
must have - Python
We are looking for a hands-on, passionate engineer and architect with strong PySpark, Python, cloud and modern data-architecture expertise.
must have - Spark
Design processing and query architectures using Databricks, Snowflake, Spark, and Trino, with open table and file formats such as Iceberg, Delta Lake, and Parquet.
must have - Streaming
Experience designing batch, streaming, API, and secure data-sharing architectures and integrating heterogeneous systems across cloud environments.
must have - AWS
Design AWS data-platform architecture using S3, IAM, EKS, networking, and relevant cloud-native data services, and Azure architecture using ADLS, Microsoft Entra ID, Azure RBAC, Key Vault, AKS, and relevant analytics services.
must have · not practised here - Java
Experience designing distributed control-plane applications using Java and/or Python, REST and gRPC APIs, asynchronous messaging, workflow and state management, authentication and authorization, multi-tenancy, idempotency, resiliency, auditability, and operational observability.
must have · not practised here - Kubernetes
Design Kubernetes platforms for portable compute, workload isolation, security, scaling, observability, and deployment across cloud, private, sovereign, and on-premises environments.
must have · not practised here - Airflow / orchestration
Own and optimize infrastructure for data storage, processing, orchestration, networking, identity, secrets, security, reliability, and cost.
- Cost & performance
evaluate federation, replication, and compute-to-data using latency, cost, regulation, residency, freshness, reliability, and operational complexity.
- Warehousing
This role will design scalable distributed and lakehouse platforms across regions and execution environments;
- Governance & security
establish federated governance, metadata, access, and query patterns;
not practised here - Azure
Design AWS data-platform architecture using S3, IAM, EKS, networking, and relevant cloud-native data services, and Azure architecture using ADLS, Microsoft Entra ID, Azure RBAC, Key Vault, AKS, and relevant analytics services.
not practised here - SQL
Design processing and query architectures using Databricks, Snowflake, Spark, and Trino, with open table and file formats such as Iceberg, Delta Lake, and Parquet.
Your plan
- The SQL screen: correct, then fast≈ 2 h
Data quality · SQL
- Airbnb: converting on a day with no published rateAdvanced
- Amazon: net revenue across two fan-outsAdvanced
- Build a percentile tableAdvanced
- Duplicate snapshot keysAdvanced
- Flag outlier ordersAdvanced
- Python: the data-wrangling round≈ 4 h
Python
- Data modelling: the round most people fail≈ 4 h
Data modelling · Warehousing
- Pipeline design: safe to run twice≈ 4 h
Streaming · Airflow / orchestration
- Spark: read the plan Spark actually ran≈ 2 h
Spark · Cost & performance
- applyInPandas per country: what Spark ships to Python, and the native rewriteAdvanced
- Filters you wrote in the wrong place: where Catalyst moves themAdvanced
- snappy, gzip or zstd: measure the Parquet codec trade-off yourselfAdvanced
- A correlated COUNT(*) subquery: the join Spark runs, and the count bug it avoidsAdvanced
- Find the four suspects in a slow pipeline (one of them is innocent)Advanced
- Say it out loud≈ 1 h
Not covered by the plan: AWS, Java, Kubernetes, Governance & security, Azure.
Readiness
Counted from drills you have completed anywhere on D8LooP.
leaves in 5 dremoved the moment Mastercard closes it
