Mastercard · mastercard.wd1.myworkdayjobs.com · checked today
Senior Data Engineer
Mastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we’re helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible.
Skills, with evidence
- Data modelling
Deep understanding of data modeling, ETL/ELT design patterns, data architecture, and data lifecycle management.
must have - Data quality
Implement and enhance data quality, observability, governance, security, lineage, and monitoring capabilities across data ecosystems.
must have - Python
Proficiency in one or more programming languages such as Python, Java, Scala, or SQL.
must have - SQL
Proficiency in one or more programming languages such as Python, Java, Scala, or SQL.
must have - Spark
Strong expertise in modern data engineering technologies including Spark, Kafka, Hadoop ecosystem technologies, cloud-native data services, and large-scale data processing frameworks.
must have - AWS
Experience building and supporting solutions on cloud platforms such as AWS, Azure, or GCP.
must have · not practised here - Governance & security
Knowledge of data governance, security, privacy, lineage, metadata management, and regulatory compliance requirements.
must have · not practised here - Streaming
Design, build, deploy, and maintain scalable batch, streaming, and real-time data pipelines supporting business-critical products and analytics workloads.
- Airflow / orchestration
Develop reusable frameworks and services for data ingestion, transformation, orchestration, and data delivery across multiple platforms and environments.
- Failure handling
Implement and enhance data quality, observability, governance, security, lineage, and monitoring capabilities across data ecosystems.
- Warehousing
Build and support enterprise data lake, warehouse, and lakehouse solutions that enable large-scale data processing and analytics.
- Azure
Experience building and supporting solutions on cloud platforms such as AWS, Azure, or GCP.
not practised here - GCP
Experience building and supporting solutions on cloud platforms such as AWS, Azure, or GCP.
not practised here - Java
Proficiency in one or more programming languages such as Python, Java, Scala, or SQL.
not practised here
Your plan
- The SQL screen: correct, then fast≈ 2 h
Data quality · SQL
- Median delivery time per cityIntermediate
- Bucket deliveries into quartilesIntermediate
- Median order value without a median functionIntermediate
- New and repeat orders by monthIntermediate
- Every order against its customer's averageIntermediate
- Python: the data-wrangling round≈ 3 h
Python
- Diff two snapshots of a tableIntermediate
- Explode an array column into rowsIntermediate
- Flatten nested event payloadsIntermediate
- Pivot a long metrics table to wideIntermediate
- Choose what an incremental run should readIntermediate
- Data modelling: the round most people fail≈ 3 h
Data modelling · Warehousing
- Addresses that stay true to the pastIntermediate
- Seat holds and the release-night raceIntermediate
- Subscription warehouse grainIntermediate
- Campaign efficiencyIntermediate
- Catalogue: products, variants and sellersIntermediate
- Pipeline design: safe to run twice≈ 4 h
Streaming · Airflow / orchestration · Failure handling
- The source will not let youIntermediate
- Parcel tracking pipelineIntermediate
- Is this change safe?Intermediate
- Marketplace transactions at scaleIntermediate
- Five minutes behind the sourceIntermediate
- Spark: read the plan Spark actually ran≈ 2 h
Spark
- broadcast() with auto-broadcast off, and the case where Spark ignores itIntermediate
- autoBroadcastJoinThreshold compares an estimate: flip a join with select() and one settingIntermediate
- Does Spark really run your EXISTS subquery once per row?Intermediate
- left_semi and left_anti: "customers who did / never did" without a full joinIntermediate
- A self-join on a real key that still multiplies rowsIntermediate
- Say it out loud≈ 1 h
25 drills · Intermediate + Advanced≈ 14 hours
Not covered by the plan: AWS, Governance & security, Azure, GCP, Java.
Readiness
Counted from drills you have completed anywhere on D8LooP.
leaves in 7 dremoved the moment Mastercard closes it