Amazon · amazon.jobs · checked today
Data Engineer III, Data Engineering India
The Amazon Delivery and Shopping Experience Product Team is looking for a Data Engineer-III who will develop data solutions to optimize customer experience by disproportionately improving discoverability and speed. As part of Amazon’s DEX product team, you have the opportunity to be at the forefront of DEX product thought leadership - working on some of the most difficult Customer Experience problems while partnering with research scientists, software developers, and business leaders. With the focus on driving real impact on Amazon's long-term profitability, we build new data sets & analysis f
Skills, with evidence
- Data modelling
Experience with data modeling, warehousing and building ETL pipelines
must have - Python
Optimize data pipelines, build and manage Python-based automations/tools for data pipelines
must have - SQL
Interface with other technology teams to extract, transform, and load data from a wide variety of data sources using Redshift SQL, Spark SQL and other AWS big data technologies
must have - Spark
Experience with big data technologies such as: Hadoop, Hive, Spark, EMR
must have - Warehousing
Experience operating large data warehouses
must have - Cost & performance
The Amazon Delivery and Shopping Experience Product Team is looking for a Data Engineer-III who will develop data solutions to optimize customer experience by disproportionately improving discoverability and speed.
- AWS
- Interface with other technology teams to extract, transform, and load data from a wide variety of data sources using Redshift SQL, Spark SQL and other AWS big data technologies
not practised here - Java
- Experience in at least one modern scripting or programming language, such as Python, Java, Scala, or NodeJS
not practised here - Scala
- Experience in at least one modern scripting or programming language, such as Python, Java, Scala, or NodeJS
not practised here
Your plan
- The SQL screen: correct, then fast≈ 2 h
SQL
- Median delivery time per cityIntermediate
- Bucket deliveries into quartilesIntermediate
- Median order value without a median functionIntermediate
- New and repeat orders by monthIntermediate
- Every order against its customer's averageIntermediate
- Python: the data-wrangling round≈ 3 h
Python
- Diff two snapshots of a tableIntermediate
- Explode an array column into rowsIntermediate
- Flatten nested event payloadsIntermediate
- Keep only the latest row for each eventIntermediate
- Pivot a long metrics table to wideIntermediate
- Data modelling: the round most people fail≈ 3 h
Data modelling · Warehousing
- Addresses that stay true to the pastIntermediate
- Seat holds and the release-night raceIntermediate
- Subscription warehouse grainIntermediate
- Campaign efficiencyIntermediate
- Catalogue: products, variants and sellersIntermediate
- Spark: read the plan Spark actually ran≈ 2 h
Spark · Cost & performance
- broadcast() with auto-broadcast off, and the case where Spark ignores itIntermediate
- autoBroadcastJoinThreshold compares an estimate: flip a join with select() and one settingIntermediate
- Does Spark really run your EXISTS subquery once per row?Intermediate
- left_semi and left_anti: "customers who did / never did" without a full joinIntermediate
- A self-join on a real key that still multiplies rowsIntermediate
- Say it out loud≈ 1 h
Not covered by the plan: AWS, Java, Scala.
Readiness
Counted from drills you have completed anywhere on D8LooP.
leaves in 5 dremoved the moment Amazon closes it