Amazon · amazon.jobs · checked today
Data Engineer, ShipTech Analytics
Join ShipTech Analytics (STA) as a Data Engineer to build the data and analytics backbone powering Amazon's global transportation network. You'll contribute to our vision of building an intelligent, autonomous, and scalable performance management and decision support system across Amazon Transportation. As a Data Engineer II, you'll work on building standardized data solutions that empower operations teams worldwide.
Skills, with evidence
- Data modelling
Experience with data modeling, warehousing and building ETL pipelines
must have - Python
Experience in at least one modern scripting or programming language, such as Python, Java, Scala, or NodeJS
must have - SQL
Experience with SQL
must have - Spark
Experience with big data technologies such as: Hadoop, Hive, Spark, EMR
must have - Streaming
Knowledge of batch and streaming data architectures like Kafka, Kinesis, Flink, Storm, Beam
must have - AWS
Experience with AWS technologies like Redshift, S3, AWS Glue, EMR, Kinesis, FireHose, Lambda, and IAM roles and permissions
must have · not practised here - Data quality
- Own data quality and implement enhancements for datasets that enable operational excellence and improve customer experience
- Idempotency & backfills
- Build data products and data tools that streamline the complete data lifecycle including data ingestion, processing, backfill, storage, vending, and operational support modules.
- Governance & security
- Drive continuous improvement through code reviews, design discussions, operational best practices, and security compliance
not practised here - Java
- Experience in at least one modern scripting or programming language, such as Python, Java, Scala, or NodeJS
not practised here - Scala
- Experience in at least one modern scripting or programming language, such as Python, Java, Scala, or NodeJS
not practised here
Your plan
- The SQL screen: correct, then fast≈ 1 h
SQL · Data quality
- Average order value by countryFoundations
- Line items per orderFoundations
- Revenue by buyer countryFoundations
- Customers who never orderedFoundations
- Hiring funnel by roleFoundations
- Python: the data-wrangling round≈ 2 h
Python
- Clean spreadsheet headers into column namesFoundations
- Events normalization jobFoundations
- List the distinct composite keysFoundations
- Deduplicate rows, first one winsFoundations
- Remove repeats from a sorted list of partitionsFoundations
- Data modelling: the round most people fail≈ 2 h
Data modelling
- Marketplace core entitiesFoundations
- Cinema seat bookingFoundations
- City parking baysFoundations
- Dating app matchesFoundations
- Food delivery ordersFoundations
- Pipeline design: safe to run twice≈ 3 h
Streaming · Idempotency & backfills
- Run it for last TuesdayFoundations
- Counting it onceFoundations
- dbt: the DAG that never leaves the warehouseFoundations
- Too slow by morningFoundations
- Which day does it belong to?Foundations
- Spark: read the plan Spark actually ran≈ 1 h
Spark
- HAVING vs WHERE: where does a filter after groupBy actually run?Foundations
- countDistinct vs approx_count_distinct: what the extra shuffle buysFoundations
- COUNT(*) vs COUNT(column): the null trapFoundations
- Grouping by two columns: what changes in the shuffle?Foundations
- Does the join type change the join strategy?Foundations
- Say it out loud≈ 1 h
25 drills · Foundations + Intermediate≈ 10 hours
Not covered by the plan: AWS, Governance & security, Java, Scala.
Readiness
Counted from drills you have completed anywhere on D8LooP.
leaves in 8 dremoved the moment Amazon closes it