Amazon · amazon.jobs · checked today
Data Engineer I, SPIV Data Engineering
Selling Partner Identity Verification is working to identify and prevent fraudulent behavior through all stages of a seller's lifecycle. We are seeking a skilled Data Engineer to join our team and help us build and manage the Data for the business domains within the Selling Partner Identity Verification space. As a Data Engineer, you will play a crucial role in designing, implementing, and maintaining our data infrastructure and pipelines.
Skills, with evidence
- Data modelling
design and implement scalable data models using medallion architecture
must have - Python
Experience with one or more scripting language (e.g., Python, KornShell)
must have - SQL
Experience with SQL
must have - Spark
Experience with big data technologies such as: Hadoop, Hive, Spark, EMR
must have - Warehousing
Experience with data modeling, warehousing and building ETL pipelines
must have - Failure handling
Implement data quality checks and monitoring systems to ensure data integrity and reliability.
- Cost & performance
Optimize data storage and retrieval systems for improved performance and cost-effectiveness.
- Data quality
Implement data quality checks and monitoring systems to ensure data integrity and reliability.
- Scala
- Experience with one or more query language (e.g., SQL, PL/SQL, DDL, MDX, HiveQL, SparkSQL, Scala)
not practised here
Your plan
- The SQL screen: correct, then fast≈ 1 h
SQL · Data quality
- Average order value by countryFoundations
- Line items per orderFoundations
- Revenue by buyer countryFoundations
- Customers who never orderedFoundations
- Hiring funnel by roleFoundations
- Python: the data-wrangling round≈ 2 h
Python
- Clean spreadsheet headers into column namesFoundations
- Events normalization jobFoundations
- List the distinct composite keysFoundations
- Deduplicate rows, first one winsFoundations
- Remove repeats from a sorted list of partitionsFoundations
- Data modelling: the round most people fail≈ 2 h
Data modelling · Warehousing
- Marketplace core entitiesFoundations
- Cinema seat bookingFoundations
- City parking baysFoundations
- Dating app matchesFoundations
- Food delivery ordersFoundations
- Pipeline design: safe to run twice≈ 3 h
Failure handling
- After the first one finishesFoundations
- The history nobody keptFoundations
- The spreadsheet is a dependencyFoundations
- Waiting for the fileFoundations
- Where did the rows go?Foundations
- Spark: read the plan Spark actually ran≈ 1 h
Spark · Cost & performance
- HAVING vs WHERE: where does a filter after groupBy actually run?Foundations
- countDistinct vs approx_count_distinct: what the extra shuffle buysFoundations
- COUNT(*) vs COUNT(column): the null trapFoundations
- Grouping by two columns: what changes in the shuffle?Foundations
- Does the join type change the join strategy?Foundations
- Say it out loud≈ 1 h
25 drills · Foundations≈ 10 hours
Not covered by the plan: Scala.
Readiness
Counted from drills you have completed anywhere on D8LooP.
leaves in 11 dremoved the moment Amazon closes it