Amazon · amazon.jobs · checked today
Data Engineer II, Eva Trust Dev Team
AWS Infrastructure Services owns the design, planning, delivery, and operation of all AWS global infrastructure. In other words, we’re the people who keep the cloud running. We support all AWS data centers and all of the servers, storage, networking, power, and cooling equipment that ensure our customers have continual access to the innovation they rely on.
Skills, with evidence
- Data modelling
1+ years of developing and operating large-scale data structures for business intelligence analytics using data modeling experience
must have - Data quality
Set and enforce data quality, freshness, and monitoring standards, including alarms and runbooks
must have - SQL
1+ years of developing and operating large-scale data structures for business intelligence analytics using SQL experience
must have - AWS
Experience with AWS technologies like Redshift, S3, AWS Glue, EMR, Kinesis, FireHose, Lambda, and IAM roles and permissions
must have · not practised here - Airflow / orchestration
Lead the migration of manual, high-maintenance data processes and legacy dashboards to automated, monitored pipelines
- Dashboards & BI
You'll lead the move of operational data from manual processes to automated pipelines, dashboards that generate themselves, and knowledge Eva can use the moment someone needs it.
not practised here - Cost & performance
You’ll collaborate with people across AWS to help us deliver the highest standards for safety and security while providing seemingly infinite capacity at the lowest possible cost for our customers.
- Failure handling
- Set and enforce data quality, freshness, and monitoring standards, including alarms and runbooks
- Spark
- Experience with AWS technologies like Redshift, S3, AWS Glue, EMR, Kinesis, FireHose, Lambda, and IAM roles and permissions
- Streaming
- Experience with AWS technologies like Redshift, S3, AWS Glue, EMR, Kinesis, FireHose, Lambda, and IAM roles and permissions
Your plan
- The SQL screen: correct, then fast≈ 1 h
Data quality · SQL
- Average order value by countryFoundations
- Line items per orderFoundations
- Revenue by buyer countryFoundations
- Customers who never orderedFoundations
- Hiring funnel by roleFoundations
- Data modelling: the round most people fail≈ 2 h
Data modelling
- Marketplace core entitiesFoundations
- Cinema seat bookingFoundations
- City parking baysFoundations
- Dating app matchesFoundations
- Food delivery ordersFoundations
- Pipeline design: safe to run twice≈ 3 h
Airflow / orchestration · Failure handling · Streaming
- After the first one finishesFoundations
- Run it for last TuesdayFoundations
- The history nobody keptFoundations
- The spreadsheet is a dependencyFoundations
- Too slow by morningFoundations
- Spark: read the plan Spark actually ran≈ 1 h
Cost & performance · Spark
- HAVING vs WHERE: where does a filter after groupBy actually run?Foundations
- countDistinct vs approx_count_distinct: what the extra shuffle buysFoundations
- COUNT(*) vs COUNT(column): the null trapFoundations
- Grouping by two columns: what changes in the shuffle?Foundations
- Does the join type change the join strategy?Foundations
- Say it out loud≈ 1 h
20 drills · Foundations + Intermediate≈ 8 hours
Not covered by the plan: AWS, Dashboards & BI.
Readiness
Counted from drills you have completed anywhere on D8LooP.
leaves in 11 dremoved the moment Amazon closes it