OKX · job-boards.greenhouse.io · checked today
Staff Engineer, Machine Learning & Data Infrastructure
At OKX, we believe that the future will be reshaped by crypto, and ultimately contribute to every individual's freedom. OKX is a leading crypto exchange, and the developer of OKX Wallet, giving millions access to crypto trading and decentralized crypto applications (dApps). OKX is also a trusted brand by hundreds of large institutions seeking access to crypto markets.
Skills, with evidence
- Data modelling
Design and optimize offline data pipelines on Databricks, Apache Spark, and warehouse/lakehouse technologies, addressing data modeling, partitioning, joins, shuffles, skew, incremental processing, and cost.
must have - Incremental loads
Design and optimize offline data pipelines on Databricks, Apache Spark, and warehouse/lakehouse technologies, addressing data modeling, partitioning, joins, shuffles, skew, incremental processing, and cost.
must have - Python
Strong Python, SQL, and at least one backend or systems language such as Java, Scala, Go, or C++.
must have - SQL
Strong Python, SQL, and at least one backend or systems language such as Java, Scala, Go, or C++.
must have - Spark
Deep understanding of Databricks and Spark, or comparable offline data platforms, including execution plans, distributed query optimization, storage layouts, data correctness, and workload cost management.
must have - AWS
Build reusable infrastructure for model packaging, versioning, deployment, serving, monitoring, and rollback across cloud platforms and regions.
must have · not practised here - Azure
Build reusable infrastructure for model packaging, versioning, deployment, serving, monitoring, and rollback across cloud platforms and regions.
must have · not practised here - GCP
Build reusable infrastructure for model packaging, versioning, deployment, serving, monitoring, and rollback across cloud platforms and regions.
must have · not practised here - Cost & performance
Improve model inference latency, throughput, resource utilization, and cost.
- Streaming
You will help teams move models from experimentation into reliable production services, improve model execution performance, and build scalable data foundations for training, evaluation, and real-time decisions.
- Data quality
Experience with the model lifecycle, including training/serving integration, validation, deployment, observability, and rollback.
- Failure handling
Build reusable infrastructure for model packaging, versioning, deployment, serving, monitoring, and rollback across cloud platforms and regions.
- Idempotency & backfills
Connect offline and online feature pipelines, ensuring point-in-time correctness, consistent feature definitions, freshness, lineage, and reproducible backfills.
- Warehousing
Design and optimize offline data pipelines on Databricks, Apache Spark, and warehouse/lakehouse technologies, addressing data modeling, partitioning, joins, shuffles, skew, incremental processing, and cost.
Your plan
- The SQL screen: correct, then fast≈ 2 h
SQL · Data quality
- Airbnb: converting on a day with no published rateAdvanced
- Amazon: net revenue across two fan-outsAdvanced
- Build a percentile tableAdvanced
- Duplicate snapshot keysAdvanced
- Flag outlier ordersAdvanced
- Python: the data-wrangling round≈ 4 h
Python
- Data modelling: the round most people fail≈ 4 h
Data modelling · Warehousing
- Pipeline design: safe to run twice≈ 4 h
Incremental loads · Streaming · Failure handling · Idempotency & backfills
- Fourteen months of wrong numbersAdvanced
- Creator payout runAdvanced
- Delete me from everywhereAdvanced
- Realtime events pipelineAdvanced
- You fixed it. Who already used it?Advanced
- Spark: read the plan Spark actually ran≈ 2 h
Spark · Cost & performance
- applyInPandas per country: what Spark ships to Python, and the native rewriteAdvanced
- Filters you wrote in the wrong place: where Catalyst moves themAdvanced
- snappy, gzip or zstd: measure the Parquet codec trade-off yourselfAdvanced
- A correlated COUNT(*) subquery: the join Spark runs, and the count bug it avoidsAdvanced
- Find the four suspects in a slow pipeline (one of them is innocent)Advanced
- Say it out loud≈ 1 h
Not covered by the plan: AWS, Azure, GCP.
Readiness
Counted from drills you have completed anywhere on D8LooP.
leaves in 12 dremoved the moment OKX closes it
