Learning level
Foundations
Python For Data Engineering Mindset
Think in records, transforms, contracts, failure paths, and reviewable code instead of isolated syntax drills.
Records, Collections & Keyed Data
Lists, dicts, sets, tuples, grouping, lookup maps, dedupe keys, and nested records used in real transforms.
Strings, Regex & Text Normalization
Clean, parse, and normalize messy text: encodings, whitespace, casing, delimiters, and regex for real-world fields.
Files, Paths & Payload Ingestion
Read files and payloads safely with pathlib, CSV, JSON, NDJSON, encodings, manifests, and malformed-row handling.
Types, Time, Nulls & Numeric Correctness
Handle None, datetime values, time zones, Decimal math, rounding, optional fields, and type hints deliberately.
Learning level
Intermediate
Functions, Modules & Configuration
Structure transform code with small functions, modules, config objects, environment rules, and dependency boundaries.
Iterators, Generators & Streaming Data
Process data that does not fit in memory using lazy evaluation, yield, itertools, and streaming file pipelines.
Pure Python Batch Transform Patterns
Use map/filter/reduce-style flows, in-memory joins, aggregations, stateful scans, and sorted-window logic.
Serialization & File Formats
Move data between CSV, JSON, Parquet, Avro, and ORC: row vs columnar, compression, and schema-aware reads/writes.
Validation, Contracts & Error Handling
Design required-field checks, schema validation, rejected-row reports, warnings, exceptions, and contract evidence.
Learning level
Advanced
APIs, Databases & External Boundaries
Pull from APIs and databases with pagination, retries, rate limits, secrets boundaries, and idempotent source reads.
Concurrency, Parallelism & Performance
Choose threads, processes, or asyncio with the GIL in mind, then profile and tune before reaching for a cluster.
Testing, Debugging, Logging & Operability
Use pytest-style thinking, fixtures, golden outputs, logging, metrics, and traceable failures for production confidence.
Incremental, Idempotent & Orchestrated Pipelines
Build cursor, watermark, replay-window, checkpoint, manifest, and task-boundary habits for rerunnable jobs.
Scaling Python & Interview Capstone
Know when to move from pure Python to pandas, Polars, or Spark, and how to narrate those tradeoffs in interviews.