Tools / Spreadsheets, BI and data engineering
Large-scale processing with Spark
Processing data that doesn't fit on one machine, with DataFrames that look a lot like pandas.
About 20 hours of study. It builds on Data wrangling with pandas and SQL.
Where to learn it
Apache Software Foundation
Quick start, then the SQL and DataFrames guide
Section not checked yet
No practice set picked yet.
Suggest oneWrong chapter, dead link, or know something better? Tell us on GitHub.
Your route to Large-scale processing with Spark
Pick where you’re starting from. Anything your background already covers is left out, and the rest is put in order.
4 stops to go, about 95 hours of study.
Open a stop to see the book, the video course and the practice for it. Tick its circle when you've finished.
- 40 h
Programming. Ready to start.
- Tool25 h
Data. After Programming in Python.
- 10 h
Data. Ready to start.
- Tool20 h
Data. After Data wrangling with pandas and SQL.
That's the whole route to Large-scale processing with Spark.
Which roles ask for it
- Data engineer to get hired
- Machine learning engineer good to have
- MLOps engineer good to have
- Analytics engineer good to have
- Data scientist good to have
What it needs and what it leads to: Large-scale processing with Spark on the map.