This week, we're starting Module 6: Batch Processing.

Reminder: the previous homework deadline is in less than 24 hours.

In this module, you'll learn how batch processing works with Spark and PySpark.

You'll cover:

• Batch processing fundamentals and Spark basics
• Installing and running Spark locally or in Colab
• Working with Spark SQL and DataFrames
• Handling schemas and processing NYC taxi data
• How Spark clusters, joins, and groupBy work internally
• Running Spark in the cloud with Dataproc and BigQuery

Homework deadline: 10 March, 12 AM CET

We also recently had a workshop with dlt on AI-assisted data ingestion. Watch the recording and check out the code if you missed it. Practice what you learned in the homework assignment, and submit it here.

We uploaded the video about creating projects to YouTube, as some of you reported having problems with watching in on Loom

Here it is: https://www.youtube.com/watch?v=BL0E8xO8OnE

We'll also update it in the repo

And by the way, don't forget about our docs: https://datatalks.club/docs/courses/data-engineering-zoomcamp/

There's a lot of useful information there

We're starting our stream about streaming!

This is going to be a part of module 7 about streaming - and reworked workshop from the last year with the latest versions of PyFlink

Stream: https://www.youtube.com/watch?v=YDUgFeHQzJU
Workshop: https://github.com/DataTalksClub/data-engineering-zoomcamp/tree/main/07-streaming/workshop

Watch now or later in recording!

https://alexeyondata.substack.com/p/how-i-dropped-our-production-database

The post about the database incident last week

We're starting module 7 on stream processing.

Reminder: the previous homework deadline is in less than 24 hours.

The materials were created by Zach Wilson, who ran a Flink stream for the course last year. Alexey recorded an updated Apache Flink workshop to reflect support for Flink 2.x and modern Python versions (3.12, 3.9, 3.8).

It covers:

• Streaming fundamentals in Data Engineering Zoomcamp
• Kafka/Redpanda, Python producers and consumers
• Writing streaming events to PostgreSQL
• Apache Flink setup with Docker
• Flink jobs for stream processing
• Windowing, watermarks, and late events
• Real-time aggregations with Flink

Homework deadline: 17 March, 12 AM CET

Learn here: https://github.com/DataTalksClub/data-engineering-zoomcamp/tree/main/07-streaming

I just realized we haven't shared the homework for streaming

https://github.com/DataTalksClub/data-engineering-zoomcamp/blob/main/cohorts/2026/07-streaming/homework.md

The deadline is moved to 20 March

It’s time to apply everything from the course.

Build your final project with a complete end-to-end data pipeline.

It takes you through the full workflow:

🔸 Choose a dataset you’re interested in
🔸 Build a pipeline to ingest data into a data lake
🔸 Move the data to a data warehouse
🔸 Transform the data to prepare it for analysis
🔸 Build a dashboard to visualize the results

🎥 Watch the Projects how-to video for the full walkthrough and start building.

🏆 Tip: Use Bruin in your project to participate in the competition and win prizes. Details below.

Build your project using Bruin for ingestion, transformation, orchestration, and analysis, share it with the community, and compete for prizes.

Prizes:

🔸 Mac Mini for an outstanding project
🔸 1 year Claude Pro for the top 3 projects
🔸 1 month Claude Pro for participants

To participate:

🔸 Build your Zoomcamp project using Bruin
🔸 Publish it on GitHub with a README
🔸 Share it in #projects on Slack

Winners will be determined by community votes on Slack.

Learn more here: https://getbruin.com/zoomcamp-project/

A quick update: the Bruin project competition is now open to everyone!

This means your submission does not have to be the same as your Zoomcamp final project.

Even if you are not participating in the course, or if you are using other tech for your Zoomcamp final project, you can still build a separate project with Bruin, submit it to the competition, and compete for prizes.

Prizes:

🔸 Mac Mini for an outstanding project
🔸 1 year Claude Pro for the top 3 projects
🔸 1 month Claude Pro for participants

Deadline: Monday, June 1st, 12:00 UTC

More details here: getbruin.com/competition

Great job working on your projects!

Now it's time to learn from your peers

If you submitted your project for attempt, you will find your review assignments here:

https://courses.datatalks.club/de-zoomcamp-2026/project/project1/eval

If not - you still have time for attempt 2. Have fun!

We have just scored project attempt 1

Congratulations to the 269 course participants who passed it! We will release the certificates for you later along with the second batch

If you haven't passed it, you can improve it and submit one more time

Note that if you did pass the project, submitting it again (even with improvements) is considered self-plagiarism. Please don't do it. But you can submit another project if you want

Also if you haven't made a submission for attempt 1, attempt 2 is still open:

https://courses.datatalks.club/de-zoomcamp-2026/project/project2

Have fun building!

The form for submitting the second attempt is now closed.

Now time to learn from your peers!

If you submitted a project for attempt 2, you'll find the project review assignments here:

https://courses.datatalks.club/de-zoomcamp-2026/project/project2/eval

Have fun!

The certificates are ready! You'll find them in your enrollment profile for the course

Don't forget to share them on LinkedIn!

Thanks everyone for being a part of this course!

Some of you forgot to submit the project, and some of you forgot to do peer reviews. I see from Telegram and Slack that there are quite a few of you in this situation.

For those who already have a project but haven't submitted it yet, we have opened a third attempt.

It has a very strict deadline, so you won't have time to develop a new project. You can only submit the existing one.

https://courses.datatalks.club/de-zoomcamp-2026/project/project3

Good luck!

Judging by the questions in Slack, I realize that our documentation may be not complete or straightforward

I reorganized it a bit and here it is

https://datatalks.club/docs/courses/zoomcamp-logistics/

I hope you fill find the answers to your questions (especially around projects and peer reviews)

If you submitted your project for the last attempt, now it's time to evaluate your peers

You can find the assignments here:

https://courses.datatalks.club/de-zoomcamp-2026/project/project3/eval

Have fun!

Now Data Engineering Zoomcamp is almost over and I'm focusing on updating LLM Zoomcamp - the course about using GenAI, RAG and Agents

We will have multiple workshops prior to the course start date which will be used as the videos for the course when it starts

The first one is today. Join me if you're interested in GenAI!

I'm starting a workshop series to update the content of LLM Zoomcamp.

Each workshop will update the content of one course module:

​May 11: Build Your First RAG Application
May 18: From RAG to AI Agents: Function Calling and Tool Use
May 21: Vector Databases
May 27: RAG and Agents Evaluation
June 3: Monitoring LLM Applications

The Data Engineering Zoomcamp 2026 has officially wrapped up!

You had 3 chances to submit your final project and review 3 peers.

Congratulations to the 581 participants who successfully completed the course and earned their certificates!

If you're one of the graduates, you can find your well-deserved certificate in your course enrollment profile. We encourage you to celebrate this milestone by sharing it on LinkedIn!

Thank you goes out to everyone who joined us on this journey. We understand that not everyone was able to finish the course this time, and that’s perfectly fine. You’ll have another opportunity to complete it in a new cohort next year.

Thank you all for contributing to the learning experience of this course and DataTalks.Club community overall. We appreciate your hard work!