We're starting: https://www.youtube.com/watch?v=L2BHFnZ6XjE

Have fun =)

Homework for workshop 2: stream processing

Materials: https://github.com/DataTalksClub/data-engineering-zoomcamp/blob/main/cohorts/2024/workshops/rising-wave.md

Form: https://courses.datatalks.club/de-zoomcamp-2024/homework/workshop2

RisingWave will also give away swag for those who complete the homework!

Hi everyone!

We want to make homework 6 more interesting and helpful for you, but we need a bit more time to prepare it. Through this homework, you will learn how to use PySpark Streaming with Kafka. We will publish it by Monday.

You can already start with Module 6 videos. There are Java videos that focus more on the theory as well as Python videos that focus more on practice. We didn't have a chance to re-record the Python ones this year, but let us know if you want to volunteer and help with it.

Link to the module: https://github.com/DataTalksClub/data-engineering-zoomcamp/tree/main/06-streaming

In the meantime, you can work on the 2nd workshop and its homework.

The streaming homework is ready

https://github.com/DataTalksClub/data-engineering-zoomcamp/blob/main/cohorts/2024/06-streaming/homework.md

It took some time for us to prepare it, and it messed up with your plans, so you will have quite a bit of time to complete it (the deadline will overlap with projects). We will announce the deadlines later, but if you're now on vacations, you should be able to pick it up when you come back.

We will publish the form soon too.

Also, we updated the python videos from the streaming module - now they have much better audio quality

Thanks Ella Sahnan and Artsiom Turevich for help with the homework and the videos.

Finally, next week, we should start working on the projects. So on Monday, at 16:30 CET, we will have office hours to discuss it. You can already ask your questions here: https://app.sli.do/event/su9wCLiM9nHnCwtGBfgicX

We're starting the webinar about starting a career in data engineering

https://www.youtube.com/watch?v=xxSu2y7q9cg

(The recording will stay on the channel and is available for watching later)

We're starting office hours in 5 minutes

We will talk about your projects

Questions: https://app.sli.do/event/su9wCLiM9nHnCwtGBfgicX

Stream: https://www.youtube.com/watch?v=Of0ZqLrZXmc (recorded, available for replay later)

Projects: https://github.com/DataTalksClub/data-engineering-zoomcamp/blob/main/cohorts/2024/project.md

Remember that awesome-data-engineering document we mentioned at the launch stream but never actually used?

We now put it to github:

https://github.com/DataTalksClub/data-engineering-zoomcamp/blob/main/awesome-data-engineering.md

Hopefully it's helpful! PRs are welcome as always

Hi everyone!

How is it going with the project?

We're extending the deadline for submitting the project attempt 1 till Wednesday 23:00. Use this time wisely.


We also opened the form for submitting attempt 2

https://courses.datatalks.club/de-zoomcamp-2024/project/project2


Have fun!

Now it's time to evaluate your peers!

You can find the assignments here:

https://courses.datatalks.club/de-zoomcamp-2024/project/project1/eval

Make sure you're logged in

We finally scored the first project attempt

You can see the results here: https://courses.datatalks.club/de-zoomcamp-2024/project/project1/results

We still need to implement a few things, like displaying the feedback from your peers, but at least now you can see if you passed or failed

Also, the deadline for attempt 2 is extended till Thursday, 18 of April. Have fun!

We started preparing the certificates for those who passed the first project attempt (it's around 150 people! great work!)

Please make sure the name on the certificate is the correct one

Go to https://courses.datatalks.club/de-zoomcamp-2024/enrollment and update it (you need to be logged in)

If you passed the first project attempt, your certificate should be available!

https://certificate.datatalks.club/dezoomcamp/2024/fe629854d45c559e9c10b3b8458ea392fdeb68a9.pdf

For now, use this instruction to get the certificate: https://github.com/DataTalksClub/data-engineering-zoomcamp/blob/main/certificates.md

If you didn't specify the name for the certificate, we used your leaderboard display name (sorry, Compassionate Perlman, Awesome Cray, Frosty Ellis and others)

Also, the peer review assignments for attempt 2 are now available at https://courses.datatalks.club/de-zoomcamp-2024/project/project2/eval

Enjoy learning from your peers and thanks everyone for your hard work!

A reminder:

You need to update the "certificate name" field for your certificate, otherwise we will have to use your leaderboard display name

You can do it here

https://courses.datatalks.club/de-zoomcamp-2024/enrollment

(You have to be logged in)

We don't have the bandwidth to individually re-issue certificates if you forget to do it, so make sure you do it before the deadline for the second attempt is over

Don't forget to evaluate your peers! The deadline is over, but you have a few more hours to do it if you have forgotten. We'll close the form tonight

We have closed the evaluation form - now the course has officially ended! Yay!

Congratulations to everyone who took part - especially the 359 people who graduated. It was tough yet you made it - great work!

You can get your certificate using the instructions from here https://github.com/DataTalksClub/data-engineering-zoomcamp/blob/main/certificates.md

If your certificate previously had leaderboard display name there, but you updated the certiciate name - now the pdf should be updated.

If you didn't update your certificate name - it seems you'll need to update your legal name now. Sorry, Nervous Jang, Confident Brown, Competent Brown, Inspiring Fermi, Data Engineering Zoomcamp 2024 and others.

Congratulations to the top-100 leaderboard! You worked really hard to get there!

We will publish your names publicly on the course github repo - like here https://github.com/DataTalksClub/machine-learning-zoomcamp/blob/master/cohorts/2023/leaderboard.md

To do it, fill in this form - https://courses.datatalks.club/de-zoomcamp-2024/homework/leaderboard

This is also your last chance to update the certificate name. We will regenerate certificates one more time in one week and after that we will not do it again.

We're starting a stream on getting your first data engineering job now:

https://www.youtube.com/watch?v=1AchMU9xf0Q

Join now or watch later in replay

We have updated the names on your certificates. If you haven't changed the certificate name, sorry, but we won't regenerate certificates anymore

Also unfortunately there was a mistake in scoring the second project batch. We had to recall 6 certificates for people who didn't pass the project. Apologies for the error on our side.