Today we start module 2 about workflow orchestration and Mage!

Course materials: https://github.com/DataTalksClub/data-engineering-zoomcamp/tree/main/02-workflow-orchestration

Homework: https://github.com/DataTalksClub/data-engineering-zoomcamp/blob/main/cohorts/2024/02-workflow-orchestration/homework.md

Form for submitting: https://courses.datatalks.club/de-zoomcamp-2024/homework/hw2

Deadline: February 5th (Monday), 23:00 CET

Also we're extending the deadline for homework 1 for a couple of days. You can submit it till Wednesday 23:00 CET

"What's homework URL?"

It's a very common question. I know you don't have time to check the FAQ, so citing Michael here:

It's your repository or any other location you have your code where a reasonable person would look at it and think yes, you went through the week and exercises.

By the way, here's the link to the FAQ:

https://docs.google.com/document/d/19bnYs80DwuUimHM65UV3sylsCn2j1vziPOwzBwQrebw/edit?usp=sharing

Looks like the code we use for scoring the questions is not working well for hundreds of submissions, so we need to optimize it. (We'll need to put these data engineering skills into practice.)

In the meantime, the form for submitting the first homework remains open, so you can still submit it.

Homework 1 is scored. The bottleneck is identified but not fixed yet =) You should now see the answers in the form

Tomorrow we have two streams:

- The Data Ingestion workshop (dlt)
- Office Hours with Matt - we will answer your questions about Mage and the course. You can ask questions here: https://app.sli.do/event/rNeAkUXsCEQ6dGQ1jEZMdc

We will stream to our YouTube channel (https://www.youtube.com/c/datatalksclub). The streams will be recorded and the videos will stay on our channel.

See you tomorrow

Good news: we're extending the deadline for homework 2 by a couple of days: now the due date is Thursday, 23:00 CET.

Don't forget that you need to do homework 3 and also workshop homework if you plan to take part. Refer to the deadline calendar if you need help with planning your time.

Also, the workshop starts in 2 hours 30 minutes. We will share the link to the stream here in chat. See you soon!

We're starting the workshop in 10 minutes

Stream: https://www.youtube.com/watch?v=oLXhBM7nf2Q (it's being recorded and you can re-watch it later at any time)

Workshop page: https://github.com/DataTalksClub/data-engineering-zoomcamp/blob/main/cohorts/2024/workshops/dlt.md

Dlt workshop links:

- Workshop page and homework: https://github.com/DataTalksClub/data-engineering-zoomcamp/blob/main/cohorts/2024/workshops/dlt.md
- Form for submitting the homework: https://courses.datatalks.club/de-zoomcamp-2024/homework/workshop1

We're starting our office hours session in 8 minutes

Stream (available for replay later): https://www.youtube.com/watch?v=7PBvH6dHVxc
Questions: https://app.sli.do/event/rNeAkUXsCEQ6dGQ1jEZMdc

In addition to the workshop, this week, we also work on the Data Warehouse module

Content: https://github.com/DataTalksClub/data-engineering-zoomcamp/tree/main/03-data-warehouse

Homework: https://github.com/DataTalksClub/data-engineering-zoomcamp/blob/main/cohorts/2024/03-data-warehouse/homework.md

The due date for this homework is February 12.

You will have a bit of extra time to finish the workshop homework (February 15)

Have fun!

We want to organize a webinar with career consultation advice for the students of this course.

We have some ideas on what to talk about. But we also want to know from you what the most helpful content would be. What would you like to know?

We're extending homework 3 by two days. The new deadline is Wednesday 23:00 CET

There was a minor production accident - the form was displaying only one option for each question for ~20 minutes

We were rolling out a new update and migrating data, but it broke midway, so the database was in an inconsistent state. Now it's all back to normal.

Please check that your answers for workshop 1 (especially question 3) and homework 3 are not affected. If they are, please update them

Let us know if you come across any issues. Thanks! Happy learning!

Hi everyone!

Today we started module 4: Analytics Engineering (but many of you started it earlier this week)

Content: https://github.com/DataTalksClub/data-engineering-zoomcamp/tree/main/04-analytics-engineering

Homework: https://github.com/DataTalksClub/data-engineering-zoomcamp/blob/main/cohorts/2024/04-analytics-engineering/homework.md

Form for submitting the homework: https://courses.datatalks.club/de-zoomcamp-2024/homework/hw4

Also, we scored homework 3, the leaderboard is updated: https://courses.datatalks.club/de-zoomcamp-2024/leaderboard

And there's still time to continue working on workshop 1 homework.

Have fun!

If you want to learn more about the internals of the Q&A bot, join the stream where Alex (the creator) talks about it

https://www.youtube.com/watch?v=QhFLeZV-PVk

We will leave the form for submitting the workshop homework open for a few more days, so if you haven't finished it yet, you have time till Thursday

We're extending the deadline for homework 4 till Saturday 23:00 CET. Have fun!

We started module 5: batch processing and spark

Course materials: https://github.com/DataTalksClub/data-engineering-zoomcamp/tree/main/05-batch

Homework: https://github.com/DataTalksClub/data-engineering-zoomcamp/blob/main/cohorts/2024/05-batch/homework.md

Have fun!

Hi everyone!

How is it going?

Just a reminder that on Monday at 17:00 CET we have a workshop about streaming with RisignWave

It's already in the calendar, but it's also possible (not mandatory) to sign up for it

https://lu.ma/24re7c0h

See you soon and enjoy the weekend!