Spyke

Syndicated from the fediverse. Read and engage on the original instance.

View original on lemmy.world
data_engineering·Data EngineeringbyAndy

Building a real-time data pipeline - Technical article and GitHub repo

If you're a Data Engineer, before long you'll be asked to build a real-time pipeline.

In my latest article, I build a real-time pipeline using Kafka, Polars and Delta tables to demonstrate how these can work together. Everything is available to try yourself in the associated GitHub repo. So if you're curious, take a moment to check out this technical post.

https://medium.com/@nydas/building-a-real-time-data-pipeline-5eff6c6d8a3c?source=friends_link&sk=8d8792af4527b0d6f91f430ebdc40fc7Open linkView original on lemmy.world
20

No replies yet

No comments on the original post yet.
Building a real-time data pipeline - Technical article and GitHub repo | Spyke