Working for a while with Jupyter, both for ML processes and Data Engineering ETL’s, we came up with interactive solution to make our ETL’s stable, reliable and maintainable — read more about how we build our interactive platform in the following story:
https://medium.com/@benm_23166/tech-story-spark-jupyter-on-k8s-64b9d43a37ba
After having all infrastructure ready to work - we started writing our ETL pipelines in Jupyter and schedule it in Google Cloud Composer. In our ETL’s we came across multiple problems — we wanted to resolved them with Jupyter Magics infrastructure. Our goal was to create more modular, maintainable and flexible source of code. …
Introduction:
If you were in the data engineering industry for a while you probably get know all the existing notebooks solutions out there. in this article I’m gonna focus on what and how we use it in our Data & M.L platform.
First let talk about us…
Plarium is a leading gaming company in the industry — Plarium’s games aren’t just play-for-fun it’s totally a way of life — with millions of active players every day our games plays top 1# gaming company in RPG & Strategy games. Raid Shadow Legends is just one example of a game that blows the charts every day.
All our…
Ben Mizrahi
Bugs always exists between the keyboard to the screen :)
