What skills do you need as a Data Engineer?
All developers share a common insecurity...
And it's whether or not we have enough of the right skills or if we’re falling behind.
It doesn't matter if you’re brand new to data engineering or a seasoned vet.
There’s always something new to learn.
In today’s video I want to break down some of the specific skills to know as a data engineer.
So that you can identify what you need to learn to land your first job or maybe some things to take that next step in your career.
►► The Starter Guide for The Modern Data Stack (Free PDF)
Simplify the “modern” data stack + better understand common tools & components → bit.ly/starter-mds
Timestamps:
0:00 - Intro
0:52 - Junior Engineer
5:46 - Senior Engineer
8:47 - Lead/Staff Engineer
Title & Tags:
What skills do you need as a Data Engineer?
#kahandatasolutions #dataengineering #analytics
►► The Starter Guide for The Modern Data Stack (Free PDF)→ bit.ly/starter-mds Simplify “modern” architectures + better understand common tools & components
Great content! I appreciate the work you are doing. Well I was expecting a pdf following this link but I get an email (Lesson 1 of 5) which asks me to watch another video. Could you please clarify if there is no single text source as in a pdf?
You have no idea how helpful your channel is for young data engineers.
Exactly
This video has been so helpful! Thank you
Awesome, thanks for watching!
Hey Man thanks for the outstanding videos . your videos' has been a huge help for me . Can't wait for you to do an apache airflow video
Thanks Omar! Glad to help
Great video! Very useful information 😊
Glad it was helpful!
Very useful information,thanks for sharing 🙏
You bet, thanks for watching
Nice vid. I love being a lead. Some days I wish I was a jr.
Interesting video. Though, I think you forgot about privacy. This is as important as security.
This is so helpful. Thank you so much!
Glad it was helpful!
Great video! Thanks😊
Glad you liked it!
Hello! Thank you for your invaluable video! I find it extremely useful for beginners! I would like to ask about one thing regarding this skill. I learnt Pandas in terms of Data Wrangling and Transformation. Therefore, how about Pandas for Data Engineers? Is it useful tool for ETL/ELT transformations? Obviously, the next step will be PySpark, but I would like to start learninig Pandas. It seems it is a good path for the next one. What do you think about it ? I would appreciate it if you could share your views about it.
Man, I don't know what I'm doing wrong then. I am a Python wizard (5 years) applying to 100s of these junior engineer positions. I only know PySQL however, so I wonder if that is the main thing hurting me. I have data analytics and viz experience, and some ETL projects I made to fill out more, but nothing. I don't really know what to do and feel like giving up. sorry if that's a lot lol.
Awesome video
Thanks!
Learn fundamentals. How to set up a data stack with the right environment, plateform. (Terraform, docker, ainsible etc.) and learn about NETWORKS communication between the stack . Know where data metadata will be store etc.
I've got a job interview as a data coordinator and i really don't know what they actually require lol i mean i have some sql skills and database skills like data modelling but hardly any and they want someone who can do data transformations/ collections etc ? what should i learn i have a week till the first phone interview?
is there any specific bit of python you'd recommend learning for juniors? Like pandas etc, same with sql I guess. Thanks
I would recommend pandas, numpy, scipy and at least a SQL database management library (MySQL-Python) and a noSQL one (pymongo)
Hey Kahan. Is it possible to become only Data Warehouse engineer? I’m actually avoiding having to learn Python😭😭😭
Learn sql, then dbt and snowflake. You will be good to go.
What is dbt
@@mihirranade815 data building tool for it is used for ELT it working with the transformation part
Started my data engineering career with exactly these 3... And I couldn't agree more with you, these 3 are best to get started
@@mihirranade815data build tool, it's the T of ELT, i.e, helps with the transformation part of the data journey. Infact this channel has quite amazing dbt content, good starting point.
I am not finished my high school can I become Data Engineering
Of course! You have a long career ahead of you. I didn't start learning any of this until I after I already graduated college and worked in a different job for nearly 3 years. You are way ahead
👋👋👋😉
Nice video bro! I have a quick question; Why did you didn't talk about the "Mid Engineer" ? (I'm a Junior Engineer close to be moving to the next level in my actual Job, and I hope it be the "Mid Engineer") Grettings from Colombia! 🟡🔵🔴
I've always wanted to visit Colombia - thanks for tuning in! Regarding your question - these positions are very much on a spectrum and the actual job titles will vary a ton depending on where you work. For that reason I decided to stick to just 3. Perhaps a "Mid Engineer" at your company is the same as what I'm referring to "Senior Engineer" (But I'm not sure). Either way, I would see if you can find concepts in the video that you are unfamiliar with and try to learn a bit more about it. That will help you level up in general. Good luck!