I recently started a new chapter in my tech journey by joining LuxDev, an institution focused on practical, in-depth training in data analysis, data science, and data engineering.
We kicked off our classes on March 31st, and after just one week, I’m already feeling the momentum. If you're curious about what diving into data engineering looks like — especially from day one — here’s a recap of what we covered in our Week 1.
Getting Oriented: What is Data Engineering?
Before jumping into the heavy tools and tech, we took time to understand what data engineering really is. From data pipelines to ETL processes, we discussed:
- The role of a data engineer in the modern data stack
- How data engineering connects to data science and analytics
- Real-world use cases where solid data infrastructure is a game-changer
Tooling up
We then moved straight into setting up our working environments. Here's what we installed:
Python — Our go-to language for scripting and automations
PostgreSQL — A robust relational database
DBeaver — A universal database tool that makes it easy to interact with PostgreSQL (and others)
AWS CLI — To interface with Amazon Web Services directly from the terminal
Aiven.io — For managed cloud data infrastructure
Git Bash — Our preferred terminal on Windows systems
Connecting to Servers, Cloud & Terminal
Things got real-world quickly when we started connecting to actual remote and cloud-based servers:
We used Linux systems and command-line tools to SSH into servers.
Connected to a LuxDev-hosted cloud server — this involved working in a real Linux environment.
Established remote connections from our terminal to AWS and Aiven instances.
Set up secure, terminal-only connections between a local machine and cloud-hosted PostgreSQL databases.
All of this was done without a GUI — just pure terminal power 💪.
🔜 Up Next
Looking forward to next week, we'll be diving deeper into:
Data modeling
Schema design
ETL pipelines
and probably... some Python scripting magic!
