Posts tagged "dataengineering"

100daysofcode 100daysofdevops 100pay 10mistakes 10yearworkanniversary 11 11labs 11tly 11ty 127001

Why Pi-Shaped Teams Matter in This AI Era

Let me tell you something that's been bugging me for years... This post was born out of frustration: the persistent myth that data engineers are merely "pipeline builders" – – and not true softwar...

19.03.2025 0 Read More

Building an NBA Data Lake with Azure: A Breakdown of the Project Structure

Introduction Data lakes are essential for modern data engineering, enabling efficient storage and processing of raw data. In this blog post, we will explore how to build an NBA Data Lake usin...

20.03.2025 0 Read More

Databricks Optimistic Concurrency Control: Enabling Efficient Collaboration in Big Data Environments

As organizations increasingly adopt data-driven approaches to solve complex business problems, the need for collaborative data engineering and analytics platforms has never been greater. Databricks, a...

20.03.2025 0 Read More

🐼 Pandas Too Slow? Try These Fast Python Libraries for Data Analysis

Pandas is great — until it crashes with large data. Here’s what to use instead 🚀In my latest post, I explore modern, high-performance Python libraries that can handle huge datasets faster and m...

21.03.2025 0 Read More

Slash your cost by 90% with Apache Doris Compute-Storage Decoupled Mode

In the age of data-driven decision-making, the exponential growth of data volume and the ever-evolving demands for analytics pose great challenges. Data streams in from diverse sources (such as applic...

21.03.2025 0 Read More

Azure Data Lake Storage

Key Concepts Data Lakehouse It is a modern data management system that combines the benefits of data lakes and data warehouses. It enables efficient data storage, processing, and ana...

22.03.2025 0 Read More

Unlocking the Power of Delta Lake UNIFORM in Databricks

Organizations are constantly seeking more efficient ways to manage their big data ecosystems. If you're working with data lakes, data warehouses, or the increasingly popular data lakehouse architectur...

22.03.2025 0 Read More

[Snowflake's New Feature] Alert on New Data, and try with Monitoring Tasks and Dynamic Tables

※This is an English translation of my original article in Japanese.Snowflake has introduced a new feature that allows you to execute specified condition expressions and trigger alerts when new recor...

23.03.2025 0 Read More

Personal Picks: Data Product News (March 19, 2025)

※This is an English translation of my original article in Japanese.Hello, I'm Sagara.As a consultant specializing in Modern Data Stack, I'm constantly keeping up with the flood of information in thi...

23.03.2025 0 Read More

Calculate a Pair of Minimum Values that Meet the Criteria within the Group — From SQL to SPL #12

Problem description & analysis: A table stores events that occur for multiple accounts on multiple dates.Task: Now we need to find a pair of events that meet the criteria under each accou...

23.03.2025 0 Read More

Why Pi-Shaped Teams Matter in This AI Era

Building an NBA Data Lake with Azure: A Breakdown of the Project Structure

Databricks Optimistic Concurrency Control: Enabling Efficient Collaboration in Big Data Environments

🐼 Pandas Too Slow? Try These Fast Python Libraries for Data Analysis

Slash your cost by 90% with Apache Doris Compute-Storage Decoupled Mode

Azure Data Lake Storage

Unlocking the Power of Delta Lake UNIFORM in Databricks

[Snowflake's New Feature] Alert on New Data, and try with Monitoring Tasks and Dynamic Tables

Personal Picks: Data Product News (March 19, 2025)

Calculate a Pair of Minimum Values that Meet the Criteria within the Group — From SQL to SPL #12

#reading

#popular