"Designing Data-Intensive Applications": Chapter 2 - Data Models - Part I

"Bad programmers focus too much on writing code. Great programmers care more about how data is organized and connected."

Let’s be honest: We’ve all picked a database just because it was popular. But here’s the truth—your data model isn’t just a place to store stuff. It’s the backbone of your app. If you get it wrong, you’ll be stuck patching things up late into the night. Don’t worry—we’re in this together.

🕰️ A Quick Look Back at Data Chaos

The Rise of Relational Databases (1970s–Present)

Relational databases have become the foundation of modern data management—trusted, familiar, and widely adopted.

• Tables and Rows: Structured like spreadsheets, but far more powerful and flexible.
• SQL: The standard query language that introduced concepts like "WHERE" to everyday development.
• Everyday Impact: Powers critical systems, from banking ATMs to dating applications.

Why it became the standard:

✔️ Simple and intuitive data structure

✔️ Reliable ACID transactions ensuring data integrity

✔️ A proven, dependable choice for developers across industries

The following is an image representing the LinkedIn profile of Bill Gates using a relational schema.

Other Early Database Models

Before SQL and relational databases became dominant, several other database models tried to address data management challenges:

The NoSQL Revolution: Moving Beyond Traditional Tables

By the 2010s, the limitations of rigid, predefined table structures became more apparent, especially with the rise of large-scale web applications. Developers began exploring more flexible ways to store and retrieve data.

Document Databases (e.g., MongoDB, CouchDB):

• Flexible Structure: Schemas are dynamic, allowing the data model to evolve over time.
• Nested Data: Supports hierarchical data storage, similar to embedded structures like JSON.
• High Performance for Simple Queries: Optimized for rapid retrieval of documents, though complex relationships across documents can be more challenging to manage.

Real-World Use Case:

// Perfect for that "move fast and break things" phase
{
  "startup_phase": "pre-revenue",
  "tech_stack": ["Blockchain", "AI", "NFT"],
  "burn_rate": "$100k/month",
  "exit_strategy": "Acquired by Google (please)"
}

This shift increased the one-to-many relationships in the data. Each JSON document with one-to-many relationships started to look like this :

The Great Debate: Relational vs. Document Databases

When Document Databases Are a Better Fit:

• Your data is naturally nested, resembling hierarchical structures (like nested folders or Matryoshka dolls).
• You prefer flexibility, avoiding the need to constantly update a fixed database schema.
• Your application's main task is to retrieve and display entire records or related data at once.

When Relational Databases Excel:

• Your data is highly interconnected, requiring complex relationships (similar to a social network where everyone is connected).
• You need to perform detailed queries that combine and filter data across multiple entities ("Find all customers who purchased a product and live in a specific region").
• You require strict consistency and integrity of your data, ensuring reliable and accurate transactions.

This is one of the key reasons why relational databases are typically preferred over document databases when managing many-to-many relationships.

Mixing Approaches: Combining the Best of Both Database Styles

Modern databases are evolving, blending the strengths of both relational and document models:

• PostgreSQL: Primarily a relational database, but now offers powerful support for flexible JSON data types, allowing for hybrid structured and semi-structured storage.
• MongoDB: Originally focused on document storage, it has introduced robust transaction support to ensure greater reliability and consistency.
• Google Spanner: Designed to offer the best of both worlds—combining relational structure with horizontal scalability, and enabling global, distributed consistency.

-- Yes, this actually works in modern PostgreSQL
SELECT users->'profile'->>'twitter_handle'
FROM hybrid_db
JOIN social_graph ON (users->>'id' = social_graph->>'user_id')
WHERE transactions->'amount' > 1000;

Key Considerations When Choosing a Database

• Will your data remain understandable if your organizational needs change over time?
  
  Document databases offer flexibility for evolving data models, whereas relational databases may require significant restructuring.

• How interconnected is your data?
  
  Highly linked data often benefits from relational models, while more independent records can thrive in document-based systems.

• Who defines the structure of your data—the application or the database?
  
  This decision greatly impacts how you design, scale, and maintain your system.

Conclusion

Balancing the Best Tools for the Job

The key takeaway today? Using different types of databases for different purposes—like a chef selecting the right tool for each ingredient:

• Relational: The reliable, versatile tool for structured data.
• Document: The flexible choice for handling nested or evolving data.
• Graph: A specialized tool for complex, interconnected data.

Pro Tip:

"Choose your databases as carefully as you choose your shoes—what works for hiking won't work for ballet.

Now, take a step back and plan your data strategy... maybe sketch out your ideas first? 🎨