Introduction
Remember that time you had to wait 10 seconds for your app to fetch data? Your users remember too, and they weren't happy about it. In the world of web applications, speed isn't just a nice-to-have feature—it's the difference between a successful product and an abandoned tab.
As an old-timer in the Go ecosystem (I've been writing Go since it wore diapers), I've seen countless projects brought to their knees by database load or API bottlenecks. Today, we're going to solve that with the programming equivalent of a supermarket express lane: in-memory caching.
Let's build a memory cache system in Go that will make your applications zippy and your users happy!
1. Memory Cache 101: Fundamentals of Go Caching
A memory cache is like that friend who remembers every embarrassing story about you—quick to recall, but sometimes needs to forget things to make room for new embarrassing moments. In technical terms, it's a data store that keeps frequently accessed information in RAM for lightning-fast retrieval.
Why Cache in Go?
Using a database for every request is like going to the grocery store every time you need a single ingredient. A cache is your kitchen pantry—it keeps frequently used items close at hand. In Go, implementing a cache is surprisingly straightforward thanks to the language's built-in maps and concurrency primitives.
💡 Lesser-known fact: In Go 1.9, sync.Map was introduced specifically to address high lock contention issues with mutexes guarding maps in concurrent scenarios. However, it's actually not always faster than a mutex-protected map for write-heavy workloads!
Let's start with the simplest possible cache implementation:
type SimpleCache struct {
data map[string]interface{}
mu sync.RWMutex
}
func NewSimpleCache() *SimpleCache {
return &SimpleCache{
data: make(map[string]interface{}),
}
}
func (c *SimpleCache) Set(key string, value interface{}) {
c.mu.Lock()
defer c.mu.Unlock()
c.data[key] = value
}
func (c *SimpleCache) Get(key string) (interface{}, bool) {
c.mu.RLock()
defer c.mu.RUnlock()
val, exists := c.data[key]
return val, exists
}This works! But it's like using a hammer to build an entire house—technically possible, but not exactly optimal. Our simple cache has no way to expire old data, handle memory pressure, or provide performance metrics.
2. Building a Thread-Safe Cache Engine in Go
Dealing with cache concurrency issues is like trying to coordinate toddlers sharing toys—without proper supervision (mutexes), there will be tears (race conditions). Let's level up our cache with TTL support and automatic cleanup.
Adding TTL Support
Every good cache needs an expiration mechanism. Otherwise, your cache is like a refrigerator without cleaning—eventually, you'll end up with mysterious objects from the distant past taking up valuable space.
type CacheItem struct {
Value interface{}
Expiration int64
}
type Cache struct {
items map[string]CacheItem
mu sync.RWMutex
}
func NewCache() *Cache {
cache := &Cache{
items: make(map[string]CacheItem),
}
// Start the janitor
go cache.startJanitor()
return cache
}
func (c *Cache) Set(key string, value interface{}, ttl time.Duration) {
c.mu.Lock()
defer c.mu.Unlock()
expiration := time.Now().Add(ttl).UnixNano()
c.items[key] = CacheItem{
Value: value,
Expiration: expiration,
}
}
func (c *Cache) Get(key string) (interface{}, bool) {
c.mu.RLock()
defer c.mu.RUnlock()
item, exists := c.items[key]
if !exists {
return nil, false
}
// Check if the item has expired
if item.Expiration > 0 && time.Now().UnixNano() > item.Expiration {
return nil, false
}
return item.Value, true
}
func (c *Cache) startJanitor() {
ticker := time.NewTicker(time.Minute)
defer ticker.Stop()
for range ticker.C {
c.DeleteExpired()
}
}
func (c *Cache) DeleteExpired() {
now := time.Now().UnixNano()
c.mu.Lock()
defer c.mu.Unlock()
for k, v := range c.items {
if v.Expiration > 0 && now > v.Expiration {
delete(c.items, k)
}
}
}🔍 Pro tip: The janitor goroutine in this example runs every minute, which might be too infrequent for some applications. Adjust the ticker interval based on your specific needs and the typical TTL values you set.
Understanding the Tradeoffs
⚠️ Lesser-known fact: The sync.Map implementation in Go uses a clever "read-mostly" optimization technique with two internal maps—one for read operations and another for writes. This reduces contention but adds complexity and memory overhead, making it worse than a regular mutex-protected map for many common caching scenarios!
For a read-heavy cache, where items are written once and read many times, a sharded approach might be better:
type ShardedCache struct {
shards []*Cache
shardCount int
shardMask int
}
func NewShardedCache(shardCount int) *ShardedCache {
// Ensure shard count is a power of 2
if shardCount <= 0 || (shardCount&(shardCount-1)) != 0 {
shardCount = 16 // Default to 16 shards
}
sc := &ShardedCache{
shards: make([]*Cache, shardCount),
shardCount: shardCount,
shardMask: shardCount - 1,
}
for i := 0; i < shardCount; i++ {
sc.shards[i] = NewCache()
}
return sc
}
func (sc *ShardedCache) getShard(key string) *Cache {
// Simple hash function
h := fnv.New32a()
h.Write([]byte(key))
return sc.shards[h.Sum32()&uint32(sc.shardMask)]
}
func (sc *ShardedCache) Set(key string, value interface{}, ttl time.Duration) {
shard := sc.getShard(key)
shard.Set(key, value, ttl)
}
func (sc *ShardedCache) Get(key string) (interface{}, bool) {
shard := sc.getShard(key)
return shard.Get(key)
}Sharding a cache is like having multiple refrigerators in your house—sure, it's more efficient, but now you can't remember if the milk is in the kitchen fridge or the garage fridge. The key benefit is reducing lock contention by distributing keys across multiple underlying caches.
3. Advanced Optimization Techniques: Eviction Policies & Benchmarking
Choosing the right eviction policy is like deciding which items to throw out when your closet is full—it's all fun and games until you realize you just tossed something you needed tomorrow.
Implementing LRU Eviction
Let's implement a Least Recently Used (LRU) cache using Go's container/list package:
type LRUCache struct {
capacity int
items map[string]*list.Element
list *list.List
mu sync.RWMutex
}
type entry struct {
key string
value interface{}
}
func NewLRUCache(capacity int) *LRUCache {
return &LRUCache{
capacity: capacity,
items: make(map[string]*list.Element),
list: list.New(),
}
}
func (c *LRUCache) Get(key string) (interface{}, bool) {
c.mu.Lock()
defer c.mu.Unlock()
if element, exists := c.items[key]; exists {
c.list.MoveToFront(element)
return element.Value.(*entry).value, true
}
return nil, false
}
func (c *LRUCache) Set(key string, value interface{}) {
c.mu.Lock()
defer c.mu.Unlock()
// If key exists, update value and move to front
if element, exists := c.items[key]; exists {
c.list.MoveToFront(element)
element.Value.(*entry).value = value
return
}
// Add new element
element := c.list.PushFront(&entry{key, value})
c.items[key] = element
// Evict least recently used if capacity exceeded
if c.list.Len() > c.capacity {
c.removeOldest()
}
}
func (c *LRUCache) removeOldest() {
element := c.list.Back()
if element != nil {
c.list.Remove(element)
entry := element.Value.(*entry)
delete(c.items, entry.key)
}
}🎯 Lesser-known fact: The default Go map implementation uses a randomized hash seed to protect against hash-flooding attacks, which is a clever security feature but can lead to non-deterministic iteration order that surprises developers trying to debug cache behavior.
Benchmarking Your Cache
A cache without benchmarks is like a sports car without a speedometer—you have no idea if it's actually fast or just looks fast. Let's write a simple benchmark for our cache implementations:
func BenchmarkCacheGet(b *testing.B) {
// Create a cache with 1000 items
cache := NewCache()
for i := 0; i < 1000; i++ {
key := fmt.Sprintf("key-%d", i)
cache.Set(key, i, time.Hour)
}
b.ResetTimer()
b.RunParallel(func(pb *testing.PB) {
i := 0
for pb.Next() {
key := fmt.Sprintf("key-%d", i%1000)
_, _ = cache.Get(key)
i++
}
})
}
func BenchmarkShardedCacheGet(b *testing.B) {
// Create a sharded cache with 1000 items
cache := NewShardedCache(16)
for i := 0; i < 1000; i++ {
key := fmt.Sprintf("key-%d", i)
cache.Set(key, i, time.Hour)
}
b.ResetTimer()
b.RunParallel(func(pb *testing.PB) {
i := 0
for pb.Next() {
key := fmt.Sprintf("key-%d", i%1000)
_, _ = cache.Get(key)
i++
}
})
}Running these benchmarks on my machine yields some interesting results:
BenchmarkCacheGet-8 10000000 112 ns/op
BenchmarkShardedCacheGet-8 20000000 58 ns/opThat's nearly a 2x performance improvement with sharding! 🚀
Conclusion
We've gone from a simple map with a mutex to a sharded, LRU cache with TTL support—quite the journey! Here's what we've learned:
- Start simple - A basic mutex-protected map can be an excellent starting point
- Add features incrementally - TTL, cleanup, metrics can be added as needed
- Consider access patterns - Choose eviction policies that match your usage
- Measure, don't guess - Always benchmark to ensure your optimizations help
Remember, caching is both an art and a science. The perfect cache for your application depends on your specific workload, memory constraints, and performance requirements.
What parts of your application could benefit from an in-memory cache? The answer is probably "more than you think." Take the code examples from this post, adapt them to your use case, and share your results in the comments!
Until next time, happy caching! 🚀
