Redis is an in memory storing giant in the industry.

Why so fast?

Redis store data in the memory. In general, reading and writing data from memory faster than reading/writing data from the disk. Redis store everything as a key/value pairs in a hash map.

Most Redis commands are operate in O(1) or O(log N) time complexity regardless of the size of the data.

Redis architecture

Redis uses Single-Threaded Event Loop architecture. In general, Redis can handle 10,000 concurrent requests. This can be change using redis.conf maxclients directive.

Persistence of data?

Since, Redis uses memory to store data, data will be lost upon the server crash. Redis provides two approaches to persistence the data

  1. RDB - Redis Database
    In this way, snapshot will be written to the disk periodically. Can be restore data form them. But, it has few cons as well. Until data restore complete, there will be a significant latency. This would not ideal in the production.

  2. AOF (Append Only File)
    In this way, AOF persistence logs every write operation received by the server. These operations can then be replayed again at server startup, reconstructing the original dataset. There is a cons here, recovery time will be higher, when the file become larger and larger over the time

The recommended approach is use the combination of RDB and AOF

Read Replicas (High availability)

Use read replicas would be ideal in this case. It has primary node and multiple read replicas. Once the primary goes down, one of the replica start acting as new primary node (Redis Sentinel).

Read More...

Redis use cases

Redis can use in 3 different main purposes,

  1. Use as a Cache
  2. Use as a Database
  3. Use as a Message Broker

Redis core data types

  • Strings – Simple key-value pairs.
  • Lists – Doubly linked lists.
  • Hashes – Key-value maps.
  • Sets – Unordered unique elements.
  • Sorted Sets – Ordered sets with scores.

Spine Redis instance

Can use Docker to spine up a Redis server in a minute with different features.

# start redis server
$ docker run --name redis-server -d -p 6379:6379 redis
# -d: Detached mode
# -p: port mapping

# restart and stop redis server
$ docker restart redis-server
$ docker stop redis-server

Most needed command 

# go inside docker container
$ docker exec -it  bash

# go inside redis
$ redis-cli -h  -p  -a 
# eg. redis-cli -h 192.168.1.8 -p 6379 -a xxxx

# OR without go inside,
$ docker exec redis-server redis-cli GET username

# set string
$ SET username "Apache"
$ GET username
$ DEL username
$ EXPIRE username 60 # expire item in 60 seconds
$ TTL username # check TTL of key. xx seconds left
$ FLUSHDB # delete all keys in current DB
$ FLUSHALL # delete all keys in all DB

# Hashes
$ HSET user:1000 name "Bob" age 23
$ HGET user:1000 name # Bob
$ HGET user:1000 age # 23
$ HGETALL user:1000

# Lists (ordered items)
$ LPUSH tasks "task1" "task2" # push item to left
$ LPOP tasks # remove item from left
$ RPUSH tasks "task3" # push item to right
$ LRANGE tasks 0 -1 # list all items

# Sets (unordered items)
$ SADD colors "red" # add item

# monitor redis activities
$ monitor

# docker create new network
$ docker network create redis-net

# inspect docker network
$ docker network inspect redis-net

Persistence of data (RDB/AOF). How to do? 

# enable snapshots (RDB)
$ docker run --name redis-rdb -d -p 6379:6379 -v $(pwd)/redis-data:/data redis redis-server --save 60 1

# -v:  mount redis volume to host /redis-data folder.
# redis-server: this runs the redis-server executable inside the container. we use this to pass additional options
# -save 60 1: saves to disk if 1+ key changes in 60 seconds.

# enable append only (AOF)
$ docker run --name redis-rdb -d -p 6379:6379 -v $(pwd)/redis-data:/data redis redis-server --appendonly yes

Redis replicas. How to do? 

# create docker network
$ docker network create redis-net

# make sure network is there
$ docker network list

# create master node
$ docker run --name redis-master --network redis-net -d -p 6379:6379 redis redis-server --appendonly yes

# create replica 01
$ docker run --name redis-replica1 --network redis-net -d -p 6380:6379 redis redis-server --replicaof redis-master 6379 --appendonly yes

# Note: both redis nodes must be in the same docker network, in order to communicate each other

# test everything works:
$ docker exec redis-master redis-cli set username "Bob"

$ docker exec redis-replica1 redis-cli get username

# check information about the node
$ docker exec redis-master redis-cli info replication

Can be used docker compose file to spine all in one command (docker-compose.yml)

services:
  redis-master:
    image: redis
    container_name: redis-master
    restart: always
    volumes:
      - redis_master:/data
    ports:
      - 6379:6379

  redis-replica1:
    image: redis
    container_name: redis-replica1
    restart: always
    volumes:
      - redis_slave:/data
    ports:
      - 6479:6379
    command: redis-server --slaveof redis-master 6379
volumes:
  redis_master:
  redis_replica1:

Run from command line

$ docker compose -f docker-compose up

Redis automatic failover (Sentinel). How to do?

Redis does not handle failover by default. We need to config for it. Sentinel is the Redis concept we can use to handle failover along with replicas created above.

Create sentinel config file (sentinel.conf) in the current working directory:

port 26379
sentinel monitor mymaster redis-master 6379 2
sentinel down-after-milliseconds mymaster 5000
sentinel failover-timeout mymaster 10000
sentinel parallel-syncs mymaster 1

mymaster → Name of the monitored master.
redis-master 6379 → IP and port of the master.
2 → The number of Sentinels that must agree before a failover occurs.
down-after-milliseconds 5000 → Time (5 sec) before Sentinel marks the master as down.
failover-timeout 10000 → Time (10 sec) before failing over.
parallel-syncs 1 → Number of replicas syncing simultaneously.

# create sentinel conatiner
$ docker run --name redis-sentinel --network redis-net -d -p 26379:26379 -v $(pwd):/etc/redis redis redis-server /etc/redis/sentinel.conf --sentinel

# verify sentinel uses the correct master node
$ docker exec -it redis-sentinel redis-cli -p 26379 SENTINEL get-master-addr-by-name mymaster
# this should return either redis-master or ip of the master container

# check the actual role of the node
$ docker exec redis-master redis-cli info replication

How to test the failover

# stop the master node
$ docker stop redis-master

# check the role of replica
$ docker exec redis-replica1 redis-cli info replication

Redis use cases

  • As a rate limiter
    Leaky bucket algorithm

  • As a distribution lock

  • Rank/Leaderbord
    Using sorted list data type

Conclusion

Now a days, we have plenty of managed services available out there, rather manually setting up everything. Example, AWS ElastiCache. But, I think it is better to know what is going on behind the scenes.

Add anything I missed here....

Cheers... Happy coding!!!