Microservices Write Patterns focus on how to write data in a reliable way, including writing to multiple microservices.

Microservices Reliability Playbook

  1. Introduction to Risk
  2. Introduction to Microservices Reliability
  3. Microservices Patterns
  4. Read Patterns
  5. Write Patterns
  6. Multi-Service Patterns
  7. Call Patterns

Download the full Playbook at a free PDF

Writer Pattern

The writer pattern isolates write operations from processing. It is useful when write operations have to have highest reliability. The writer pattern assumes a simple write, for complex multi-table or multi-service writes consider the multi-writer pattern.

Writer Pattern Diagram

One example of an application of the writer pattern is the order writer - once an order has been paid, a system has to write the order with a high reliability service to ensure no orders are lost. Then, it can start handling all the aspects of the order, from fulfillment to updating different “my account” pages, analytics, etc.

Once the writing is done, the writer pattern triggers a call to the processor / reader service to process the write operation (e.g. the order).

Advantages:

  • Simple writer service as an ideal microservice, five-nines reliability.
  • No processing on write - less chance of failure

Disadvantages:

  • The call to the processing of the order is non-garanteed, as the service first writes the order, then calls the processing. In case of failure of the processing, we have an order that has not been processed. This requires another mechanism to ensure all orders are processed.

Writer Pattern with a Queue

The writer pattern may use a persistent queue instead of a table for the write, so that both the update of the written data and the processing are driven by the queue. Persistent queues are a great tool to create reliable multi-reaction writes, which can be utilized here.

The writer does only two things - validates the write (access control and business validation) and writes the data to a persistent queue.

This pattern has a downside that it is challenging to get a client to read their own writes immediately, and it is challenging to propagate processing failure to the client. One technique to resolve both is to return a “ticket number” on the write operation, and another API on the processor to get the status of the “ticket number” - was the processing completed and now the writer can read their own writes? Was the processing successful, and if not why?

Writer Pattern with a Queue diagram

For instance, applied to the same order service, the order writer writes the order to a queue, which then the processor dequeues, gets the order, writes the order to the database as well as performs any needed processing.

The main advantage of this pattern compared to the Writer pattern is the error handling and persistence we get from the queue. All we need to ensure the data (an order) will be processed is the writer service and the queue to be operational.

In case the processor / reader is not online, or has a problem, the queue saves the written data (orders) to be processed later, or retried.

Also, we note that the writer service is very simple, decoupling the risk of change related to all the processing (order processing).

Multi-Writer Pattern

The multi-writer pattern is a generalization of the Writer Pattern with a Queue, in case we have to write to multiple microservices in a reliable way.

Multi-writer pattern diagram

Consider a write operation for the product catalog, at which the catalog, inventory and categories are each managed by a different microservice. The multi-writer pattern allows one to create a single product writer that gets the details of the product, category and inventory, and writes it all to a queue. Each processing service accepts the message from the queue and handles the writes to its database.

The advantage of this pattern is that it decouples writes from processing and reads, while allowing multiple services to be reliably updated.

However, this pattern also introduces a risk that some of the writes have completed successfully, while other writes have failed or are pending due to some delay. The pattern does not guarantee transactional write to multiple microservices, creating the risk of inconsistencies. Such inconsistencies can be mitigated into an eventual consistent system using the persistent queue and a retry mechanism, one that eventually will complete all the writes and restore the system into a consistent state.