๐๐ผ๐ฎ๐ฑ ๐๐ฎ๐น๐ฎ๐ป๐ฐ๐ถ๐ป๐ด โ spreads incoming traffic across nodes to avoid bottlenecks.
๐๐ผ๐ฎ๐ฑ ๐๐ฒ๐๐ฒ๐น๐ถ๐ป๐ด โ smooths out spikes by queuing work for later processing.
๐๐ผ๐ฎ๐ฑ ๐ฆ๐ต๐ฒ๐ฑ๐ฑ๐ถ๐ป๐ด โ drops non-critical requests to keep the core alive.
Pick the wrong one, and youโre either wasting resources or crashing hard.
๐ช๐ต๐ฒ๐ป ๐ง๐ผ ๐จ๐๐ฒ ๐๐ข๐๐ ๐๐๐๐๐ก๐๐๐ก๐
โ Evenly distribute requests to ensure consistent response times.
โ Scale horizontally during peak hours (e.g. market open/close).
โ Prevent single points of failure with smart routing.
Distributes traffic across servers to prevent overload and keep performance steady. Smart routing avoids bottlenecks and single points of failure. Works best when resources are healthy and scalable.
๐ช๐ต๐ฒ๐ป ๐ง๐ผ ๐จ๐๐ฒ ๐๐ข๐๐ ๐๐๐ฉ๐๐๐๐ก๐
โ Request peaks that come in waves? Buffer them.
โ Use queues to decouple services (e.g., order matching from settlement).
โ Works when a little delay is acceptable for long-term system health.
A messaging channel is set up between clients and the service. This channel helps manage the flow for requests, allowing the service to handle them at its own pace. But beware of queue growth and latency creep.
๐ช๐ต๐ฒ๐ป ๐ง๐ผ ๐จ๐๐ฒ ๐๐ข๐๐ ๐ฆ๐๐๐๐๐๐ก๐
โ Market crash? Order spike? Can't handle everything?
โ Prioritize order execution over non-critical analytics or notifications.
โ Drop 20% of traffic if it means saving the core 80%.
A server can become overwhelmed with requests, leading to slow performance or ever unavailability. To manage this, it can reject excess requests when it reaches its capacity. Better to shed load than go down completely.
๐ช๐ต๐ฒ๐ฟ๐ฒ ๐๐ ๐๐น๐น ๐๐ผ๐บ๐ฒ๐ ๐ง๐ผ๐ด๐ฒ๐๐ต๐ฒ๐ฟ
In a stock exchange system:
โข Balance incoming traffic to keep nodes healthy.
โข Shed non-critical updates during volatile spikes.
โข Level settlement processing using queues to avoid crash.
๐ช๐ต๐ฒ๐ฟ๐ฒ ๐ง๐ฒ๐ฎ๐บ๐ ๐ข๐ณ๐๐ฒ๐ป ๐๐ฒ๐ ๐๐ ๐ช๐ฟ๐ผ๐ป๐ด
โข Relying only on balancingโwhen all nodes are overloaded, it fails.
โข Over-sheddingโleads to lost revenue and frustrated users.
โข Leveling without limitsโqueues grow endlessly, latency explodes.
Load isnโt the problem. Mismanaging it is.
