Latency number that you should know

Ashkrit SharmaMay 28th, 2013Last Updated: May 27th, 2013

0 47 2 minutes read

Latency number that you should know

Many of you work on low latency & high throughput systems. The key to developing such systems is understanding latency, be it of CPU cache, RAM, disk or network. I found some interesting latency numbers, understanding these numbers is very important because all these are based on speed of light and we all know that we will never be able to get faster than light.

L1 cache reference ……………………. 0.5 ns
Branch mispredict ………………………. 5 ns
L2 cache reference ……………………… 7 ns
Mutex lock/unlock ……………………… 25 ns
Main memory reference …………………. 100 ns
Compress 1K bytes with Zippy …………. 3,000 ns = 3 µs
Send 2K bytes over 1 Gbps network ……. 20,000 ns = 20 µs
SSD random read …………………… 150,000 ns = 150 µs
Read 1 MB sequentially from memory ….. 250,000 ns = 250 µs
Round trip within same datacenter …… 500,000 ns = 0.5 ms
Read 1 MB sequentially from SSD* ….. 1,000,000 ns = 1 ms
Disk seek ……………………… 10,000,000 ns = 10 ms
Read 1 MB sequentially from disk …. 20,000,000 ns = 20 ms
Send packet CA->Netherlands->CA …. 150,000,000 ns = 150 ms

CPU Cache latency cost

L1 cache is nearest to CPU core and as you move away you can see that latency is taking a hit and if you are doing it billion times then it is going to get converted to human noticeable delay, here is what it will look like.

Minute

L1 cache reference                  0.5 s         One heart beat (0.5 s)

Branch mispredict                   5 s           Yawn

L2 cache reference                  7 s           Long yawn

Mutex lock/unlock                   25 s          Taking couple of deep breath.

Hour

Main memory reference               100 s         Answering nature call

Week

Read 1 MB sequentially from memory 2.9 days
Welcome long weekend

So to get better performance you have to try to come close to L1/L2 cache which is really difficult things to do. You have develop Cache oblivious algorithm to get super performance.

I/O latency cost

Read 1 MB sequentially from disk …. 20,000,000
ns =
20 ms

Read 1 MB sequentially from SSD(~1GB/sec) ..... 1,000,000 ns  =   1 ms

So a normal disk on an average can read 50 MB/Sec

For SSD we have 1000 MB/Sec

50 MB/Sec is pretty fast and there are many techniques with which you can increase sequential read more by adjusting buffer size of read. So, before making decision on what type of disk you use, you should make sure you are you able to process data at the rate it is read from normal Disk. If you can’t process as fast as that, then there is no point in getting a SSD disk.