After understanding cache terminology, it is important to understand the effects that a cache has on access time and traffic to the lower level.

A Cache Level

The following algorithm is used by a cache in order to take advantage of both temporal and spatial locality.

    if the requested data is in the local store (cache hit, cache access time)
        return the requested data
    else (cache miss, lower-level access time)
        get the requested and nearby data from the lower level
        save it in the local store
        return the requested data
      

The fraction or percentage of accesses that result in a hit is called the hit rate.

The fraction or percentage of accesses that result in a miss is called the miss rate.

hit rate + miss rate = 1.0 (100%)

The difference between lower level access time and cache access time is called the miss penalty.

A Cache Level

Effective access time is a standard effective average.

    effective-access-time = hit-rate * cache-access-time
                          + miss-rate * lower-level-access-time
      

Miss penalty is defined as the difference between lower level access time and cache access time. Then the above equation becomes

    effective-access-time = cache-access-time + miss-rate * miss-penalty
      
A Cache Level

Due to locality of reference, many requests are not passed on to the lower level store. The following equation gives an approximation to the traffic to the lower level.

    lower-level-traffic = miss-rate * requested-traffic
      

In order to make this more accurate, consideration must be given to data that is fetched from the lower level but not used by the requester. The requester might not need all of the data in a cache line. In any case, the lower level must have the throughput capacity to support the lower level traffic.