After understanding cache terminology, it is important to understand the effects that a cache has on access time and traffic to the lower level.
The following algorithm is used by a cache controller in order to take advantage of both temporal and spatial locality.
if the requested data is in the local store // cache hit, cache access time return the requested data else // cache miss, lower-level access time get the requested and nearby data from the lower level save it in the local store return the requested data
The fraction or percentage of accesses that result in a hit is called the hit rate.
The fraction or percentage of accesses that result in a miss is called the miss rate.
It follows that hit rate + miss rate = 1.0 (100%).
The difference between lower level access time and cache access time is called the miss penalty.
Effective access time is a standard effective average.
effective-access-time = hit-rate * cache-access-time + miss-rate * lower-level-access-time
Miss penalty is defined as the difference between lower level access time and cache access time. Then the above equation becomes
effective-access-time = cache-access-time + miss-rate * miss-penalty
Due to locality of reference, many requests are not passed on to the lower level store. The following equation gives an approximation to the traffic to the lower level.
lower-level-traffic = miss-rate * requested-traffic
In order to make this more accurate, consideration must be given to data that is fetched from the lower level but not used by the requester. The requester might not need all of the data in a cache line. In any case, the lower level must have the throughput capacity to support the lower level traffic.