Caches are tiny but memory prefetchers can make them appear much larger. The prefetcher predicts you will access access contiguous memory in linear order, forward or backward, and will therefor start transferring the next block of memory to cache before you even ask for it. Under ideal conditions your tiny cache can appear to be infinitely large as every access will hit the cache due to perfect prefetching giving you multiple orders of magnitude improvement in throughput.