Question

Many multi-core systems have different levels of cache on the chip, one level is local to each processing core (L1 and L2 on Intel systems), and another level is shared among all processing cores (L3 on Intel systems). Why are caching systems designed this way.

Answer 1

Let's first try to understand why do we need cache memory first of all. We know that CPU is much faster when compared to main memory. To avoid latency between memory and CPU we needed faster memory compared to RAM. Now there are multiple factors which needs to be considered to increase the memory access speed.

1. Memory must be faster means fast technology to be used for example SRAM is used for Cache.
2. Memory must be near to all main components like ALU, MMU etc. More near the memory more faster the access.

Now we can make RAM also faster by using SRAM but it is very costly. Cache is comparatively very less in size so cost factor is neglected. One more thing which affects the cache size is space as SRAM needs lots of space due to more transistors per bit, we can not place it near to small components like ALU, MMU on chip so to avoid that cache is always very small in size. That's how cache memory came into picture.

Now a days each processor comes with multiple cores. If only single cache is shared between all the cores we lose the benifit of locality of reference. That's where multi-level cache design came into picture where first level cache or first and second level cache will be individual to cores and third level cache will be shared among all cores. That is why size of L1 cache is very small so that it can be placed on chip near to individual cores and L2 is slightly bigger and L3 biggest. This kind of cache design not only improved individual core speed but also overall total speed of system.

Answer 2

Caching systems are designed with multiple levels of cache (L1, L2, and L3) on multi-core systems to optimize data access and improve overall system performance. Each level of cache serves a specific purpose and helps in mitigating the performance bottleneck caused by the speed difference between the processor and main memory (RAM).

Here's why caching systems are designed this way:

1. Proximity to the Processor: The primary reason for having multiple levels of cache is to bring the most frequently used data closer to the processor. As we move from L1 to L3 cache, the size of the cache increases, but the access speed to each level becomes slower. Having smaller and faster L1 and L2 caches local to each processing core ensures that the processor can quickly access critical data needed for its immediate operations.

2. Low Latency Access: L1 cache is the closest to the processor and has the lowest access latency. It is a small but extremely fast cache, holding only a limited amount of data. By having the most critical data stored in the L1 cache, the processor can retrieve it quickly without having to wait for slower memory accesses to main memory (RAM).

3. Cache Hierarchy: The multi-level cache hierarchy is designed based on the principle of spatial and temporal locality. Spatial locality refers to the tendency of the processor to access nearby memory locations, while temporal locality refers to the reuse of recently accessed data. The L1 cache focuses on spatial locality, while the larger L2 and L3 caches target temporal locality.

4. Shared Cache: The L3 cache, being shared among all processing cores, serves as a larger pool of data that can be accessed by any core when needed. It helps in reducing data duplication and improving overall efficiency by allowing data to be shared among cores, promoting inter-core communication, and reducing the need to access the main memory frequently.

5. Cost-Efficiency: Larger caches, like the L3 cache, are more expensive to implement on a chip. By having smaller L1 and L2 caches dedicated to each processing core and a shared L3 cache, the design achieves a balance between cost and performance. The L3 cache can be shared, reducing the overall cost of cache implementation on the chip while providing a substantial performance boost compared to relying solely on main memory.

In conclusion, the design of multi-level caching systems with local L1 and L2 caches per core and a shared L3 cache aims to optimize data access patterns, improve processor efficiency, and bridge the performance gap between the processor and main memory, ultimately enhancing the overall performance of multi-core systems.