High bandwidth memory (HBM) rather than graphics processor unit (GPU) will become the key factor determining AI performance, a leading semiconductor researcher at KAIST predicted.   Starting as early as next year, HBM4 is expected to evolve from a simple memory chip into a hybrid design that includes basic computing functions. This shift would help offload some of the GPU’s workload and speed up data processing.   Professor Kim Joung-ho of KAIST's School of Electrical Engineering says this transformation will reshape the future of Korea’s semiconductor industry. Korea already leads the world in memory chip manufacturing, and the expanded role of HBM could further strengthen that position.   “I envision a future, perhaps one or two decades from now, where HBM, not the GPU, becomes the central computing unit,” Kim said on Wednesday during an online presentation outlining the next-generation HBM road map. Kim has led HBM design research for over 20 years and heads Kaist’s Tera Lab, which focuses on semiconductor design and integration. According to Kim, HBM4 will be a turning point. For the first time, the base layer of the HBM chip — known as the base die — will include lightweight computing functions such as data compression, error correction and basic communication processing. This means GPU customers like Nvidia, Google, AMD and Intel will no longer want one-size-fits-all memory chips. Instead, they’ll demand custom HBM chips tailored to their specific workloads.   “This is a major opportunity for Korean chipmakers like Samsung and SK hynix,” Kim said. “Until now, global tech giants set the standards and Korean firms followed. But with HBM5 and beyond, we can lead, setting our own architectures and becoming global pioneers.” From HBM 6 onward, the traditional single-stack memory design will give way to what Kim calls a “multi-tower” architecture — multiple HBM stacks grouped like apartment buildings, all connected to a shared computing base. This design dramatically increases both memory bandwidth and storage capacity.   For the scale-up to be possible, thermal bottlenecks need to be resolved, to which Kim outlined several cooling solutions such as immersion cooling, which involves submerging the entire chip or system board in a special nonconductive cooling liquid to efficiently absorb heat, or embedded cooling, which allows heat to be removed from the package through building micro channels for coolant fluids to pass directly inside the chip or interposer layers.   Until such advanced cooling technologies are ready, moving the computing components to the top of the memory stack to be closer to the heat sink could also help prevent overheating, Kim said.   “Cooling will be the ultimate bottleneck,” he warned. “No matter how fast or large HBM becomes, if we can’t manage heat, performance will hit a wall.”   (ICE SEOUL)

Fonte notizia: KOREA JOONGANG DAILY