The meteoric rise of generative AI, with applications like ChatGPT capturing the public imagination, is driving an unprecedented surge in demand for high bandwidth memory (HBM). This critical component is essential for accelerating the training and inference of complex AI models, particularly large language models that underpin these transformative applications.
The HBM market is predicted to experience exponential growth in the coming years, far outpacing the broader DRAM market. A February 2024 report by Yole Intelligence, a market research firm, forecasts that the HBM sector will grow at a staggering compound annual growth rate (CAGR) of 50% between 2022 and 2028, reaching a market value of US$16.7 billion by 2028 . TrendForce, another market research firm, echoed this sentiment, predicting that memory makers would need to double their HBM bit output in 2023 and further increase it in 2024 to meet the escalating demand.
HBM’s unique architecture is central to its appeal. By stacking multiple layers of memory chips vertically and connecting them with through-silicon vias (TSVs), HBM offers unmatched bandwidth and energy efficiency compared to traditional memory structures. This design is tailor-made for the memory-intensive workloads of generative AI models, which often involve billions of parameters. In the world of AI, memory is the key to learning, decision-making, and adaptation. HBM enables AI accelerators to access vast amounts of data with exceptional speed, resulting in faster model training and inference, and ultimately, improved performance and reduced deployment time for AI applications.
Lam Research, in a recent article, highlighted HBM’s potential to redefine AI capabilities. They emphasised that advancements in semiconductor manufacturing, especially in areas like etch and deposition, and advanced packaging, are crucial to unlock the full potential of HBM. The relentless march of AI progress demands faster data access and lower energy consumption, making HBM an indispensable technology in meeting these requirements.
However, the growing reliance on HBM presents challenges. The design complexity and manufacturing costs associated with HBM are significant, primarily due to the use of silicon interposers to connect memory stacks and processors. The high demand for HBM is also straining supply chains, leading to potential shortages and price increases, as highlighted in SemiEngineering. The limited number of suppliers capable of producing HBM further exacerbates this issue, creating a bottleneck in the production of AI-powered devices.
Despite these hurdles, the future of HBM remains undeniably bright. Emerging applications like generative AI and data processing are still in their nascent stages, indicating a vast untapped market potential for HBM. As AI continues to evolve and permeate various sectors, the demand for high bandwidth memory is expected to soar. Furthermore, the industry is already witnessing the development of next-generation HBM technologies, such as HBM3, which promise even higher bandwidth and lower power consumption, further solidifying HBM’s role as a cornerstone of the AI revolution. SK Hynix, a leading memory manufacturer, recently announced mass production of HBM3, signaling a significant step towards meeting the growing demand. However, analysts at The Register warn that despite such efforts, the demand for HBM3 is so high that it could lead to a severe undersupply until 2025.
This insatiable demand for HBM3, coupled with its technical challenges and limited supply, paints a picture of an industry in flux. As AI continues its rapid evolution, the role of HBM will only become more critical, driving innovation in memory technology and shaping the future of the entire electronics industry.
Thousands of senior engineers and procurement professionals subscribe to our LinkedIn Market Intel newsletter – get yours here
For more help with looking at supply chain options, contact Astute Electronics
