AI’s Next Feat: The Shift from the Cloud to Edge Computing

It has been two years since ChatGPT burst onto the scene, igniting a wildfire of investment in generative artificial intelligence (AI). This AI frenzy has propelled the valuations of startups like OpenAI, the creator of ChatGPT, and tech giants whose cloud computing platforms enable these powerful models. However, signs of strain are starting to surface, indicating that AI’s next growth phase may very well lie in the palm of your hand.

The Current Landscape of Generative AI

At present, much of the generative AI technology exists within the realm of the cloud. Take OpenAI, for instance; the company relies heavily on Microsoft’s Azure cloud platform to train and deploy its large language models (LLMs). Anyone with internet access can tap into ChatGPT, utilizing Azure’s expansive data centers around the globe.

Yet, as these models become increasingly complex and data-rich, the infrastructure necessary to train them and manage user queries escalates accordingly. This has caused a rush among tech companies to build bigger and more powerful data centers. For instance, OpenAI and Microsoft are currently in negotiations to launch a data center project in 2028, with a staggering price tag of $100 billion, according to The Information.

In fact, large players such as Alphabet, Microsoft, and Meta Platforms are forecasted to shell out a combined $160 billion on capital expenditures next year—an astounding increase of three-quarters compared to 2022. A significant chunk of this expenditure is likely to be spent on Nvidia’s coveted $25,000 graphic processor units (GPUs) and other essential infrastructure for model training.

The Challenges Ahead

However, the biggest challenge to this cloud-driven growth lies in technology itself. Current devices lack the necessary computing power, energy efficiency, and memory bandwidth to effectively run large models like OpenAI’s GPT-4, which boasts about 1.8 trillion parameters. Even Facebook’s relatively smaller LLaMA models require an additional 14 gigabytes of temporary storage to function effectively on a phone, while the latest iPhone 16 offers only 8GB of random access memory (RAM).

Silver Linings: Smaller Models and Specialized AI

Despite these obstacles, there is ample cause for optimism. Developers are increasingly gravitating toward smaller models tailored for specific tasks. These models necessitate far less data for training—Google’s lightweight Gemma architecture, for instance, can operate with as few as 2 billion parameters—and they often perform better than their larger counterparts with decreased margin for error.

Moreover, most everyday applications of AI, such as personal assistants and photo-editing tools, likely won’t require hefty models. In fact, some smartphones are already capable of live translation and real-time transcription functions. Therefore, it is logical for cloud service providers to begin shifting basic AI tasks to the edge, thereby allowing robust data centers to focus on more complex operations.

Future Directions: The Rise of Edge AI

As semiconductor manufacturers strive to cram more processing capability into personal devices, we can expect substantial shifts in the consumer market. Research firm Yole Group anticipates that the number of smartphones capable of supporting an LLM with 7 billion parameters will increase to 11% this year, up from 8% last year. Trailblazing chipmakers like Taiwan’s TSMC and South Korea’s Samsung Electronics are also innovating new methods to enhance semiconductor performance.

A former TSMC executive has warned, however, that achieving optimal performance and efficiency will require a paradigm shift. In the next decade, advanced packaging could yield a “multichiplet” with over 1 trillion transistors, creating immense opportunities for growth.

Investing in Edge AI: Opportunities Ahead

For savvy investors, the rise of edge AI presents a wealth of opportunities. While larger tech firms and Nvidia have dominated the conversation thus far, the evolution of AI could lead consumers to invest in newer, more sophisticated smartphones and personal computers. According to UBS analysts, combined sales in these markets are expected to surpass $700 billion by 2027, indicating a 14% growth from current levels.

Traditional tech brands, from Apple to Lenovo, along with their suppliers, stand to benefit from this surge. In semiconductors, while Nvidia’s advanced GPUs continue to dominate, companies like Qualcomm and MediaTek are also poised for significant gains.

Conclusion

Success in the burgeoning edge AI market will hinge on creating compelling applications that consumers find indispensable. If executed effectively, the next big wave of AI innovation will shift from the commercial cloud to smaller, more capable devices, creating a landscape rife with opportunities for investors, businesses, and consumers alike. The edge may not only be where AI applications move—it may very well be where the money moves too.