The startup Groq said on Monday that it has raised $640 million in a fresh investment round headed by Blackrock. Groq is building chips that run generative AI models faster than standard CPUs. Participating companies included Neuberger Berman, Samsung Catalyst Fund, Cisco, KDDI, and Type One Ventures.
Groq, which was allegedly hoping to raise $300 million at a somewhat lower ($2.5 billion) valuation, is celebrating a significant victory with this tranche, which puts the company’s total raised to over $1 billion and values it at $2.8 billion. Groq raised approximately $1 billion in April 2021 and raised $300 million in a funding round headed by D1 Capital Partners and Tiger Global Management. This valuation more than doubles that amount.
Groq has also announced today that Stuart Pann, the former chief operating officer of HP and president of Intel’s foundry division, will become chief operating officer of the startup. Yann LeCun, the main AI scientist at Meta, will advise Groq technically. LeCun’s appointment is somewhat of a surprise considering Meta’s investments in in-house AI processors, but it definitely provides Groq a strong ally in a competitive market.
Groq, which came out of stealth in 2016, is developing an inference engine for language processing units, or LPUs. According to the business, its LPUs can execute generative AI models that are now available at ten times the speed and one tenth of the energy consumption of OpenAI’s ChatGPT and GPT-4o.
Jonathan Ross, the CEO of Groq, is renowned for his contribution to the creation of the tensor processing unit (TPU), Google’s proprietary AI accelerator processor used for model training and execution. Nearly ten years ago, Ross co-founded Groq with Douglas Wightman, an entrepreneur and former engineer at Google parent firm Alphabet’s X moonshot lab.
Groq offers an LPU-powered developer platform called GroqCloud, which includes an API that lets users use its chips in cloud instances, as well as “open” models like Google’s Gemma, OpenAI’s Whisper, Mistral’s Mixtral, and Meta’s Llama 3.1 series. (Groq also runs GroqChat, an AI-powered chatbot playground that it introduced at the end of last year.) More than 356,000 developers were using GroqCloud as of July. According to Groq, some of the money raised during this round will be utilized to expand the company’s capacity and introduce new models and features.
“Many of these developers are at large enterprises,” Stuart Pann, Groq’s COO, told TechCrunch. “By our estimates, over 75% of the Fortune 100 are represented.”
As generative AI becomes more popular, Groq will have to contend with competition from rival AI chip startups as well as Nvidia, the industry leader in AI hardware.
Nvidia is attempting to preserve its dominance in the market for AI processors, which are needed to train and implement generative AI models. The company is expected to dominate between 70% and 95% of this industry.
like opposed to every other year like in the past, Nvidia has pledged to release a new AI chip architecture annually. Additionally, it is apparently starting a new business unit dedicated to creating custom chips for cloud computing companies as well as other businesses, including AI devices.
Other than Nvidia, Groq’s competitors are Amazon, Google, and Microsoft, all of whom already offer or plan to offer customized processors for cloud-based AI workloads. Customers of Google Cloud can use the aforementioned TPUs as well as Google’s Axion chip in due course. Microsoft recently introduced Azure instances in preview for its Cobalt 100 CPU, with Maia 100 AI Accelerator instances to follow in the coming months. Amazon offers its Trainium, Inferentia, and Graviton processors through AWS.
Analysts predict that in the next five years, the AI chip market might exceed $400 billion in sales, and Groq may see competition from Arm, Intel, AMD, and an increasing number of startups. Due in large part to cloud vendors’ increasing capital expenditures to meet the capacity demand for generative AI, Arm and AMD in particular have seen their AI chip businesses flourish.
In order to commercialize what company describes as a first-of-its-kind inference compute platform, D-Matrix received $110 million late last year. With $120 million, Etched came out of stealth in June to unveil a processor designed specifically to accelerate the transformer, the most popular generative AI model architecture available today. Masayoshi Son of SoftBank is allegedly trying to raise $100 billion for a chip business to take on Nvidia. Additionally, it’s said that OpenAI is in discussions to start a chip-making venture with investment firms.
Groq is significantly spending on industry and government outreach in an effort to carve out its niche.
To create Groq Systems, a new business unit, Groq bought Definitive Intelligence, a Palo Alto-based company that provides a variety of business-oriented AI solutions, in March. Serving entities, such as sovereign states and U.S. government agencies, that want to construct new data centers employing Groq processors or integrate Groq chips into already-existing ones is under Groq Systems’ jurisdiction.
More recently, the startup Groq signed an agreement to install tens of thousands of its LPUs in the Norway data center of European company Earth Wind & Power. Additionally, Groq teamed with Carahsoft, a government IT contractor, to market its solutions to public sector clients through Carahsoft’s reseller partners.
Additionally, Groq and Aramco Digital, a Saudi Arabian consulting business, are working together to install LPUs in upcoming Middle Eastern data centers.
Groq, a company located in Mountain View, California, is developing new ties with customers while simultaneously moving forward with the development of its microprocessor. The company said in August of last year that it will be working with Samsung’s foundry division to produce 4nm LPUs, which are anticipated to outperform Groq’s initial 13nm chips in terms of efficiency and performance.
By the end of Q1 2025, Groq intends to deploy over 108,000 LPUs.