Groq’s AI chips are quicker than Nvidia’s? AI startup hits the highlight with ‘lightning-fast’ engine

AI startup Groq (now not Elon Musk’s Grok) has unveiled its new synthetic intelligence (AI) chip with a Language Processing Unit (LPU) structure that says to ship prompt reaction instances. This new innovation comes at a time when AI is witnessing a growth, and corporations similar to OpenAI, Meta and Google are arduous at paintings growing their suite of AI gear similar to Sora, Gemma and extra. On the other hand, Groq outright claims that it delivers “the arena’s quickest massive language fashions.”

Groq claims its LPUs are quicker than Nvidia’s Graphics Processing Gadgets (GPUs). Taking into consideration that Nvidia has grabbed the highlight thus far with regards to AI chips, this side is startling. On the other hand, to again that up, Gizmodo stories that the demonstrations made through Groq had been “Lightning-fast” they usually even made “…present variations of ChatGPT, Gemini or even Grok glance slow.”

Groq AI chip

The AI chip evolved through Groq has specialised processing gadgets that run Huge Language Fashions (LLMs) turning in just about prompt reaction instances. The brand new novel processing unit, referred to as Tensor Streaming Processor (TSP), has been categorized as an LPU and now not a Graphics Processing Unit (GPU). The corporate says it supplies the “quickest inference for computationally extensive programs with a sequential element to them”, similar to AI programs or LLMs.

What are the advantages? 

It removes the desire for advanced scheduling {hardware} and favours a extra streamlined solution to processing, the corporate claims. Groq’s LPU is designed to conquer compute density and reminiscence bandwidth – two issues that plague LLMs. The corporate says relating to LLMs, LPU has a better compute capability than a GPU and CPU, thus, lowering the volume of calculation time in keeping with phrase. This leads to a lot quicker textual content technology.

Calling it an “Inference Engine”, the corporate says its new AI processor helps usual system studying (ML) frameworks similar to PyTorch, TensorFlow, and ONNX for inference. On the other hand, its LPU Inference Engine does now not recently enhance System Finding out (ML) coaching.

Groq allows quicker and extra environment friendly processing, with decrease latency and constant throughput. On the other hand, it isn’t an AI chatbot and isn’t supposed to switch one. As a substitute, it claims to cause them to run quicker. Those that want to take a look at Groq can make the most of open-source LLMs similar to Llama-2 or Mixtral 8x7B.


In a demo shared through HyperWrite CEO Matt Shumer on X, the Groq equipped a couple of responses to a question, whole with citations in seconds. Every other demo of Groq in a side-by-side comparability with GPT-3.5 published that it finished the similar job as GPT, most effective just about 4 instances quicker. In line with benchmarks, Groq can hit nearly 500 tokens a 2d, in comparison to 30-50 tokens treated through GPT-3.5.

Additionally learn different most sensible tales nowadays:

Call for for Deepfake legislation! Synthetic intelligence professionals and business executives, together with ‘AI godfather’ Yoshua Bengio, have signed an open letter calling for extra legislation across the advent of deepfakes. Some attention-grabbing main points on this article. Test it out right here.

Sora raises fears! Since OpenAI rolled out its text-to-video AI technology platform, main content material creators are fearing if they’re the newest pros about to get replaced through algorithms. Take a look at all of the main points right here.

Microsoft to construct a home-grown processor! Microsoft has change into a buyer of Intel’s made-to-order chip industry. The corporate will use Intel’s 18A production era to make a imminent chip that the tool maker designed in-house. Learn all about it right here.

Yet another factor! We at the moment are on WhatsApp Channels! Observe us there so that you by no means leave out any updates from the arena of era. ‎To apply the HT Tech channel on WhatsApp, click on right here to sign up for now!

Leave a Comment