With the birth of ChatGPT, competition in the field of artificial intelligence (AI) has entered the same heat.Nvidia's high -end graphic processing unit (GPU) chip is fluttered by major technology companies.At the same time, some startups have taken a different approach to focus on the development of another chip -AI reasoning chip to inject new impetus into the booming and application of AI products.
According to a recent report by the physicist organization network, these AI reasoning chips aim to reduce the high calculation costs required to generate AI, and better meet the daily operation requirements of AI tools.The continuous exploration of such chip costs and the continuous improvement of performance is expected to set off a new wave of AI applications, so that more complex and powerful AI applications have entered millions of households.
Reasoning Calculation Demand Water Rose Boat
Training and reasoning are the solid cornerstone of the two core capabilities of AI large language models.
During the application process, the trained ChatGPT and other generatory AI tools will absorb new information, which can be reasonably reasoned and generates response, such as writing documents and generating images.This type of AI tools can be applied in the fields of medical diagnosis, autonomous driving, and natural language understanding.
With the widespread application of AI models, the number of hardware and calculation of reasoning is increasing, and the demand for reasoning chips will also rise.International data companies (IDC) reports show that in the next few years, the proportion of AI servers on the reasoning side will continue to rise.It is estimated that by 2027, the workload for reasoning will occupy more than 70 %.
Science and technology company bidding new products
Start-up companies such as CEREBRAS, GROQ and D-Matrix, as well as traditional giants such as Chaowei Semiconductor Corporation (AMD) and Intel, have launched AI reasoning chips.These companies keenly captured the opportunity to show their skills in AI reasoning chips.
According to CEREBRAS's official website, on August 28, 2024, the company launched the AI reasoning chip of the same name.This chip implements 1800TOKEN/second reasoning speed on the LLAMA 3.1-8b model; 450Token/second reasoning speed is achieved on Llama 3.1 70B, which is about 20 times the Nvidia GPU reasoning speed.Token refers to the smallest unit or basic element of AI processing text, such as a word, a character, etc.
CEREBRAS explained that this excellent performance is due to its innovative AI chip design solution.Its wafer -grade engine (WSE) is like a huge computing factory, the biggest feature is that the size is amazing -a single chip occupies almost the area of a whole piece of wafer.On this oversized chip, the calculation unit and memory unit are highly integrated to form a dense grid structure.Such a design allows data to transmit between the calculation unit and the storage unit within a very short distance, which fundamentally reduces the cost of data movement and solves the width bottleneck that the GPU reasoning cannot be avoided.Such large chips can process information faster, so that the answer is given in a shorter time.
As early as February last year, GroQ released its own AI reasoning chip Groqcloud.It realizes 250Token/second reasoning services on the LLAMA 3.1 70B model, which is almost an increase than the GPU.
On November 19 last year, the Silicon Valley startup D-Matrix announced that its first AI reasoning chip CORSAIR has begun to ship, aiming to provide services such as chat robots and video generation.In a single server environment, Corsair can enable the LLAMA3 8B model to achieve 60,000Token/second processing capacity, and the latency of each token is only 1 millisecond, which fully demonstrates its excellent performance in high -speed processing of large -scale data.It is worth mentioning that Corsair can significantly reduce energy consumption and cost while providing the same performance compared to GPUs and other solutions.
Application development on the new track
Technology companies such as Amazon, Google, the Yuan universe platform, Microsoft, and other technology companies have scolded the expensive GPUs, with a view to getting off the AI development track.At the same time, the AI reasoning chip manufacturer has aimed at a wider customer group, hoping to show his skills in this new blue ocean.
These potential customers are lacking in the Fortune 500 companies that eager to use emerging generation AI technology but do not want to spend Zhouzhang's self -built AI infrastructure.Moreover, buying AI reasoning chips is cheaper than buying GPUs from Nvidia.The AI reasoning chip aims to optimize the speed and efficiency of reasoning calculation, and is particularly good at intelligent recommendations, voice recognition, natural language processing and other fields.
Industry experts said that once the reasoning speed increases to thousands of token per second, the AI model will be able to complete the thinking and answering process in the blink of an eye.This will not only make the existing application of interaction efficiency achieve a qualitative leap, but also bring a series of refreshing human -computer interaction scenarios.For example, in the field of voice dialogue, delay will be compressed to millisecond -level, which can achieve a near natural dialogue experience; in the field of virtual reality/augmented reality, AI will be able to generate and adjust the virtual environment, role dialogue and interactive logic in real time.Bring a personalized and immersive experience.