▲ Cerebras Systems WSE processor
In August 2019, Cerebras’ first wafer-level chip WSE made a sensation in the global technology circle as soon as it came out. It is made from a single wafer and integrates 1.2 trillion transistors on an area of 46225mm². Its second-generation WSE-2 launched in 2021 went one step further, using a 7nm process, setting a new record of 2.6 trillion transistors integrated.
In contrast, the Nvidia A100 GPU, regarded by the industry as the benchmark for cloud AI chips, also uses a 7nm process and has a total of 54 billion transistors.
Next, Cerebras plans to use this financing to expand its global business and engineering team, as well as design the next generation of processors based on TSMC’s 5nm node.
The benefits of this article: Cerebras Systems will introduce the giant chip WSE-2 speech PPT on Hot Chips2021, which can be obtained by replying to the keyword [Core Things 191] in the chat bar of the official account.
01. Venture partners are on the road again, investors claim that they are redefining the possibility of AI
Cerebras Systems is located in California, USA and was founded in 2015 by Andrew Feldman, Gary Lauterbach and others.
Andrew Feldman and Gary Lauterbach are the CEO and CTO of Cerebras Systems, respectively. The two have been working together for more than 12 years.
Andrew Feldman has an MBA degree from Stanford University and has completed many acquisitions and listings as a company executive. Gary Lauterbach is a well-known computer architect in the industry. He was the chief architect of SPARC Ⅲ and UltraSPARC Ⅳ microprocessors.
In 2007, Andrew Feldman and Gary Lauterbach founded the micro server company SeaMicro. In 2012, SeaMicro wasAMDWith the acquisition of 334 million US dollars, the two also joined AMD. Andrew Feldman served as Vice President at AMD for two and a half years.
▲ Cerebras Systems CEO Andrew Feldman (left) and CTO Gary Lauterbach (right)
Cerebras Systems has completed 6 rounds of financing since its establishment, with a total amount of US$720 million. As early as 2016, Cerebras Systems completed its first US$64.5 million financing; in January of the following year, Cerebras Systems received US$25 million in Series B financing; 6 months later, it raised another US$60 million; November 2018 , Completed the D round of financing of 88 million US dollars.
With continuous financing, Cerebras Systems has also become a new AI chip unicorn from a company valued at $245 million. Twitter, Benchmark, which funded Snap, legendary chip designer, former AMD CTO Fred Weber, AI scientist of the famous non-profit laboratory OpenAI, and Ilya Sutskever, co-founder of AlexNet, have all invested in it. .
In 2019, Cerebras Systems completed its E round of financing, with a company valuation of approximately US$2.4 billion. Today, the new F round of financing has raised another US$250 million in funding for the company, which is equivalent to approximately 6% of Cerebras Systems, and its valuation has exceeded US$4 billion.
According to technology media AnandTech, the US$250 million in financing will support Cerebras’ layout in the next 2-3 years, including designing chips at the 5nm node and new memory expansion solutions. Cerebras Systems currently has about 400 employees in Sunnyvale, USA, San Diego, Toronto, Canada, Tokyo, Japan and other places, and hopes to expand to 600 by the end of 2022, mainly increasing the number of engineers and focusing on full-stack product development.
Rick Gerson, co-founder and chairman of Alphawave Ventures, said: “Cerebras Systems is redefining the possibilities of artificial intelligence and has a top-notch performance in accelerating innovation in several fields such as pharmaceuticals and life sciences. We are proud to work with Andrew Work with the Cerebras team to support them in introducing high-performance AI computing to new markets and regions around the world.”
02. The second-generation processor has 2.6 trillion transistors, and its performance is more than doubled
In 2019, Cerebras Systems released its first-generation WSE chip, which has 400,000 cores and 1.2 trillion transistors and uses TSMC’s 16nm process.
In April of this year, Cerebras Systems launched the second-generation processor WSE-2, with a record-breaking 2.6 trillion transistors (the largest GPU on the market has only 54 billion transistors) and 850,000 AI-optimized cores, similar in size to a dinner plate , Using TSMC’s 7nm process. Compared with the first generation of WSE processors, WSE-2 has more than doubled the number of AI cores, number of transistors, density, memory bandwidth and other parameters.
▲ Comparison of Cerebras two-generation processor parameters (Source: AnandTech)
Unlike many current chips, Cerebras Systems’ WSE-1 and WSE-2 are not made from a small part of the wafer, but process the entire wafer with a diameter of 300mm into one chip. Although this chip is relatively large, it consumes less power and occupies less space than a GPU cluster with the same computing power due to the number of transistors and cores and the advantages of interconnection.
If a traditional GPU cluster wants to achieve the same computing power, it needs dozens of racks to carry hundreds or even thousands of GPU chips. The CS-2 is only 26 inches tall, 1/3 of the standard data center rack.
Cerebras Systems’ technology has also played a role in many fields such as medicine, astronomy, and scientific research.
Whether it is the United States Argonne National Laboratory, Lawrence Livermore National Laboratory, Pittsburgh Supercomputing Center, University of Edinburgh Supercomputing Center and other scientific research institutions, as well as GlaxoSmithKline, Tokyo Electronic Devices and other manufacturers have become loyal customers of Cerebras Systems .
03. The parameter scale has been increased by 100 times, and deep learning services are also provided in the cloud
On August 24 this year, Cerebras Systems also launched the world’s first brain-scale AI solution.
The human brain contains about 100 trillion synaptic structures, while the previous largest artificial intelligence hardware cluster has only about 1 trillion parameters, which is similar to the same number of synapses, which is only 1% of the size of the human brain. The single CS-2 of Cerebras Systems supports more than 120 trillion parameters, reaching the level of the human brain in scale and promoting the development of AI neural network technology.
Rick Stevens, deputy director of Argonne National Laboratory, said: “Cerebras’ invention will increase the parameter capacity by 100 times, and it has the potential to change the industry. For the first time, we will be able to explore models of the size of the brain, opening up new avenues for broad research and insights. .”
The solution includes four core technologies, codenamed Weight Streaming, MemoryX, SwarmX and Sparsity.
Weight Streaming allows the AI model parameters to be stored off the chip, while providing the same training and inference performance capabilities as the chip, simplifying the workload distribution model.
MemoryX is a new memory expansion technology that can achieve 2.4 PTAB high-performance memory and supports 120 trillion parameter models.
SwarmX is a high-performance, AI-optimized communication interconnection structure that can interconnect up to 163 million AI cores, spanning 192 CS-2 systems to work together to train a single neural network.
Sparsity allows users to select the weight sparsity of the model, reducing the FLOP computing power and time required for model processing.
On September 16, Cirrascale Cloud Services, a deep learning cloud service provider in the United States, announced the use of the CS-2 system and WSE-2 processor.
In its application, the 8GPU server is 9.5 times slower than the CS-2 system to train the natural language processing NLP BERT model. In terms of training accuracy, users need more than 120 GPUs to match the training accuracy of a single CS-2 system.
▲ Cerebras Systems system in the computer room
04. Conclusion: Cerebras Systems is recognized for its development of giant chip routes
What needs to be pointed out is that Cerebras Systems’ WSE series chips are not simply amplifying the chip size. Large-size chips require chip companies to have unique technologies and solutions in terms of interconnection, chip packaging, and heat dissipation. These technical solutions are also reflected in Cerebras Systems’ brain-scale AI solutions.
Although its chips are large in size and cannot be used in PCs, mobile devices and other fields, Cerebras Systems has also shown us the broad application prospects of giant chips, and its products can occupy a place in supercomputing, cloud and other institutions or corporate solutions. The new round of financing represents capital’s recognition of this technical route.