Microsoft is reportedly engaged on a brand new giant language mannequin (LLM) to tackle Google’s Gemini and OpenAI’s GPT-4.
Codenamed MAI-1, the brand new LLM is at present within the growth section and is being led by Mustafa Suleyman, co-founder of Google DeepMind and Inflection AI, The Info reported citing two sources.
Suleyman joined Microsoft in March together with Karen Simonyan, the opposite co-founder of Inflection AI, with the intention to lead the corporate’s copilot effort, in line with a weblog publish authored by Microsoft Chief Government Satya Nadella.
Microsoft had additionally paid $650 million to Inflection AI to license its software program. Suleyman and Simonyan together with different Inflection AI employees becoming a member of Microsoft are a part of the identical deal.
Whereas the sources cited by the Info didn’t reveal the aim behind constructing the 500-billion parameter LLM, they stated the brand new LLM might be launched on the firm’s Construct convention later this month.
Reportedly, the corporate is dedicating an enormous quantity of computing assets to coach the mannequin, together with utilizing information from the web and information generated from GPT-4.
To place issues into context, OpenAI’s GPT-4 reportedly has 1.76 trillion parameters and the corporate spent over $100 million on compute assets to coach it.
Whereas Microsoft could also be engaged on the behemoth mannequin, the corporate final month launched a brand new household of small language fashions (SLMs) — Phi-3 household — as a part of its plan to make light-weight but high-performing generative AI know-how obtainable throughout extra platforms, together with cellular gadgets.
The Phi-3 household consists of three fashions — the three.8-billion-parameter Phi-3 Mini, the 7-billion-parameter Phi-3 Small, and the 14-billion-parameter Phi-3 Medium.
The previous couple of months have seen a flurry of LLMs being introduced by a number of distributors, corresponding to Snowflake, Databricks, Cohere, Mistral, Anthropic, Meta, Google, and AWS.
Whereas Snowflake launched its Arctic LLM, Databricks launched its DBRX mannequin. Individually, Meta had launched its Llama 3 mannequin. Simply days later, Cohere had launched iterations of its Command household of fashions.
