Stranger in a Strange Land
understanding
🔷 Subscribe to get breakdowns of the most important developments in AI in your inbox every morning.
Here’s today at a glance:
Grok Is Released
Elon Musk, fresh from filing a lawsuit accusing OpenAI of being closed, promised to open source Twitter/X.AI’s LLM Grok this week:
As Pi Day, 3.14 March 14th passed an accusatory peanut gallery had sprung up. Where, where is Grok, we cried out.
We got to Sunday, and finally:
The joke, being of course, that Twitter has an annoying porn spam problem, with ░P░U░S░S░Y░I░N░B░I░O░ being the first response to any popular tweet.
The bio in question led to a torrent link, with the model weight.
ChatGPT hilariously responded:
In any case, Grok is middle-open-source?
The architecture:
Grok-1 is a 314B big Mixture-of-Experts (MoE) transformer. 🧐 What we know so far:🧠 Base model, not fine-tuned⚖️ Apache 2.0 license🧮 314B MoE with 25% active on a token📊 According to the initial announcement; 73% on MMLU, 62.9% GMSK, and 63.2% on HumanEval.@_philschmid
This is a big model
Notes:
It’s a base model, meaning it’s raw material that can be shaped by both instruction and fine-tuning (for anything from moderation to profanity, from left wing to right wing)
It’s not a terribly capable model (given its pre-instruction and fine-tuning, not surprising)
The size… is so large that you’d need 8 Nvidia H100s to do inference, roughly $300k of hardware unless some optimizations lower the requirements in the future
🗞️ Things Happen
NVIDIA GTC kicks off in San Jose today. Jensen is going to appear on stage, and will all the Transformer paper authors.
🖼️ AI Artwork Of The Day
