Google Moves The World Forward
Jeff Dean announces Gemini Pro 1.5 in developer preview
Jeff Dean announces Gemini Pro 1.5 in developer preview, in Google’s usual style, preventing many people from actually using the tech while declaring how great it actually is.
It does seem to be a breakthrough:
Looong Context - Up to 10 million tokens, with perfect recall in needle in the haystack tests
Multimodal - hours of video and audio
Reasoning about code, audio, and video
Working with large codebases
It’s a large mixture of experts
Maybe in 6 months when Google finally allows open access, we’ll finally be able to figure out whether it’s better than GPT-4. But if they actually solved the large codebase problem without burning too many GPUs at inference, we may be on the precipice of the next great leap forward.