Google Gemini Pro 1.5 Roundup

Gemini is finally an OpenAI beating model.

🔷 Subscribe to get breakdowns of the most important developments in AI in your inbox every morning.

Just to be clear, Gemini Pro 1.5 (hereafter referred to as “Gemini”) is not Bard and is also not Gemini Ultra, the subscription ChatGPT competitor launched last week. Instead, Gemini is currently a developer preview API of a very competent multi-modal language model.

TLDR

• Gemini can take in lots of data—text, audio, and video—into its context window

• Can retrieve single facts from the data near flawlessly

• I can retrieve and make sense of multiple facts from data quite well

• Given its strength in long context, you can give it lots of examples on how to do something, and its corresponding performance increases

• It’s better than GPT-4 for many business use cases

• It’s slow (up to 90 seconds for inference)

• Most of its strengths and weaknesses stem from the long context

Let’s examine Gemini in detail below:

Long Context - Needle in a Haystack

Gemini has near-perfect recall in needle-in-…