Gemini 1.5 Generative AI Model Unveiled by Google
Google Gemini has only been around for about two months, but the search giant has already released its successor, and it’s its newest Large Language Model (LLM) to date – Gemini 1.5. This version is currently only available to businesses and developers, with a full consumer rollout expected soon.
Google CEO Sundar Pichai says Gemini 1.5 features “dramatic improvements in multiple dimensions” while achieving quality comparable to Gemini 1.0 Ultra, its most advanced LLM, and using less computing.
Additionally, Pichai added that this new generation achieved a breakthrough in understanding long context, and now Google has been able to “increase the amount of data our model can handle – running up to a million tokens consistently and achieving the longest context window of any large-scale foundational model yet.”
Gemini 1.5: What’s new?
First, Gemini 1.5 features a new Mixture-of-Experts (MoE) architecture, making it more efficient and easier to service.
Initially, Google will release the 1.5 Pro version for early testing, and it will perform at the same level as the 1.0 Ultra. Gemini 1.5 Pro is available with a standard context window of 128,000 tokens, but a limited number of developers and companies can try it with a context window of up to one million tokens.
Google also emphasized that Gemini 1.5, built on Transformer and MoE architectures, is more efficient because the MoE models are divided into smaller “expert” neural networks.
Additionally, context understanding is another key area where Gemini 1.5 has been improved. “The 1.5 Pro can handle massive amounts of data at once – including 1 hour of video, 11 hours of audio, codebases containing more than 30,000 lines of code, or more than 7,00,000 words. In our research, we’ve also managed to test up to 10 million characters,” Google said.
Simply put, if you give Gemini 1.5 a large chunk of information, such as a major novel, a research paper, or as Google mentions, the 402-page Apollo 11 mission transcript, and ask it to summarize it, it can do it. And later you can also ask detailed questions based on what Gemini 1.5 understands about it.
The show sees a jump
Compared to the Gemini 1.0 Pro, the new 1.5 Pro model outperforms it in 87% of the benchmarks used to develop Google’s LLM and performs similarly to the Gemini 1.0 Ultra.
Another big change is the ability to display “in-context learning” skills. This means that Gemini 1.5 Pro can “learn a new skill from information given in a long prompt without any extra fine-tuning.”
