Gemini 1.5 – a breakthrough in AI from Google
Last week, Google launched its most advanced model, Gemini 1.0 Ultra, taking a significant step forward in improving Google’s products, starting with Gemini Advanced. Today, Cloud developers and customers can also start building with 1.0 Ultra thanks to the Gemini APIs in AI Studio and Vertex AI. Google teams are constantly pushing the boundaries of the latest models, with security as a core component. They are making rapid progress. In fact, Google is ready to introduce the next generation: Gemini 1.5. It shows dramatic improvements in many dimensions, and 1.5 Pro achieves comparable quality to 1.0 Ultra, using less computing resources. This new generation also brings a breakthrough in understanding long context. Google has been able to significantly increase the amount of information that models can process – consistently handling up to 1 million tokens, achieving the longest context window of any major core model. Longer context windows show the promise of what is possible. They will enable entirely new capabilities and help developers build much more useful models and applications. Google is excited to offer a limited trial of this experimental feature to developers and enterprise customers.
A deeper look at Gemini 1.5
Gemini 1.5 is built on Google’s leading research on Transformer and MoE architectures. While the traditional Transformer functions as one large neural network, MoE models are divided into smaller “expert” neural networks. Depending on the type of input given, MoE models learn to selectively activate only the most appropriate expert paths in their neural network. This specialization greatly increases the efficiency of the model. Google was an early adopter and pioneer of MoE techniques for deep learning through research such as Sparsely-Gated MoE, GShard-Transformer, Switch-Transformer, M4 and others. Google’s latest innovations in model architecture allow Gemini 1.5 to learn complex tasks faster and maintain quality while being more efficient to train and operate. These efficiencies help Google teams iterate, train and deliver more advanced versions of Gemini faster than ever before, and they are working on further optimizations.
Security and ethics in Gemini 1.5
In line with its AI Principles and robust security policies, Google ensures that models undergo extensive ethics and security testing. It then integrates these learnings from testing into its management processes and model development and evaluation to continually improve AI systems. Since the introduction of 1.0 Ultra in December, Google teams have continued to refine the model, making it safer for a wider release. They have also conducted new research on security risks and developed red-teaming techniques to test for a range of potential harms.
Building and experimenting with Gemini models
Google is committed to responsibly delivering each new generation of Gemini models to billions of people, developers and enterprises around the world. Starting today, Google is offering a limited trial of 1.5 Pro to developers and enterprise customers through AI Studio and Vertex AI. For more information, visit Google’s developer blog and Google Cloud blog. Google will introduce 1.5 Pro with a standard 128,000 token context window when the model is ready for a wider release. Pricing tiers that start with a standard context window of 128,000 tokens and scale up to 1 million tokens when the model is upgraded are planned soon. All in all, Gemini 1.5 represents a significant step forward in the development of artificial intelligence, offering not only improved performance, but also groundbreaking capabilities in understanding long context. With its innovative architecture and advanced features, Gemini 1.5 opens up new opportunities for developers and enterprises to create more advanced and useful applications. Google’s commitment to security and ethics ensures that these advanced technologies will be developed and deployed responsibly, which is critical to the future of artificial intelligence.
