Google Unveils Gemma 3n: A Multimodal Powerhouse for Developers
Google has recently announced that its innovative multimodal small language model, Gemma 3n, is now available in preview on the new LiteRT Hugging Face community. This new model is not just a step up from its predecessor but brings a plethora of features that cater to a variety of use cases. Supporting text, image, video, and audio inputs, Gemma 3n allows for greater versatility in applications and interactions.
Enhanced Model Variants: Gemma 3n 2B and 4B
Gemma 3n comes in two parameter variants—Gemma 3n 2B and Gemma 3n 4B—both of which support text and image inputs, with audio support promised soon. Compared to the earlier Gemma 3 1B, which launched earlier this year, Gemma 3n marks a significant leap in model size and capabilities. The earlier version required just 529MB to process up to 2,585 tokens per second on a mobile GPU, showcasing a much more compact performance.
Real-World Use Cases
One of Gemma 3n’s standout features is its applicability in enterprise environments. Google has positioned this model as an ideal solution for developers who can leverage the full resources of the device for larger models. For instance, field technicians with limited connectivity can capture images of components and pose questions directly from their mobile devices. Similarly, workers in warehouses or kitchens can manage inventory through voice commands while their hands are occupied.
Selective Parameter Activation for Optimal Efficiency
Gemma 3n employs a technique called selective parameter activation. This method enables efficient parameter management, allowing the two models to contain more parameters than the stated 2B or 4B that are actually active during inference. This intelligent design not only optimizes performance but also ensures that developers can utilize a model tailored for their specific needs.
Advanced Quantization Tools
In a significant move to enhance efficiency, Google has introduced new quantization tools as part of this release. These tools allow developers to fine-tune the base model, convert, and quantize it for performance optimization. Google states that with the latest versions of its quantization tools, new schemes can provide much higher quality int4 post-training quantization. This approach can reduce the size of language models by 2.5 to 4 times when compared to the traditional bf16 data type, thus lowering latency and peak memory consumption.
More Read
Retrieval Augmented Generation (RAG)
Beyond fine-tuning, Gemma 3n allows for on-device Retrieval Augmented Generation (RAG), which bolsters the language model with application-specific data. The AI Edge RAG library, which is currently available only on Android, enhances this capability.
The RAG pipeline includes several straightforward steps: data import, chunking and indexing, embeddings generation, information retrieval, and finally, response generation using a large language model (LLM). This setup allows for full customization, including support for custom databases, chunking strategies, and retrieval functions tailored to specific applications.
AI Edge On-device Function Calling SDK
Additionally, Google rolled out the AI Edge On-device Function Calling SDK, which is also available exclusively on Android for now. This innovative tool enables models to execute specific actions in response to commands, moving beyond mere text generation. For instance, an LLM can generate structured calls to functions, such as searching for current information, setting alarms, or making reservations.
To seamlessly integrate an LLM with an external function, developers can describe the function by specifying its name, providing guidance on when the LLM should invoke it, and detailing the required parameters. This metadata forms a Tool object, which is then passed to the LLM via the GenerativeModel constructor. The function calling SDK supports receiving function calls from the LLM based on the descriptive information provided, along with sending execution results back, enhancing interactivity and utility.
Explore the New Tools
For those interested in diving deeper into these groundbreaking tools, the best starting point is the Google AI Edge Gallery. This experimental app showcases a range of models and supports image, audio, and text processing, offering developers a hands-on experience with the latest advancements.
By integrating robust features and capabilities, Gemma 3n represents a significant advancement in the landscape of multimodal AI. Its applications and ongoing development pave the way for enhanced user experiences and innovative solutions in various sectors.
Inspired by: Source
