Skip to content
AI.info

The Pulse

Google details EmbeddingGemma 2’s multimodal design in report

Google has highlighted a technical report on EmbeddingGemma 2, its open model for representing text, images, audio and video in a shared space. The report describes its training and resource-saving features, while Google’s model card provides benchmark results.

Google details EmbeddingGemma 2’s multimodal design in report

Google has pointed developers to a technical report detailing how its EmbeddingGemma 2 model brings text, images, audio and video into a shared representation space. In a Nov. 17 post, the company said the report covers the model’s training strategy, evaluation methods and experimental results.

The open model represents individual types of input or combinations of them in the same space, extending Google’s earlier text-only embedding model. The report is a preprint, first posted to arXiv on Oct. 27 and revised on Nov. 9.

One space for different inputs

Embedding models turn inputs into numerical representations that can be used for tasks such as search and classification. EmbeddingGemma 2 produces representations for text, including code, as well as images, audio and video in a 768-dimensional vector space.

The model is designed for consumer hardware such as mobile devices and laptops. Its possible applications include search, retrieval-augmented generation, classification and clustering, according to Google DeepMind’s model card.

It has 740 million total parameters: a 270 million-parameter text model and modular vision and audio encoders with 170 million and 300 million parameters respectively. Developers can selectively load the encoders needed for their application.

The model has an 8K-token context window that can process minutes of audio or video. Short text instructions can steer its representations toward tasks such as search, classification, clustering or semantic similarity.

Training and smaller representations

The researchers describe a multi-stage training curriculum that progresses through different input types and objectives. Their report also discusses simplified per-layer embeddings and experiments examining key design decisions.

The model supports shortened representations through a technique called Matryoshka Representation Learning. Alongside its native 768 dimensions, it supports outputs truncated to 512, 256 or 128 dimensions and then re-normalized.

According to the model card, shortening vectors can reduce storage requirements, with minimal quality impact down to 256 dimensions; it describes 128 dimensions as best suited to text-only workloads. The report also lists adjustable vision token budgets and quantized weights among the options for resource-constrained environments.

Results and trade-offs

Google’s model card reports tests across text, code, image, visual-document, video and audio benchmarks using the full-precision checkpoint. On the multilingual version of the Massive Text Embedding Benchmark, EmbeddingGemma 2 scored 61.36 on the mean task metric, compared with 61.15 for EmbeddingGemma 1.

On the code version of that benchmark, the new model scored 78.68 on mean task NDCG@10, versus 68.76 for its predecessor. The card also reports results for the newer model on image, video and audio benchmarks, without predecessor scores in those rows.

The card’s truncation results show a trade-off between representation size and benchmark scores. At full size, the model scored 61.36 on the multilingual text benchmark; at 128 dimensions, it scored 57.89, with a listed compression ratio of 1:6.

Sources

  1. Google
  2. arXiv
  3. Google DeepMind on Hugging Face

Companies in this story

Related stories

Explore

More articles