Soynade Logo
Back to Research
paper

Oolel-Embed: Cross-lingual Speech-Text Retrieval for Wolof and French

2026-04-20
Yaya Sy, Dioula Doucouré, Christophe Cerisara, Irina Illina
Read Original

Cross-lingual Matryoshka Representation Learning across Speech and Text explores how spoken Wolof queries can retrieve relevant documents written in French through a shared representation space.

Useful information for Wolof speakers is often available in written French, while speech may be the most natural way to search for it. Conventional retrieval systems transcribe the Wolof query, translate it into French, and then search the document collection. Each stage adds computation and can introduce errors that affect the final result.

Oolel-Embed connects the original Wolof speech directly with French text at retrieval time, without passing the query through a separate transcription-and-translation chain.

Shared representation across speech and text

The model combines features from a Wolof HuBERT speech encoder with a Qwen3-based text embedding model. Its strongest architecture, Late-Fusion, learns the connection between these representations while keeping the underlying speech and text models frozen.

The resulting embedding space supports both spoken Wolof and written Wolof queries against French documents. Semantically related speech and text are placed close together even when they come from different languages and modalities.

Matryoshka embeddings

Oolel-Embed produces nested embeddings at 128, 256, 512, and 1,024 dimensions. A single model can therefore support different retrieval configurations: smaller embeddings reduce storage and search costs, while larger embeddings preserve more retrieval quality.

This flexibility is particularly useful when deploying retrieval systems across environments with different compute, memory, and latency constraints.

Training data

Text training uses 1,176,908 Wolof-query and French-document pairs, representing more than 593 million French document tokens.

The speech foundation comes from 1.4 TB of natural Wolof audio collected from public web sources. Source separation, diarization, voice activity detection, and quality filtering produced 860 hours of high-quality speech before further transcription filtering was applied to construct the paired training data.

Retrieval benchmarks

The model is evaluated in two cross-lingual retrieval settings:

  • Kallaama-Retrieval-Eval contains 150 natural Wolof speech and text queries. Each query is associated with three generated French documents written as a dialogue, a blog post, and a story.
  • Fleurs-Retrieval-Eval contains 166 natural Wolof-speech and French-web-document pairs.

Together, these benchmarks test retrieval with both generated and naturally occurring French documents.

Retrieval results

On Kallaama-Retrieval-Eval, the 1,024-dimensional Late-Fusion model reaches 69.85 nDCG@5 and 74.49 nDCG@10.

  • The ASR-to-text retrieval pipeline reaches 57.09 nDCG@5 and 62.82 nDCG@10.
  • NLLB-LLM2Vec, using 4,096-dimensional embeddings, reaches 57.98 nDCG@5 and 61.53 nDCG@10.

On Fleurs-Retrieval-Eval, Late-Fusion reaches 57.89 nDCG@5 and 61.19 nDCG@10, compared with 55.98 and 59.43 for NLLB-LLM2Vec.

At 128 dimensions, Late-Fusion reaches 56.13 nDCG@5 and 62.30 nDCG@10 on Kallaama. The result illustrates the trade-off between compact embeddings and retrieval quality.

Transfer to speech tasks

The learned representations also transfer to speech tasks beyond document retrieval:

  • Keyword spotting: 88.79 F1 and 89.79 recall.
  • Intent detection without task examples: 44.79 F1 and 50.64 recall.
  • Intent detection after fine-tuning with 16 examples per class: 96.11 F1 and 96.10 recall.

The intent-detection results distinguish performance without task examples from performance after task-specific fine-tuning.

Scope and limitations

The experiments focus on Wolof and French. Most of the paired French training documents are generated, and the two principal retrieval benchmarks contain 150 and 166 queries. Further evaluation across larger collections, additional domains, and other language pairs will be needed to measure how well the approach generalizes.

The results also show that reducing the embedding dimension affects retrieval quality, particularly at 128 dimensions. The appropriate representation size therefore depends on the storage, latency, and accuracy requirements of the retrieval system.

Resources