Vector storage is coming to Meilisearch to empower search through AI

Vector search enables efficient retrieval of objects sharing similar characteristics. This AI-powered search technique uses embedding vectors. These vectors are mathematical representations of objects generated by machine learning models (like LLMs). Starting with the 1.3 release, Meilisearch supports storing and searching vectors.

Powering the Future of Search

Vector search might be the dawn of a new era for search. Its use cases are numerous. In ecommerce, it enables offering recommendations for similar products. It also allows building multi-modal search, like image or audio search. Building upon conversational AI technologies enables the creation of Q&A applications. Combining vector search with user-provided information like geolocation and search history can power even more contextual search experiences.

Vector search is also the foundation for semantic search, which aims to understand the query’s meaning. Conversely, traditional lexical search only matches keywords. With semantic search, a warm clothes query could give results like gloves, coat, and more results related to winter clothing.

Vector search unlocks a world of new capabilities for search. Take a look at what some users have already implemented:

• similar videos recommendations on a site listing Ruby conferences
• a documentation chatbot proof of concept using GPT3.5 and LangChain

Getting Started with Meilisearch Vector Store

Starting with v1.3, you can use Meilisearch as a vector store. Meilisearch allows you to store vector embeddings alongside your documents conveniently. You will need to create the vector embeddings using your third-party tool of choice (Hugging Face, OpenAI).

First, spin up a Meilisearch instance. You can install Meilisearch locally or create a Meilisearch Cloud account.

Then, enable the vector store experimental feature:

curl \
  -X PATCH 'http://localhost:7700/experimental-features/' \
  -H 'Content-Type: application/json'  \
  --data-binary '{
    "vectorStore": true
  }'

Meilisearch now accepts a _vector field in your documents. Use it to store the vector embeddings corresponding to your document in this field.

curl -X POST -H 'content-type: application/json' \
     'localhost:7700/indexes/songs/documents' \
     --data-binary '[
         { "id": 0, "_vectors": [0, 0.8, -0.2], "title": "Across The Universe" },
         { "id": 1, "_vectors": [1, -0.2, 0], "title": "All Things Must Pass" },
         { "id": 2, "_vectors": [0.5, 3, 1], "title": "And Your Bird Can Sing" }
     ]'

After storing your vectorized documents in Meilisearch, you can query them using the search or multi-search route. To do this, you need to compute the vector of your query (using a third-party tool) and send it in the vector field.

curl -X POST -H 'content-type: application/json' \
   'localhost:7700/indexes/songs/search' \
   --data-binary '{ "vector": [0, 1, 2] }'

When using vector search, returned documents include a semanticScore field:

{
  "hits": [
    { "id": 0, "_vectors": [0, 0.8, -0.2], "title": "Across The Universe", "_semanticScore": 0.6754 },
    { "id": 1, "_vectors": [1, -0.2, 0], "title": "All Things Must Pass", "_semanticScore": 0.7546 },
    { "id": 2, "_vectors": [0.5, 3, 1], "title": "And Your Bird Can Sing", "_semanticScore": 0.78 }
  ],
  "query": "",
  "vector": [0, 1, 2],
  "processingTimeMs": 0,
  "limit": 20,
  "offset": 0,
  "estimatedTotalHits": 2
}

A First Step Toward Semantic Search

Vector search is our first step towards semantic search. But our long-term goal is to provide hybrid search—combining the benefits of full-text and semantic search to provide the most relevant search experience. Clément Renault, founder and CTO of Meilisearch, shared his thoughts on Github about exploring semantic search — read for a founder’s perspective. We can’t wait to share more with you!

Drop your email below to learn more about our progress with AI-powered search. We’ll keep you posted on all updates regarding vector search & semantic search.

We’re excited to walk our first steps toward semantic search. We can’t wait to hear your thoughts on integrating Meilisearch as a vector store. You can give your feedback in this Github discussion.

You can stay in the loop by subscribing to our newsletter. To learn more about Meilisearch's future and help shape it, look at our roadmap and participate in our Product Discussions.

For anything else, join our developers community on Discord.

I’ll see you there.