Full-Text Search | turbopuffer

Overview

turbopuffer implements BM25 full-text search over string attributes. Enable it in your schema, then rank documents using the BM25 operator.

Enable full-text search

Set fullTextSearch: true (or pass a configuration object) when defining your schema:

await client.namespaces.write("my-namespace", {
  schema: {
    content: {
      type: "string",
      fullTextSearch: {
        language: "english",
        stemming: true,
        removeStopwords: true,
      },
    },
  },
  upsertRows: [{ id: "1", attributes: { content: "turbopuffer vector search" } }],
});

Query with BM25

const results = await client.namespaces.query("my-namespace", {
  rankBy: ["content", "BM25", "vector search engine"],
  topK: 10,
});

Configuration options

Option	Default	Description
`language`	`english`	Tokenization language for stemming and stopword removal
`stemming`	`false`	Reduce words to their root form
`removeStopwords`	`true`	Filter common words (the, a, is, …)
`asciiFolding`	`false`	Normalize accented characters (à → a)
`caseSensitive`	`false`	Case-sensitive matching
`k1`	`1.2`	BM25 term saturation parameter
`b`	`0.75`	BM25 document length normalization

Supported languages

arabic, danish, dutch, english, finnish, french, german, greek, hungarian, italian, norwegian, portuguese, romanian, russian, spanish, swedish, tamil, turkish