A knowledge base lives on your server. The phone sends a question and shows an answer.

Where do the embeddings and the vector index actually live?

In a column, in your database, in the same row as the text. An embedding, in OpenAI's own words, is a vector or list of floating point numbers, and the distance between two vectors measures their relatedness. That is the whole trick. Nothing clever happens at question time. The clever part already happened when you embedded the documents, and what is left is arithmetic over numbers you stored.

Supabase's guide shows the shape in Postgres. You enable the extension, then declare the column, which in its example reads embedding extensions.vector(384). The number in the brackets is not decoration. Supabase tells you to change it to match the dimensions your embedding model produces, and the models do not agree with each other. OpenAI's table lists 1536 dimensions for text-embedding-3-small and 3072 for text-embedding-3-large. Pick the model before you create the table, because the column is typed to the model.

Searching is a sort, not a lookup. Supabase documents a Postgres function called match_documents that takes a query embedding, a match threshold and a match count, and returns the rows that clear the threshold, closest first. Three distance measures are available on a vector column: cosine distance, Euclidean distance and negative inner product. Supabase notes that dot product tends to be the fastest if your vectors are normalised. Once the table starts to grow you add a vector index so the query stops reading every row. All of that is a database object, which is the plainest answer to where the index lives: wherever your Postgres lives.

None of this is special to one vendor. Any Postgres with pgvector does it, and dedicated vector databases are the same arrangement with different names for the pieces. For an app-sized corpus, keeping your documents, your vectors and your user accounts in one Postgres is one less system to operate and one less place for data to drift, which is the practical case for Supabase as a backend and for the managed Postgres that comes with most app platforms.

Supabase, storing embeddings in Postgres with pgvector, read 10 October 2026

Why is this a server job and not something the phone can do?

Three reasons, and the first one is the key. The embeddings endpoint is authenticated: OpenAI's own examples send an authorization header carrying your API key with every request. An app binary is a file anybody can download and unpack, so a key compiled into it is a key you have published. Obfuscating it moves the problem by about a week. The phone therefore calls an endpoint you own, that endpoint holds the key, and the same goes for the second call, the one that writes the answer.

The second reason is that the query vector and the stored vectors have to sit in the same space. Distances only mean anything between vectors from the same model. Change the model and every number moves. Even one model can hand back a shorter vector, because OpenAI documents a dimensions parameter that shortens an embedding by dropping numbers from the end of the sequence. If the document vectors were built on a server last month and the query vector comes from whatever small model happens to be on the handset, the two sets of numbers are not comparable, and nothing in the system will tell you so. The results just quietly get worse.

The third reason is the corpus. A nearest row search ranks every row, so whichever machine runs the search needs every row. Putting the knowledge base on the device means shipping your documents to everyone who installs the app, and fixing one sentence in one document means shipping an app update. Keep it on the server and an edit is one re-embed of one chunk, live for everybody in seconds. This is the same boundary you would draw for adding search, and for the same reason: the index belongs next to the data.

What the phone does get is worth saying out loud, because it is not nothing. It holds the conversation on screen. It caches the last answers, so a repeated question costs you nothing. It shows which documents an answer came from, which is the single feature that makes people trust the thing. And it can tell someone honestly that this screen needs a connection. An app that pretends to answer offline is worse than one that admits it cannot.

OpenAI, vector embeddings guide, read 10 October 2026

What does one question cost in practice?

Less than people expect on the retrieval side, and the bill is somewhere else. OpenAI prices its embedding models with a pages per dollar figure and notes the assumption of roughly 800 tokens to a page. Read on 10 October 2026, that table gives 62,500 pages per dollar for text-embedding-3-small, 12,500 for text-embedding-ada-002 and 9,615 for text-embedding-3-large. Work it backwards and a page costs about one six-hundredth of a cent with the small model.

So the one-off build is cheap. Ten thousand pages of documentation embedded once with the small model is about sixteen cents, and about a dollar with the large one. The question itself is cheaper still. A one-line question is maybe twenty tokens, a fortieth of a page, so embedding it rounds to nothing. You would be answering millions of questions before that line on the invoice was worth reading.

The cost that scales with traffic is the second call, the one that reads the retrieved chunks and writes the answer. You pay for everything you paste into that prompt. Ask for the ten nearest chunks at 500 tokens each and every question carries 5,000 tokens of input before the person has finished typing. That is the dial to tune. Dropping the match count from ten to four cuts it by more than half and often makes answers better, because the model stops being handed six paragraphs that were merely nearby. Prices for answer models move often enough that a number here would be stale, so take it from the model page you are actually calling.

The real trap is not per-question cents. It is re-embedding. Change your embedding model and the whole corpus has to be rebuilt, because the old vectors describe the old space. The same is true of chunk size: change it and you rebuild. That makes both choices stickier than a price list suggests, so settle them while you are still sketching the backend and database, then leave them alone.

What goes wrong first, and how do you catch it?

Chunking, almost always. People embed a whole handbook as one vector and wonder why retrieval feels vague. One vector for forty pages is an average of forty pages: close to everything, useful for nothing. There is a hard ceiling as well. OpenAI lists 8,192 tokens of maximum input for its current embedding models, so a long document cannot go in as a single call anyway. Cut on headings, keep each chunk readable on its own, and keep the heading inside the chunk text so a retrieved snippet still says what it is about.

Then dimension mismatches. A column declared at 384 and a model returning 1536 gives you an error, which is the good outcome. The bad outcome is two models that both return 1536. Nothing fails, half your rows sit in one space and half in another, and the only symptom is answers that are subtly wrong. Store the model name and a chunking version on every row, so you can always tell which rows are stale and re-embed just those.

After that, a missing floor. The match threshold exists because the nearest row is always some row. Ask a knowledge base with no refund policy about refunds and it hands back whatever was least unrelated, and the answer model writes a confident paragraph from it. With a threshold the query comes back empty instead, and an empty result is what lets your app say it does not know. Return the title and the row id with every match and show them under the answer, so a person can check it without trusting you.

The rest is ordinary product work. A knowledge base is one tool bolted onto a chat screen, so everything in adding an AI chatbot still applies: streaming, retries, a visible failure state, a cap on what one user can spend. Retrieval removes none of it. It only decides what the model is allowed to read before it answers.

Supabase, the match_documents function and its threshold argument, read 10 October 2026