Embeddings

Embeddings place text into a numerical space for similarity comparison. Retrieval quality also depends on chunking, filtering and source organization—not only the model.

#Basic Python request

pythonCopy
import os
from openai import OpenAI

client = OpenAI(timeout=20.0)
response = client.embeddings.create(
    model=os.environ["OPENAI_EMBEDDING_MODEL"],
    input=["A guide to Python generators", "A guide to database indexes"],
)
vectors = [item.embedding for item in response.data]
print(len(vectors), len(vectors[0]))

#JavaScript

javascriptCopy
import OpenAI from "openai";
const client = new OpenAI({ timeout: 20000 });
const result = await client.embeddings.create({
  model: process.env.OPENAI_EMBEDDING_MODEL,
  input: "A guide to Python generators"
});
console.log(result.data[0].embedding.length);

#Retrieval sequence

Chunk documents, embed those chunks, embed the query in the same space, compare with the appropriate metric, apply authorized filters and return the relevant source passages.

pythonCopy
from math import sqrt

def cosine(a, b):
    if len(a) != len(b):
        raise ValueError("Vector dimensions must match")
    scale = sqrt(sum(x*x for x in a)) * sqrt(sum(y*y for y in b))
    if scale == 0:
        raise ValueError("Zero vector has no cosine direction")
    return sum(x*y for x, y in zip(a, b)) / scale

This local calculation is not itself an embedding model or a retrieval-quality benchmark.

#Evaluate in order

Build a small baseline, inspect chunk size/overlap and metadata, then compare candidate models on the same queries. The source's small/large model examples are candidates to evaluate, not guaranteed accuracy rankings.

Keep model, dimensions and preprocessing consistent across ingestion and query. Do not mix spaces merely because their vector lengths happen to match.

#References

• Embedding creation↗
• Embedding guide↗