Understanding Bias Across Text and Image Embeddings
We are building an interactive research prototype to understand how different AI models organize products—and whether gendered patterns found in product titles also appear in product imagery.
The hypothesis
Our initial hypothesis is that model architecture influences how strongly gendered wording shapes product embeddings. Text models may place products closer together because their titles contain words such as “men’s,” “women’s,” “boys’,” or “girls’.” If similar separation appears in an image-only model, however, wording cannot be the only explanation. Visual characteristics such as color, shape, styling, and presentation may also contribute to the pattern.
Understanding the clusters
The prototype lets us compare where the same products appear across multiple embedding spaces. The two-dimensional clusters are exploratory views—not definitive measurements—but they help us identify patterns worth testing.
When products form a cluster, we want to understand what is driving that proximity:
- Are the products functionally similar?
- Are they grouped primarily by gendered language?
- Do visual characteristics create comparable groupings?
- Does the pattern remain consistent across model architectures?
Linked product selection and nearest-neighbor inspection allow us to move from an aggregate pattern to the individual products responsible for it.
Models currently in place
We are comparing five representations of the same product cohort:
- TF-IDF: a lexical baseline based on shared words.
- MPNet: a bidirectional text encoder.
- BGE-base: a second bidirectional text encoder.
- Qwen3-0.6B: a causal language model using last-token pooling.
- CLIP ViT-L/14: a vision encoder that receives only product-image pixels.
This combination lets us compare simple lexical similarity, semantic text representations, causal encoding, and image-only representation within one evaluation framework.
The outcome we are working toward
Our goal is to create a practical evaluation tool for understanding how model choice affects product similarity. Alongside visual exploration, we are developing metrics for neighborhood agreement, separation beyond chance, and matched-product comparisons.
Ultimately, we want to help teams ask a better question before selecting an embedding model: not only “How well does it perform?” but also “What relationships does it preserve, amplify, or introduce?”
The choices behind comparison views connect to data visualization as a product design decision. For another example of making model limits visible, see the PhysaFlow calculator’s explanation of assumptions. Find this research alongside my data science and product experiments.