PolarMem: Training-Free Polarized Latent Graph Memory

This skill teaches you to implement PolarMem-style memory systems that transform noisy VLM confidence scores into discrete logical states (verified-positive, verified-negative, uncertain) and store them in a polarized graph where inhibitory edges explicitly encode what is not true. At retrieval time, negative constraints dominate semantic similarity, categorically suppressing hallucination-prone results regardless of how "close" they appear in embedding space. The technique is training-free and wraps around any frozen vision-language model.

When to Use

• When building a multimodal RAG system and the agent hallucinates by retrieving semantically similar but factually wrong evidence (e.g., retrieving a photo of a robin when asked about a cardinal because both are "red birds")
• When the user asks to add negation or "verified absence" tracking to a retrieval-augmented agent
• When implementing a memory layer for a long-horizon agent that must distinguish between "I saw X" and "I confirmed X is not here"
• When building a knowledge graph where edges must represent both presence and absence relationships
• When the user wants to reduce false positives in image retrieval by encoding what each image does not contain
• When creating a verifiable agent whose answers can be traced back to logically grounded evidence, not just cosine similarity

Key Technique

The core problem: Standard dense retrieval (cosine similarity over embeddings) conflates semantic affinity with factual existence. An image of a forest may score highly for the query "tiger in forest" simply because forests are semantically close to tiger habitats -- even if no tiger is present. Probabilistic VLMs compound this by returning soft scores that never firmly say "no."

PolarMem's solution has three stages. First, non-parametric distributional partitioning runs ensemble VQA queries against each stored image (e.g., "Does this image contain a tiger?" across multiple prompt templates), collects confidence scores, then applies Otsu's method (maximum inter-class variance thresholding) to split the score distribution into three bins: verified-positive (above threshold + margin), verified-negative (below threshold - margin), and uncertain. The dynamic margin delta = kappa * sigma_weighted adapts to each image's score spread. This converts fuzzy likelihoods into discrete logical states without any learned parameters.