Tackling AI Image Hallucinations by Amplifying Their Flaws

Home

News

December 22, 2025

RalphRoberts

Vision models like ChatGPT frequently invent elements missing from an image. A novel approach reduces these errors by generating exaggerated versions of the model's own hallucinated details from its captions, then prompting it to revise. This technique requires no retraining or additional data, making it broadly applicable to various models and architectures.

A new study from China tackles the stubborn issue of hallucinations in AI-generated images and videos—details that clearly contradict the user's prompt.

The process starts conventionally: the model describes an image. That caption is then fed into a text-to-image model to produce a new image—any extra objects or features in this reconstruction directly reveal the model's initial hallucinations. By comparing the original and generated images, the system guides the model to avoid repeating those mistakes in future attempts.

An illustration of how the new method identifies and reduces hallucinations in image captions. The regular model describes birds that do not exist in the original image, leading to a reconstructed image that adds them in. These errors are marked in red. By contrast, the proposed method avoids these invented details while keeping the caption specific and fluent. Source: https://arxiv.org/pdf/2509.21997

This diagram shows how the new technique detects and minimizes caption hallucinations. A standard model incorrectly adds birds to its description of an image, and the reconstructed version inserts them visually (highlighted in red). The new method avoids these fabrications while maintaining descriptive accuracy. Source: https://arxiv.org/pdf/2509.21997

The method begins by having the model describe real images, sometimes including objects or details not actually present. These inaccurate captions generate synthetic images that highlight the errors. By comparing real and synthetic images, the system identifies internal patterns that lead to invented content.

Once these error patterns are recognized, they are stored for future use. When captioning a new image, the system adjusts the model's internal signals, steering it away from known hallucination triggers. This correction happens in a single pass without extra data, retraining, or image generation during testing.

The Challenge of Entanglement

In the paper's example, "entanglement" likely explains why birds were added to an image containing none.

Entanglement occurs when a model strongly links certain concepts because they often appear together in its training data. Here, the model may have frequently encountered planes with birds, creating an association that wrongly influences the caption.

While ending training earlier can reduce entanglement (increasing model flexibility), it also diminishes conceptual detail and resolution. Developers face a persistent trade-off: prioritize a flexible, disentangled model or a more generative one prone to associated hallucinations?

Ideally, source image captions would itemize every object present, allowing the model to store them as separate, disentangled entries. However, SEO-driven captioning practices and large-scale web scraping—common in training powerful generative models—often fall short of this standard.

An illustration of how weak captions limit the usefulness of LAION images for training models like Stable Diffusion. Many of the text labels are shallow, vague, or optimized for SEO rather than accurate description, making it harder for the model to learn fine-grained visual concepts such as facial features (original source was https://rom1504.github.io/, now defunct).

Weak captions reduce the value of LAION images for training models like Stable Diffusion. Labels are often shallow, vague, or SEO-focused rather than descriptive, hindering the model's ability to learn detailed visual concepts like facial features. (Original source was https://rom1504.github.io/, now defunct).

Since a foundational fix is impractical, workarounds to reduce hallucinations in LLMs and VLMs have become a significant research focus. The new Chinese method, tested across various architectures and conditions, shows promise in curbing "hallucination pollution."

The authors state:

‘Extensive experiments across multiple benchmarks show that our method significantly reduces hallucinations at the object, attribute, and relation levels while largely preserving recall and caption [richness].’

The paper, titled Exposing Hallucinations To Suppress Them: VLMs Representation Editing With Generative Anchors, comes from researchers at the University of Science and Technology of China and Nanjing University.

How the Method Works

The researchers developed an end-to-end pipeline to expose and suppress caption hallucinations:

An illustration of the full pipeline. A vision-language model first generates a caption from the input image, which may include hallucinated content. This caption is then used to produce a reconstructed image via a text-to-image model, making any hallucinations easier to spot. Embeddings from both the original and reconstructed images are extracted and used to guide adjustments inside the decoder, helping the model suppress hallucinated details while preserving caption quality.

The full pipeline illustration. A vision-language model generates a caption from an input image, potentially including hallucinations. That caption is used to create a reconstructed image via a text-to-image model, making errors visible. Embeddings from both images guide internal adjustments, helping the model reduce invented details without losing caption quality.

A vision-language model first captions a real image, possibly inventing objects or relationships. That caption then generates a reconstructed image, revealing any fabrications as visual discrepancies. Comparing the two images turns subtle text errors into measurable, correctable signals.

To discourage inventions, the system compares the original image (a reliable reference) with the reconstructed one (highlighting errors). Each is converted into a compact embedding. By adjusting internal representations to align more closely with the original and diverge from the reconstruction, the model self-corrects in a fully self-supervised manner.

The paper explains:

‘Hallucinations in MLLMs are intrinsically difficult to detect because they are linguistically well-formed and often indistinguishable from faithful descriptions at the text level. The discrepancy lies not in language plausibility but in the misalignment with visual evidence, which the model itself is typically insensitive to.

‘To address this, we introduce a hallucination exposure mechanism that leverages generative reconstruction to convert implicit inconsistencies into explicit and observable signals.’

Using the FLUX.1-dev text-to-image model, the system recreates an image from the caption, exaggerating any false details. These amplified errors help the model recognize and correct its mistakes.

To validate the approach, researchers injected hallucinations into captions, generated reconstructed images, and then re-captioned them with LLaVA. They measured semantic similarity between original and hallucinated captions:

An illustration of how the hallucination amplification mechanism makes subtle errors visible. Each point shows the similarity between captions of the original and reconstructed images for one image-caption pair. The orange line represents similarity measured directly between original and hallucinated captions, which stays high and masks small mistakes; the blue line represents similarity after reconstruction, which drops sharply, showing that the process turns hidden hallucinations into clear semantic markers that can be detected and corrected.

The hallucination amplification mechanism visualizes subtle errors. Each point shows caption similarity for an image-caption pair. The orange line (direct comparison) remains high, masking mistakes; the blue line (post-reconstruction) drops sharply, revealing hidden hallucinations as detectable semantic markers.

Similarity declined significantly after reconstruction, demonstrating the method's ability to expose subtle errors.

Data and Testing

Effectiveness was validated using three benchmarks: Caption Hallucination Assessment with Image Relevance (CHAIR); MLLM Evaluation (MME); and Pooling-based Object Probing Evaluation (POPE).

From the CHAIR release paper: examples of hallucinated objects generated by two leading captioning systems, TopDown and NBT, where each model invents visual elements not actually present in the image, such as laptops, sinks, or surfboards. Source: https://arxiv.org/pdf/1809.02156

From the CHAIR release paper: examples of hallucinated objects generated by captioning systems TopDown and NBT, which invent elements like laptops, sinks, or surfboards not present in the image. Source: https://arxiv.org/pdf/1809.02156

Standard metrics like hallucination rate or recall can be misleading—models might avoid errors by producing vague captions. To balance accuracy and completeness, the combined metric Hallucination and Recall (HAR@β) was used, scoring captions on both factors with adjustable weighting.

POPE assessed context-sensitive object hallucinations, and MME evaluated attribute-level hallucinations, both as Yes/No tasks.

Tests used the Flux model and LLaVA-v1.5-7B on datasets including Microsoft COCO, A-OKVQA, and GQA. Latent editing targeted the models' second layer, with consistent hyperparameters and temperature across all tests.

Initial CHAIR results are shown below*:

Performance on CHAIR benchmark for hallucination mitigation, evaluated using multiple metrics.

Performance on the CHAIR benchmark for hallucination mitigation, assessed with multiple metrics.

The authors note:

‘[Our method consistently outperforms other baselines on both CHAIR_S and CHAIR_I[*] , demonstrating its superior effectiveness in suppressing hallucinations. Meanwhile, although nearly all methods inevitably reduce recall while suppressing hallucinations, reflecting a trade-off between faithfulness and informativeness, our approach achieves the smallest drop.

‘This demonstrates that our method captures a broad range of ground-truth objects. With the HAR@β metric, our method achieves the highest score, highlighting its ability to reduce hallucinations while maintaining coverage.’

Strong results are attributed to dual-supervision: reinforcing clean semantics from the original image while suppressing misleading signals from the reconstruction. By targeting only hallucination-related directions, the system corrects errors without losing detail.

Comparison of performance on the POPE benchmark under various configurations and datasets.

Performance comparison on the POPE benchmark across different configurations and datasets.

Regarding POPE results, the paper states:

‘It can be observed that our method consistently achieves the best performance across all settings. Notably, our method can achieve up to +5.95% accuracy and +6.85% F1 score on average, outperforming other training-free approaches by a large margin.

‘Therefore, these results demonstrate that our method provides a reliable and generalizable solution across different levels of difficulty.’

From the third testing round, performance comparisons over MME.

Performance comparisons from the third testing round on MME.

The final main test was on MME, with results shown above. However, the paper mentions the method "OPERA" without definition in the main text or appendix. While the authors claim strong MME performance, the lack of method details suggests caution in interpreting these results.