Hallo4D: Multi-Modal Hallucination Mitigation
for Consistent Spatio-Temporal Generation

Abstract

While recent progress in 3D generation has enabled impressive visual synthesis, most existing methods still primarily rely on 2D diffusion-based supervision without mechanisms for enforcing geometric consistency, often resulting in spatial hallucinations such as duplicated structures or misaligned geometry. These challenges intensify in 4D generation, where maintaining consistency across viewpoints and temporal progression is substantially more difficult and often leads to temporal artifacts such as jitter, identity flicker, and structural drift. To address these limitations, we present Hallo4D, a unified and model-agnostic framework for mitigating spatiotemporal hallucinations in both 3D and 4D content generation. Hallo4D introduces a generation-detection-correction paradigm that leverages the reasoning capabilities of large multimodal language models (LMMs) to locate and summarize spatial and temporal inconsistencies from multi-view, multi-frame renderings. To prevent compounding errors from single-pass edits and ensure robust geometric fidelity, these insights guide a consensus-driven image-space consistency optimization, where an LMM selector evaluates multiple candidate corrections via multi-model voting. This process is achieved without requiring retraining or architectural modifications. To further enhance temporal consistency, Hallo4D incorporates a motion-saliency-driven keyframe sampling strategy based on optical flow, enabling more targeted and efficient refinement. The framework also includes an LMM-guided initialization scheme and an attention-based appearance alignment module to improve early optimization and cross-view fidelity. Additionally, we address exposure instability with two losses: Contrastive Semantic Exposure Alignment (CSEA), a foreground-masked contrastive objective that favors well-exposed semantics while penalizing over- and under-exposure, and a log-dynamic-range (LDR) loss that regularizes luminance contrast. Together with union-of-frusta visibility pruning to remove out-of-view clutter and reduce pseudo under-exposure, these additions mitigate exposure-driven collapse under non-frontal views. Extensive experiments demonstrate that Hallo4D consistently outperforms strong baselines across diverse generation settings, offering a scalable and generalizable solution for consistency-aware 3D and 4D content generation.

Across multiple representative text-to-3D baselines, Hallo4D consistently improves structural integrity, reduces hallucinated artifacts, and delivers more faithful prompt alignment.

Under diverse image-conditioned generation settings, Hallo4D yields cleaner geometry and more stable cross-view appearance than strong baseline methods.

Reference

BaselineOurs

Reference

BaselineOurs

Reference

BaselineOurs

Reference

BaselineOurs

Reference

BaselineOurs

Reference

BaselineOurs

Reference

BaselineOurs

Reference

BaselineOurs

We substantially enlarge the 4D evaluation with denser and more diverse cases to comprehensively assess temporal coherence, structural stability, and cross-view appearance consistency.

Reference

BaselineOurs

Boy

Reference

BaselineOurs

Bear

Reference

BaselineOurs

Reference

BaselineOurs

Reference

BaselineOurs

Reference

BaselineOurs

To further validate generalizability, we evaluate Hallo4D on 44 diverse scenes from the SV4D. Each video shows the baseline (left) vs. Hallo4D (right).