Kuo Yang

We present a training-free approach to cross-modal alignment that operates on independently pretrained encoders without paired data. By representing samples through anchor profiles—vectors of similarities to shared reference points—we enable direct comparison across heterogeneous representation spaces. We introduce two Mantel-based components: (i) anchor reliability weighting that down-weights anchors with inconsistent cross-modal structure, and (ii) pairwise significance testing via permutation tests that separate true correspondences from chance agreements. Experiments across vision–text and audio–text with eight encoder families show consistent improvements, especially for single-modal encoders lacking multimodal pretraining, revealing that our method extracts latent cross-modal structure from independently trained representations.

Referring Expression Comprehension (REC) links language to region-level visual perception. Standard benchmarks (RefCOCO, RefCOCO+, RefCOCOg) have progressed rapidly with multimodal LLMs but remain weak tests of visual reasoning and grounding: (i) many expressions are very short, leaving little reasoning demand; (ii) images often contain few distractors, making the target easy to find; and (iii) redundant descriptors enable shortcut solutions that bypass genuine text understanding and visual reasoning. We introduce Ref-Adv, a modern REC benchmark that suppresses shortcuts by pairing linguistically nontrivial expressions with only the information necessary to uniquely identify the target. The dataset contains expressions on real images, curated with hard distractors and annotated with reasoning facets including negation. We conduct comprehensive ablations (word order perturbations and descriptor deletion sufficiency) to show that solving Ref-Adv requires reasoning beyond simple cues, and we evaluate a broad suite of contemporary multimodal LLMs on Ref-Adv. Despite strong results on RefCOCO, RefCOCO+, and RefCOCOg, models drop markedly on Ref-Adv, revealing reliance on shortcuts and gaps in visual reasoning and grounding. We provide an in-depth failure analysis and aim for Ref-Adv to guide future work on visual reasoning and grounding in MLLMs.

Multimodal large language models (MLLMs) are making rapid strides in complex visual reasoning. This survey synthesizes the emerging paradigm of Image-Grounded Chain-of-Thought (IG-CoT), where models ground intermediate inferences by interleaving textual rationales with visual state updates. We formalize IG-CoT, present a method-centric taxonomy covering prompting, supervised fine-tuning, and reinforcement learning, and map these techniques to representative benchmarks. Our analysis identifies two domains where IG-CoT offers significant advantages: detail-oriented reasoning requiring meticulous perception, and imagined-world reasoning for simulating unseen states in games, geometry, and planning. We discuss the practical trade-offs of current methods regarding controllability, data, and compute. We conclude by highlighting key challenges—efficiency, data quality, and generative capabilities—and outlining promising future directions, including lightweight architectures, richer intermediate supervision, and method-aware evaluations that better assess faithfulness and long-horizon reasoning.

This work studies how uncertainty in Earth system forecasts can support operational decisions. A decision risk adapter converts forecast samples, context, and utility functions into risks for candidate actions. Utility-aware calibration targets the reliability of decision losses, supporting choices such as warnings, dispatch, inspection, and resource allocation. The evaluation reports lower decision regret and missed-event rates than the strongest baseline. The arXiv record reports acceptance at ADMA 2026.

AMRD transfers knowledge from multiple speech teachers into a lightweight student. It estimates teacher reliability for each batch with a one-class SVM, then combines weighted predictions with a loss that preserves relationships among samples. Evaluations on IEMOCAP and CREMA-D across four student architectures show complementary benefits from adaptive weighting and relational transfer. Both components act during training, leaving inference cost unchanged.

Recent studies have uncovered an interesting phenomenon: unimodal foundation models tend to learn convergent representations, regardless of differences in architecture, training objectives, or data modalities. However, these representations are essentially internal abstractions of samples that characterize samples independently, leading to limited expressiveness. In this paper, we propose The Indra Representation Hypothesis, inspired by the philosophical metaphor of Indra's Net. We argue that representations from unimodal foundation models are converging to implicitly reflect a shared relational structure underlying reality, akin to the relational ontology of Indra's Net. We formalize this hypothesis using the V-enriched Yoneda embedding from category theory, defining the Indra representation as a relational profile of each sample with respect to others. This formulation is shown to be unique, complete, and structure-preserving under a given cost function. We instantiate the Indra representation using angular distance and evaluate it in cross-model and cross-modal scenarios involving vision, language, and audio. Extensive experiments demonstrate that Indra representations consistently enhance robustness and alignment across architectures and modalities, providing a theoretically grounded and practical framework for training-free alignment of unimodal foundation models.

Foundation models learn highly transferable representations through large-scale pretraining on diverse data. An increasing body of research indicates that these representations exhibit a remarkable degree of similarity across architectures and modalities. In this survey, we investigate the representation potentials of foundation models, defined as the latent capacity of their learned representations to capture task-specific information within a single modality while also providing a transferable basis for alignment and unification across modalities. We begin by reviewing representative foundation models and the key metrics that make alignment measurable. We then synthesize empirical evidence of representation potentials from studies in vision, language, speech, multimodality, and neuroscience. The evidence suggests that foundation models often exhibit structural regularities and semantic consistencies in their representation spaces, positioning them as strong candidates for cross-modal transfer and alignment. We further analyze the key factors that foster representation potentials, discuss open questions, and highlight potential challenges.