Visual Features Across Modalities: SVG and ASCII Art Reveal Cross-Modal Understanding

We found that the same feature that activates over the eyes in an ASCII face also activates for eyes across diverse text-based modalities, including SVG code and prose in various languages. This is not limited to eyes – we found a number of cross-modal features that recognize specific concepts: from small components like mouths and ears within ASCII or SVG faces, to full visual depictions like dogs and cats. […]

These features depend on the surrounding context within the visual depiction. For instance, an SVG circle element activates “eye” features only when positioned within a larger structure that activates “face” features.
— Read on simonwillison.net/2025/Oct/25/visual-features-across-modalities/


Comments

Leave a Reply

Your email address will not be published. Required fields are marked *