Robotics

In my last post evaluating SigLIP 2, the takeaway was pretty clear: pure 2D appearance models fail hard when you start rearranging a room. Global VLMs over-index on object semantics (like matching a red scoop) instead of understanding the actual 3D geometry of the place itself. If we want an agent to actually understand spatial relationships—or if we want to synthetically edit a scene—we need to lift our data out of the 2.5D image plane and into a true 3D coordinate space. We need continuous geometry. ...

To perform long-horizon tasks, agents need memory that supports reasoning over time and reduces the search space. In robotics there is substantial work on spatial and episodic memory, most of which combines geometry, pose, and tracking. This post asks a simpler question: How far can a pure appearance-based approach using a modern vision-language embedding (SigLIP2) go for place recognition and basic memory under real object and viewpoint change? Overview I test a lightweight online appearance-based visual memory. Each image is embedded with SigLIP2 and either assigned to an existing place cluster or used to start a new one, based on cosine similarity. For a known place, previous observations can be retrieved. High-level change descriptions are generated with a vision-language model only as a secondary step. ...