loading a sample…
RelateAnything reads a picture and says how the things in it relate: person riding horse. The relations are words you choose. Type any list and the model scores every pair of objects against it.
The graph below is what the model produced for this photo. Nothing has been downloaded yet, and nothing is uploaded. When you want it on your own images, the models load into this page and run there.
loading a sample…
How it works
Coloured boxes are the objects a detector found; each arrow is a relation, drawn from subject to object. Amber arrows are spatial relations, about where things are: on, behind, to the left of. Green arrows are semantic ones, about what things do: riding, holding, looking at. Only the strongest few arrows are labelled on the image. The list under it has them all, and hovering a row highlights its arrow.
The number on each row is a calibrated score, σ(a·(pred + w·pair) + b). The demo, the Python
pipeline and the evaluator all use it, so 0.60 means the same thing everywhere.
The model has no fixed list of relations stored in its weights. A small distilled text encoder turns each predicate string into a direction in the relation head's text space, and the head scores every object pair against those directions. Editing the list points the model somewhere else without retraining it. Remove a word and it can never be predicted; add one the model has no direction for and it loses to the rest.
The model is a DINOv3 ViT-S+/16 backbone (53M parameters) with a pair-sampling relation head, trained on RA-4M (474k machine-annotated images), raw Visual Genome and a 5% share of HICO-DET train, with source-aware negative masking. It is never given object class labels, at training or at inference. The graph comes from pixels and box coordinates alone.
The page opens on a stored capture of the relation head's raw logits for the sample photos, so the decode you drive with the words and sliders is the real one, with nothing downloaded. Pressing Run it live fetches the relation model (105 MB), the YOLOE-11s prompt-free detector (24 MB) and a 28 MB WebAssembly runtime; everything after that runs on your machine, webcam included. The browser caches the files, and clear cached models frees the space.
YOLOE gives open-vocabulary boxes and instance masks, prompt-free over 4,585 classes or re-targeted live to any class list you type. FastSAM returns segments with no class names at all, so the page names each one by its own colour instead (brown object 2). The relation head never reads object names, so the graph is the same either way; only the way you refer to the nodes changes. YOLO26, YOLO12 and YOLO-World are in the advanced panel, from a 5 MB COCO detector upwards.
On WebAssembly a 1280-pixel frame costs roughly 300–500 ms in the detector and 580 ms in the relation head. That is fine for a still image and slow for a webcam. Two things move it: the detector you pick and, for the text-prompt model, how many class names you give it. Object count does not; the relation head costs the same for 2 boxes as for 32. WebGPU is used where the browser exposes it, and the pill in the header says which provider actually ran.
No. The models run in this page; images, video and webcam frames never leave your machine. There is no server to send them to.
Yes, once the predicate text encoder (23 MB, in the advanced panel) is loaded; it encodes any string on the spot. Before that you have the words the shipped bank already carries (243 in this build), and in precomputed mode the 263 strings captured with the sample.
That is the detector rather than the relation model. Lower the detector confidence, or switch detectors. Most of what limits a deployed graph is detector recall.