MoSE3 · result tables

Result tables

The full numbers behind the charts on the front page. Lower is better for RRE and ADD, higher for the rest.

SE(3) estimation on HO3D and iTACO

	Per-pixel — HO3D				Per-pixel — iTACO				Object / part — HO3D				Object / part — iTACO
	RRE↓	ADD↓	AUC_r↑	AUC_ADD↑	RRE↓	ADD↓	AUC_r↑	AUC_ADD↑	RRE↓	ADD↓	AUC_ADD↑	IoU↑	RRE↓	ADD↓	AUC_ADD↑	IoU↑
SpatialTrackerV2	36.45	3.73	0.376	0.633	36.71	28.45	0.408	0.309	32.56	3.46	0.670	0.385	27.25	23.01	0.421	0.239
POMATO	60.53	6.59	0.216	0.405	46.42	46.66	0.319	0.121	59.99	6.18	0.446	0.197	41.53	44.48	0.152	0.190
ZeroMSF	61.53	6.70	0.224	0.409	46.52	43.68	0.298	0.128	62.29	6.43	0.442	0.230	43.47	41.83	0.158	0.187
Any4D	57.02	5.24	0.263	0.504	43.98	31.50	0.364	0.292	55.44	5.16	0.523	0.298	37.23	28.56	0.399	0.185
V-DPM	40.37	3.65	0.342	0.641	32.70	22.60	0.460	0.378	38.34	3.56	0.660	0.421	28.25	19.92	0.448	0.258
TraceAnything	61.56	5.69	0.245	0.467	44.73	35.47	0.347	0.249	61.20	5.65	0.488	0.221	38.45	32.47	0.341	0.142
4RC	38.62	3.46	0.367	0.667	28.60	20.91	0.471	0.410	36.08	3.34	0.688	0.409	18.70	17.26	0.526	0.349
Track4World-Pi3x	33.21	3.37	0.371	0.669	28.07	24.82	0.447	0.312	30.46	3.17	0.698	0.414	18.80	20.14	0.397	0.353
Track4World-DA3	34.61	3.34	0.361	0.671	32.09	25.69	0.408	0.321	30.98	3.20	0.694	0.383	23.78	22.07	0.399	0.319
ProxyPose†	--	--	--	--	--	--	--	--	31.22	--	0.490	--	28.51	--	0.300	--
Ours	18.28	2.16	0.569	0.785	12.30	11.37	0.741	0.594	17.96	2.06	0.802	0.709	10.76	10.49	0.625	0.784

Per-pixel SE(3) evaluates a rigid transform at every pixel; object/part-level SE(3) clusters pixels into rigid bodies and recovers one transform per cluster, with IoU against ground-truth rigid masks. ADD in centimeters; AUC_r at 30°, AUC_ADD at 10 cm on HO3D and 20 cm on iTACO. Baselines cluster predicted point tracks with k-NN; ours clusters its learned rigidity embeddings. Bold best, underline second.

†ProxyPose predicts a pose per query rather than a dense per-pixel field; it is evaluated in its single-query variant under its own protocol with ground-truth intrinsics, so it is reported at the object/part level only and has no predicted mask for IoU. Its ADD is omitted: the mean is dominated by a few clips where its tracker diverges.

SE(3) estimation on YCBInEOAT

	Per-pixel				Object / part
	RRE↓	ADD↓	AUC_r↑	AUC_ADD↑	RRE↓	ADD↓	AUC_ADD↑	IoU↑
SpatialTrackerV2	21.93	4.93	0.461	0.576	22.93	6.55	0.508	0.251
POMATO	46.47	10.78	0.223	0.336	50.11	10.89	0.309	0.054
ZeroMSF	44.79	11.57	0.243	0.353	46.71	10.74	0.387	0.073
Any4D	42.30	10.28	0.251	0.432	44.67	10.45	0.455	0.267
V-DPM	30.42	7.58	0.329	0.504	31.60	8.18	0.501	0.234
TraceAnything	44.79	9.61	0.249	0.415	48.47	10.18	0.392	0.066
4RC	21.18	4.85	0.492	0.614	24.23	4.52	0.625	0.322
Track4World-Pi3x	19.92	7.68	0.441	0.562	19.91	7.53	0.549	0.360
Track4World-DA3	22.46	5.59	0.399	0.560	28.89	6.11	0.542	0.278
ProxyPose†	--	--	--	--	17.01	--	0.515	--
Ours	14.89	3.98	0.588	0.628	11.93	4.00	0.638	0.498

Nine clips of robot manipulation with rigid objects, evaluated with the protocol above. ADD in centimeters; AUC_r at 30°, AUC_ADD at 10 cm. Bold best, underline second.

World-coordinate 3D point tracking

	PointOdyssey		ADT		PStudio		Average
	L-16	L-50	L-16	L-50	L-16	L-50	L-16	L-50
SpatialTrackerV2	0.1925	0.1763	0.2456	0.2163	0.3790	0.3689	0.2724	0.2538
POMATO	0.4425	0.3905	0.3611	0.3548	0.5166	0.4713	0.4401	0.4055
ZeroMSF	0.4053	0.3505	0.4530	0.3563	0.4828	0.4386	0.4471	0.3818
Any4D	0.4769	0.4174	0.4460	0.3717	0.5707	0.5066	0.4979	0.4319
V-DPM	0.4848	0.4233	0.4783	0.3759	0.6084	0.5795	0.5238	0.4596
Track4World	0.5345	0.5162	0.6250	0.5622	0.5946	0.5422	0.5847	0.5402
Ours	0.5635	0.5650	0.5789	0.5953	0.6400	0.6235	0.5941	0.5946

World-coordinate 3D point tracking on PointOdyssey, ADT and PStudio, following the Track4World protocol: accuracy at horizons L-16 and L-50; Avg. is the unweighted mean across the three datasets. Bold best, underline second.