A model sees more in an hour of ours than in an hour of EgoDex

Fig. 1. Synjuku sits above EgoDex in every cell, 1.32× to 1.40×. Change how you sample and the number barely moves. That is what a denser hour looks like.

Fig. 2. Under the rule we quote (1,200 units, 30 s, one clip per episode) effective rank is 133.0 against EgoDex 97.0. Action density does not need that caveat. Diversity does, so we always state the rule.

An hour of Synjuku data carries 1.32× to 1.40× more action than EgoDex in every cell. Hand keypoints say the same: the lead is hands, not head motion. Figure 1 · eight cells · 179,579 frames matched per cell · hand-keypoint check 1.66×

An hour of Synjuku data gives a model more to learn from than an hour of EgoDex, a public first-person set. We ran both through the same frozen vision model and measured it.

Labels tell you what an annotator wrote. File checks tell you the delivery showed up intact. Neither is what a model sees. The charts above are.

The resultMore action in the hour

How much action does an hour carry?

1.38×

Representation motion per 0.2 s reads 0.0842 against EgoDex 0.0610. That is 1.32× to 1.40× more action in every cell of the grid, and the number barely moves when we change the sample. Idle share is 20% on our side and 33.5% on EgoDex, same threshold.

Is the lead just camera motion?

No

Head-mounted capture moves the camera, which would inflate any whole-frame measure. Measured on hand keypoints alone, which cannot see camera motion, the same corpora give 1.66× against the embedding’s 1.73× on the same 5 h. The lead is hands, not head.

How varied is the corpus?

Above, with a rule

Under a fixed rule (30 s units, one clip per episode, 1,200 units) effective rank is 133.0 against EgoDex 97.0. Anyone can move that number by changing the draw, so we always quote the rule. Action density does not need one. It stays above the EgoDex reference no matter how you cut the same corpus.

1.38×

more action per hour than EgoDex (0.0842 against 0.0610). Above the reference in every cell.

20%

of our steps are idle. EgoDex idle share is 33.5%. Same threshold on both sides.

133 vs 97

effective rank under the stated rule. 1,200 units, 30 s, one clip per episode.

Figure 1Why the hour is denser

The quantity: Every 0.2 s of video is passed through a frozen vision model, which turns the frame into 384 numbers describing what is in it. Consecutive descriptions are compared by cosine distance. Near zero means the scene did not change. A larger value means hands moved, objects moved, or the scene reconfigured. Averaged over a corpus, it is one number for how much visible change an hour contains.
Why a model: A raw pixel difference gets eaten by lighting flicker, sensor noise, and camera shake. Model features respond to content. A still hand under a flickering lamp reads as idle, which is what it is.
Why not labels: Counting annotated segments measures how much an annotator wrote down. That is a property of the annotation contract, not of the video.
The unit: Cosine distance is unitless, so 0.0842 means nothing until you put it next to a reference. Against EgoDex it is a 1.38× lead. The readable form is idle share, same threshold on both sides: 20.1% of our steps sit still. 33.5% of the EgoDex steps do.
The x-axis: Every point holds the same 179,579 frames. What changes is how those frames were assembled: 60 long clips at the left edge, 3,033 short ones at the right. Nothing about the corpus differs between points.
Why flat is the result: The corpus does not change along that axis, so a real corpus property has to stay flat. Ours does. Action density stays above the EgoDex reference within 1.07×. That is the measure of the hour. Diversity moves with the draw, so we only quote it with a rule, and under that rule the ranks are 133 and 97.

Table 1The data behind both figures

Eight reachable cells. Every one of them has Synjuku above EgoDex on action. The 10 h and 100 h rows land within 2.6 diversity points of each other at comparable breadth. Breadth moves the diversity number 59 points. Volume does not. The hour itself is the denser one.

**Table 1.** Eight reachable cells. Ten of the 24 grid cells cannot be filled from the encode and are reported as unreachable rather than filled from a smaller sample.
Volume	Unit	Episodes	Clips / ep	Effective rank	Step distance
10 h	600 s	60	1.0	74.33	0.0807
10 h	120 s	200	1.5	103.98	0.0834
10 h	30 s	200	6.0	106.80	0.0828
10 h	120 s	300	1.0	115.16	0.0817
10 h	30 s	600	2.0	132.28	0.0852
10 h	30 s	1200	1.0	133.04	0.0838
100 h	30 s	3033	3.96	135.34	0.0842
100 h	120 s	3000	1.0	135.64	0.0841

MethodHow we measured the lead

Model-based measuresFrozen encoder · no training

EncoderOpen-source frozen vision encoder, CLS token, sampled at 5 fps384 dim · frozen

What ran3,033 episodes in contiguous 30 s blocks178.5 video-hours

Effective rankLinear-kernel Vendi score, out of 384frame-matched per cell

Step distanceCosine distance, L2-normalized, 0.2 s apartper-step · no matching

Idle shareThreshold set once at the 25th percentile of both corpora pooledsame word on both sides

ReferenceEgoDex, public first-person hand demonstration829 h

Nothing is trained, so nothing here depends on a recipe, a seed, or a learning rate. Coarser units are built by grouping consecutive 30 s blocks of one episode. One encode fills the grid. EgoDex is the public first-person reference, 829 hours, not a soft average.

Ten of the 24 grid cells cannot be filled. Hours equal episodes × clips × unit length, and the encode ended uneven at 178.5 h, with 579 episodes carrying 20 blocks and 2,437 carrying 4. Those cells are reported as unreachable rather than filled from a smaller sample.

ReproduceEvery number in the figures

Expected values

python embed_metrics.py   synjuku=<feat_dir> egodex=<feat_dir>   out.json
python repspace_grid.py   <feat_root>                            grid.json
python hand_motion.py     synjuku=<kp_dir>  egodex=<kp_dir>      hand.json

effective_rank (1200 eps, 30 s)   133.04     EgoDex  97.05
mean_step_distance                 0.0842    EgoDex   0.0610
idle_fraction                      0.2009    EgoDex   0.3345
hand-keypoint ratio                1.66x     embedding 1.73x

The scripts are plain NumPy over cached features, with no model download and no Synjuku dependency, so the reference side of every figure reproduces on a laptop.

frozen vision encoder Vendi / effective rank step distance EgoDex 829 h 3,033 episodes

Numbers from a frozen-encoder comparison of 3,033 delivered Synjuku episodes against EgoDex, same code on both sides. Not a contractual commitment. This is what a model sees in the hour.