Research · Annotation

We name 98% of the hour. DROID names 46%.

A model only learns from the part of the hour that has a name. We put Synjuku kitchen, household, and office hours next to BridgeData V2, DROID-100, and HoloAssist. The public tape leaves more of the hour unnamed, and a share of the names it does write are blank.

Synjuku Research 6 min read
Words of language per hour of tape Named-span words, matched by environment. Same scale on all three rows. Synjuku Public set 0 2k 4k 6k 7.3k 7,293 3,492 Kitchen vs BridgeData V2 6,888 815 Household vs DROID-100 7,170 2,719 Office vs HoloAssist
Fig. 1. Words in named spans, per hour. +109% against Bridge, 8× against DROID, +164% against HoloAssist. One scale. Three public sets. Ours is the rust bar in every pair.
More of the Synjuku runtime carries a name. Fewer labels are empty. The names themselves are denser: 7,293, 6,888, and 7,170 words per hour against 3,492, 815, and 2,719. Figure 1 · words / h · three 12-episode samples

The labels on a Synjuku hour cover almost all of the runtime. On three public sets, more of the tape is unnamed, more of the captions are empty, and the language next to the video is thinner.

Named coverage is how much of the tape has a name. Empty-label rate is how often that name is blank. Words per hour is how much language sits on the named part. Those three numbers are the annotation product. Session length and cameras are in the table. They are not the story.

The resultDenser labels, on every row

How much of the tape is named?

98% · 98% · 99.9%

Kitchen 98% against Bridge 87%. Household 98% against DROID 46%. Office 99.9% against HoloAssist 77%. On the household row, more than half of the public tape has no name.

How often is the label empty?

2.3% and 2.9%

Kitchen empty-label rate is 2.3% against Bridge 13.1%. Household is 2.9% against DROID 54%. Office is 0.15%. HoloAssist fine spans in the dump are never empty. That row is a wash, not a selling point.

How much language sits on the hour?

+109% · 8× · +164%

Words in named spans, per hour: 7,293 against 3,492, 6,888 against 815, 7,170 against 2,719. Mean words per Synjuku span are 11.7, 11.0, and 10.4. HoloAssist spans are 2.4 words.

Are the labels time-aligned?

Yes

Synjuku spans sit on the tape. Bridge and DROID attach one caption to a short clip, so time-aligned named/h on those sets is 0. Counting that caption as one keeps silence from inflating the rate. HoloAssist is time-aligned, and those spans are short.

Does the frame support the label?

81% vs 40% and 38%

Twenty-nine midpoint stills. Match plus partial. Synjuku 7 / 6 / 3 (n=16). Bridge 0 / 2 / 3 (n=5). DROID 2 / 1 / 5 (n=8). Small n. A probe, not a delivery guarantee. Office stills were not graded.

98%
named coverage on the robot rows. Office is 99.9%. DROID covers 46% of its runtime. HoloAssist covers 77%.
2.9%
empty labels on the household sample. DROID is 54%. Kitchen is 2.3% against Bridge 13.1%.
8×
more words per hour than DROID (6,888 against 815). Kitchen +109%. Office +164%.

Figures 2 and 3Coverage and empty labels

The words-per-hour bar is the density claim. Coverage and empty-label rate say why that bar is tall: more of our tape is named, and fewer of those names are blank.

Named coverage stays high on every row
Share of runtime that carries a name
Synjuku Public set
0 25 50 75 100 98% 98% 99.9% 87% 46% 77% Kitchen Household Office vs Bridge · DROID · HoloAssist
Fig. 2. Named coverage of runtime. 98% and 98% on the robot rows against 87% and 46%. Office is 99.9% against 77%. Rust is ours.
Empty labels stay low on the robot rows
Share of labels that are blank. Lower is better.
Synjuku Public set
0 15 30 45 60 2.3% 2.9% 0.15% 13.1% 54% 0% Kitchen Household Office vs Bridge · DROID · HoloAssist
Fig. 3. Empty-label rate. 2.3% and 2.9% on the robot rows against 13.1% and 54%. Office is 0.15%. HoloAssist fine spans in the dump are never empty, so that cell is a wash.
Named coverage
Share of runtime that carries a name. It is the first annotation number. A dense hour with half the tape unnamed is not a dense hour for training.
Empty-label rate
Share of labels that are blank. 2.3% and 2.9% on our robot rows against 13.1% and 54%. Office is 0.15%. HoloAssist fine spans in the dump are never empty.
Words / h
Words inside named spans, divided by hours of tape. The unit that stays comparable when one set uses a clip-level caption and another uses short verb-noun spans.
Named / h
Time-aligned spans on Synjuku (625, 624, 687). One caption per clip on Bridge (438) and DROID (77). HoloAssist fine-grained is 1,134 because those spans are short (2.4 words). Words per hour is the comparable density number, and that lead is ours on every row.
The three rows
Kitchen against BridgeData V2. Household against DROID-100. Office against HoloAssist. Each number is one environment, one named set, one check. Nothing is blended.

Table 1The annotation numbers

Coverage, empty-label rate, and words per hour are the three annotation checks. Named/h on HoloAssist uses short verb-noun spans, so it is not the same unit as a Synjuku sentence. Session length and occupancy sit below. They are not the claim.

Table 1. Annotation first. Twelve-episode Synjuku samples against three public sets. Rust columns are Synjuku. Named/h on Bridge and DROID counts one caption per clip. HoloAssist named/h is fine-grained verb-noun.
Synjuku kitchen Bridge V2 Synjuku household DROID-100 Synjuku office HoloAssist
Named coverage98%87%98%46%99.9%77%
Empty-label rate2.3%13.1%2.9%54%0.15%0%
Words / h7,2933,4926,8888157,1702,719
Mean words / span11.7n/a11.0n/a10.42.4
Named / h625438624776871,134
Time-alignedyesnoyesnoyesyes
Frame supports label81%40%81%38%not scorednot scored
Sample12 eps · 8.99 h50,415 · 100 h12 eps · 8.78 h100 · 0.60 h12 eps · 8.54 h1,758 · 131.6 h
Mean session2,698 s7.15 s2,633 s21.5 s2,562 s270 s
Motion occupancy80%39%82%12%88%not scored

MethodHow we scored the labels

Annotation comparisonOne row · one set · one check
Synjuku sampleTwelve episodes per environment26.3 h across three rows
Public setsBridgeData V2, DROID-100, HoloAssist labelskitchen · household · office
Words / hWords inside named spans, divided by hours of tapecomparable across label styles
Named / hTime-aligned spans on Synjuku and HoloAssist. One caption per clip on Bridge and DROIDnot mixed
CoverageShare of runtime that carries a name98% · 98% · 99.9%
OccupancyOne-second motion check on the scored videoHoloAssist footage not scored
Frame supportMidpoint stills, match plus partial. 29 graded. Not a census81% vs 40% and 38%

Each number is one environment row, one named set, one check. Named actions per hour is an annotation convention, not a motion score. HoloAssist footage is not in this comparison, so occupancy on that row is blank, not zero. Frame support is 29 stills. That is a probe, not a delivery guarantee.

Bridge and DROID attach one caption to a short clip. Counting those captions as one keeps silence from inflating the rate. Time-aligned named/h on those two sets is 0. HoloAssist fine-grained is the named-action unit we quote (1,134 / h). Coarse sentences on that set are a second track (95 / h) and are not mixed in.

BridgeData V2 DROID-100 HoloAssist named coverage empty-label rate words / h time-aligned spans

Numbers from a comparison of Synjuku kitchen, household, and office samples against three public sets. Not a delivery guarantee. HoloAssist named/h uses short verb-noun spans, so it is not the same unit as ours.

Train on labels that cover the tape

Request a sample. Same cameras, same time-aligned labels, and the annotation numbers behind this post.