Figure has announced Index, an app and data-collection pipeline designed to gather videos of people performing physical tasks for training its Helix robotics system. The company says the service was operated in stealth for four months before its launch on Google Play and the App Store.
Figure reports more than 264,000 downloads across 108 countries, more than 44,000 weekly active users and over 16 million uploaded videos. It also says the system is processing 30 minutes of video uploads every second—equivalent, in Figure’s description, to 4.9 years of human work uploaded each day.
The announcement does not provide model-performance results showing that Index data makes Helix more capable or reliable.
Contents
- What Figure says Index does
- How Index data is prepared for Helix
- What the reported scale means and does not show
- Figure's planned expansion and open questions
What Figure says Index does
Index is intended to collect data from people carrying out tasks in homes and workplaces. Figure lists activities including cooking, cleaning, laundry, logistics, restaurant work, factory tasks and office work.
Users can record tasks themselves or book a Creator to perform and record tasks in a home or workplace. Figure uses “Creators” for the people contributing recordings.
For a robot-learning system, such recordings can provide observations of tasks, objects and environments. Training data that represents the conditions in which a system is expected to operate can help connect what a robot sees with the task context and the actions it may need to perform. Variation matters because the same broad task can involve different objects, layouts, surfaces and human techniques.
Figure says that, for every 1,000 hours collected, Index contains:
- 373 unique tasks;
- 1,146 unique manipulated objects; and
- 116 unique environments.
These are Figure’s reported dataset-diversity figures. The announcement does not define precisely how it counts a “unique” task, manipulated object or environment, so the figures cannot by themselves be used to compare Index with other datasets.
The company also says it has paid Creators $15 million to date. The announcement does not specify the full terms of participation, how data ownership is handled, or what consent and privacy protections apply to people and locations visible in recordings.
How Index data is prepared for Helix
Figure describes a five-stage pipeline for converting uploaded recordings into data it can use for Helix training:
- Filtering: automated technical, visual and semantic filters are used to identify unsuitable material.
- Fraud review: Figure says human audits are performed on samples from users.
- Deduplication: similar examples are removed or managed to reduce repeated material.
- Rebalancing: task quotas and embedding-based clusters are used to adjust the composition of the dataset.
- Annotation: the remaining data receives hierarchical text captions.
Filtering, deduplication, sampling and annotation are common data-engineering steps. They can help reduce unusable or repetitive examples and organise a dataset for model development. In this case, Figure does not provide quantitative information about how much data is removed at each stage or how the processing changes Helix’s results.
The use of task quotas and clusters suggests that Figure is trying to manage the distribution of examples rather than simply collect the largest possible volume. That could be important if some activities, objects or environments are overrepresented among contributors. However, the announcement does not explain the quota values, the clustering method or the criteria used to decide what a balanced dataset should contain.
It is also unclear what information accompanies each video. Figure does not specify the camera configurations, sensors, metadata or action labels used. Video can show what a person appears to do, but it does not necessarily record the forces, tactile information, robot control outcomes or other physical details needed to reproduce a task with a machine.
What the reported scale means and does not show
The reported numbers indicate that Figure is attempting to build a large source of varied physical-world data rather than relying only on a small set of laboratory demonstrations. The combination of household and workplace recordings could expose a training system to more diverse objects and environments than a tightly controlled setting.
That is a potential dataset advantage, not demonstrated evidence of better robot performance.
Figure has not supplied benchmark tasks, training experiments, error rates, success rates or comparisons with alternative robot datasets in the announcement. There is therefore no reported quantitative evidence that Index improves Helix’s ability to generalise to unfamiliar tasks or environments.
The composition of the dataset is another unresolved issue. Figure does not disclose the geographic distribution of contributors or their demographic and socioeconomic composition. Recording behaviour may also influence what enters the dataset: people may choose easier, safer or more common tasks, while the selection and quality-control process may remove other types of examples. The announcement does not describe how these factors affect representativeness or bias.
The scale figures themselves are also company-reported. Figure does not define exactly how it counts a download, weekly active user or video. Its announcement uses both “100+ countries” and the more specific figure of 108 countries; the Index section gives the latter.
Figure's planned expansion and open questions
Figure says it is pursuing a 100-fold scale-up and is committed to spending more than $1 billion on data and compute over the following 12 months.
The company also says Index is intended to lay the groundwork for a future robot-as-a-service model, in which customers would order robotic capability as an ongoing service. For that model to work, the training pipeline would need to support dependable performance across the varied conditions in which customers use the robots.
The current announcement does not establish whether it can do so. Important unanswered questions include how contributors are compensated and governed, how privacy is protected, what labels and physical measurements accompany the videos, and whether training on the resulting data produces measurable gains in Helix.