Physical AI data collection
Real-World Human Demonstrations for Physical AI
DataNexx sources, records, structures and annotates real-world human activities to support the development of robotics and physical AI systems. Our collection network spans household activities, agriculture, skilled trades, manufacturing and other physical workflows.
- Real-world task
- Egocentric recording
- Consent & authorization check
- Human review
- Structured, annotated dataset
The short answer
Physical AI is AI that senses and acts in the physical world, such as robots that pick, sort, fold or assemble. To learn, it needs physical AI data: real examples of how tasks are done, step by step, with real objects.
DataNexx collects one kind of that data: first-person video of people doing everyday physical work, filmed on a chest- or head-mounted phone with written consent. It is human demonstration data, not robot data, and a person reviews every recording before release.
What is physical AI?
Physical AI is the part of AI that works in the real world instead of on a screen. It covers robots, robotic arms and other machines that have to see a space, plan a movement and handle objects.
Language models learn from text. Physical AI has to learn things text rarely describes: how hard to grip, where to put a hand, and what to do when something slips. That knowledge comes from watching and doing real tasks.
What data does physical AI need?
Teams building physical AI usually combine several kinds of data:
- Human demonstrations: video of people doing a task, so a model can learn the steps and the hand movements. This is what DataNexx collects here.
- Robot data: logs from the machine itself, such as joint positions, sensor readings and remote-control sessions. We do not record robot data in this program. Our robotics datasets page covers the industrial machine and sensor records we source from businesses.
- Simulation: scenes generated in software. Useful for scale, but only as faithful as the simulator.
- Labels: descriptions, timestamps and object names that say what happens in each clip.
Why real-world recordings matter
Real homes, fields and workshops are messy. Light changes, tools are worn, and objects sit in odd places. Recordings of real work show that variety, and the small corrections people make without thinking.
How is human video different from robot data?
A human demonstration shows a person doing the task with their own hands. It tells a model what the goal is, which steps lead there and how objects behave. It does not record a robot's joints, motors or grippers.
Teams can use human video to teach the task and robot data to teach the machine its own body. We are clear about which one you get: everything in this program is first-person human video. Motion-sensor capture and AI-assisted annotation are still planned.
What we do here
From a task someone does every day to a dataset a robot can learn from.
Physical AI has to see how people actually do things. We organize the people, places and consent that make that footage usable.
- Source
- We recruit individual contributors and seek partnerships with farms, workshops, plants and warehouses where the work already happens.
- Record
- Contributors record real tasks from a first-person view, following a written task specification, on equipment they already own.
- Structure
- Every recording is checked, linked to its task specification and consent record, and kept with an unchanged original.
- Annotate
- Recordings are described and labelled so a model can learn what happened, when, and with which objects. AI-assisted labelling is planned; people review every result.
Egocentric video
First-person video of hands, tools and the workspace.
Egocentric video is recorded from the point of view of the person doing the task. The camera sees what they see: their hands, the tools and objects they handle, and the workspace around them, in the order the work actually unfolds.
For robotics and physical AI that viewpoint matters. It captures how a task is approached, how objects are grasped and moved, how mistakes are corrected and how the environment changes along the way — detail that third-person or staged footage leaves out.
We collect it where the work is real: kitchens, fields, workshops, production floors and warehouses, performed by people who do these tasks as part of their life or job.
What a recording contains
- Viewpoint
- First person, chest- or head-mounted camera
- In frame
- Hands, tools, objects, workspace
- Task
- One specified task, start to finish
- Attached
- Task specification version, device, consent record
- Not in frame
- Bystanders without a release, screens, documents
Industries and task categories
Everyday physical work, recorded where it happens.
Four groups to start with. Each task is defined by a written specification, so recordings from different contributors are comparable.
- 01
Household Task Data
Domestic routines performed in real homes.
Example tasks
- Washing dishes
- Folding laundry
- Chopping vegetables
- Cooking a meal
- Cleaning surfaces
- Loading and unloading appliances
- 02
Agricultural Data
Field, orchard and greenhouse work through the season.
Example tasks
- Picking fruit
- Sorting produce
- Pruning and tying plants
- Planting seedlings
- Packing harvest
- Weeding and irrigation checks
- 03
Skilled Work and Manufacturing
Trades and production tasks that depend on hand skill.
Example tasks
- Manual assembly
- Packaging
- Quality inspection
- Carpentry
- Plumbing
- Tiling
- 04
Logistics
Handling goods between arrival and dispatch.
Example tasks
- Warehouse picking
- Packing orders
- Palletizing and wrapping
- Labelling and scanning
- Loading and unloading
- Inventory counts
Collection methods
Video first. Sensors and annotation as they mature.
We are open about what is in use today and what is still being built.
- 01Core method
Egocentric Video Collection
The foundation of every dataset: first-person video recorded on a chest- or head-mounted phone camera by a contributor following a written task specification.
Specification
- 01RGB video, 1080p at 30 fps or higher
- 02Chest- or head-mounted phone camera, the contributor's own device
- 03Task specifications written, versioned and attached to every recording
- 04Original files kept unchanged, with checksums
- 02Planned / in development
Optional Motion-Sensor Capture
For tasks where hand and body motion carry information the camera cannot see, we are developing synchronized sensor capture alongside the video.
Specification
- 01Phone inertial sensors (IMU) recorded with the video
- 02Smartwatch wrist sensors for hand motion
- 03Time alignment between sensor streams and video
- 03Planned / in development
AI-Assisted Annotation
Models draft the labels; people check them. Nothing is released on the strength of an automated judgement alone.
Specification
- 01Activity descriptions for each recording
- 02Action timestamps marking the steps of the task
- 03Object labels for tools and items handled
- 04Hand-pose estimates
- 05Every annotation reviewed by a person before release
Custom dataset production
We operate as a data producer and as a collection partner.
Whichever fits your program: datasets we produce to our own specifications, or collection we run against yours.
Scope, volumes and timelines are agreed per project. We do not quote figures before we understand the task.
- 01
Datasets we produce
Task groups we select, record and annotate ourselves, documented with a dataset card, provenance record and license scope.
- 02
Collection to your specification
You define the tasks, environments, viewpoints, annotation schema and acceptance criteria. We recruit, record, review and deliver against them.
Provenance, privacy and quality
Every recording can answer who, where, under what consent — and whether a person checked it.
The rules below apply to all collection, whether we run it for ourselves or for a client.
- 01
Informed consent
Every contributor agrees to a written consent before recording. Consent is versioned, and the version in force is stored with each recording.
- 02
Worksite authorization
Recording on business premises happens only with written authorization from someone entitled to give it.
- 03
Adults only
We do not enroll minors, and recordings that show minors are not accepted.
- 04
No bystanders without a release
People other than the contributor appear only with a signed release; otherwise the recording is rejected.
- 05
No screens or documents
Recordings that capture screens, paperwork or other readable information are excluded.
- 06
Immutable originals
Original files are stored unchanged, with checksums, so every derived file can be traced back to its source.
- 07
Human review before release
Validation checks run automatically, but a person reviews each recording before it is approved for a dataset.
- 08
Anonymous contributor identifiers
Datasets refer to contributors by an anonymous identifier. Names and contact details are kept apart from the data and are never exported.
- 09
Versioned task specifications
Each task specification is frozen once published, so the instructions a contributor followed are always known.
Dataset catalog
The catalog is being built.
Catalog — in preparation
We publish a dataset only after it has passed review, with its dataset card and provenance record. Until then we list no inventory, counts or sample sizes. If you need a particular task family now, tell us and we will say plainly whether and when we can cover it.
Dataset licensing
Four ways to license.
Terms depend on the dataset, the intended use and the scope of the contributors' consent. All are set out in writing before delivery.
- 01
Evaluation license
Access to a documented subset for a defined period, to evaluate fit before a wider license.
Terms on request
- 02
Non-exclusive license
Defined uses for a defined period, alongside other licensees.
Terms on request
- 03
Exclusive license
Exclusive or field-of-use exclusive rights where the collection and its consent scope allow.
Terms on request
- 04
Custom collection
Data collected to your specification, with rights set at the outset.
Terms on request
FAQ
Questions about physical AI data.
What does a recording look like?
A single task, filmed from the contributor's point of view on a chest- or head-mounted phone camera, from start to finish, together with the task specification and consent version it was recorded under. Hands, tools and the workspace are in frame; bystanders, screens and documents are not.
Can we specify our own tasks and environments?
Yes. Custom collection runs against your task list, environments, viewpoints and annotation schema. Before starting we tell you which parts we can cover and which we cannot.
Where do contributors come from?
Individuals who apply through this site, and workers at organizations — farms, workshops, plants and warehouses — that partner with us and authorize recording on their premises. Every contributor must be an adult who has agreed to a written consent.
How are contributors compensated?
Terms for approved recordings are agreed with each contributor individually and set out in writing. We do not publish rates, and nothing on this site is an offer of work.
Is any of this data available today?
The catalog is being built, and we list no inventory until datasets have passed review. If you have a requirement now, send it; we will say whether and when we can meet it.
What is DataNexx Studio?
The platform we are building for contributors and partners: task assignments, uploads, consent records, review and release. It is in development; until it opens, applications are handled by email.
How is personal data handled?
Contributor identities are kept separate from the recordings. Datasets use anonymous identifiers, and names and contact details are never exported. Recordings that show people without a release, or screens and documents, are rejected.
Contributors
Record the work you already do.
Apply to join the collection network. Applications are reviewed, and we contact you when tasks open in your area.
Robotics and physical AI teams
Tell us the tasks your model needs to see.
Request a dataset or a custom collection, or talk to us about what is coming.