Accepted evidence
Video links from Hugging Face Spaces/Datasets (preferred) or unlisted YouTube.
Submission format
Form submission plus JSON metadata with team info, eval settings, and metrics.
Update cadence
Submissions ingest daily, verification weekly, leaderboard publish weekly (UTC).
Leaderboard
Hugging Face Static Space format using results.json.
Loading results.json...
| Rank | Team / Model | Composite | Success | Avg Return | Video | Submitted | Status |
|---|
Submission
Step 1
Prepare videos, metadata JSON, and contact details.
Step 2
Submit through the form and include all required links and metrics.
Step 3
We verify evidence and publish validated results in weekly leaderboard updates.
Submission JSON schema (draft)
{
"team_name": "Aurora Lab",
"contact_email": "team@example.com",
"affiliation": "Example University",
"model_name": "Aurora-WM",
"model_version": "v1.1",
"eval_setting": "Open-vocab tabletop; seed=5; 50 episodes",
"metrics": {
"composite_score": 89.3,
"success_rate": 83.0,
"avg_return": 71.2
},
"videos": [
{ "label": "long-horizon demo", "url": "https://huggingface.co/spaces/your-space/video1" }
],
"inference_hardware": "2xA100; 4-bit quant; 12 tokens/s",
"notes": "Any deviations from reference protocol."
}
Ranking updates and governance
- Ingest daily: new submissions are queued.
- Validate weekly: videos and metadata are checked.
- Publish weekly: rankings refresh after validation.
- Tie-break: composite score, then success rate, then earlier submission.