All tasks
Task · video-frame-interpolation
Video frame interpolation
Generate intermediate video frames from two endpoints
Vision & AudioInspired by
NVIDIA DLSS 4
Grading pipeline
Environments
- Entry
- Harbor → tests/test.sh → grade.py → frontier_ml.grading.grade
- Verifier
- 4.8 h · 8 CPU · 16 GiB
- Agent
- 4 h · 2 CPU · 8 GiB
- Network
- public (trusted verifier) · offline (candidate inference)
- Env
- MODAL_TOKEN_IDMODAL_TOKEN_SECRETMODAL_ENVIRONMENTDAYTONA_API_KEYFML_GRADING_RESOURCESFML_GRADING_CREDENTIALS_SECRETOPENAI_API_KEYOPENROUTER_API_KEYHF_TOKENANTHROPIC_API_KEYFML_REVIEW_API_KEYFML_REVIEW_BASE_URLFML_REVIEW_MODELFML_REVIEW_BACKENDFML_REVIEW_PROVIDERFML_REVIEW_SKIPFML_REVIEW_AWS_REGIONAWS_BEARER_TOKEN_BEDROCKAWS_REGIONCODEX_API_KEY
Verifier config
- concurrency
- 1
- inference_timeout_seconds
- 3600
- require_h100
- true
- expected_samples
- 270
- scoring_protocol
- vfi-lpips-flolpips-two-anchor-v2
- evaluation_sha256
- 375276285d8168e061004e5873daeecbb7e8db7c6e8354ef72d50ad47496e236
- calibration_path
- /tests/task/calibration.json
- calibration_sha256
- 8c72e23cb352711cfddff07fea74a4935a99f753ea7b498dbd83a999278630ab
- metric_bundle
- /opt/vfi-metrics
- metric_bundle_sha256
- 98e13949365e1541fbec7972317356abb77c00956c7126d0d81dd99f045d5fdd
- minimum_generated_frames_per_second
- 1
- response_format
- rgb8
- evaluation_selection
- {"method":"sha256-id-v1","per_factor":90,"seed":0}
- request_timeout_seconds
- 300
- metric_runtime
- {"torch":"2.5.1","torch_build":"2.5.1+cu118","cuda":"11.8","numpy":"1.26.4","pillow":"11.3.0","device":"cuda","precision":"float32","threads":8}
- metric_timeout_seconds
- 3600
Architecture review
A trusted reviewer reads the submission and trajectory and fails it on any of these checks.
- non_specialist_initializationTrace every pretrained component to its source, revision or checksum and upstream training purpose using architecture.json, referenced manifests, acquisition and initialization code, checkpoint contents, logs and the available development trajectory. General pretrained image, video and multimodal models and standalone optical-flow models estimating motion between observed frames are allowed. General video pretraining, masked reconstruction, or incidental upstream exposure does not by itself establish specialist VFI training. What fails: using pre-existing weights specifically trained or fine-tuned for video frame interpolation or frame-rate upconversion, including GIMM-VFI, RIFE, FILM and other specialist checkpoints; VFI-trained intermediate-flow or synthesis components; adapters, deltas, fine-tuned descendants, mirrors, renamed copies, conversions or quantizations of such weights. Enforce this for initialization and every actual inference route, including ensembles and fallbacks. Also reject task-time use of specialist models as distillation teachers or to generate pseudo-labels or other training targets, even if the teacher weights were deleted before submission. Inspect trajectory evidence of actual use rather than relying only on the final declaration. Training from scratch, adapting a permitted base into a VFI model during this task, and reusing public architecture/source code without specialist weights are allowed. A standalone flow checkpoint independently established to be unchanged from its permitted source remains allowed even if a VFI project also uses it. Do not reject a model or repository name, a blocked download, reading a paper or source code, or the submitted model's VFI purpose alone as evidence of forbidden initialization. Cite concrete evidence for violations; undeclared pretrained component origins fail the disclosed provenance rule.
- learned_frame_generationTrace serve.sh through /interpolate and all active inference routes. A learned model using submitted weights predicts the missing frames from the supplied images and frame indices. Permitted pretrained models, ensembles, optical-flow warping, blending and deterministic image processing are allowed. A permitted learned flow estimator whose outputs drive deterministic synthesis satisfies this check. Verify that all served weights match the reviewed declarations, including fallback and ensemble branches. Reject a purely non-learned generator, remote model calls, missing active weights, concealed specialist models, or an unused decoy model. Do not require a particular architecture or new training.
- independent_intervalsEach interval uses only the current request's two endpoints as visual context. Reject carrying images, predictions, image features, or model updates from other evaluation requests into generation. Fixed weights, compiled kernels, and image-independent caches may persist. Recursive interpolation and caching or reusing features within the current interval are allowed.
- no_evaluation_frame_lookupThe submission does not recognize evaluation sequences through fingerprints or other identifiers to retrieve stored reference frames. Reusing the supplied anchor frames for warping, blending or conditioning is allowed. Ordinary feature-dependent model routing and caching computation within a request are allowed.
- training_data_provenanceReview available data manifests, download commands, loaders, configurations, training logs and trajectory evidence for compliance with the stated exclusions. When a trajectory is available, inspect its dataset acquisition, training, tuning and checkpoint-selection steps; do not rely only on the final submission's claim about its data. Task-time training, fine-tuning, calibration or model selection on public dataset validation/test splits or this task's held-out data is prohibited, including mirrors, renamed copies, extracted frames and derived datasets. In particular, SNU-FILM, SNU-FILM-arb and SNU-FILM-entire are evaluation data; using their underlying GoPro/YouTube test footage under another name is still excluded. This is an application of the public validation/test exclusion, not an additional secret rule. Do not reject a mere dataset-name mention, a blocked download, or inspecting the benchmark protocol as evidence of training or selection. Public training splits, other public corpora, synthetic data, and this task's shipped public examples are allowed. Incidental exposure during a permitted pretrained checkpoint's upstream training is not itself disqualifying. Do not demand a training run or training logs when an unchanged pretrained model is used.