VIBE: Video Instruction-aligned Background music gEneration.

VIBE model architecture

Comparison with other state-of-the-art models

VIBE Ours Video + Text
Caption
A dark, atmospheric Trap track at 150 BPM in B Minor, featuring a deep, resonant 808 bass, crisp trap drums with tight kicks, snappy snares, and rapid hi-hat rolls in a half-time feel, layered with moody synth pads and a haunting bell-like melody, evoking a brooding, introspective mood that shifts from energetic momentum to subdued melancholy, rooted in a static, hypnotic B Minor tonal center with no chord changes, creating a searching, melancholic atmosphere that aligns with the dark trap sub-genre's cinematic, urban aesthetic.
CMT Video-only
GVMGen Video-only
VidMuse Video-only
Video2Music Video-only
M2UGen Generic text
VIBE Ours Video + Text
Caption
A high-energy Trap-Hip-Hop track at 93.75 BPM in F minor, featuring a deep, resonant 808 sub-bass, crisp trap drums with rapid hi-hat rolls, and a bright, melodic synth lead that carries the main hook, layered with a serene and awe-inspiring atmosphere that contrasts the energetic beat, evoking a sense of wonder and tranquility while serving as an ambient backdrop to a nighttime spectacle, blending modern technological elements with universally accessible instrumentation.
CMT Video-only
GVMGen Video-only
VidMuse Video-only
Video2Music Video-only
M2UGen Generic text
VIBE Ours Video + Text
Caption
A dark, atmospheric electronic track at 76.9 BPM in C Major with a subtle lean toward minor tonalities, featuring evolving synth pads, a deep resonant synth bass, a sparse electronic drum pattern with soft kick, crisp snare, and subtle hi-hats, and a clean, immersive stereo mix that emphasizes gentle compression and dynamic breathing, conveying an introspective and slightly ominous mood with a hypnotic, steady 4/4 groove that remains detached and non-synchronized, evoking late-night cityscapes and contemplative visual media through a minimalist, genre-blurring fusion of ambient-electronic and downtempo trip-hop sensibilities.
CMT Video-only
GVMGen Video-only
VidMuse Video-only
Video2Music Video-only
M2UGen Generic text
VIBE Ours Video + Text
Caption
An ambient electronic track at ≈81 BPM in E Major, featuring layered evolving synth pads, a deep resonant synth bass, and a simple repetitive synth melody, creating a serene, introspective atmosphere with a calm and meditative flow, devoid of emotional depth or dynamic contrast, serving as a universally accessible backdrop that evokes a sense of spacious stillness and quiet reflection.
CMT Video-only
GVMGen Video-only
VidMuse Video-only
Video2Music Video-only
M2UGen Generic text
VIBE Ours Video + Text
Caption
A relaxed, atmospheric Lo-fi Hip-Hop track at ≈83 BPM in F♯ minor, featuring a deep, smooth synth bass, a laid-back drum-machine groove, warm sustained synth pads, and a bright, bell-like synth melody in a subtle minor-key progression—F♯m → C♯m → Amaj7 → Dmaj7 → C♯m7 → F♯m7—that drifts across the stereo field, creating a calm, contemplative mood with a dreamy, introspective vibe, evoking a peaceful late-night atmosphere while subtly layering a sense of melancholy and introspection over its chillwave-inspired, low-fi production.
CMT Video-only
GVMGen Video-only
VidMuse Video-only
Video2Music Video-only
M2UGen Generic text
VIBE Ours Video + Text
Caption
A modern ambient-electronic track at ≈96.8 BPM in C♯ Major, featuring lush, evolving synth pads, a delicate arpeggiated synth line, and a deep, sustained synth bass, blending ethereal textures with a subtle downtempo groove to evoke a serene, introspective, and slightly wistful East Asian Lunar New Year ambiance, underlining a restrained cultural reverence and traditional reverence without mirroring the visual exuberance, with a calm and contemplative atmosphere layered over a gentle harmonic journey through major-sixth chords and a brief C♯ minor shift, creating a meditative, immersive soundscape rooted in the 2010s–2020s ambient-electronic tradition.
CMT Video-only
GVMGen Video-only
VidMuse Video-only
Video2Music Video-only
M2UGen Generic text

Instruction Following

Input video
DIY textured canvas painting
Same video → 2 prompts
Sad

A deeply sad, melancholic piano ballad …, quiet sorrow …

Love

A warm, romantic love theme …, gentle heartfelt melody …

Input video
Cute dog
Same video → 3 prompts
Sad

A deeply sad, melancholic piano ballad …, quiet sorrow …

Love

A warm, romantic love theme …, gentle heartfelt melody …

125 BPM

An uplifting melodic electronic track at 120 BPM

Input video
Dogs at play
Same video → 2 prompts
Happy

A bright, joyful, upbeat acoustic-pop track …, pure happiness and fun …

Motivational

A powerful, motivational cinematic anthem …, inspiring triumphant build …

Input video
Taj Mahal
Same video → 3 prompts
Sad

A deeply sad, melancholic piano ballad …, quiet sorrow …

Happy

A bright, joyful, upbeat acoustic-pop track …, pure happiness and fun …

Motivational

A powerful, motivational cinematic anthem …, inspiring triumphant build …

Mel-spectrogram comparison

Log-power mel-spectrograms (0–8 kHz) of each system's output for the same input video.

VIBE Ours
VIBE mel-spectrogram
CMT
CMT mel-spectrogram
GVMGen
GVMGen mel-spectrogram
VidMuse
VidMuse mel-spectrogram
Video2Music
Video2Music mel-spectrogram
M2UGen
M2UGen mel-spectrogram