Merge audio and video files with FFmpeg

Merging audio and video files is a common task in video production and content creation. Whether you're adding a soundtrack to a movie, replacing dialogue in a foreign language, or simply combining separately recorded audio and video tracks, FFmpeg provides powerful tools to merge these files efficiently. This process, also known as multiplexing or muxing, is essential for various creative and technical purposes.

Why merge audio and video files?

There are several reasons why you might need to merge audio and video files:

1. Post-production work: Adding separately recorded audio to video footage.
2. Language dubbing: Replacing original audio with translated dialogue.
3. Music videos: Combining high-quality audio tracks with video footage.
4. Podcast video creation: Adding video elements to audio podcasts.
5. Fixing audio issues: Replacing poor-quality audio with a better recording.
6. Educational content: Adding narration to instructional videos or presentations.
7. Creative projects: Experimenting with different audio-visual combinations.

Let's explore how to use FFmpeg to merge audio and video files in various scenarios.

Basic audio and video merging

The simplest way to merge an audio file with a video file is to use FFmpeg's audio and video mapping features:

ffmpeg -i input_video.mp4 \
  -i input_audio.mp3 \
  -c:v copy -c:a aac -map 0:v:0 -map 1:a:0 \
  output.mp4

Breakdown of the command:

• -i input_video.mp4: Specifies the input video file
• -i input_audio.mp3: Specifies the input audio file
• -c:v copy: Copies the video codec without re-encoding
• -c:a aac: Encodes the audio to AAC format
• -map 0:v:0: Maps the first video stream from the first input
• -map 1:a:0: Maps the first audio stream from the second input
• output.mp4: Name of the output file

This command will replace the original audio in the video with the new audio file.

Merging audio while preserving original video audio

In some cases, you might want to add a new audio track while keeping the original audio:

ffmpeg -i input_video.mp4 \
  -i input_audio.mp3 \
  -filter_complex "[0:a][1:a]amerge=inputs=2[a]" \
  -map 0:v -map "[a]" -c:v copy -c:a aac -ac 2 \
  output.mp4

This command uses the amerge filter to combine both audio tracks.

Advanced techniques

Adjusting audio sync

If the audio is out of sync with the video, you can adjust it:

ffmpeg -i input_video.mp4 \
  -itsoffset 1.5 -i input_audio.mp3 \
  -map 0:v -map 1:a -c:v copy -c:a aac \
  output.mp4

The -itsoffset 1.5 option delays the audio by 1.5 seconds. Use negative values to move audio earlier.

Trimming audio to match video length

If your audio is longer than the video:

ffmpeg -i input_video.mp4 \
  -i input_audio.mp3 \
  -shortest -c:v copy -c:a aac \
  -map 0:v:0 -map 1:a:0 \
  output.mp4

The -shortest option makes the output as long as the shortest input stream. Be sure to include the stream index here (0 for video, 1 for audio).

Looping short audio to match video length

For a short audio file that needs to loop:

ffmpeg -i input_video.mp4 \
  -stream_loop -1 -i input_audio.mp3 \
  -shortest -c:v copy -c:a aac \
  -map 0:v:0 -map 1:a:0 \
  output.mp4

The -stream_loop -1 option loops the audio indefinitely until the video ends. The -shortest option here is necessary as the second stream (audio) is looped forever technically. So the shortest stream would be the video.

Choosing the right approach

Different merging techniques have various benefits and drawbacks:

Basic merging:

• Benefits:
  • Simple and fast
  • Maintains original video quality
• Drawbacks:
  • Replaces original audio entirely

Preserving original audio:

• Benefits:
  • Keeps background sounds or music from the original video
  • Allows for creative mixing of audio sources
• Drawbacks:
  • May require careful balancing of audio levels
  • Increases complexity of the command

Adjusting sync or trimming:

• Benefits:
  • Allows for precise timing adjustments
  • Helps in matching audio and video durations
• Drawbacks:
  • May require trial and error to get timing right
  • Can be time-consuming for long videos

Mux tips for effective audio-video merging with

1. Check audio formats: Ensure your audio is in a format compatible with your target video container.
2. Mind the audio quality: The output quality will only be as good as your input files.
3. Consider video length: Make sure your audio is appropriate for the length of your video.
4. Test different codecs: Some codecs may produce better results or smaller file sizes.
5. Normalize audio levels: Adjust volume levels to ensure consistent sound throughout the video.
6. Use lossless inputs when possible: Working with lossless formats preserves quality during processing.
7. Preview before finalizing: Always check the output to ensure proper sync and quality.

Remember that the quality of your output depends on the quality of your input files and the processing choices you make. Always start with the highest quality source files available for the best results. As you become more comfortable with FFmpeg's capabilities, you can experiment with more advanced options to fine-tune your output and streamline your workflow.

Audio and video merging FAQs

Can I merge audio and video without re-encoding?

Yes, if your audio format is already compatible with your video container. Use -c:v copy -c:a copy to stream copy both tracks without re-encoding, which is much faster. However, this only works if the audio codec is supported by your output container format—for example, MP4 containers support AAC and MP3 audio, while WebM requires Opus or Vorbis.

Why is my merged audio out of sync with the video?

Audio sync issues typically occur when the audio and video have different frame rates or when one file has variable frame rate (VFR). You can use the -itsoffset parameter to manually adjust timing, or specify -async 1 to stretch/squeeze audio to match video timing. For precise sync, identify the exact offset needed by comparing specific moments in both files.

How do I add multiple audio tracks to one video?

FFmpeg supports multiple audio tracks in containers like MP4 and MKV. Use multiple -map commands to include each audio stream: -map 0:v -map 1:a -map 2:a maps video from the first input and audio from the second and third inputs. You can set metadata to label each track, like -metadata:s:a:0 language=eng -metadata:s:a:1 language=spa for English and Spanish tracks.

What audio format should I use when merging with video?

AAC is the most universally compatible audio codec for web video and works with MP4 containers on all platforms. For highest quality, use AAC at 192-256 kbps for stereo or 384-512 kbps for 5.1 surround. MP3 also works but is less efficient. Avoid formats like FLAC or WAV in web videos—they create unnecessarily large files without perceptible quality improvement for most viewers.

Can I adjust audio volume while merging?

Yes, use the volume filter to adjust gain. For example, -af "volume=2.0" doubles the volume, while -af "volume=0.5" halves it. You can also normalize audio levels with -af "loudnorm" to ensure consistent volume. When merging multiple audio sources, use the amix filter with weights to balance relative volumes: -filter_complex "[0:a][1:a]amix=inputs=2:weights=1 0.5".

What happens if my audio is shorter than the video?

By default, FFmpeg will output a video that ends when the shortest stream (audio or video) ends. If your audio is shorter, use -stream_loop -1 on the audio input to loop it until the video ends, combined with -shortest to prevent infinite output. Alternatively, you can extend the audio with silence using the apad filter.

How do I replace only part of the audio track?

This requires more complex filtering with the adelay and amix filters to time-align and blend audio segments. A simpler approach is to split your workflow: extract the segments you want to keep from the original audio, merge them with the new audio segment externally, then merge the complete audio track back with the video. FFmpeg can handle this, but the command complexity increases significantly.

Back to Articles