Batch Processing with FFmpeg: Automate Video Tasks for Multiple Files

Thu Sep 05 2024

Process hundreds of videos automatically with shell scripts, parallel encoding, and error handling for production workflows.

Processing one video at a time is tedious. FFmpeg's command-line nature makes it perfect for batch operations—transcode entire directories, apply watermarks to hundreds of files, or normalize audio across your entire library. This guide shows you how to automate repetitive video tasks efficiently.

Basic Shell Loop

#!/bin/bash
# Convert all MOV files to MP4
for file in *.mov; do
  ffmpeg -i "$file" -c:v libx264 -crf 23 -c:a aac -b:a 192k "${file%.mov}.mp4"
done

# Process with different naming
for file in *.avi; do
  output="converted_$(basename "$file" .avi).mp4"
  ffmpeg -i "$file" -c:v libx264 -preset medium -crf 22 -c:a aac "$output"
done

The ${file%.mov} syntax removes the extension, letting you replace it with .mp4. Always quote "$file" to handle filenames with spaces.

Recursive Processing

#!/bin/bash
# Process all MP4 files in subdirectories
find . -name "*.mp4" -type f | while read -r file; do
  dir=$(dirname "$file")
  base=$(basename "$file" .mp4)
  ffmpeg -i "$file" -c:v libx264 -crf 23 -c:a copy "$dir/compressed_$base.mp4"
done

Use find to locate files recursively, then process each one while preserving directory structure.

Parallel Processing

Single FFmpeg instances don't utilize all CPU cores efficiently. Process multiple videos simultaneously:

#!/bin/bash
# Using GNU parallel (install via: apt-get install parallel or brew install parallel)
parallel -j 4 'ffmpeg -i {} -c:v libx264 -crf 23 -c:a aac -b:a 192k {.}_converted.mp4' ::: *.mov

# Or with xargs
ls *.mp4 | xargs -P 4 -I {} ffmpeg -i {} -c:v libx264 -crf 23 -c:a copy compressed_{}

-j 4 or -P 4 runs 4 jobs in parallel. Adjust based on your CPU cores (typically use 50-75% of available cores).

Progress Tracking and Logging

#!/bin/bash
# Process with progress output
total=$(ls -1 *.mp4 | wc -l)
current=0

for file in *.mp4; do
  ((current++))
  echo "Processing $current/$total: $file"
  ffmpeg -i "$file" -c:v libx264 -crf 23 -c:a aac "output_$file" 2>&1 | tee -a conversion.log
done

echo "Batch complete: $total files processed"

The tee command saves FFmpeg output to a log file while still displaying it on screen.

Error Handling

#!/bin/bash
# Skip files that fail, continue processing
failed_files=()

for file in *.mkv; do
  echo "Processing: $file"
  if ffmpeg -i "$file" -c:v libx264 -crf 23 -c:a aac "${file%.mkv}.mp4" 2>&1 | tee -a batch.log; then
    echo "✓ Success: $file"
  else
    echo "✗ Failed: $file"
    failed_files+=("$file")
  fi
done

# Report failures
if [ ${#failed_files[@]} -gt 0 ]; then
  echo "Failed files:"
  printf '%s\n' "${failed_files[@]}"
fi

Conditional Processing

#!/bin/bash
# Only process files larger than 100MB
for file in *.mp4; do
  size=$(stat -f%z "$file" 2>/dev/null || stat -c%s "$file")  # macOS/Linux compatible
  if [ "$size" -gt 104857600 ]; then  # 100MB in bytes
    echo "Compressing large file: $file ($size bytes)"
    ffmpeg -i "$file" -c:v libx264 -crf 28 -c:a aac -b:a 128k "compressed_$file"
  else
    echo "Skipping small file: $file"
  fi
done

Resolution-Based Batch Processing

#!/bin/bash
# Downscale only videos over 1080p
for file in *.mp4; do
  height=$(ffprobe -v error -select_streams v:0 -show_entries stream=height -of csv=p=0 "$file")
  
  if [ "$height" -gt 1080 ]; then
    echo "Downscaling $file (${height}p → 1080p)"
    ffmpeg -i "$file" -vf "scale=-2:1080" -c:v libx264 -crf 23 -c:a copy "1080p_$file"
  else
    echo "Skipping $file (already ${height}p or lower)"
  fi
done

Create Web-Optimized Versions

#!/bin/bash
# Generate web, mobile, and thumbnail versions for each video
for file in *.mp4; do
  base="${file%.mp4}"
  
  # 1080p web version
  ffmpeg -i "$file" -vf "scale=-2:1080" -c:v libx264 -crf 23 -preset fast -movflags +faststart \
    -c:a aac -b:a 128k "${base}_1080p.mp4"
  
  # 720p mobile version
  ffmpeg -i "$file" -vf "scale=-2:720" -c:v libx264 -crf 26 -preset fast -movflags +faststart \
    -c:a aac -b:a 96k "${base}_720p.mp4"
  
  # Thumbnail at 5 seconds
  ffmpeg -ss 00:00:05 -i "$file" -frames:v 1 -q:v 2 "${base}_thumb.jpg"
done

Batch Watermarking

#!/bin/bash
# Add watermark to all videos
logo="watermark.png"

if [ ! -f "$logo" ]; then
  echo "Error: $logo not found"
  exit 1
fi

for file in *.mp4; do
  ffmpeg -i "$file" -i "$logo" \
    -filter_complex "[0:v][1:v]overlay=W-w-20:H-h-20" \
    -c:v libx264 -crf 23 -c:a copy "watermarked_$file"
done

Smart Re-Encoding (Skip Already Processed)

#!/bin/bash
# Only process files that haven't been converted yet
for file in *.mov; do
  output="${file%.mov}_converted.mp4"
  
  if [ -f "$output" ]; then
    echo "Skipping $file (already processed)"
    continue
  fi
  
  echo "Converting $file → $output"
  ffmpeg -i "$file" -c:v libx264 -crf 23 -c:a aac "$output"
done

Organize Output into Directories

#!/bin/bash
# Create organized output structure
mkdir -p output/{1080p,720p,480p}

for file in *.mp4; do
  base=$(basename "$file" .mp4)
  
  ffmpeg -i "$file" -vf "scale=-2:1080" -c:v libx264 -crf 23 -c:a aac "output/1080p/${base}.mp4"
  ffmpeg -i "$file" -vf "scale=-2:720" -c:v libx264 -crf 25 -c:a aac "output/720p/${base}.mp4"
  ffmpeg -i "$file" -vf "scale=-2:480" -c:v libx264 -crf 27 -c:a aac "output/480p/${base}.mp4"
done

Hardware-Accelerated Batch Processing

#!/bin/bash
# Use NVIDIA NVENC for faster batch encoding
for file in *.mp4; do
  ffmpeg -hwaccel cuda -i "$file" \
    -c:v h264_nvenc -preset p5 -cq 23 -c:a copy "hw_$file"
done

# Or Intel QuickSync (Linux)
for file in *.mp4; do
  ffmpeg -hwaccel qsv -c:v h264_qsv -i "$file" \
    -c:v h264_qsv -preset medium -global_quality 23 -c:a copy "qsv_$file"
done

CSV Report Generation

#!/bin/bash
# Create a report of all processed files
echo "filename,duration,resolution,filesize,bitrate" > report.csv

for file in *.mp4; do
  duration=$(ffprobe -v error -show_entries format=duration -of csv=p=0 "$file")
  width=$(ffprobe -v error -select_streams v:0 -show_entries stream=width -of csv=p=0 "$file")
  height=$(ffprobe -v error -select_streams v:0 -show_entries stream=height -of csv=p=0 "$file")
  size=$(stat -f%z "$file" 2>/dev/null || stat -c%s "$file")
  bitrate=$(ffprobe -v error -show_entries format=bit_rate -of csv=p=0 "$file")
  
  echo "$file,$duration,${width}x${height},$size,$bitrate" >> report.csv
done

echo "Report generated: report.csv"

Advanced: Queue System

#!/bin/bash
# Process files from a queue file
queue_file="to_process.txt"

if [ ! -f "$queue_file" ]; then
  echo "Creating queue from current directory..."
  ls *.mp4 > "$queue_file"
fi

while IFS= read -r file; do
  if [ -f "$file" ]; then
    echo "Processing: $file"
    ffmpeg -i "$file" -c:v libx264 -crf 23 -c:a aac "done_$file"
    
    # Remove from queue on success
    if [ $? -eq 0 ]; then
      sed -i.bak "/^${file}$/d" "$queue_file"
      echo "✓ Completed and removed from queue"
    fi
  fi
done < "$queue_file"

echo "Queue processing complete"

Pitfalls

  • Always test your script on a few files before running on hundreds—mistakes can be costly.
  • Parallel processing can overwhelm storage I/O; monitor disk usage and adjust parallelism.
  • Don't overwrite original files; use different names or directories for output.
  • Hardware acceleration may produce slightly different quality; test before batch processing.
  • Large batches can take hours or days; use screen or tmux to prevent interruption from terminal disconnects.
  • Check disk space before starting; you'll need room for both input and output files.

Production Workflow Example

#!/bin/bash
# Complete production workflow
set -e  # Exit on error

SOURCE_DIR="raw_footage"
OUTPUT_DIR="processed"
BACKUP_DIR="backup"

mkdir -p "$OUTPUT_DIR" "$BACKUP_DIR"

echo "Starting batch processing at $(date)"

# Process all files with progress tracking
find "$SOURCE_DIR" -name "*.mp4" -type f | while read -r file; do
  filename=$(basename "$file")
  
  echo "Processing: $filename"
  
  # Backup original
  cp "$file" "$BACKUP_DIR/"
  
  # Process
  ffmpeg -i "$file" -c:v libx264 -preset medium -crf 23 \
    -c:a aac -b:a 192k -movflags +faststart \
    "$OUTPUT_DIR/$filename" 2>&1 | tee -a processing.log
  
  if [ ${PIPESTATUS[0]} -eq 0 ]; then
    echo "✓ Success: $filename"
  else
    echo "✗ Failed: $filename - check processing.log"
  fi
done

echo "Batch processing complete at $(date)"
echo "Check processing.log for details"

With these batch processing techniques, you can handle video production at scale, automating repetitive tasks and maintaining consistency across your entire video library. Whether processing dozens or thousands of files, these patterns provide a solid foundation for production workflows.