blog: Parallel YouTube Download

Since youtube-dl still doesn’t incorporate parallel downloads, here is a one-liner (sort of) parallelizing playlist download with Unix tools:

youtube-dl "PLAYLIST_URL" --flat-playlist -j | awk '{print $2}' | tr -d ',"' | awk '{print "" $0}' | xargs -n 1 -P 10 youtube-dl

Only works with newer versions of youtube-dl, older ones seem to shuffle the utilized JSON dump nondeterministically. xargs’ parameter P would be the knob to adjust the number of parallel processes.

Posted in snippets
2018-07-18 19:56