by Minu » Wed Oct 19, 2022 12:26 am
what kind of data do you archive and hoard??
Just about everything that I find interesting.
I'll try to keep what I talk about in this post to web videos. It's 99% YouTube, though.
I currently have 2.1TiB of web videos, when I come across a channel I find interesting I'll archive the whole thing.
I have an elaborate system to do so, though it has several rough edges. In addition to channels, I also have similar systems for playlists and individual videos.
They all download new videos (meaning uploaded within 30 days) once every day, and then redownloads them once they're more than 30 days old, always at the highest quality available and with metadata. The reason to redownload after a month is because Google transcodes them in the background and it can take a few weeks to finish. This can cause dupes when the uploader slightly changes the title after uploading (to "hack the algorithm"), because my system is based off of filenames. For example, the Veritasium video "The 4 things you need to be an expert" that I downloaded on 2022-08-02 is a 1080p h264, and the renamed "The 4 things it takes to be an expert" that I downloaded on 2022-09-02 is a 4k AV1.
Currently there is one channel on YouTube that I want to archive, but it has been around since 2014 and has LOTS of videos. I remember the first time I blindly tried to archive it and a week later realized it was several TB in, still downloading, and only halfway through the channel's history. I'm just using playlists of the content I like on it for now, but maybe someday I'll grab all of it.
[quote]what kind of data do you archive and hoard??[/quote]
Just about everything that I find interesting.
I'll try to keep what I talk about in this post to web videos. It's 99% YouTube, though.
I currently have 2.1TiB of web videos, when I come across a channel I find interesting I'll archive the whole thing.
I have an elaborate system to do so, though it has several rough edges. In addition to channels, I also have similar systems for playlists and individual videos.
They all download new videos (meaning uploaded within 30 days) once every day, and then redownloads them once they're more than 30 days old, always at the highest quality available and with metadata. The reason to redownload after a month is because Google transcodes them in the background and it can take a few weeks to finish. This can cause dupes when the uploader slightly changes the title after uploading (to "hack the algorithm"), because my system is based off of filenames. For example, the Veritasium video "The 4 things you need to be an expert" that I downloaded on 2022-08-02 is a 1080p h264, and the renamed "The 4 things it takes to be an expert" that I downloaded on 2022-09-02 is a 4k AV1.
Currently there is one channel on YouTube that I want to archive, but it has been around since 2014 and has LOTS of videos. I remember the first time I blindly tried to archive it and a week later realized it was several TB in, still downloading, and only halfway through the channel's history. I'm just using playlists of the content I like on it for now, but maybe someday I'll grab all of it.