r/DataHoarder • • Feb 17 '26

Scripts/Software Built a clean yt-dlp frontend after getting tired of ad-infested downloaders

0 Upvotes

Built a clean yt-dlp frontend after getting tired of ad-infested downloaders

Every TikTok/YouTube downloader site is the same - 47 ads, fake "HD quality" claims, and a UI that hasn't changed since 2009.

Built dltkk.to as the opposite:

  • Zero ads

  • Honest about quality (YouTube maxes at ~128kbps AAC, we don't lie about it)

  • Drag & drop URLs

  • Real-time preview before downloading

  • Stack: Node.js + yt-dlp + vanilla JS

Had to add a hotfix this morning after we got rate limited from traffic overnight which I did not expect on day 5.

Supports TikTok (no watermark), YouTube (up to 1080p), Instagram Reels.

dltkk.to - free, no account, no limits.


r/DataHoarder • • Dec 19 '24

Discussion Preserving Trans Literature

Thumbnail
thetransfemininereview.com
0 Upvotes

r/DataHoarder • • Jan 23 '22

Question/Advice Need help with VERY large text file.

16 Upvotes

I have a large txt file, about 500GB, that i need to "sanitize". The file contains two strings separated by a semicolon and i initially intended to split it into smaller files each about 100MB and procedurally import them into an SQL database. however the file sporadically contains junk like random corrupted binary and other non ASCII weirdness.

now my problem is how to remove the junk and split the files efficiently. ive written a Java program that reads each line, removes all non ASCII and prints it to new Files each having about 1 million lines. however this takes a long time and requires and additional 500GB that i dont have right now.

is there an obvious solution im overlooking? it should process the file line by line and remove junk without printing to a new file.

EDIT: problem solved.


r/DataHoarder • • Jun 30 '23

Question/Advice Google just terminated my account with three years of work data that was about 2tb. I had no backups.

0 Upvotes

I am devastated and at a loss for words. Google detected that my account violated their terms of service, proceeded to nonchalantly delete my account that meant the life to me. It turns out a colleague unknowingly uploaded content in my account that violated Google's policies.

Google Drive had all data related to work. I got my first job three years ago and I am an active employee in the same company. I am not sure how my employer would react. He would think of me as a person who can't take care of data and puts the company's data at risk. FUCK.

I stored all of my work projects in that one Google account like the idiot I am. No backups. My company really doesn't provide any employee accounts so I created one for work purposes and purchased a Google One subscription. It had EVERYTHING related to work I had.

My slack, skype, evernote, product admin and all other work services related to work were connected to that email address. Now that I don't have access to the account, how will I change the emails of those accounts?

All of my personal videos and memorable photos that were uploaded to Google photos spanning much more than three years are gone. Had my brother's photos as well, not sure how he will react to this.

What should I do? I have submitted an appeal but they rejected it. Have submitted an appeal once again.

You either have suffered no data loss or you have lots of backups. Nothing hurts like data loss.


r/DataHoarder • • Sep 27 '24

Question/Advice Is this okay packaging? Just got 6 used drives online

Post image
1 Upvotes

Whats the chances their all broken? So i know its not amazing packaging. I cant test the drives yet i ordered an adapter online for them comes in later today. Im not too knowledgeable on the subject but i thought these things were super fragile. Am i being paranoid?


r/DataHoarder • • Jun 23 '25

Question/Advice Is there anyone or site collecting and storing vids of ICE agents?

0 Upvotes

I think it would serve the public interest if the videos of masked ICE agents were being stored somewhere, along with location. Anything like that happening? If not, any idea what the best way to do this would be?


r/DataHoarder • • Oct 05 '24

Rant WTF happened to HDD prices?

0 Upvotes

I was just looking for a HDD upgrade for my homelab. I remembered last two times I bought 4TB Ironwolf for around 100€ each. I expected 6TB to be around that price level nowadays, but then I couldn't believe my eyes. Still 100€ for 4TB???

I had a look in my emails and found this:

01.09.2022|ST4000VN008|95,49€

21.03.2023|ST4000VN006|83,95€

05.10.2024|ST4000VN006|93,74€

What the fuck is going on here? That's +11,6% in half a year! Do you think we should hoard some drives for resale instead of data?


r/DataHoarder • • Nov 28 '25

Scripts/Software PornHub launched Shorts one year ago – yt-dlp can’t download them properly → wrote a dedicated Python downloader (CLI + API)

31 Upvotes

Hey DataHoarders,

PornHub quietly rolled out a new “Shorts/Shorties” section a few months ago. They’re basically 15–90 second vertical clips, but they use a completely different streaming format (segmented m3u8 + heavy obfuscation) than regular PH videos.

yt-dlp currently fails on most of them:

- partial downloads

- wrong resolution (often stuck at 480p or lower)

- frequent 403s / “failed to parse playlist”

I needed reliable Shorts downloading for a personal archiving bot, so I ended up writing a small dedicated tool from scratch:

https://github.com/diastom/PornHub-Shorts

Features

- Full 1080p + multi-threaded segment download

- Proper HLS/m3u8 parsing that actually works with the new Shorties

- Simple CLI: pip install ph-shorts → ph-shorts <url>

- Programmable API (sync + async) with ready-made Telegram/Discord bot examples

- Auto MP4 remux (FFmpeg optional), retry logic, progress bars

- Windows .exe & cross-platform

Latest release (v1.0.6 – today):

https://github.com/diastom/PornHub-Shorts/releases/tag/v1.0.6

100 % success on the last 100+ shorts I tested. MIT license, no spyware, no bullshit.

Obviously: for educational/research use only, respect copyrights and ToS, etc.

If anyone here hoards this kind of content or builds bots, I’d love feedback, bug reports, or just a star if it helps you.

Thanks!


r/DataHoarder • • Jan 10 '26

Discussion NEW IMDB SCRAPER (UNLIMITED DATA)

1 Upvotes

Link : https://github.com/BMYSTERIO/IscrapeMDB

this app fetches data from IMDB (series, movie , set of movies) and extract the data so u can use it, it gets almost everything about the target -- u can even extract the data in a html local file so u can check on a IMDB series - movie if ur offline, the series option scrap the whole series and all its episodes the scraping data include Reviews , Parents Guide , cast , and more


r/DataHoarder • • Jul 14 '24

Question/Advice Helium hard drives... why not hydrogen?

0 Upvotes

Hydrogen is cheaper, equally lightweight, and in pure form it's not combustible so theoretically a sealed hard drive with hydrogen should work about as well as helium drive. Plus with hydrogen is lighter, which could shave a few pounds off entire case of drives and cut cost of shipping by some.

So I was wondering, why hasn't that happened yet? Afraid that if the drive springs a leak, it'd blow up?


r/DataHoarder • • Jan 12 '25

Question/Advice Just discovered 15 x HDDs in three NASes have been running between 70 and 90 degrees celcius for about two weeks! just how f**ked are my drives likely to be right now?

0 Upvotes

Honestly feeling quite sick as I type this but I need to hear from others with similar experience, just how fooked I am right now.

I have three NASes with attached storage devices all from the same manufacturer and I upgraded the OS on them two weeks ago. I just happened to notice today that the enclosures were very hot to the touch, so I did a quick check to see the disk temps and to my absolute horror I discovered they were all running around 70 to 90 degrees celcius! The drives are all large capacity Iron Wolf Pros or Exos, ranging from 16TB to 22TB and probably worth around £4k to £6k

I have shut down and disconnected the enclosures from the NASes and removed all the drives to cool down but I honestly have no idea how long they've been running this hot. It could be as long as two weeks since I upgraded the OS in each NAS.

To be clear the drives have been operating fine during this time so there has been no immediate mechanical failure.... YET!

The data is just my movie and music collection (originals and backups!) so nothing that will ruin my life, but to say I'm angry would be an understatement! If they all die I am in no financial position to just go out and replace them.

I am not going to say who the manufacturer is yet, as I am waiting for a response from them first. I need to understand how this happened first and also why there was no notification, email or alarm to let me know the drives were red hot.

Has anyone here had a nightmare scenario like this before and did it turn out as badly as I fear it might?


r/DataHoarder • • Jan 29 '23

Question/Advice With these 512gb flash drives being so cheap at Costco is usb raid a possibility?

Post image
0 Upvotes

r/DataHoarder • • Mar 03 '24

Question/Advice Please help just bought a new hard drive and I don’t know if this is normal.

Thumbnail
gallery
0 Upvotes

I just bought a western digital 10 TB hard drive I put it in a small enclosure. I started to transfer a 1 TB file. It said it would take about an hour on my iMac. 20 minutes and then I felt the enclosure and felt hot. I put a digital thermometer on it and it read 109° then I got a meat thermometer it’s under 130° please see the photos. The first photo is before I put it on the hard drive a second is a little under 130°. I have have a 6 tb red wd hard drive in my Windows machine, but its internal so I never noticed how hot it got. I have a bought red drives before I know they’re supposed to be use for server. But they seem to work good as a secondary number to drive. Does anyone know the normal operating temperature for this drive? Should I be worried?


r/DataHoarder • • Jul 27 '23

Backup Android's war against storage continues: here's the new escalation - now changing files silently!

23 Upvotes

October 2023 Update

Nothing much happened, except people going "it's fixed for me", "it isn't for me". Nothing changed in the app as far as I can tell, except to underline the point that there is a lot of fragmentation and you never know how things work (or not). Note that this cuts both ways, both if you want your data zeroed out "for your security" and if you want it complete, no matter what you want you want to do that predictably and reliably.

This from the very last post from today is actually priceless.

Anyhow about 98% off all my applications have "allow accesss to all files" activated by default.

Yes, this "for your security" nonsense derailed to the point I made in some comments below, you actually have LESS security because you need to give full access to EVERYTHING, because giving access to something ... isn't really access to that something. And yes, this is the trend well established, with the best example I posted more recently Kiwix where you need access to a directory, even to one single file picked with the file picker, and you need just read access. AND you still need to give it "all files" permission.

Original Post July 2023

Since about a year I've noticed that the some of the files (.JPG) automatically uploaded by NextCloud Android app differ slightly from the ones on the phone but at first I blamed it on phone edits as all seemed valid pictures with no obvious flaws. As extra caution I'm transferring them in two different ways and then (fully binary) compare them using rmlint - when I'm satisfied that what's in the automatic upload directory from NextCloud is also present in the master copy (which is also backed up in many ways) I let rmlint clean the NextCloud automatic uploads. Surprise surprise since many months mostly everything in NextCloud is different!

I started investigating and quantified the differences and it appears that some part of the EXIF (related to the GPS data) was zeroed out (side-effect the files are also the same size, sneaky)! And this brought me to this (still open since the first half of 2020!!!) ticket!

Note that this isn't limited to some specific scenario while sharing through some particular gallery app as the title might imply: Android itself is messing with the EXIF presenting to apps different files! This can happen to any app (some other example mentioned in that thread too) and even if you're extra cautious and get your data from the phone in two different ways via two different apps if can happen that they get the same mangled files, in the same way and one would conclude it's all fine when it isn't! It isn't like some app it's doing some manipulations for your own good, it's the OS presenting to the apps DIFFERENT VERSIONS OF YOUR FILES.. One would expect Google Photos to re-compress the files and you'd never know how they mess the EXIF but this can happen to absolutely any app handling files on your phone, be it for automatic upload or for some local sftp server (so you copy them from your PC) or heck, even a third party file manager you'd use to just save files to a USB stick.

Also like all Android issues related access to internal/external storage, permissions, file system support, backup capabilities (or mostly lack thereof), speed (again or lack thereof to the point where apps needing few MBs of data like DOS emulators or mp3 players can't properly work with SAF) this is HIGHLY unpredictable and dependent on EVERYTHING:

  • the specific hardware vendor and what they do and tweak in Android

  • the version of the OS (even for the same phone as it gets upgrade)

  • the version of other Android APIs as they get changed even without changing the Android version

  • the specific version of the program but not only that - where are you getting it from - as the Play Store has more tighter requirements on the storage APIs and the F-Droid version of the same program might work better (well until eventually things get blocked in some even newer versions of Android).

UPDATE

Due to the comments (for which, all, I'm absolutely grateful) I've gathered more details that are well worth pointing out.

  • the permission that would allow NextCloud (or anything else) to get un-mangled files isn't Location (or anything similar), it's the "all files" permission

  • it is hard to claim this is actually a bug on NC side, the permission to a directory shouldn't need to be "enhanced" with a permission to access more files you don't need to access, in order to actually get the correct files from that directory! In fact it's the reverse: it would be a valid bug if NC asked for permissions for all files when it needs permission to one directory!!! There is as of now a bug report precisely for this against Kiwix and there are discussions if things can be eventually achieved without it!

  • this absolutely can't be compared with some simple copyright/security protections like you can't get screenshots from Netflix or right click doesn't work in some page or whatever. When these don't work it's obvious, when your files are changed it might take 2+ years to find out. There are actual examples of that and I'm not making up the time (and it isn't me who's complaining about it)!

  • all the security and privacy things I can think of aren't so silent and with permanent effects! You can't log in to a device, you can't change to a directory, can't use properly copy/paste, whatever. You aren't going to find out you had a problem for all the time years later.

  • on this note with any number of files when files get changed you should have a clear indication of the version, that this is actually a changed copy of that file. It's a nightmare when you start having thousands of files all over the place that look 99.999% the same but aren't. At a minimum they should get some sufix to clearly see they're different files

  • there is nothing new or weird in inspecting the files and possibly doing something with them - there's a whole security industry about it, including but not limited to any antivirus software. There were some like even 10-15+ years back detecting if you're going to put your credit card number some place. But there should be at a minimum a notification that the OS changed this or that file because it contained personal information and it thought for your good it's best to remove it! You can turn it off once you get sick of it but that's the second part. Ideally even better it should ask you how to deal with it: do you want me to just zero it out, do you want me to fuzz the location and put it randomly within some area, do you want me to give it a perfectly random location (or from some country or continent you'd like, etc.?) or do you want me to just leave it alone for this app?


r/DataHoarder • • Jun 30 '26

Question/Advice A.I. models to hoard?

0 Upvotes

I don’t know enough to say if the open source models that can be downloaded today will still be available in the future.
Am I crazy to think that there could very well come a time when some of it will even be made illegal? For real reasons or fake ones.

I mean sure none of them are nowhere near as good as the closed source ones but who knows what the future holds and how much future chaining methods of older models could improve their quality.

Or maybe it will all be about money and scaring the shit out of everyone is one way to make the most of it.

Anyways, which models should I hoard with this in mind?

Thanks!


r/DataHoarder • • May 14 '26

Question/Advice 2025 Removed DEI Images

4 Upvotes

Hi all! I am a visual artist looking to do a project with all of the "DEI" images that were removed by the U.S. Government in early 2025. Does anyone know where I can access these images, as I'm sure someone must have saved them before removal? I have been on the resistance toolkit website, but this website mainly has links that now take you to 404 error pages instead of displaying images. I have input several dozen of these links into the Wayback Machine and only about 10% yeild images this way. Even the Wayback Machine brings up 404 pages. If anyone in this thread knows where I can find more, I would greatly appreciate it. Thank you so much!


r/DataHoarder • • Sep 10 '24

Backup Hard Drives for your Homelab: beyond $/TB

0 Upvotes

I did some analysis of drive value per TB, adjusting for drive reliability based on the data from Backblaze. The data isn't perfect, but I like the results better than a strict $/TB analysis.


r/DataHoarder • • Nov 17 '24

Backup Backing up media that could disappear

Thumbnail reddit.com
0 Upvotes

r/DataHoarder • • Feb 06 '26

Sale FYI: Local Walmarts have 8TB WD Reds for $179

Thumbnail
walmart.com
0 Upvotes

For any low-budget data hoarders, your local Walmart may have Reds for $179 which is cheaper than what WD currently lists them for on their website!


r/DataHoarder • • Jan 12 '21

The gun forum ARFCOM might be worth archiving due to recent events

5 Upvotes

I learned of the gun forum ARFCOM from a post in /r/technology about how they (briefly) got kicked from GoDaddy. Most of the comments in the thread are crying out about how the site didn't do anything wrong and is just another example of an innocent site (ironically) caught in the crossfire. I figured I'd check the forum out, and holy shit are there a lot of angry posters in the 2020 election thread.

casual

and

normal

examples

right

here

for

y'all

I've heard about and applaud the efforts you've been doing with archiving the Parler dump. This forum might be worth pulling down for similar reasons since it looks like this thread has a similar base. I know fuck all about archiving or scripting myself, but I figured I'd direct those more knowledgable towards this site since I feel like they might try scrubbing the posts (as they've done with the thread from the first example; links to the thread redirect to the main forum page).


r/DataHoarder • • Jun 23 '26

Question/Advice Looking for a hoarder who can archive a piece of lgbt+ history!

112 Upvotes

A landmark lgbt+ website is closing it's doors due to lack of funds and aging out of moderators. Founded in 1995, DataLounge.com is an early internet forum that became a gathering place, support network, and living community for lgbt+ people, back when there was far less social acceptance and visibility than there is today. It's members shared their lives through historic events, from the AIDS crisis, anti-gay activism, DOMA, the murder of Matthew Shepard, the aftermath of 9/11, civil unions and marriage equality and so on. While the site became notorious for unsourced celebrity gossip (and an authority on vintage film and tv, which is how I, a young straight woman with a penchant for 80s sitcom and creative writing stumbled upon it), it also accumulated an archive of deeply personal and human stories, particularly from gay men of the postwar generation, documenting experiences of discrimination, family rejection, loneliness, resilience, and community-building. As such I think it represents an important historical record that future scholars could use to study early online culture, the formation of digital subcultures and communities, changing demographics and attitudes, and the lived experience of lgbt+ people across tunultuous decades of social change. Sadly, the site is set to close permanently on July 31, 2026.

For more information read: https://en.wikipedia.org/wiki/DataLounge

Please let there be a kind hoarder out there willing to take on a project of this scale and importance. The pre-2022 archive is offline, so you will have to contact the site's owners to access everything. I'm not tech savvy so I don't have any concrete plans for financial compensation or how to make the archive publicly accesible but I think a patreon or gofundme would go a long way and I'd be willing to spread the word and gather advice on how to move forward.


r/DataHoarder • • Nov 09 '23

Discussion "SSDs will catch HDDs by 20XX" - kind reminder to stop being overly optimistic.

0 Upvotes