Just got done recovering ~5TB worth of data

Hello all. I'm new to the DataHoarder community, but not new to having numerous hard drives in my desk.

TLDR: Sent in my 4TB External HDD for recovery because I deleted files to save space. Deleted the corrupted or broken recovered files, organized the remaining good files, de-duplicated them, moved them to my main 8TB HDD, formatted the 8TB transfer drive, then moved all the files back onto the freshly formatted 8TB transfer drive.

Edit: A lot of people are blaming me for spending $400 on a data recovery when I could've done it for free with Recuva...

Guys, I've already tried Recuva last year to recover the files I was looking for. It didn't work at all, hence why I went with sending it in. I said last year because I needed to save up for this recovery.

Last night I used Recuva to recover a video when my hand accidentally bumped the cable and disconnected one of the drives during the transfer. It successfully recovered the video file then, since It was almost immediate after the problem happened.

While it's great for same day deletions, it doesn't fare well against something I deleted almost 5 months of running it before finally retiring it.

Edit 2:

2017: 4TB Seagate External from Costco for $100 2021: 8TB Seagate External from Amazon for $150

2023: 8TB Internal HGST from eBay for $60 Also 2023: 4TB Internal HGST from eBay for $20

I could've bought 12TB instead of 4TB back in 2017...

​

Here's the full story:

Back in October 2020, I decided to delete a bunch of videos off my measly 4TB Seagate External HDD since it was getting full. Later in February 2021, I got an 8TB External HDD that I still use to this day for videography and other purposes that I use daily. I moved all my data from my 4TB HDD to my 8TB HDD, basically leaving the 4TB completely empty. However (and luckily), I did not use it at all since that day.

Just this past November, I saved up enough to hire $300 Dollar Data Recovery to recover ALL of the data on the 4TB, specifically the videos that I deleted way back in 2020. I filled out their forms, and sent another 8TB Internal HDD I had, and the 4TB from before.

When they shipped back both drives, I was surprised to find that there was a little over 5TB worth of data, all coming from a drive that has 3.63TB of storage. Everything from videos to really small files 1KB or less files, totaling about a million files and several tens of thousands of folders.

The FIRST thing I had to do was painstakingly sort out all the corrupted or "broken" files that weren't playable or were distorted beyond recognition. So I had to carefully plan and make the process as efficient and fast as possible.

For videos and photo, I opened up File Explorer in full screen and put it in TILE view. I used Ctrl + Scroll and zoomed out as far as I could, just before going into a different view, to have as many icons as possible on my screen.

I also sorted the files by size, because most of the corrupt files were either ~1KB or gigantic (400+ MB for a corrupted photo). Doing this would group all the small and large files together, making it so I could simply drag around the blue box and select a bunch of them at once, and unselect the few good ones from my selection while holding down Ctrl.

I would let File Explorer load all the thumbnails for the photos/videos, and whatever didn't have a thumbnail I simply selected and moved to a folder called "Broken Files", since opening them would simply give me a corrupted file error 99% of the time. Then I would simply scroll, wait a couple seconds for File Explorer to load the thumbnails, quickly scan for any files missing a thumbnail, then slowly scroll again.

For MP3 or other audio files, since all of them combined was only about ~100GB, I decided it wasn't worth my time to comb through lots of more files to free up less space. For HTML, SVG, WAV, RTF, etc... all of them combined was only about ~40GB, with over 200k files, so again, it wasn't worth my time trying to free up such little space with so many small files to comb through.

After weeding out the corrupted or broken files, I made folders that were organized by year (2017, 2018, 2019, etc etc). I sorted the video/photos by their dates, and moved them into their respective year folders. I did this by again, having File Explorer in fullscreen, then putting it into DETAIL view (so I could see the date), sorted by date, then dragged the mouse all the way down, letting it autoscroll down until I've selected all the files for that year.

For the other files (HTML, WAV, RTF, etc), I again used File Explorer in full screen, but this time put it in LIST view instead of tile view (although you could use small icon view), and then sorted by file type. I then did the same for above, but instead of dates, it was file type (duh). I then put all the selected files into their respective folders (HTML, WAV, etc).

At this point in the recovery (or organization, really), I had a bunch of folders ranging from 2015 to 2021, and a few other folders for different file types. I put all of these folders in one big giant folder called "good files", then used DupeGuru to remove any duplicates from that folder. I set my 8TB HDD as a reference, and let DupeGuru sort through 300k+ files overnight.

The next morning I came back and saw a lot of duplicates, all with a 100% match. I made sure to save the results on my computer just in case. I went ahead and sorted by file size, and saw that the video files and all others were all matching up by year, file size, and even filename. The small ~10kB files I knew luckily weren't that important to me, so I just let it slide, since they were probably some program files or PNG's that aren't family pictures or selfies.

I moved all the duplicate files into another folder called "good duplicates". I then moved all of the "good files" to my 8TB HDD, totaling 1.5TB being moved via File Explorer.

I then ran DupeGuru once more for good measure, setting the "good duplicates" folder against my 8TB HDD and the "good files". I saw one photo slide, and ~200 small files like RTF, HTML, etc, after moving the duplicates to yet another folder called "good duplicates 2". I then moved the remaining non-duplicates to my 8TB HDD, and called it a day.

I then went ahead and formatted the 8TB TRANSFER drive (not my 8TB HDD mentioned previously of course!) to get rid of any pesky files or anything that could interfere with storing data. Before I did any deletion or formatting, I made sure that the only files that were on the transfer drive were either broken files, or duplicates, which I confirmed they were.

​

The data recovery itself took a little over a week from shipping to getting it back in the mail. The organization took about 2 weeks to complete.

I formatted the 8TB transfer drive yesterday, and starting moving all the files I wish to move from my 8TB HDD to the 8TB transfer drive. It's about 5.3TB worth of data being moved over to my transfer drive, which will now be used as extra storage. As I'm typing this story down this morning, my data is still being transferred over, and has been doing so since last night, so I will give it probably a full 24 hours or so to fully transfer 5.3TB worth of data.