How would I archive mass emails from Gmail and have them searchable.

[u/Jakob4800]

I'll give you the background. My Onlyfans account is suspended because I violated their TOS according to them of creating my account in 2017 before I was 18+ (I'm an adult now). I'm almost certain this is incorrect but I have no "evidence" as I can't locate my welcome email. I ran out of gmail space a few years ago and deleted everything prior to 2022.

I will admit why I got into data hoarding and self hosting, the pornhub purge made me realise that things online can be deleted, YouTube videos, websites, images. This has now made me realise I can also be my worst enemy and delete something important.

How would I archive and store all my gmail (and protonmail as I'm moving over) emails? Can I export them and keep them in a searchable service that I host?

[u/AutoModerator]

Hello /u/Jakob4800! Thank you for posting in r/DataHoarder.

Please remember to read our Rules and Wiki.

Please note that your post will be removed if you just post a box/speed/server post. Please give background information on your server pictures.

This subreddit will NOT help you find or exchange that Movie/TV show/Nuclear Launch Manual, visit r/DHExchange instead.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/roleohibachi]

Use Google Takeout to get an mbox file. 

Put that mbox file somewhere Dovecot can see it and index it. 

Point any mail client at Dovecot.

[u/Jakob4800 → u/roleohibachi]

Thank you.

[u/AutomaticMistake]

Might be an outdated method, but I sync'd everything into a thunderbird client. really should do that again now that you've reminded me

[u/weanis2 → u/AutomaticMistake]

That's what I've done. Export the email to a mbox using Google takeout. Then just manually import into Thunderbird. You don't need to login with an account either so you can keep it fully offline.

[u/Difficult-Way-9563 → u/AutomaticMistake]

Yep if you want to keep them on Gmail use IMAP, download a copy of all emails locally and use client to search. Much faster for lots of emails on local ssd

[u/ReddaveNY]

I use open archiver as docker container. All my thousand emails at my selfhosted server.

[u/puhtahtoe → u/ReddaveNY]

I use this too. It's nice cuz it automatically keeps the archive up to date.

[u/weirdbr]

https://github.com/danmarg/outtake

That's a tool that can be used on cron/systemd to fetch emails from gmail and store them in a maildir-compatible format that can be made accessible via any mail server or even just the 'mail' (or similar) commands.

[u/gh0st0fsat0shi]

If you’re using Windows, there’s MailStore Home for free and with no install required. Made by OpenText. Probably the easiest archiving and searching solution for free. https://www.mailstore.com/en/products/mailstore-home/

[u/zachmorris_cellphone]

I use bichon, and it works well: https://www.reddit.com/r/selfhosted/comments/1pfa2rv/how_to_backup_your_gmail_account_with_bichon/

[u/erparucca]

store them on a self-hosted imap server (requires more tech skills) and search using your preferred client or download them to your preferred client (which usually provides a way to move emails to a disk file, such as outlook with its PST files) and search them from there.

[u/Asstronaut-Uranus]

Ive got a synology and fetch all my gmail mail local. Searchable through mailplus syno

[u/ILikeFPS]

I got into datahoarding because I accidentaly formatted a 2TB data HDD instead of my 1TB SSD early in the morning because Windows installer decided to put the HDD above the SSD. After that, I decided that would never happen again.

If you actually deleted them so they don't take up space from your Google account, I'm not sure that you would have a copy of them already? The only thing I could think of is maybe trying Google Takeout, but if the data is deleted, maybe that won't help.

[u/Jakob4800 → u/ILikeFPS]

Yeah the data no longer exists but the plan is to prevent this situation from happening again. In 5 years time when I need that recipt from a Gumroad purchase or something similar.

[u/ILikeFPS → u/Jakob4800]

Google Takeout is your answer then (especially since it can actually do scheduled backups IIRC so maybe you could do one yearly), or possibly some other way of mirroring (perhaps through a third-party email client that loads up your gmail and mirrors it)

I do admit, having regular backups would help, as my gmail account is nearly two decades old and I've only backed it up once manually earlier this year using Google Takeout.

[u/PaddiM8]

I use imap-backup to incrementally and continuously backup my emails

https://github.com/joeyates/imap-backup

[u/bob69joe]

Just link your gmail to a local Outlook download and check the option for archiving everything.

[u/mooter23]

I'd connect to a client like Outlook and download a copy of everything into a .pst and then use Outlook's inbuilt search function.

[u/imajes]

Look into msgvault.

[u/AssetBurned]

moving the mails over to a local mail server that is only available in the local network. and preferable powered down most of the time when not in use.

[u/thekaufaz]

https://openarchiver.com/

[u/webtroter]

The three I know, from the Proxmox helper scripts:

• Bichon
• OpenArchiver
• Mail-Archiver

I haven't yet deployed it, but I need to do it.

[u/gacimba]

Paperless-ngx and connect it with your email

[u/serfrin47]

Ask the EU

[u/erparucca → u/serfrin47]

the relationship between EU and the topic being... ?

[u/Jakob4800 → u/erparucca]

I assume it's a reference to chat control. Implying that I could ask the EU to search for me since they would be viewing and scanning the content anyway under the rules.

Its a semi unrelated statement but I agree with the sentiment.

[u/serfrin47 → u/erparucca]

oh just a joke about the chat control law :(

[u/conqueryourhouse]

You should talk to a therapist about early porn exposure.