I have a remote backup of my home partition so I don't loose any documents, videos, etc.
How do you test these backups? I can browse them using Pika (GUI for BorgBackup) and I can do an integrity check, is there any more to it? I mean if my drive fails, I just copy those files back into my new drive's home directory. But I don't see how a test on that would be beneficial
Recently read a comment here: "Think you have a backup? Delete your drives now. If you're hesitant, you don't really have a backup."
As true as that is: I'd be in the "hesitant" category because I only back up my data / containers, not my bare metal system and its config. Do you guys? (Proxmox users will probably do, but what about the rest?)
I backup "everything" from my raspberries, and just \home for laptops but for an actual disaster recovery I'd just do a clean install, get an IP address and then reinstall everything via Ansible, then drag my home data back off the NAS
But... I like that statement. I'll use that at work.
I actually did that. Or well, I was planning to upgrade the Debian version and instead of doing it in place I replaced the drive (kept the old around for a while just in case) and did a clean install. I didn't make any new backups or moved configs or anything outside what I already had, the system was back to operations faily quickly. The main reason, I have a markdown file describing every configuration and recovery step needed to get a fresh system to a fully working state in case of a disaster, and my test showed that it worked great. It does take a bit of effort to keep it up-to-date, but as shown, really worth it, and it douples as a small wiki for accumulated knowledge. I do wish to play with Ansible at some point if I have the time, but this works for now.
whenever the self hosting mood strikes, i try so much off the wall shit that it'd all be irrevocably broken if i wasn't able to scrap the whole day and reload last nights back up!
i keep each group of services in their own lxc so at least only a part goes down if so!
I tested mine involuntarily. I enabled full disk encryption on my the main drive of my server, ignored all the warnings and then noticed, that the whole drive was formatted.
Luckily, all the important data was in my backups, I lost some compose files and the database of my paperless ngx seems to be broken (although theoretically, I am sure I could recover that data, I just don't have that time and there is nothing that important on there).
You reapply/ reload database and do all the recovery procedure on a clean new installation for everything. Disaster recovery is much a thing together with backup. Depends on your infra how to proceed in concrete steps.
21 replies
I have a remote backup of my home partition so I don't loose any documents, videos, etc. How do you test these backups? I can browse them using Pika (GUI for BorgBackup) and I can do an integrity check, is there any more to it? I mean if my drive fails, I just copy those files back into my new drive's home directory. But I don't see how a test on that would be beneficial
Recently read a comment here: "Think you have a backup? Delete your drives now. If you're hesitant, you don't really have a backup."
As true as that is: I'd be in the "hesitant" category because I only back up my data / containers, not my bare metal system and its config. Do you guys? (Proxmox users will probably do, but what about the rest?)
Nice philosophy.
I'm kinda in the middle ground here.
I backup "everything" from my raspberries, and just \home for laptops but for an actual disaster recovery I'd just do a clean install, get an IP address and then reinstall everything via Ansible, then drag my home data back off the NAS
But... I like that statement. I'll use that at work.
I actually did that. Or well, I was planning to upgrade the Debian version and instead of doing it in place I replaced the drive (kept the old around for a while just in case) and did a clean install. I didn't make any new backups or moved configs or anything outside what I already had, the system was back to operations faily quickly. The main reason, I have a markdown file describing every configuration and recovery step needed to get a fresh system to a fully working state in case of a disaster, and my test showed that it worked great. It does take a bit of effort to keep it up-to-date, but as shown, really worth it, and it douples as a small wiki for accumulated knowledge. I do wish to play with Ansible at some point if I have the time, but this works for now.
whenever the self hosting mood strikes, i try so much off the wall shit that it'd all be irrevocably broken if i wasn't able to scrap the whole day and reload last nights back up!
i keep each group of services in their own lxc so at least only a part goes down if so!
Congrats on becoming one of the 5% of ppl who backup, but actually also test them.
5 %, that many?
There are two kinds of self-hosters: those who backup, and those who will start testing em after their first failed restore.
I tested mine involuntarily. I enabled full disk encryption on my the main drive of my server, ignored all the warnings and then noticed, that the whole drive was formatted.
Luckily, all the important data was in my backups, I lost some compose files and the database of my paperless ngx seems to be broken (although theoretically, I am sure I could recover that data, I just don't have that time and there is nothing that important on there).
How do you check?
You reapply/ reload database and do all the recovery procedure on a clean new installation for everything. Disaster recovery is much a thing together with backup. Depends on your infra how to proceed in concrete steps.
rm -rf /(don't actually do this obv)
Here, you dropped this.
--no-preserve-root
Its OK, root was preserved
Restore a backup and make sure everything is where you left it.
yep! restore a lxc on the backup machine and shut down the main one for a second
Thats good. So many places I see just accept that backup job completed without errors means backups are ok.
What are you using to run backups?
proxmox style
apologies if that is not a lot of info!
Proxmox VM snapshots and backups, plus proxsave I assume?
Huge! Feels good, huh?