2019-01-11 15:31:02 +00:00
Motivation:
All the backup systems are either slow or crashing or both on my backup.
Tried duply:
2019-02-25 21:51:56 +00:00
< details >
< summary > Works but is very slow< / summary >
2019-01-11 15:31:02 +00:00
```
--------------[ Backup Statistics ]--------------
StartTime 1547198362.85 (Fri Jan 11 09:19:22 2019)
EndTime 1547209509.04 (Fri Jan 11 12:25:09 2019)
ElapsedTime 11146.19 (3 hours 5 minutes 46.19 seconds)
SourceFiles 3065438
SourceFileSize 585041709586 (545 GB)
NewFiles 0
NewFileSize 0 (0 bytes)
DeletedFiles 0
ChangedFiles 0
ChangedFileSize 0 (0 bytes)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 0
RawDeltaSize 0 (0 bytes)
TotalDestinationSizeChange 111 (111 bytes)
Errors 0
-------------------------------------------------
--- Finished state OK at 12:25:15.000 - Runtime 03:06:43.000 ---
```
2019-02-25 21:51:56 +00:00
< / details >
2019-01-11 15:31:02 +00:00
Tried restic:
* crashes with OOM
Goals for bakare:
2018-08-17 17:58:18 +01:00
* fast
* using max bandwidth
2019-01-12 14:50:23 +00:00
* use max cpu
* use max disk I/O
2018-08-17 17:58:18 +01:00
* memory usage limit
* encryption by default - asymmetric, creates a keypair for you
2019-01-12 14:50:23 +00:00
* deduplication of file data
2018-08-18 18:39:40 +01:00
* fuzzy find by file name in stored files
2019-01-12 14:50:23 +00:00
* failure to process one file should not affect any other files
2019-02-18 11:38:20 +00:00
* intermittent network failures should not make the whole process fail (test with random packet drop)
2018-08-17 17:58:18 +01:00
2019-02-18 11:40:05 +00:00
Nice to have:
2019-02-25 21:49:47 +00:00
* daemon that listens for file events and updates a list of files to be backed up on the next backup run - or a `continous backup` mode - the daemon uploads the file whenever it sees the change
* peer2peer mode - people storing encrypted backups for each other
2019-02-25 22:24:55 +00:00
* relay mode, where daemon works on one or more central points with local storage (e.g. NAS) and various computers sync with that central location. Then though the central locaiton uploads everything to the other location, typically the cloud.
2019-02-18 11:40:05 +00:00
2018-08-18 18:39:40 +01:00
Implementation:
2019-01-12 14:50:23 +00:00
* test with randomly created dirs and files, with property based tests and fuzzer
2019-02-18 11:38:20 +00:00
* see if we can use `salsa` for recomputaiton
2019-02-25 21:49:47 +00:00
* index corruption tests - mutate random byte and see if everything is readable
* network packet drop tests
