2019-01-11 15:31:02 +00:00
|
|
|
Motivation:
|
|
|
|
|
All the backup systems are either slow or crashing or both on my backup.
|
|
|
|
|
|
|
|
|
|
Tried duply:
|
|
|
|
|
* works but very slow:
|
|
|
|
|
```
|
|
|
|
|
--------------[ Backup Statistics ]--------------
|
|
|
|
|
StartTime 1547198362.85 (Fri Jan 11 09:19:22 2019)
|
|
|
|
|
EndTime 1547209509.04 (Fri Jan 11 12:25:09 2019)
|
|
|
|
|
ElapsedTime 11146.19 (3 hours 5 minutes 46.19 seconds)
|
|
|
|
|
SourceFiles 3065438
|
|
|
|
|
SourceFileSize 585041709586 (545 GB)
|
|
|
|
|
NewFiles 0
|
|
|
|
|
NewFileSize 0 (0 bytes)
|
|
|
|
|
DeletedFiles 0
|
|
|
|
|
ChangedFiles 0
|
|
|
|
|
ChangedFileSize 0 (0 bytes)
|
|
|
|
|
ChangedDeltaSize 0 (0 bytes)
|
|
|
|
|
DeltaEntries 0
|
|
|
|
|
RawDeltaSize 0 (0 bytes)
|
|
|
|
|
TotalDestinationSizeChange 111 (111 bytes)
|
|
|
|
|
Errors 0
|
|
|
|
|
-------------------------------------------------
|
|
|
|
|
|
|
|
|
|
--- Finished state OK at 12:25:15.000 - Runtime 03:06:43.000 ---
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Tried restic:
|
|
|
|
|
* crashes with OOM
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Goals for bakare:
|
2018-08-17 17:58:18 +01:00
|
|
|
* fast
|
|
|
|
|
* using max bandwidth
|
2019-01-12 14:50:23 +00:00
|
|
|
* use max cpu
|
|
|
|
|
* use max disk I/O
|
2018-08-17 17:58:18 +01:00
|
|
|
* memory usage limit
|
|
|
|
|
* encryption by default - asymmetric, creates a keypair for you
|
2019-01-12 14:50:23 +00:00
|
|
|
* deduplication of file data
|
2018-08-18 18:39:40 +01:00
|
|
|
* fuzzy find by file name in stored files
|
2019-01-12 14:50:23 +00:00
|
|
|
* failure to process one file should not affect any other files
|
2019-02-18 11:38:20 +00:00
|
|
|
* intermittent network failures should not make the whole process fail (test with random packet drop)
|
2018-08-17 17:58:18 +01:00
|
|
|
|
2018-08-18 18:39:40 +01:00
|
|
|
Implementation:
|
2019-01-12 14:50:23 +00:00
|
|
|
* test with randomly created dirs and files, with property based tests and fuzzer
|
2019-02-18 11:38:20 +00:00
|
|
|
* see if we can use `salsa` for recomputaiton
|
