Commit Graph

184 Commits

Author SHA1 Message Date
Erwan LE PRADO
3661a4b49b
feat: Add storage class header (#119)
* feat: Add storage class header

* doc: change the readme

* chore: Remove the unnecessary default  value
2022-07-06 13:18:12 +02:00
e738bd0539
Make crond log to stderr so Docker can forward it (#120) 2022-07-06 13:16:43 +02:00
342ae5910e
Add env template helper (#121) 2022-07-06 13:16:32 +02:00
c2a8cc92fc
Untangle tests (#112)
* Isolate S3 test case

* Isolate webdav test case

* Isolate SSH test case

* Isolate local storage test case

* Isolate gpg test case

* Add missing volume mount

* Fix file locations for local test case

* Remove compose test case, use utils

* Use test utils throughout

* Use dedicated tmp dir

* Fix link location that is being tested

* Use dedicated tmp_dirs when working on host fs

* Force delete artifact

* Fix expected filename

* Provide helpful messages on failing tests

* Fix filename

* Use proper volume names

* Fix syntax error, use large resource class

* Use named Docker volumes when referencing them in test scripts

* Add name of test case to logging output
2022-06-23 14:40:29 +02:00
1892d56ff6
Change default value for SSH identity file (#108)
* Change default value for SSH identity file

* Force remove write protected file in tests
2022-06-17 11:28:29 +02:00
İbrahim Akyel
0b205fe6dc
SSH Backup Storage Support (#107)
* SSH Client implemented

* Private key auth implemented
Code refactoring

* Refactoring

* Passphrase renamed to IdentityPassphrase
Default private key location changed to .ssh/id
2022-06-17 11:06:15 +02:00
8c8a2fa088
Update vulnerable containerd dependency (#104) 2022-06-07 09:21:40 +02:00
a850bf13fe Fix broken link in README 2022-05-12 08:18:12 +02:00
b52b271bac
Allow for the exclusion of files from backups (#100)
* Hoist walking of files so it can be used for features other than archive creation

* Add option to ignore files from backup using glob patterns

* Use Regexp instead of glob for exclusion

* Ignore artifacts

* Add teardown to test

* Allow single Re for filtering only

* Add documentation

* Use MatchString on re, add bad input to message in case of error
2022-05-08 11:20:38 +02:00
cac5777e79 Add documentation on using multiple configs for complex retention schemes 2022-04-20 18:01:41 +02:00
94a1edc4ad
Allow disabling of certificate verification for WebDAV (#98) 2022-04-20 14:16:59 +02:00
a654097e59
Update webdav client library (#97) 2022-04-20 10:56:26 +02:00
1b1fc4856c
List objects recursively when selecting candidates from S3 (#92) 2022-04-15 11:05:52 +02:00
e81c34b8fc
Consider S3 Path when selecting candidates for pruning (#91) 2022-04-13 17:09:37 +02:00
Simon Dünhöft
9c23767fce
Fixed wrong env name for S3 bucket in README (#89)
The README was using `AWS_BUCKET_NAME` instead of `AWS_S3_BUCKET_NAME` in the recipes. 
This resulted in no data being uploaded to S3.
2022-04-12 19:38:15 +02:00
51af8c3c77
Deprecate BACKUP_FROM_SNAPSHOT (#81) 2022-03-25 18:28:58 +01:00
1ea0b51b23
Tag releases with major version too (#82) 2022-03-25 18:27:00 +01:00
da8c63f755
Support identical cron schedule (#87)
* Retry on lock being unavailable

* Refactor locking to return plain error

* Collect LockedTime in stats

* Add test case

* Add documentation for LOCK_TIMEOUT

* Log in case lock needs to be awaited

* Release resources created for awaiting lock
2022-03-25 18:26:34 +01:00
9bc8db0f7c
Build using Go 1.18 (#86) 2022-03-17 11:22:41 +01:00
508bc07b4f
Disable healthcheck in swarm test (#85) 2022-03-17 11:13:07 +01:00
b8f71b04a1
Use errgroup for running commands in parallel (#83) 2022-03-10 11:09:39 +01:00
5f3832d621
Consider prefix rules when pruning WebDAV storages (#79) 2022-03-05 13:33:15 +01:00
4b1127b8c4 Document BACKUP_SOURCES 2022-03-04 19:57:32 +01:00
ae50a3ac4f Add attribution to code taken from moby repository 2022-03-04 16:40:34 +01:00
bad22eee93 Fix syntax highlighting in container 2022-03-04 14:08:31 +01:00
c9ebb9e14e
Allow multiple schedules in the same container (#78)
* Allow mounting of config directory for multiple schedules

* Add docs for conf.d feature

* Fix behavior on multiple files

* Define default case first in entrypoint script
2022-03-04 13:51:26 +01:00
6e1b8553e6 Remove superfluous --update flag from cert install 2022-02-26 16:45:29 +01:00
5ec2b2c3ff Install ca-certs with --no-cache to reduce image size 2022-02-25 08:54:07 +01:00
Rajan Patel
3bbeba5b83
update custom docker host documentation for pre/post commands (#77) 2022-02-24 05:31:36 +01:00
9155b4d130 Add missing print directive, fix go.mod 2022-02-23 10:12:57 +01:00
Kazi
2a17e84ab6
snapshot-style restore example (#76)
* snapshot-style restore example

* manual backup recommendation
2022-02-23 07:58:09 +01:00
Rajan Patel
00f2359461
Add DOCKER_HOST documentation (#74)
* add DOCKER_HOST documentation

* add which endpoints are required for DOCKER_HOST

* Update README.md

Co-authored-by: Frederik Ring <frederik.ring@gmail.com>
2022-02-22 08:00:26 +01:00
0504a92a1f
Add option to run pre/post commands for any container (#73)
* Add option to run pre commands on arbitrary container

* Correctly handle quoted args in commands

* Provide defaults for test version arg

* Allow filtering of target containers

* Add documentation on exec commands

* Use mysqldump in exec test

* Add mysqldump section to recipes

* Also run commands test in swarm mode

* Use name instead of id

* Add syntax highlighting

* Add missing license headers
2022-02-22 07:53:33 +01:00
3ded77448c
Do not skip directories when creating tar archive (#72)
* Update targz library to include potential ownership fix

* Move archive logic to main repo

* Remove assertions for debugging

* Use relative path in assertion

* Strip local part from archive location

* Log when extracting in tests

* Fix trimming of prfix

* Add license info to archive.go file

* Undo change in test assertion

* Add test checking for preserved file ownership

* use same postgres version in tests

* Wrap errors when archiving, handle deletion at script layer
2022-02-22 07:49:24 +01:00
58b42b9036
Supporting proxied Docker APIs through DOCKER_HOST (#70) 2022-02-18 09:08:21 +01:00
180438f1fc
Update ubuntu image used for running integration tests (#67) 2022-02-15 21:05:03 +01:00
Mauro Molin
30265c14ba
Fixed TookTime (#66) 2022-02-14 17:32:05 +01:00
a57e93d01e
Split source into multiple files, deduplicate pruning logic, do not parse templates when notifications are not used (#63)
* Split code into multiple files

* Deduplicate logic for pruning backups against different storages

* Only parse templates when notifications are enabled

* Use better description
2022-02-13 10:52:19 +01:00
3e17d1b123
Ensure end time is recorded for unsuccessful runs too (#62)
* Ensure end time is also recorded for unsuccessful runs

* Clean up integration tests
2022-02-13 09:41:36 +01:00
0e248010a8 Add note about how notifications can be customized to config reference 2022-02-12 20:34:51 +01:00
e6af6efd8a
Add test setup for notification feature (#61)
* Add test case for notification feature

* Fix template data

* bash is needed to interpret test

* Do not use bashisms in test

* Only print FullPath

* Fix assertion
2022-02-12 20:28:38 +01:00
34d04211eb Update README TOC 2022-02-11 20:06:23 +01:00
Mauro Molin
8dfdd14527
Added custom notification messages using text/template (#60)
* Added custom notification messages using text/template

* Change notification template path and removed automatic newline trim

* Added stats and changed structure of template params

* Stat file hotfix

* Embedded and fixed default notification templates


Fix

* Changed Output to LogOutput

* Changed stats integer to unsigned

* Bytes formatting in template func


fix

* Changed Archives to Storages

* Removed unecessary sleep for pruning leeway

* Set EndTime after pruning is completed

* Added custom notifications documentation

* Added 5s sleep in swarm test

* Fixed documentation

* Dockerfile copies all files in cmd/backup
2022-02-11 20:05:16 +01:00
3bb99a7117 Update package targz 2022-02-08 15:12:46 +01:00
Fridgemagnet
ddc34be55d
Updated README.md "Restoring..." section example (#56)
Edited the "Restoring a volume from a backup" section example to be able to better differentiate between the names of the temporary restore container and the name of the mounted restore volume.
New example container name: temp_restore_container
The restore volume name remains the same: backup_restore

Also changed "one-off container" to "once-off container".
2022-02-04 11:52:59 +01:00
Joshua Noble
cb9b4bfcff
Add support for Filebase (#54) 2022-02-02 17:22:42 +01:00
62bd2f4a5a
Update base docker image to alpine 3.15 (#53) 2022-01-27 14:40:56 +01:00
6fe629ce87
Allow path to be set for bucket storage (#52) 2022-01-25 21:16:16 +01:00
1db896f7cf Tweak README, improve client naming, tidy go.mod file 2022-01-22 13:35:13 +01:00
Kaerbr
6ded00aa06
Support Nextcloud / WebDav (#48)
* add studio-b12/gowebdav to be able to upload to webdav server

* make sure all env variables are present for webdav upload

* implement file upload to WebDav server

directory defaults to the base directory

* docs: add the new feature to the documentation

* if no WebDav env variable are given throw no error

* docs: use more elegant english :D

Co-authored-by: Frederik Ring <frederik.ring@gmail.com>

* docs: use official spelling of "WebDAV"

* perf: golang likes to return early instead of having an else block

* use WEBDAV_PATH instead of WEBDAV_DIRECTORY

* use split_words for more convenience

like shown here: https://github.com/kelseyhightower/envconfig#struct-tag-support

* simplify

* feat: add pruning of files in WebDAV remote

Based on / Inspired by the minio/S3 implementation of pruning remote files.

* remove logging from the development

* test: first try implementing tests

Sandly I have to use the remote pipeline -- local wont work for me.

* test: adapt used volume names

* test: specify image only once!

* test: minio AND webdav data should be present

* test: backups on WebDAV remote are laying in the root directory

* test: the webdav server stores date in /var/lib/dav

* trying with data subfolder

* test: 1 container was added so the number raised from 3 to 4

* webdav  subfolder is "data" not "backup"

* fix: password AND username must be defined

not password OR username

* improve logging

* feat: if the given path on the server isnt preset it will be created

* test: add creation of new folder for webdav to tests

Co-authored-by: Frederik Ring <frederik.ring@gmail.com>
2022-01-22 13:29:21 +01:00