Commit Graph

152 Commits

Author SHA1 Message Date
dependabot[bot]
a077f12c11
Bump google.golang.org/protobuf from 1.31.0 to 1.33.0 (#392) 2024-03-14 06:06:22 +00:00
dependabot[bot]
cb5a38a1b7
Bump github.com/minio/minio-go/v7 from 7.0.68 to 7.0.69 (#390) 2024-03-11 22:13:04 +00:00
dependabot[bot]
b8995dbc51
Bump golang.org/x/oauth2 from 0.17.0 to 0.18.0 (#389) 2024-03-11 22:11:37 +00:00
dependabot[bot]
e8562b1785
Bump github.com/minio/minio-go/v7 from 7.0.67 to 7.0.68 (#382) 2024-03-05 05:01:42 +00:00
dependabot[bot]
5d7451410b
Bump github.com/ProtonMail/go-crypto from 1.1.0-alpha.0 to 1.1.0-alpha.1 (#381) 2024-03-05 05:00:55 +00:00
dependabot[bot]
6c8b0ccce5
Bump github.com/klauspost/compress from 1.17.6 to 1.17.7 (#377) 2024-02-27 05:57:17 +00:00
060a6daa7a Use proper path expansion 2024-02-22 17:42:53 +01:00
02ba9939a2
Revert "Values without a backing env var should not be expanded (#368)" (#371)
This reverts commit 911fc5a223.
2024-02-21 18:43:02 +01:00
911fc5a223
Values without a backing env var should not be expanded (#368)
* Values without a backing env var should not be expanded

* Add unit tests for sourcing behavior

* Replace godotenv with shell lib
2024-02-21 17:44:37 +01:00
dependabot[bot]
241b5d2f25
Bump github.com/docker/cli (#353)
Bumps [github.com/docker/cli](https://github.com/docker/cli) from 24.0.1+incompatible to 24.0.9+incompatible.
- [Commits](https://github.com/docker/cli/compare/v24.0.1...v24.0.9)

---
updated-dependencies:
- dependency-name: github.com/docker/cli
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-13 09:44:32 +01:00
dependabot[bot]
aab47509d9
Bump golang.org/x/oauth2 from 0.16.0 to 0.17.0 (#355) 2024-02-13 08:33:16 +00:00
dependabot[bot]
9b52c1f63e
Bump github.com/robfig/cron/v3 from 3.0.0 to 3.0.1 (#354) 2024-02-12 21:26:49 +00:00
dependabot[bot]
164d6df3b4
Bump github.com/minio/minio-go/v7 from 7.0.66 to 7.0.67 (#352) 2024-02-12 21:26:36 +00:00
pixxon
1d45062100 Move cron scheduling inside application (#338)
* Move cron scheduling inside application

* Make envvar a fallback and check for errors

* Panic significantly less

* propagate error out of runBackup

* Add structured logging

* FIx error propagation to exit

* Enable the new scheduler by default

* Review fixes

* Added docs and better error propagation
2024-02-12 16:04:12 +01:00
dependabot[bot]
64d934102d
Bump github.com/klauspost/compress from 1.17.5 to 1.17.6 (#345) 2024-02-06 05:48:05 +00:00
c3daeacecb
Improve Swarm support (#333)
* Query for labeled services as well

* Try scaling down services

* Scale services back up

* Use progress tool from Docker CLI

* In test, label both services

* Clean up error and log messages

* Document scale-up/down approach in docs

* Downgrade Docker CLI to match client

* Document services stats

* Do not rely on PreviousSpec for storing desired replica count

* Log warnings from Docker when updating services

* Check whether container and service labels collide

* Document script behavior on label collision

* Add additional check if all containers have been removed

* Scale services concurrently

* Move docker interaction code into own file

* Factor out code for service updating

* Time out after five minutes of not reaching desired container count

* Inline handling of in-swarm container level restart

* Timer is more suitable for timeout race

* Timeout when scaling down services should be configurable

* Choose better filename

* Reflect changes in naming

* Rename and deprecate BACKUP_STOP_CONTAINER_LABEL

* Improve logging

* Further simplify logging
2024-01-31 12:17:41 +01:00
dependabot[bot]
2065fb2815
Bump github.com/klauspost/compress from 1.17.4 to 1.17.5 (#336) 2024-01-30 05:45:11 +00:00
dependabot[bot]
67e3b79709
Bump github.com/ProtonMail/go-crypto (#331) 2024-01-22 21:30:42 +00:00
dependabot[bot]
b51b25997b
Bump github.com/Azure/azure-sdk-for-go/sdk/azidentity (#330) 2024-01-22 21:30:30 +00:00
dependabot[bot]
bf44369915
Bump golang.org/x/sync from 0.5.0 to 0.6.0 (#326) 2024-01-09 06:03:13 +00:00
dependabot[bot]
7e1ee21ef9
Bump golang.org/x/oauth2 from 0.15.0 to 0.16.0 (#325) 2024-01-09 05:46:30 +00:00
dependabot[bot]
0fbc0637ed
Bump github.com/cloudflare/circl from 1.3.3 to 1.3.7 (#324) 2024-01-08 17:05:11 +00:00
06792eb1f0
Package crypto 0.16.0 is vulnerable to CVE-2023-48795 (#320) 2023-12-20 20:44:35 +01:00
dependabot[bot]
da6683a98f
Bump github.com/Azure/azure-sdk-for-go/sdk/storage/azblob (#318) 2023-12-18 21:25:04 +00:00
dependabot[bot]
be1901d181
Bump github.com/minio/minio-go/v7 from 7.0.65 to 7.0.66 (#317) 2023-12-18 21:24:27 +00:00
dependabot[bot]
84990ed6bd
Bump github.com/klauspost/compress from 1.17.3 to 1.17.4 (#311) 2023-12-05 06:40:04 +00:00
dependabot[bot]
94f0975a30
Bump github.com/minio/minio-go/v7 from 7.0.64 to 7.0.65 (#312) 2023-12-05 06:23:44 +00:00
dependabot[bot]
e5c3b47ec9
Bump github.com/minio/minio-go/v7 from 7.0.63 to 7.0.64 (#309) 2023-11-28 06:02:52 +00:00
dependabot[bot]
619624f0d0
Bump golang.org/x/oauth2 from 0.14.0 to 0.15.0 (#308) 2023-11-28 06:02:32 +00:00
dependabot[bot]
52cd70c7a9
Bump github.com/klauspost/compress from 1.17.2 to 1.17.3 (#305) 2023-11-20 21:54:22 +00:00
dependabot[bot]
55bcd90c2d
Bump golang.org/x/oauth2 from 0.13.0 to 0.14.0 (#303) 2023-11-14 05:53:23 +00:00
dependabot[bot]
382a613cbc
Bump golang.org/x/sync from 0.4.0 to 0.5.0 (#302) 2023-11-07 06:45:49 +00:00
d3e1d1531b
Bump Docker client (#296) 2023-10-30 19:10:44 +01:00
dependabot[bot]
2d81ac046b
Bump github.com/klauspost/compress from 1.17.1 to 1.17.2 (#289) 2023-10-24 05:02:31 +00:00
dependabot[bot]
e8ac4e1da6
Bump github.com/klauspost/compress from 1.17.0 to 1.17.1 (#284) 2023-10-17 06:07:47 +00:00
dependabot[bot]
3477c12b9d
Bump github.com/Azure/azure-sdk-for-go/sdk/storage/azblob (#285) 2023-10-17 05:43:33 +00:00
dependabot[bot]
264c2e3089
Bump github.com/Azure/azure-sdk-for-go/sdk/azidentity (#286) 2023-10-17 05:15:26 +00:00
dependabot[bot]
e079eeafa0
Bump golang.org/x/net from 0.16.0 to 0.17.0 (#282) 2023-10-12 04:34:56 +00:00
dependabot[bot]
e1e2843f87
Bump golang.org/x/sync from 0.3.0 to 0.4.0 (#281) 2023-10-10 05:28:48 +00:00
dependabot[bot]
2e1f65b0df
Bump golang.org/x/oauth2 from 0.12.0 to 0.13.0 (#280) 2023-10-10 05:11:52 +00:00
dependabot[bot]
e35164628c
Bump github.com/otiai10/copy from 1.11.0 to 1.14.0 (#279) 2023-10-03 07:12:56 +00:00
dependabot[bot]
40bbf2c919
Bump github.com/minio/minio-go/v7 from 7.0.62 to 7.0.63 (#274) 2023-09-19 13:01:54 +00:00
dependabot[bot]
c87dc09ad4
Bump golang.org/x/oauth2 (#273) 2023-09-19 06:39:24 +00:00
dependabot[bot]
9be3a1861b
Bump github.com/klauspost/compress from 1.16.7 to 1.17.0 (#275) 2023-09-19 06:38:39 +00:00
MaxJa4
336c5bed71
Replace Gzip with PGzip (#266)
* Replace Gzip with optimized PGzip. Add concurrency option.

* Add shortened timeout for 'dc down' too.

* Add NaturalNumberZero to allow zero.

* Add test for concurrency=0

* Rename to GZIP_PARALLELISM

* Fix block size. Fix compression level. Fix CI.

* Refactor compression writer fetching. Renamed WholeNumber
2023-09-03 16:49:52 +02:00
MaxJa4
43c4961116
Support _FILE based configuration values (#264)
* Replace envconfig with env

* Adjust config options and processing

* Added _FILE variant for all password vars.

* Try pathenvconfig

* Revert everything so far

* Use our fork of envconfig with custom lookup

* Use our fork of envconfig with custom lookup

* Test compose timeout option

* Remove secret resolving and specific _FILE config

* Fix timing issue in swarm tests

* Revert "Test compose timeout option"

This reverts commit ab50b21748, reversing
changes made to 0282514b2b.

Revert "Test compose timeout option"

This reverts commit 0282514b2b.

* Use offen/envconfig v1.5.0

* Add info about _FILE in README

* Value > File. Panic on file error. Panic on duplicate presence.

* Test panic on duplicate vars and panic on file error.
2023-08-30 20:14:24 +02:00
MaxJa4
5fcc96edf9
Cleanup: Lint warnings and deprecated packages (#263)
* Fix lint warnings and std lib deprecations

* Replace deprecated std lib with maintained drop-in replacement fork

Backwartds compatible with original package and suggested by std lib due to security and stability issues.

* OAuth2 is now a direct dependency due to Dropbox

* Undo change

* Revert "Replace deprecated std lib with maintained drop-in replacement fork"

This reverts commit 2887bd409f.

* Update channel handling

* Add linter for PRs

* Rename CI, fetch all issues, add govet
2023-08-27 18:14:55 +02:00
dependabot[bot]
b90fc9ea4d
Bump github.com/minio/minio-go/v7 from 7.0.61 to 7.0.62 (#255)
Bumps [github.com/minio/minio-go/v7](https://github.com/minio/minio-go) from 7.0.61 to 7.0.62.
- [Release notes](https://github.com/minio/minio-go/releases)
- [Commits](https://github.com/minio/minio-go/compare/v7.0.61...v7.0.62)

---
updated-dependencies:
- dependency-name: github.com/minio/minio-go/v7
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-24 19:37:50 +02:00
MaxJa4
e08a3303bf
Add new storage backend: Dropbox (#103) (#251)
* Add new storage backend: Dropbox (#103)

* Remove duplicate check

* Add concurrency level for parallel upload to dropbox.

* Fixed some instabilites. Changed default concurrency to 6.

* Added some env config vars to readme. WIP

* Wrap errors for storage backend creation.

* Fixed token issue, added OAuth2 including recipe and docs.

* Readme typo fix

* Test for dropbox integration

* Update info and TOC

* Missed a file

* Docker-compose fix

* Fix endpoint connection

* Fix container names

* Fix log fetching

* Fix log fetching (again)

* Print command output to logs

* Addressing comments part 1

* Address comments part 2

* OpenAPI Mock spec path adjusted
* Dropbox FileMetadata reflection refactored
* NaturalNumber type added

* Add OAuth2 mock server for CI testing

* Fix env name of oauth2 endpoint

* Remove hostname

* Add forgotten change to commit...

* Fix oauth2 endpoint

"Worked on my machine"

* Try again

* Try suggested hostname again

* Fix docker internal DNS resolving issues (as suggested by oauth2 mock docs)

* Add docker network, remove hostname

* Network not external

* Last hostname try

* Add more delay, add oauth2 endpoint log

* Temp CI log output of command even when failing

* Try different config and method

* Add custom server-hostname. Rename test folder to accellerate debugging

* Try that fix again

* Adding quotes

* Port fix attempt

* Try localhost

* Try extra hosts

* Change network mode

* Undo some changes

* Use static IP

* Remove specific IP binding

* Change to default net driver

* Fix static IP

* Squash for revert

* Revert "Squash for revert"

This reverts commit e9b617be9a.

* Actual fix for CI testing from #257
2023-08-24 19:33:47 +02:00
dependabot[bot]
1765b06835
Bump github.com/pkg/sftp from 1.13.5 to 1.13.6 (#248) 2023-08-15 04:38:55 +00:00
a93ff6fe09
Build in Go 1.21 (#246) 2023-08-10 16:03:59 +02:00
1c6f64e254
Current Docker client breaks in newer Go versions (#241)
* Current Docker client breaks in newer Go versions

* Cater for breaking API changes in Docker client

* Update Docker client

* Unpin Go version used for build

* Tidy sum file
2023-07-25 19:46:57 +02:00
dependabot[bot]
085d2c5dfd
Bump github.com/minio/minio-go/v7 from 7.0.59 to 7.0.61 (#240) 2023-07-24 19:16:02 +00:00
dependabot[bot]
b1382dee00
Bump github.com/Azure/azure-sdk-for-go/sdk/storage/azblob (#239) 2023-07-24 19:15:56 +00:00
dependabot[bot]
d288c87c54
Bump github.com/minio/minio-go/v7 from 7.0.58 to 7.0.59 (#238) 2023-07-04 07:53:19 +00:00
dependabot[bot]
47491439a1
Bump github.com/studio-b12/gowebdav (#235) 2023-06-27 07:46:34 +00:00
dependabot[bot]
94f71ac765
Bump github.com/minio/minio-go/v7 from 7.0.57 to 7.0.58 (#236) 2023-06-27 05:25:42 +00:00
dependabot[bot]
2addf1dd6c
Bump golang.org/x/sync from 0.2.0 to 0.3.0 (#234) 2023-06-20 12:29:11 +00:00
dependabot[bot]
c07990eaf6
Bump github.com/minio/minio-go/v7 from 7.0.56 to 7.0.57 (#233) 2023-06-20 12:28:06 +00:00
dependabot[bot]
9d5b897ab4
Bump golang.org/x/sync from 0.1.0 to 0.2.0 (#229) 2023-06-13 07:17:58 +00:00
dependabot[bot]
30bf31cd90
Bump github.com/Azure/azure-sdk-for-go/sdk/storage/azblob (#228) 2023-06-13 07:04:03 +00:00
dependabot[bot]
32e9a05b40
Bump github.com/minio/minio-go/v7 from 7.0.44 to 7.0.56 (#227) 2023-06-13 07:03:38 +00:00
dependabot[bot]
b302884447
Bump golang.org/x/crypto from 0.3.0 to 0.9.0 (#223) 2023-06-10 13:15:41 +00:00
dependabot[bot]
b3e1ce27be
Bump github.com/Azure/azure-sdk-for-go/sdk/azidentity (#225) 2023-06-10 12:57:44 +00:00
dependabot[bot]
66518ed0ff
Bump github.com/sirupsen/logrus from 1.9.0 to 1.9.3 (#226) 2023-06-10 12:57:39 +00:00
dependabot[bot]
14d966d41a
Bump github.com/otiai10/copy from 1.10.0 to 1.11.0 (#224) 2023-06-10 12:57:30 +00:00
dependabot[bot]
144e65ce6f
Bump github.com/docker/distribution (#215) 2023-05-11 21:07:45 +00:00
ba-tno
07afa53cd3
Update shoutrrr to 0.7 (#213)
* Replace docker-compose reference with docker[space]compose

* Update shoutrrr only to 0.7.1

* modules after go mod tidy

* Refer to v0.7 docs of shoutrrr

* Replace docker-compose reference with docker[space]compose

* Update shoutrrr only to 0.7.1

* modules after go mod tidy

* Refer to v0.7 docs of shoutrrr

* Remove 'v' from shoutrrr doc link
2023-04-29 20:14:04 +02:00
5715d9ff9b
Update of package copy does not fail on deleted files (#206) 2023-04-07 11:28:36 +02:00
dependabot[bot]
6ba173d916
Bump github.com/docker/docker (#205) 2023-04-05 04:58:07 +00:00
dependabot[bot]
a06ad1957a
Bump github.com/docker/distribution (#195) 2023-03-07 06:56:56 +00:00
dependabot[bot]
15786c5da3
Bump golang.org/x/net from 0.2.0 to 0.7.0 (#191) 2023-02-18 06:34:40 +00:00
dependabot[bot]
641a3203c7
Bump github.com/containerd/containerd from 1.6.6 to 1.6.18 (#190) 2023-02-16 19:32:46 +00:00
dependabot[bot]
201a983ea4
Bump github.com/containrrr/shoutrrr from 0.5.2 to 0.6.0
Bumps [github.com/containrrr/shoutrrr](https://github.com/containrrr/shoutrrr) from 0.5.2 to 0.6.0.
- [Release notes](https://github.com/containrrr/shoutrrr/releases)
- [Changelog](https://github.com/containrrr/shoutrrr/blob/main/goreleaser.yml)
- [Commits](https://github.com/containrrr/shoutrrr/compare/v0.5.2...v0.6.0)

---
updated-dependencies:
- dependency-name: github.com/containrrr/shoutrrr
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-02-14 18:57:32 +00:00
5c04e11f10
Add support for Azure Blob Storage (#171)
* Scaffold Azure storage backend that does nothing yet

* Implement copy for Azure Blob Storage

* Set up automated testing for Azure Storage

* Implement pruning for Azure blob storage

* Add documentation for Azure Blob Storage

* Add support for remote path

* Add azure to notifications doc

* Tidy go.mod file

* Allow use of managed identity credential

* Use volume in tests

* Auto append trailing slash to endpoint if needed, clarify docs, tidy mod file
2023-01-11 21:40:48 +01:00
XF
08bafdb054
Update MinIO Go SDK to v7.0.44 (#167)
Adds support for the eu-south-2, eu-central-2, me-central-1, ap-southeast-3 AWS endpoints
2022-11-21 20:30:13 +01:00
Ä°brahim Akyel
0b205fe6dc
SSH Backup Storage Support (#107)
* SSH Client implemented

* Private key auth implemented
Code refactoring

* Refactoring

* Passphrase renamed to IdentityPassphrase
Default private key location changed to .ssh/id
2022-06-17 11:06:15 +02:00
8c8a2fa088
Update vulnerable containerd dependency (#104) 2022-06-07 09:21:40 +02:00
a654097e59
Update webdav client library (#97) 2022-04-20 10:56:26 +02:00
9bc8db0f7c
Build using Go 1.18 (#86) 2022-03-17 11:22:41 +01:00
b8f71b04a1
Use errgroup for running commands in parallel (#83) 2022-03-10 11:09:39 +01:00
0504a92a1f
Add option to run pre/post commands for any container (#73)
* Add option to run pre commands on arbitrary container

* Correctly handle quoted args in commands

* Provide defaults for test version arg

* Allow filtering of target containers

* Add documentation on exec commands

* Use mysqldump in exec test

* Add mysqldump section to recipes

* Also run commands test in swarm mode

* Use name instead of id

* Add syntax highlighting

* Add missing license headers
2022-02-22 07:53:33 +01:00
3ded77448c
Do not skip directories when creating tar archive (#72)
* Update targz library to include potential ownership fix

* Move archive logic to main repo

* Remove assertions for debugging

* Use relative path in assertion

* Strip local part from archive location

* Log when extracting in tests

* Fix trimming of prfix

* Add license info to archive.go file

* Undo change in test assertion

* Add test checking for preserved file ownership

* use same postgres version in tests

* Wrap errors when archiving, handle deletion at script layer
2022-02-22 07:49:24 +01:00
a57e93d01e
Split source into multiple files, deduplicate pruning logic, do not parse templates when notifications are not used (#63)
* Split code into multiple files

* Deduplicate logic for pruning backups against different storages

* Only parse templates when notifications are enabled

* Use better description
2022-02-13 10:52:19 +01:00
3bb99a7117 Update package targz 2022-02-08 15:12:46 +01:00
Kaerbr
6ded00aa06
Support Nextcloud / WebDav (#48)
* add studio-b12/gowebdav to be able to upload to webdav server

* make sure all env variables are present for webdav upload

* implement file upload to WebDav server

directory defaults to the base directory

* docs: add the new feature to the documentation

* if no WebDav env variable are given throw no error

* docs: use more elegant english :D

Co-authored-by: Frederik Ring <frederik.ring@gmail.com>

* docs: use official spelling of "WebDAV"

* perf: golang likes to return early instead of having an else block

* use WEBDAV_PATH instead of WEBDAV_DIRECTORY

* use split_words for more convenience

like shown here: https://github.com/kelseyhightower/envconfig#struct-tag-support

* simplify

* feat: add pruning of files in WebDAV remote

Based on / Inspired by the minio/S3 implementation of pruning remote files.

* remove logging from the development

* test: first try implementing tests

Sandly I have to use the remote pipeline -- local wont work for me.

* test: adapt used volume names

* test: specify image only once!

* test: minio AND webdav data should be present

* test: backups on WebDAV remote are laying in the root directory

* test: the webdav server stores date in /var/lib/dav

* trying with data subfolder

* test: 1 container was added so the number raised from 3 to 4

* webdav  subfolder is "data" not "backup"

* fix: password AND username must be defined

not password OR username

* improve logging

* feat: if the given path on the server isnt preset it will be created

* test: add creation of new folder for webdav to tests

Co-authored-by: Frederik Ring <frederik.ring@gmail.com>
2022-01-22 13:29:21 +01:00
1f9582df51
Fix handling of empty directories (#44)
* Add test checking whether empty directories are included in backups

* Update targz library to include fix
2021-12-29 10:10:12 +01:00
08d78a0bd6 allow sending notifications to multiple channels 2021-12-18 10:31:12 +01:00
5a6ce81b58 update github.com/otai/copy, use PreserveOwner option 2021-11-29 08:40:55 +01:00
dfd0d617e4 install bugfix releases where available 2021-11-28 20:12:23 +01:00
schwannden
411c39ee72
create a snapshot before creating tar archive (#32)
* create a snapshot before creating tar archive

* safeguard snapshot removal and make snapshot optional

* fix typo, make sure remove snapshot failure triggers failure hook

Co-authored-by: Schwannden Kuo <schwannden@mobagel.com>
2021-11-08 08:39:18 +01:00
88368197c1 implement email notifications on failed backup runs 2021-09-09 09:00:23 +02:00
fefc34c6aa tidy go mod file 2021-09-04 15:54:09 +02:00
5922820ada add test for checking behavior on symlinks 2021-09-04 10:30:34 +02:00
8aba98c012 use forked version of package targz 2021-09-04 10:08:06 +02:00
e0c4adc563 move handling of config to script layer 2021-08-24 09:01:44 +02:00
2469597848 fix lockfile mechanism 2021-08-23 18:46:49 +02:00
4c80494433 use go native strftime version 2021-08-23 14:48:33 +02:00
435583168b add logging 2021-08-23 14:48:33 +02:00
f2739b583e add gpg encryption 2021-08-23 14:48:32 +02:00