Commit Graph

127 Commits

Author SHA1 Message Date
dependabot[bot]
292d47eb19
Bump github.com/docker/cli from 24.0.9+incompatible to 26.1.0+incompatible (#411)
* Bump github.com/docker/cli

Bumps [github.com/docker/cli](https://github.com/docker/cli) from 24.0.9+incompatible to 26.1.0+incompatible.
- [Commits](https://github.com/docker/cli/compare/v24.0.9...v26.1.0)

---
updated-dependencies:
- dependency-name: github.com/docker/cli
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Upgrade docker/docker to matching version

* Tidy go.mod

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Frederik Ring <frederik.ring@gmail.com>
2024-04-25 17:33:07 +02:00
f2ef48803c
Print stack trace when encountering unexpected panic (#406) 2024-04-15 15:12:10 +02:00
8b69566291
Result of query for services is used before handling possible error (#405)
* Result of query for services is used before handling possible error

* Return early when a non-replicated service is matched
2024-04-15 15:08:37 +02:00
1af472077c
Update author reference in license statements (#393) 2024-03-15 11:42:22 +01:00
baf34ec1f7
Allow authentication using connection string when targeting Azure Blob Storage (#383)
* Allow authentication using connection string when targeting Azure Blob Storage

* Bail on ambiguous configuration
2024-03-08 20:23:30 +01:00
2d3e79cf5e
Also forward exec output when failing to demultiplex (#379) 2024-03-01 09:18:39 +01:00
5abfe5bb39
Swarm mode check fails on non-standard Info responses (#376)
* Swarm mode check fails on non-standard Info responses

* Add unit test

* Remove balena tests, add note to docs
2024-02-27 21:12:36 +00:00
9b768c71e6
Lines from conf files that are comments should not be passed to shell.Expand (#374) 2024-02-23 17:53:04 +01:00
060a6daa7a Use proper path expansion 2024-02-22 17:42:53 +01:00
02ba9939a2
Revert "Values without a backing env var should not be expanded (#368)" (#371)
This reverts commit 911fc5a223.
2024-02-21 18:43:02 +01:00
911fc5a223
Values without a backing env var should not be expanded (#368)
* Values without a backing env var should not be expanded

* Add unit tests for sourcing behavior

* Replace godotenv with shell lib
2024-02-21 17:44:37 +01:00
52c22a1891 Auto prepend caller when wrapping errors 2024-02-16 20:19:58 +01:00
83fa0aae48 Refactor handling of runtime configuration to prepare for reloading 2024-02-16 20:19:58 +01:00
a01fc3df3f
Conf files should expand env vars (#363) 2024-02-15 12:04:44 +01:00
5c8bc107de
Remove stray log statement (#359) 2024-02-13 19:54:18 +01:00
9a1e885138
Env vars should propagate when using conf.d (#358)
* Extend confd test case to test for env var propagation

* Env vars set in conf.d files are expected to propagate

* Lock needs to be acquired when instantiating script
2024-02-13 15:43:04 +01:00
4c74313222 Periodically collect runtime info when requested 2024-02-12 16:04:12 +01:00
de03d4f704 Docker client expects to be closed after usage in long running program 2024-02-12 16:04:12 +01:00
65626dd3d4 Hoist control for exiting script a level up (#348)
* Hoist control for exiting script a level up

* Do not accidentally nil out errors

* Log when running schedule

* Remove duplicate log line

* Warn on cron schedule that will never run
2024-02-12 16:04:12 +01:00
pixxon
1d45062100 Move cron scheduling inside application (#338)
* Move cron scheduling inside application

* Make envvar a fallback and check for errors

* Panic significantly less

* propagate error out of runBackup

* Add structured logging

* FIx error propagation to exit

* Enable the new scheduler by default

* Review fixes

* Added docs and better error propagation
2024-02-12 16:04:12 +01:00
0f224e4fb8
Document socket-proxy permissions, return early when update failed on scaling down (#343)
* Do not await containers when there was an error on scaling

* Add test case for usage with socket proxy

* Add documentation on required permissions for docker-socket-proxy

* Add full list of used Docker APIs to doc

* CONTAINER_START and CONTAINER_STOP is not needed
2024-02-05 14:27:06 +01:00
63b545787e
Exclusive file lock is released prematurely (#339) 2024-02-01 18:14:18 +01:00
c3daeacecb
Improve Swarm support (#333)
* Query for labeled services as well

* Try scaling down services

* Scale services back up

* Use progress tool from Docker CLI

* In test, label both services

* Clean up error and log messages

* Document scale-up/down approach in docs

* Downgrade Docker CLI to match client

* Document services stats

* Do not rely on PreviousSpec for storing desired replica count

* Log warnings from Docker when updating services

* Check whether container and service labels collide

* Document script behavior on label collision

* Add additional check if all containers have been removed

* Scale services concurrently

* Move docker interaction code into own file

* Factor out code for service updating

* Time out after five minutes of not reaching desired container count

* Inline handling of in-swarm container level restart

* Timer is more suitable for timeout race

* Timeout when scaling down services should be configurable

* Choose better filename

* Reflect changes in naming

* Rename and deprecate BACKUP_STOP_CONTAINER_LABEL

* Improve logging

* Further simplify logging
2024-01-31 12:17:41 +01:00
ed5abd5ba8
Panic handling does not log reason for script being halted (#334) 2024-01-26 20:02:09 +01:00
810c8871ec
Adopt v2 API for openpgp package (#332) 2024-01-23 20:19:42 +01:00
Alexander
64daf7b132
Add method for converting object to JSON in template (#319)
* Add method for converting object to JSON in template

* Split  to ,

* Rename toJson function
2023-12-20 20:44:45 +01:00
0325889ac4
Pruning method logs nonsensical configuration values (#301)
* Pruning method logs nonsensical configuration values

* Adjust test assertion about log output
2023-11-04 12:19:44 +01:00
1d549042fc
Error message on demultiplexing command output receives bad error arg (#295) 2023-10-29 15:43:34 +01:00
MaxJa4
336c5bed71
Replace Gzip with PGzip (#266)
* Replace Gzip with optimized PGzip. Add concurrency option.

* Add shortened timeout for 'dc down' too.

* Add NaturalNumberZero to allow zero.

* Add test for concurrency=0

* Rename to GZIP_PARALLELISM

* Fix block size. Fix compression level. Fix CI.

* Refactor compression writer fetching. Renamed WholeNumber
2023-09-03 16:49:52 +02:00
MaxJa4
43c4961116
Support _FILE based configuration values (#264)
* Replace envconfig with env

* Adjust config options and processing

* Added _FILE variant for all password vars.

* Try pathenvconfig

* Revert everything so far

* Use our fork of envconfig with custom lookup

* Use our fork of envconfig with custom lookup

* Test compose timeout option

* Remove secret resolving and specific _FILE config

* Fix timing issue in swarm tests

* Revert "Test compose timeout option"

This reverts commit ab50b21748, reversing
changes made to 0282514b2b.

Revert "Test compose timeout option"

This reverts commit 0282514b2b.

* Use offen/envconfig v1.5.0

* Add info about _FILE in README

* Value > File. Panic on file error. Panic on duplicate presence.

* Test panic on duplicate vars and panic on file error.
2023-08-30 20:14:24 +02:00
MaxJa4
ad4e2af83f
Exclude specific backends from pruning (#262)
* Skip backends while pruning

* Add pruning test step and silence download log for better readability

* Add test cases for pruning in all backends

Also add -q or --quiet-pull to all tests.

* Add test case for skipping backends while pruning

* Adjusted test logging, generate new test spec file

* Gitignore for temp test file
2023-08-27 19:19:11 +02:00
MaxJa4
5fcc96edf9
Cleanup: Lint warnings and deprecated packages (#263)
* Fix lint warnings and std lib deprecations

* Replace deprecated std lib with maintained drop-in replacement fork

Backwartds compatible with original package and suggested by std lib due to security and stability issues.

* OAuth2 is now a direct dependency due to Dropbox

* Undo change

* Revert "Replace deprecated std lib with maintained drop-in replacement fork"

This reverts commit 2887bd409f.

* Update channel handling

* Add linter for PRs

* Rename CI, fetch all issues, add govet
2023-08-27 18:14:55 +02:00
3d7677f02a
Print context in log field instead of prepending to message (#260)
* Print context in log field instead of prepending to message

* Log messages on pruning do not need a description anymore

* Remove redundant information from logs and errors
2023-08-25 12:44:43 +02:00
MaxJa4
e08a3303bf
Add new storage backend: Dropbox (#103) (#251)
* Add new storage backend: Dropbox (#103)

* Remove duplicate check

* Add concurrency level for parallel upload to dropbox.

* Fixed some instabilites. Changed default concurrency to 6.

* Added some env config vars to readme. WIP

* Wrap errors for storage backend creation.

* Fixed token issue, added OAuth2 including recipe and docs.

* Readme typo fix

* Test for dropbox integration

* Update info and TOC

* Missed a file

* Docker-compose fix

* Fix endpoint connection

* Fix container names

* Fix log fetching

* Fix log fetching (again)

* Print command output to logs

* Addressing comments part 1

* Address comments part 2

* OpenAPI Mock spec path adjusted
* Dropbox FileMetadata reflection refactored
* NaturalNumber type added

* Add OAuth2 mock server for CI testing

* Fix env name of oauth2 endpoint

* Remove hostname

* Add forgotten change to commit...

* Fix oauth2 endpoint

"Worked on my machine"

* Try again

* Try suggested hostname again

* Fix docker internal DNS resolving issues (as suggested by oauth2 mock docs)

* Add docker network, remove hostname

* Network not external

* Last hostname try

* Add more delay, add oauth2 endpoint log

* Temp CI log output of command even when failing

* Try different config and method

* Add custom server-hostname. Rename test folder to accellerate debugging

* Try that fix again

* Adding quotes

* Port fix attempt

* Try localhost

* Try extra hosts

* Change network mode

* Undo some changes

* Use static IP

* Remove specific IP binding

* Change to default net driver

* Fix static IP

* Squash for revert

* Revert "Squash for revert"

This reverts commit e9b617be9a.

* Actual fix for CI testing from #257
2023-08-24 19:33:47 +02:00
MaxJa4
47326c7c59
Fix storage backends not outputting any info logs (#250) 2023-08-20 20:35:25 +02:00
Michal Middleton
67e7288855
Add support for zstd compression (#249)
Co-authored-by: Michal Middleton <jafa81@gmail.com>
2023-08-19 19:20:13 +02:00
67d978f515
Drop logrus dependency, log using slog package from stdlib (#247) 2023-08-10 19:41:03 +02:00
1c6f64e254
Current Docker client breaks in newer Go versions (#241)
* Current Docker client breaks in newer Go versions

* Cater for breaking API changes in Docker client

* Update Docker client

* Unpin Go version used for build

* Tidy sum file
2023-07-25 19:46:57 +02:00
Erwan LE PRADO
5ea9a7ce15
feat: add better handler for part size (#214)
* feat: add better handler for part size


fix: use local file 


fix: try with another path


fix: use bytes 


chore: go back


go back readme


goback


goback


goback

* chore: better handling

* fix: typo readme

* chore: wrong comparaison

* fix: typo
2023-06-02 16:30:02 +02:00
37c01a578c
TaskTemplate.ForceUpdate is a counter (#209) 2023-04-26 08:45:12 +02:00
Diulgher Artiom
7f20036b15
Possibility to use -u (user) option in docker exec (#203)
* Add user option for docker exec

* Add test for user option

* Return test version for image

* remove gitea config file

* refactor tests

* remove comments & fix image name

* add docs

* cleanup

* Update README.md with suggested correction

Co-authored-by: Frederik Ring <frederik.ring@gmail.com>

* fix backup command & bind folder instead of volume

---------

Co-authored-by: tao <generaltao.md@gmail.com>
Co-authored-by: Frederik Ring <frederik.ring@gmail.com>
2023-04-02 11:12:10 +02:00
66ad124ddd any can be used instead of interface{} 2023-03-16 19:48:12 +01:00
2d37e08743
Use go 1.20, join errors using stdlib (#182)
* Use go 1.20, join errors using stdlib

* Use go 1.20 proper
2023-02-02 21:07:25 +01:00
5c04e11f10
Add support for Azure Blob Storage (#171)
* Scaffold Azure storage backend that does nothing yet

* Implement copy for Azure Blob Storage

* Set up automated testing for Azure Storage

* Implement pruning for Azure blob storage

* Add documentation for Azure Blob Storage

* Add support for remote path

* Add azure to notifications doc

* Tidy go.mod file

* Allow use of managed identity credential

* Use volume in tests

* Auto append trailing slash to endpoint if needed, clarify docs, tidy mod file
2023-01-11 21:40:48 +01:00
63961cd826
Pass file location to lifecycle commands (#173)
* Add test case for extending image and calling through to rsync

* Keep backup file location env var

* Add documentation

* Work against untared content in test
2022-12-30 16:07:34 +01:00
9534cde7d9
Allow use of a custom ca cert when working against S3 storages (#170) 2022-12-22 14:37:51 +01:00
5c7856feb3 Consider failed casting to error response, use established minio bootstrap in tests 2022-10-13 19:40:41 +02:00
pixxon
b5cc1262e2
add aws secret handling (#161)
* add aws secret handling

* make it look go-ish

* fix tests

* whitespace

* sleep a bit
2022-10-12 19:14:57 +02:00
00c83dfac7 Fix more error strings 2022-09-15 10:49:45 +02:00
a6ec128cab
Run copying and pruning against multiple storages in parallel (#152) 2022-09-01 14:38:04 +02:00