docker-volume-backup/cmd/backup/script.go

531 lines
16 KiB
Go
Raw Normal View History

// Copyright 2022 - Offen Authors <hioffen@posteo.de>
// SPDX-License-Identifier: MPL-2.0
package main
import (
"bytes"
"fmt"
"io"
"io/fs"
"log/slog"
"os"
"path"
"path/filepath"
"slices"
"strings"
"text/template"
"time"
Added abstract helper interface for all storage backends (#135) * Added abstract helper interface and implemented it for all storage backends * Moved storage client initializations also to helper classes * Fixed ssh init issue * Moved script parameter to helper struct to simplify script init. * Created sub modules. Enhanced abstract implementation. * Fixed config issue * Fixed declaration issues. Added config to interface. * Added StorageProviders to unify all backends. * Cleanup, optimizations, comments. * Applied discussed changes. See description. Moved modules to internal packages. Replaced StoragePool with slice. Moved conditional for init of storage backends back to script. * Fix docker build issue * Fixed accidentally removed local copy condition. * Delete .gitignore * Renaming/changes according to review Renamed Init functions and interface. Replaced config object with specific config values. Init func returns interface instead of struct. Removed custom import names where possible. * Fixed auto-complete error. * Combined copy instructions into one layer. * Added logging func for storages. * Introduced logging func for errors too. * Missed an error message * Moved config back to main. Optimized prune stats handling. * Move stats back to main package * Code doc stuff * Apply changes from #136 * Replace name field with function. * Changed receiver names from stg to b. * Renamed LogFuncDef to Log * Removed redundant package name. * Renamed storagePool to storages. * Simplified creation of new storage backend. * Added initialization for storage stats map. * Invert .dockerignore patterns. * Fix package typo
2022-08-18 08:52:09 +02:00
"github.com/offen/docker-volume-backup/internal/storage"
"github.com/offen/docker-volume-backup/internal/storage/azure"
Add new storage backend: Dropbox (#103) (#251) * Add new storage backend: Dropbox (#103) * Remove duplicate check * Add concurrency level for parallel upload to dropbox. * Fixed some instabilites. Changed default concurrency to 6. * Added some env config vars to readme. WIP * Wrap errors for storage backend creation. * Fixed token issue, added OAuth2 including recipe and docs. * Readme typo fix * Test for dropbox integration * Update info and TOC * Missed a file * Docker-compose fix * Fix endpoint connection * Fix container names * Fix log fetching * Fix log fetching (again) * Print command output to logs * Addressing comments part 1 * Address comments part 2 * OpenAPI Mock spec path adjusted * Dropbox FileMetadata reflection refactored * NaturalNumber type added * Add OAuth2 mock server for CI testing * Fix env name of oauth2 endpoint * Remove hostname * Add forgotten change to commit... * Fix oauth2 endpoint "Worked on my machine" * Try again * Try suggested hostname again * Fix docker internal DNS resolving issues (as suggested by oauth2 mock docs) * Add docker network, remove hostname * Network not external * Last hostname try * Add more delay, add oauth2 endpoint log * Temp CI log output of command even when failing * Try different config and method * Add custom server-hostname. Rename test folder to accellerate debugging * Try that fix again * Adding quotes * Port fix attempt * Try localhost * Try extra hosts * Change network mode * Undo some changes * Use static IP * Remove specific IP binding * Change to default net driver * Fix static IP * Squash for revert * Revert "Squash for revert" This reverts commit e9b617be9a15016f8abb724709a6a1aef321cdb9. * Actual fix for CI testing from #257
2023-08-24 19:33:47 +02:00
"github.com/offen/docker-volume-backup/internal/storage/dropbox"
Added abstract helper interface for all storage backends (#135) * Added abstract helper interface and implemented it for all storage backends * Moved storage client initializations also to helper classes * Fixed ssh init issue * Moved script parameter to helper struct to simplify script init. * Created sub modules. Enhanced abstract implementation. * Fixed config issue * Fixed declaration issues. Added config to interface. * Added StorageProviders to unify all backends. * Cleanup, optimizations, comments. * Applied discussed changes. See description. Moved modules to internal packages. Replaced StoragePool with slice. Moved conditional for init of storage backends back to script. * Fix docker build issue * Fixed accidentally removed local copy condition. * Delete .gitignore * Renaming/changes according to review Renamed Init functions and interface. Replaced config object with specific config values. Init func returns interface instead of struct. Removed custom import names where possible. * Fixed auto-complete error. * Combined copy instructions into one layer. * Added logging func for storages. * Introduced logging func for errors too. * Missed an error message * Moved config back to main. Optimized prune stats handling. * Move stats back to main package * Code doc stuff * Apply changes from #136 * Replace name field with function. * Changed receiver names from stg to b. * Renamed LogFuncDef to Log * Removed redundant package name. * Renamed storagePool to storages. * Simplified creation of new storage backend. * Added initialization for storage stats map. * Invert .dockerignore patterns. * Fix package typo
2022-08-18 08:52:09 +02:00
"github.com/offen/docker-volume-backup/internal/storage/local"
"github.com/offen/docker-volume-backup/internal/storage/s3"
"github.com/offen/docker-volume-backup/internal/storage/ssh"
"github.com/offen/docker-volume-backup/internal/storage/webdav"
openpgp "github.com/ProtonMail/go-crypto/openpgp/v2"
"github.com/containrrr/shoutrrr"
"github.com/containrrr/shoutrrr/pkg/router"
"github.com/docker/docker/client"
"github.com/leekchan/timeutil"
"github.com/offen/envconfig"
"github.com/otiai10/copy"
"golang.org/x/sync/errgroup"
)
// script holds all the stateful information required to orchestrate a
// single backup run.
type script struct {
Added abstract helper interface for all storage backends (#135) * Added abstract helper interface and implemented it for all storage backends * Moved storage client initializations also to helper classes * Fixed ssh init issue * Moved script parameter to helper struct to simplify script init. * Created sub modules. Enhanced abstract implementation. * Fixed config issue * Fixed declaration issues. Added config to interface. * Added StorageProviders to unify all backends. * Cleanup, optimizations, comments. * Applied discussed changes. See description. Moved modules to internal packages. Replaced StoragePool with slice. Moved conditional for init of storage backends back to script. * Fix docker build issue * Fixed accidentally removed local copy condition. * Delete .gitignore * Renaming/changes according to review Renamed Init functions and interface. Replaced config object with specific config values. Init func returns interface instead of struct. Removed custom import names where possible. * Fixed auto-complete error. * Combined copy instructions into one layer. * Added logging func for storages. * Introduced logging func for errors too. * Missed an error message * Moved config back to main. Optimized prune stats handling. * Move stats back to main package * Code doc stuff * Apply changes from #136 * Replace name field with function. * Changed receiver names from stg to b. * Renamed LogFuncDef to Log * Removed redundant package name. * Renamed storagePool to storages. * Simplified creation of new storage backend. * Added initialization for storage stats map. * Invert .dockerignore patterns. * Fix package typo
2022-08-18 08:52:09 +02:00
cli *client.Client
storages []storage.Backend
logger *slog.Logger
Added abstract helper interface for all storage backends (#135) * Added abstract helper interface and implemented it for all storage backends * Moved storage client initializations also to helper classes * Fixed ssh init issue * Moved script parameter to helper struct to simplify script init. * Created sub modules. Enhanced abstract implementation. * Fixed config issue * Fixed declaration issues. Added config to interface. * Added StorageProviders to unify all backends. * Cleanup, optimizations, comments. * Applied discussed changes. See description. Moved modules to internal packages. Replaced StoragePool with slice. Moved conditional for init of storage backends back to script. * Fix docker build issue * Fixed accidentally removed local copy condition. * Delete .gitignore * Renaming/changes according to review Renamed Init functions and interface. Replaced config object with specific config values. Init func returns interface instead of struct. Removed custom import names where possible. * Fixed auto-complete error. * Combined copy instructions into one layer. * Added logging func for storages. * Introduced logging func for errors too. * Missed an error message * Moved config back to main. Optimized prune stats handling. * Move stats back to main package * Code doc stuff * Apply changes from #136 * Replace name field with function. * Changed receiver names from stg to b. * Renamed LogFuncDef to Log * Removed redundant package name. * Renamed storagePool to storages. * Simplified creation of new storage backend. * Added initialization for storage stats map. * Invert .dockerignore patterns. * Fix package typo
2022-08-18 08:52:09 +02:00
sender *router.ServiceRouter
template *template.Template
hooks []hook
hookLevel hookLevel
file string
stats *Stats
encounteredLock bool
c *Config
}
// newScript creates all resources needed for the script to perform actions against
// remote resources like the Docker engine or remote storage locations. All
// reading from env vars or other configuration sources is expected to happen
// in this method.
func newScript() (*script, error) {
stdOut, logBuffer := buffer(os.Stdout)
s := &script{
c: &Config{},
logger: slog.New(slog.NewTextHandler(stdOut, nil)),
stats: &Stats{
StartTime: time.Now(),
LogOutput: logBuffer,
Added abstract helper interface for all storage backends (#135) * Added abstract helper interface and implemented it for all storage backends * Moved storage client initializations also to helper classes * Fixed ssh init issue * Moved script parameter to helper struct to simplify script init. * Created sub modules. Enhanced abstract implementation. * Fixed config issue * Fixed declaration issues. Added config to interface. * Added StorageProviders to unify all backends. * Cleanup, optimizations, comments. * Applied discussed changes. See description. Moved modules to internal packages. Replaced StoragePool with slice. Moved conditional for init of storage backends back to script. * Fix docker build issue * Fixed accidentally removed local copy condition. * Delete .gitignore * Renaming/changes according to review Renamed Init functions and interface. Replaced config object with specific config values. Init func returns interface instead of struct. Removed custom import names where possible. * Fixed auto-complete error. * Combined copy instructions into one layer. * Added logging func for storages. * Introduced logging func for errors too. * Missed an error message * Moved config back to main. Optimized prune stats handling. * Move stats back to main package * Code doc stuff * Apply changes from #136 * Replace name field with function. * Changed receiver names from stg to b. * Renamed LogFuncDef to Log * Removed redundant package name. * Renamed storagePool to storages. * Simplified creation of new storage backend. * Added initialization for storage stats map. * Invert .dockerignore patterns. * Fix package typo
2022-08-18 08:52:09 +02:00
Storages: map[string]StorageStats{
Add new storage backend: Dropbox (#103) (#251) * Add new storage backend: Dropbox (#103) * Remove duplicate check * Add concurrency level for parallel upload to dropbox. * Fixed some instabilites. Changed default concurrency to 6. * Added some env config vars to readme. WIP * Wrap errors for storage backend creation. * Fixed token issue, added OAuth2 including recipe and docs. * Readme typo fix * Test for dropbox integration * Update info and TOC * Missed a file * Docker-compose fix * Fix endpoint connection * Fix container names * Fix log fetching * Fix log fetching (again) * Print command output to logs * Addressing comments part 1 * Address comments part 2 * OpenAPI Mock spec path adjusted * Dropbox FileMetadata reflection refactored * NaturalNumber type added * Add OAuth2 mock server for CI testing * Fix env name of oauth2 endpoint * Remove hostname * Add forgotten change to commit... * Fix oauth2 endpoint "Worked on my machine" * Try again * Try suggested hostname again * Fix docker internal DNS resolving issues (as suggested by oauth2 mock docs) * Add docker network, remove hostname * Network not external * Last hostname try * Add more delay, add oauth2 endpoint log * Temp CI log output of command even when failing * Try different config and method * Add custom server-hostname. Rename test folder to accellerate debugging * Try that fix again * Adding quotes * Port fix attempt * Try localhost * Try extra hosts * Change network mode * Undo some changes * Use static IP * Remove specific IP binding * Change to default net driver * Fix static IP * Squash for revert * Revert "Squash for revert" This reverts commit e9b617be9a15016f8abb724709a6a1aef321cdb9. * Actual fix for CI testing from #257
2023-08-24 19:33:47 +02:00
"S3": {},
"WebDAV": {},
"SSH": {},
"Local": {},
"Azure": {},
"Dropbox": {},
Added abstract helper interface for all storage backends (#135) * Added abstract helper interface and implemented it for all storage backends * Moved storage client initializations also to helper classes * Fixed ssh init issue * Moved script parameter to helper struct to simplify script init. * Created sub modules. Enhanced abstract implementation. * Fixed config issue * Fixed declaration issues. Added config to interface. * Added StorageProviders to unify all backends. * Cleanup, optimizations, comments. * Applied discussed changes. See description. Moved modules to internal packages. Replaced StoragePool with slice. Moved conditional for init of storage backends back to script. * Fix docker build issue * Fixed accidentally removed local copy condition. * Delete .gitignore * Renaming/changes according to review Renamed Init functions and interface. Replaced config object with specific config values. Init func returns interface instead of struct. Removed custom import names where possible. * Fixed auto-complete error. * Combined copy instructions into one layer. * Added logging func for storages. * Introduced logging func for errors too. * Missed an error message * Moved config back to main. Optimized prune stats handling. * Move stats back to main package * Code doc stuff * Apply changes from #136 * Replace name field with function. * Changed receiver names from stg to b. * Renamed LogFuncDef to Log * Removed redundant package name. * Renamed storagePool to storages. * Simplified creation of new storage backend. * Added initialization for storage stats map. * Invert .dockerignore patterns. * Fix package typo
2022-08-18 08:52:09 +02:00
},
},
}
s.registerHook(hookLevelPlumbing, func(error) error {
s.stats.EndTime = time.Now()
2022-02-14 17:32:05 +01:00
s.stats.TookTime = s.stats.EndTime.Sub(s.stats.StartTime)
return nil
})
envconfig.Lookup = func(key string) (string, bool) {
value, okValue := os.LookupEnv(key)
location, okFile := os.LookupEnv(key + "_FILE")
switch {
case okValue && !okFile: // only value
return value, true
case !okValue && okFile: // only file
contents, err := os.ReadFile(location)
if err != nil {
s.must(fmt.Errorf("newScript: failed to read %s! Error: %s", location, err))
return "", false
}
return string(contents), true
case okValue && okFile: // both
s.must(fmt.Errorf("newScript: both %s and %s are set!", key, key+"_FILE"))
return "", false
default: // neither, ignore
return "", false
}
}
if err := envconfig.Process("", s.c); err != nil {
return nil, fmt.Errorf("newScript: failed to process configuration values: %w", err)
}
s.file = path.Join("/tmp", s.c.BackupFilename)
tmplFileName, tErr := template.New("extension").Parse(s.file)
if tErr != nil {
return nil, fmt.Errorf("newScript: unable to parse backup file extension template: %w", tErr)
}
var bf bytes.Buffer
if tErr := tmplFileName.Execute(&bf, map[string]string{
"Extension": fmt.Sprintf("tar.%s", s.c.BackupCompression),
}); tErr != nil {
return nil, fmt.Errorf("newScript: error executing backup file extension template: %w", tErr)
}
s.file = bf.String()
if s.c.BackupFilenameExpand {
s.file = os.ExpandEnv(s.file)
s.c.BackupLatestSymlink = os.ExpandEnv(s.c.BackupLatestSymlink)
s.c.BackupPruningPrefix = os.ExpandEnv(s.c.BackupPruningPrefix)
}
s.file = timeutil.Strftime(&s.stats.StartTime, s.file)
_, err := os.Stat("/var/run/docker.sock")
_, dockerHostSet := os.LookupEnv("DOCKER_HOST")
if !os.IsNotExist(err) || dockerHostSet {
cli, err := client.NewClientWithOpts(client.FromEnv, client.WithAPIVersionNegotiation())
if err != nil {
return nil, fmt.Errorf("newScript: failed to create docker client")
}
s.cli = cli
}
2023-03-16 19:48:12 +01:00
logFunc := func(logType storage.LogLevel, context string, msg string, params ...any) {
Added abstract helper interface for all storage backends (#135) * Added abstract helper interface and implemented it for all storage backends * Moved storage client initializations also to helper classes * Fixed ssh init issue * Moved script parameter to helper struct to simplify script init. * Created sub modules. Enhanced abstract implementation. * Fixed config issue * Fixed declaration issues. Added config to interface. * Added StorageProviders to unify all backends. * Cleanup, optimizations, comments. * Applied discussed changes. See description. Moved modules to internal packages. Replaced StoragePool with slice. Moved conditional for init of storage backends back to script. * Fix docker build issue * Fixed accidentally removed local copy condition. * Delete .gitignore * Renaming/changes according to review Renamed Init functions and interface. Replaced config object with specific config values. Init func returns interface instead of struct. Removed custom import names where possible. * Fixed auto-complete error. * Combined copy instructions into one layer. * Added logging func for storages. * Introduced logging func for errors too. * Missed an error message * Moved config back to main. Optimized prune stats handling. * Move stats back to main package * Code doc stuff * Apply changes from #136 * Replace name field with function. * Changed receiver names from stg to b. * Renamed LogFuncDef to Log * Removed redundant package name. * Renamed storagePool to storages. * Simplified creation of new storage backend. * Added initialization for storage stats map. * Invert .dockerignore patterns. * Fix package typo
2022-08-18 08:52:09 +02:00
switch logType {
case storage.LogLevelWarning:
s.logger.Warn(fmt.Sprintf(msg, params...), "storage", context)
case storage.LogLevelError:
s.logger.Error(fmt.Sprintf(msg, params...), "storage", context)
Added abstract helper interface for all storage backends (#135) * Added abstract helper interface and implemented it for all storage backends * Moved storage client initializations also to helper classes * Fixed ssh init issue * Moved script parameter to helper struct to simplify script init. * Created sub modules. Enhanced abstract implementation. * Fixed config issue * Fixed declaration issues. Added config to interface. * Added StorageProviders to unify all backends. * Cleanup, optimizations, comments. * Applied discussed changes. See description. Moved modules to internal packages. Replaced StoragePool with slice. Moved conditional for init of storage backends back to script. * Fix docker build issue * Fixed accidentally removed local copy condition. * Delete .gitignore * Renaming/changes according to review Renamed Init functions and interface. Replaced config object with specific config values. Init func returns interface instead of struct. Removed custom import names where possible. * Fixed auto-complete error. * Combined copy instructions into one layer. * Added logging func for storages. * Introduced logging func for errors too. * Missed an error message * Moved config back to main. Optimized prune stats handling. * Move stats back to main package * Code doc stuff * Apply changes from #136 * Replace name field with function. * Changed receiver names from stg to b. * Renamed LogFuncDef to Log * Removed redundant package name. * Renamed storagePool to storages. * Simplified creation of new storage backend. * Added initialization for storage stats map. * Invert .dockerignore patterns. * Fix package typo
2022-08-18 08:52:09 +02:00
default:
s.logger.Info(fmt.Sprintf(msg, params...), "storage", context)
}
Added abstract helper interface for all storage backends (#135) * Added abstract helper interface and implemented it for all storage backends * Moved storage client initializations also to helper classes * Fixed ssh init issue * Moved script parameter to helper struct to simplify script init. * Created sub modules. Enhanced abstract implementation. * Fixed config issue * Fixed declaration issues. Added config to interface. * Added StorageProviders to unify all backends. * Cleanup, optimizations, comments. * Applied discussed changes. See description. Moved modules to internal packages. Replaced StoragePool with slice. Moved conditional for init of storage backends back to script. * Fix docker build issue * Fixed accidentally removed local copy condition. * Delete .gitignore * Renaming/changes according to review Renamed Init functions and interface. Replaced config object with specific config values. Init func returns interface instead of struct. Removed custom import names where possible. * Fixed auto-complete error. * Combined copy instructions into one layer. * Added logging func for storages. * Introduced logging func for errors too. * Missed an error message * Moved config back to main. Optimized prune stats handling. * Move stats back to main package * Code doc stuff * Apply changes from #136 * Replace name field with function. * Changed receiver names from stg to b. * Renamed LogFuncDef to Log * Removed redundant package name. * Renamed storagePool to storages. * Simplified creation of new storage backend. * Added initialization for storage stats map. * Invert .dockerignore patterns. * Fix package typo
2022-08-18 08:52:09 +02:00
}
Added abstract helper interface for all storage backends (#135) * Added abstract helper interface and implemented it for all storage backends * Moved storage client initializations also to helper classes * Fixed ssh init issue * Moved script parameter to helper struct to simplify script init. * Created sub modules. Enhanced abstract implementation. * Fixed config issue * Fixed declaration issues. Added config to interface. * Added StorageProviders to unify all backends. * Cleanup, optimizations, comments. * Applied discussed changes. See description. Moved modules to internal packages. Replaced StoragePool with slice. Moved conditional for init of storage backends back to script. * Fix docker build issue * Fixed accidentally removed local copy condition. * Delete .gitignore * Renaming/changes according to review Renamed Init functions and interface. Replaced config object with specific config values. Init func returns interface instead of struct. Removed custom import names where possible. * Fixed auto-complete error. * Combined copy instructions into one layer. * Added logging func for storages. * Introduced logging func for errors too. * Missed an error message * Moved config back to main. Optimized prune stats handling. * Move stats back to main package * Code doc stuff * Apply changes from #136 * Replace name field with function. * Changed receiver names from stg to b. * Renamed LogFuncDef to Log * Removed redundant package name. * Renamed storagePool to storages. * Simplified creation of new storage backend. * Added initialization for storage stats map. * Invert .dockerignore patterns. * Fix package typo
2022-08-18 08:52:09 +02:00
if s.c.AwsS3BucketName != "" {
s3Config := s3.Config{
Endpoint: s.c.AwsEndpoint,
AccessKeyID: s.c.AwsAccessKeyID,
SecretAccessKey: s.c.AwsSecretAccessKey,
IamRoleEndpoint: s.c.AwsIamRoleEndpoint,
EndpointProto: s.c.AwsEndpointProto,
EndpointInsecure: s.c.AwsEndpointInsecure,
RemotePath: s.c.AwsS3Path,
BucketName: s.c.AwsS3BucketName,
StorageClass: s.c.AwsStorageClass,
CACert: s.c.AwsEndpointCACert.Cert,
PartSize: s.c.AwsPartSize,
}
s3Backend, err := s3.NewStorageBackend(s3Config, logFunc)
if err != nil {
Add new storage backend: Dropbox (#103) (#251) * Add new storage backend: Dropbox (#103) * Remove duplicate check * Add concurrency level for parallel upload to dropbox. * Fixed some instabilites. Changed default concurrency to 6. * Added some env config vars to readme. WIP * Wrap errors for storage backend creation. * Fixed token issue, added OAuth2 including recipe and docs. * Readme typo fix * Test for dropbox integration * Update info and TOC * Missed a file * Docker-compose fix * Fix endpoint connection * Fix container names * Fix log fetching * Fix log fetching (again) * Print command output to logs * Addressing comments part 1 * Address comments part 2 * OpenAPI Mock spec path adjusted * Dropbox FileMetadata reflection refactored * NaturalNumber type added * Add OAuth2 mock server for CI testing * Fix env name of oauth2 endpoint * Remove hostname * Add forgotten change to commit... * Fix oauth2 endpoint "Worked on my machine" * Try again * Try suggested hostname again * Fix docker internal DNS resolving issues (as suggested by oauth2 mock docs) * Add docker network, remove hostname * Network not external * Last hostname try * Add more delay, add oauth2 endpoint log * Temp CI log output of command even when failing * Try different config and method * Add custom server-hostname. Rename test folder to accellerate debugging * Try that fix again * Adding quotes * Port fix attempt * Try localhost * Try extra hosts * Change network mode * Undo some changes * Use static IP * Remove specific IP binding * Change to default net driver * Fix static IP * Squash for revert * Revert "Squash for revert" This reverts commit e9b617be9a15016f8abb724709a6a1aef321cdb9. * Actual fix for CI testing from #257
2023-08-24 19:33:47 +02:00
return nil, fmt.Errorf("newScript: error creating s3 storage backend: %w", err)
}
s.storages = append(s.storages, s3Backend)
}
if s.c.WebdavUrl != "" {
webDavConfig := webdav.Config{
URL: s.c.WebdavUrl,
URLInsecure: s.c.WebdavUrlInsecure,
Username: s.c.WebdavUsername,
Password: s.c.WebdavPassword,
RemotePath: s.c.WebdavPath,
}
webdavBackend, err := webdav.NewStorageBackend(webDavConfig, logFunc)
if err != nil {
Add new storage backend: Dropbox (#103) (#251) * Add new storage backend: Dropbox (#103) * Remove duplicate check * Add concurrency level for parallel upload to dropbox. * Fixed some instabilites. Changed default concurrency to 6. * Added some env config vars to readme. WIP * Wrap errors for storage backend creation. * Fixed token issue, added OAuth2 including recipe and docs. * Readme typo fix * Test for dropbox integration * Update info and TOC * Missed a file * Docker-compose fix * Fix endpoint connection * Fix container names * Fix log fetching * Fix log fetching (again) * Print command output to logs * Addressing comments part 1 * Address comments part 2 * OpenAPI Mock spec path adjusted * Dropbox FileMetadata reflection refactored * NaturalNumber type added * Add OAuth2 mock server for CI testing * Fix env name of oauth2 endpoint * Remove hostname * Add forgotten change to commit... * Fix oauth2 endpoint "Worked on my machine" * Try again * Try suggested hostname again * Fix docker internal DNS resolving issues (as suggested by oauth2 mock docs) * Add docker network, remove hostname * Network not external * Last hostname try * Add more delay, add oauth2 endpoint log * Temp CI log output of command even when failing * Try different config and method * Add custom server-hostname. Rename test folder to accellerate debugging * Try that fix again * Adding quotes * Port fix attempt * Try localhost * Try extra hosts * Change network mode * Undo some changes * Use static IP * Remove specific IP binding * Change to default net driver * Fix static IP * Squash for revert * Revert "Squash for revert" This reverts commit e9b617be9a15016f8abb724709a6a1aef321cdb9. * Actual fix for CI testing from #257
2023-08-24 19:33:47 +02:00
return nil, fmt.Errorf("newScript: error creating webdav storage backend: %w", err)
}
s.storages = append(s.storages, webdavBackend)
}
if s.c.SSHHostName != "" {
sshConfig := ssh.Config{
HostName: s.c.SSHHostName,
Port: s.c.SSHPort,
User: s.c.SSHUser,
Password: s.c.SSHPassword,
IdentityFile: s.c.SSHIdentityFile,
IdentityPassphrase: s.c.SSHIdentityPassphrase,
RemotePath: s.c.SSHRemotePath,
}
sshBackend, err := ssh.NewStorageBackend(sshConfig, logFunc)
if err != nil {
Add new storage backend: Dropbox (#103) (#251) * Add new storage backend: Dropbox (#103) * Remove duplicate check * Add concurrency level for parallel upload to dropbox. * Fixed some instabilites. Changed default concurrency to 6. * Added some env config vars to readme. WIP * Wrap errors for storage backend creation. * Fixed token issue, added OAuth2 including recipe and docs. * Readme typo fix * Test for dropbox integration * Update info and TOC * Missed a file * Docker-compose fix * Fix endpoint connection * Fix container names * Fix log fetching * Fix log fetching (again) * Print command output to logs * Addressing comments part 1 * Address comments part 2 * OpenAPI Mock spec path adjusted * Dropbox FileMetadata reflection refactored * NaturalNumber type added * Add OAuth2 mock server for CI testing * Fix env name of oauth2 endpoint * Remove hostname * Add forgotten change to commit... * Fix oauth2 endpoint "Worked on my machine" * Try again * Try suggested hostname again * Fix docker internal DNS resolving issues (as suggested by oauth2 mock docs) * Add docker network, remove hostname * Network not external * Last hostname try * Add more delay, add oauth2 endpoint log * Temp CI log output of command even when failing * Try different config and method * Add custom server-hostname. Rename test folder to accellerate debugging * Try that fix again * Adding quotes * Port fix attempt * Try localhost * Try extra hosts * Change network mode * Undo some changes * Use static IP * Remove specific IP binding * Change to default net driver * Fix static IP * Squash for revert * Revert "Squash for revert" This reverts commit e9b617be9a15016f8abb724709a6a1aef321cdb9. * Actual fix for CI testing from #257
2023-08-24 19:33:47 +02:00
return nil, fmt.Errorf("newScript: error creating ssh storage backend: %w", err)
}
s.storages = append(s.storages, sshBackend)
}
if _, err := os.Stat(s.c.BackupArchive); !os.IsNotExist(err) {
localConfig := local.Config{
ArchivePath: s.c.BackupArchive,
LatestSymlink: s.c.BackupLatestSymlink,
}
localBackend := local.NewStorageBackend(localConfig, logFunc)
s.storages = append(s.storages, localBackend)
}
Added abstract helper interface for all storage backends (#135) * Added abstract helper interface and implemented it for all storage backends * Moved storage client initializations also to helper classes * Fixed ssh init issue * Moved script parameter to helper struct to simplify script init. * Created sub modules. Enhanced abstract implementation. * Fixed config issue * Fixed declaration issues. Added config to interface. * Added StorageProviders to unify all backends. * Cleanup, optimizations, comments. * Applied discussed changes. See description. Moved modules to internal packages. Replaced StoragePool with slice. Moved conditional for init of storage backends back to script. * Fix docker build issue * Fixed accidentally removed local copy condition. * Delete .gitignore * Renaming/changes according to review Renamed Init functions and interface. Replaced config object with specific config values. Init func returns interface instead of struct. Removed custom import names where possible. * Fixed auto-complete error. * Combined copy instructions into one layer. * Added logging func for storages. * Introduced logging func for errors too. * Missed an error message * Moved config back to main. Optimized prune stats handling. * Move stats back to main package * Code doc stuff * Apply changes from #136 * Replace name field with function. * Changed receiver names from stg to b. * Renamed LogFuncDef to Log * Removed redundant package name. * Renamed storagePool to storages. * Simplified creation of new storage backend. * Added initialization for storage stats map. * Invert .dockerignore patterns. * Fix package typo
2022-08-18 08:52:09 +02:00
if s.c.AzureStorageAccountName != "" {
azureConfig := azure.Config{
ContainerName: s.c.AzureStorageContainerName,
AccountName: s.c.AzureStorageAccountName,
PrimaryAccountKey: s.c.AzureStoragePrimaryAccountKey,
Endpoint: s.c.AzureStorageEndpoint,
RemotePath: s.c.AzureStoragePath,
}
azureBackend, err := azure.NewStorageBackend(azureConfig, logFunc)
if err != nil {
Add new storage backend: Dropbox (#103) (#251) * Add new storage backend: Dropbox (#103) * Remove duplicate check * Add concurrency level for parallel upload to dropbox. * Fixed some instabilites. Changed default concurrency to 6. * Added some env config vars to readme. WIP * Wrap errors for storage backend creation. * Fixed token issue, added OAuth2 including recipe and docs. * Readme typo fix * Test for dropbox integration * Update info and TOC * Missed a file * Docker-compose fix * Fix endpoint connection * Fix container names * Fix log fetching * Fix log fetching (again) * Print command output to logs * Addressing comments part 1 * Address comments part 2 * OpenAPI Mock spec path adjusted * Dropbox FileMetadata reflection refactored * NaturalNumber type added * Add OAuth2 mock server for CI testing * Fix env name of oauth2 endpoint * Remove hostname * Add forgotten change to commit... * Fix oauth2 endpoint "Worked on my machine" * Try again * Try suggested hostname again * Fix docker internal DNS resolving issues (as suggested by oauth2 mock docs) * Add docker network, remove hostname * Network not external * Last hostname try * Add more delay, add oauth2 endpoint log * Temp CI log output of command even when failing * Try different config and method * Add custom server-hostname. Rename test folder to accellerate debugging * Try that fix again * Adding quotes * Port fix attempt * Try localhost * Try extra hosts * Change network mode * Undo some changes * Use static IP * Remove specific IP binding * Change to default net driver * Fix static IP * Squash for revert * Revert "Squash for revert" This reverts commit e9b617be9a15016f8abb724709a6a1aef321cdb9. * Actual fix for CI testing from #257
2023-08-24 19:33:47 +02:00
return nil, fmt.Errorf("newScript: error creating azure storage backend: %w", err)
}
s.storages = append(s.storages, azureBackend)
}
Add new storage backend: Dropbox (#103) (#251) * Add new storage backend: Dropbox (#103) * Remove duplicate check * Add concurrency level for parallel upload to dropbox. * Fixed some instabilites. Changed default concurrency to 6. * Added some env config vars to readme. WIP * Wrap errors for storage backend creation. * Fixed token issue, added OAuth2 including recipe and docs. * Readme typo fix * Test for dropbox integration * Update info and TOC * Missed a file * Docker-compose fix * Fix endpoint connection * Fix container names * Fix log fetching * Fix log fetching (again) * Print command output to logs * Addressing comments part 1 * Address comments part 2 * OpenAPI Mock spec path adjusted * Dropbox FileMetadata reflection refactored * NaturalNumber type added * Add OAuth2 mock server for CI testing * Fix env name of oauth2 endpoint * Remove hostname * Add forgotten change to commit... * Fix oauth2 endpoint "Worked on my machine" * Try again * Try suggested hostname again * Fix docker internal DNS resolving issues (as suggested by oauth2 mock docs) * Add docker network, remove hostname * Network not external * Last hostname try * Add more delay, add oauth2 endpoint log * Temp CI log output of command even when failing * Try different config and method * Add custom server-hostname. Rename test folder to accellerate debugging * Try that fix again * Adding quotes * Port fix attempt * Try localhost * Try extra hosts * Change network mode * Undo some changes * Use static IP * Remove specific IP binding * Change to default net driver * Fix static IP * Squash for revert * Revert "Squash for revert" This reverts commit e9b617be9a15016f8abb724709a6a1aef321cdb9. * Actual fix for CI testing from #257
2023-08-24 19:33:47 +02:00
if s.c.DropboxRefreshToken != "" && s.c.DropboxAppKey != "" && s.c.DropboxAppSecret != "" {
dropboxConfig := dropbox.Config{
Endpoint: s.c.DropboxEndpoint,
OAuth2Endpoint: s.c.DropboxOAuth2Endpoint,
RefreshToken: s.c.DropboxRefreshToken,
AppKey: s.c.DropboxAppKey,
AppSecret: s.c.DropboxAppSecret,
RemotePath: s.c.DropboxRemotePath,
ConcurrencyLevel: s.c.DropboxConcurrencyLevel.Int(),
}
dropboxBackend, err := dropbox.NewStorageBackend(dropboxConfig, logFunc)
if err != nil {
return nil, fmt.Errorf("newScript: error creating dropbox storage backend: %w", err)
}
s.storages = append(s.storages, dropboxBackend)
}
if s.c.EmailNotificationRecipient != "" {
emailURL := fmt.Sprintf(
"smtp://%s:%s@%s:%d/?from=%s&to=%s",
s.c.EmailSMTPUsername,
s.c.EmailSMTPPassword,
s.c.EmailSMTPHost,
s.c.EmailSMTPPort,
s.c.EmailNotificationSender,
s.c.EmailNotificationRecipient,
)
s.c.NotificationURLs = append(s.c.NotificationURLs, emailURL)
s.logger.Warn(
"Using EMAIL_* keys for providing notification configuration has been deprecated and will be removed in the next major version.",
)
s.logger.Warn(
"Please use NOTIFICATION_URLS instead. Refer to the README for an upgrade guide.",
)
}
hookLevel, ok := hookLevels[s.c.NotificationLevel]
if !ok {
return nil, fmt.Errorf("newScript: unknown NOTIFICATION_LEVEL %s", s.c.NotificationLevel)
}
s.hookLevel = hookLevel
if len(s.c.NotificationURLs) > 0 {
sender, senderErr := shoutrrr.CreateSender(s.c.NotificationURLs...)
if senderErr != nil {
return nil, fmt.Errorf("newScript: error creating sender: %w", senderErr)
}
s.sender = sender
tmpl := template.New("")
tmpl.Funcs(templateHelpers)
tmpl, err = tmpl.Parse(defaultNotifications)
if err != nil {
return nil, fmt.Errorf("newScript: unable to parse default notifications templates: %w", err)
}
if fi, err := os.Stat("/etc/dockervolumebackup/notifications.d"); err == nil && fi.IsDir() {
tmpl, err = tmpl.ParseGlob("/etc/dockervolumebackup/notifications.d/*.*")
if err != nil {
return nil, fmt.Errorf("newScript: unable to parse user defined notifications templates: %w", err)
}
}
s.template = tmpl
// To prevent duplicate notifications, ensure the regsistered callbacks
// run mutually exclusive.
s.registerHook(hookLevelError, func(err error) error {
if err == nil {
return nil
}
return s.notifyFailure(err)
})
s.registerHook(hookLevelInfo, func(err error) error {
if err != nil {
return nil
}
return s.notifySuccess()
})
}
return s, nil
}
// createArchive creates a tar archive of the configured backup location and
// saves it to disk.
func (s *script) createArchive() error {
backupSources := s.c.BackupSources
if s.c.BackupFromSnapshot {
2022-03-25 18:28:58 +01:00
s.logger.Warn(
"Using BACKUP_FROM_SNAPSHOT has been deprecated and will be removed in the next major version.",
)
s.logger.Warn(
"Please use `archive-pre` and `archive-post` commands to prepare your backup sources. Refer to the documentation for an upgrade guide.",
2022-03-25 18:28:58 +01:00
)
backupSources = filepath.Join("/tmp", s.c.BackupSources)
// copy before compressing guard against a situation where backup folder's content are still growing.
s.registerHook(hookLevelPlumbing, func(error) error {
if err := remove(backupSources); err != nil {
2022-09-15 10:49:45 +02:00
return fmt.Errorf("createArchive: error removing snapshot: %w", err)
}
s.logger.Info(
fmt.Sprintf("Removed snapshot `%s`.", backupSources),
)
return nil
})
if err := copy.Copy(s.c.BackupSources, backupSources, copy.Options{
PreserveTimes: true,
PreserveOwner: true,
}); err != nil {
return fmt.Errorf("createArchive: error creating snapshot: %w", err)
}
s.logger.Info(
fmt.Sprintf("Created snapshot of `%s` at `%s`.", s.c.BackupSources, backupSources),
)
}
tarFile := s.file
s.registerHook(hookLevelPlumbing, func(error) error {
if err := remove(tarFile); err != nil {
return fmt.Errorf("createArchive: error removing tar file: %w", err)
}
s.logger.Info(
fmt.Sprintf("Removed tar file `%s`.", tarFile),
)
return nil
})
backupPath, err := filepath.Abs(stripTrailingSlashes(backupSources))
if err != nil {
return fmt.Errorf("createArchive: error getting absolute path: %w", err)
}
var filesEligibleForBackup []string
if err := filepath.WalkDir(backupPath, func(path string, di fs.DirEntry, err error) error {
if err != nil {
return err
}
if s.c.BackupExcludeRegexp.Re != nil && s.c.BackupExcludeRegexp.Re.MatchString(path) {
return nil
}
filesEligibleForBackup = append(filesEligibleForBackup, path)
return nil
}); err != nil {
return fmt.Errorf("createArchive: error walking filesystem tree: %w", err)
}
if err := createArchive(filesEligibleForBackup, backupSources, tarFile, s.c.BackupCompression.String(), s.c.GzipParallelism.Int()); err != nil {
return fmt.Errorf("createArchive: error compressing backup folder: %w", err)
}
s.logger.Info(
fmt.Sprintf("Created backup of `%s` at `%s`.", backupSources, tarFile),
)
return nil
}
// encryptArchive encrypts the backup file using PGP and the configured passphrase.
// In case no passphrase is given it returns early, leaving the backup file
// untouched.
func (s *script) encryptArchive() error {
if s.c.GpgPassphrase == "" {
return nil
}
gpgFile := fmt.Sprintf("%s.gpg", s.file)
s.registerHook(hookLevelPlumbing, func(error) error {
if err := remove(gpgFile); err != nil {
return fmt.Errorf("encryptArchive: error removing gpg file: %w", err)
}
s.logger.Info(
fmt.Sprintf("Removed GPG file `%s`.", gpgFile),
)
return nil
})
outFile, err := os.Create(gpgFile)
if err != nil {
return fmt.Errorf("encryptArchive: error opening out file: %w", err)
}
Added abstract helper interface for all storage backends (#135) * Added abstract helper interface and implemented it for all storage backends * Moved storage client initializations also to helper classes * Fixed ssh init issue * Moved script parameter to helper struct to simplify script init. * Created sub modules. Enhanced abstract implementation. * Fixed config issue * Fixed declaration issues. Added config to interface. * Added StorageProviders to unify all backends. * Cleanup, optimizations, comments. * Applied discussed changes. See description. Moved modules to internal packages. Replaced StoragePool with slice. Moved conditional for init of storage backends back to script. * Fix docker build issue * Fixed accidentally removed local copy condition. * Delete .gitignore * Renaming/changes according to review Renamed Init functions and interface. Replaced config object with specific config values. Init func returns interface instead of struct. Removed custom import names where possible. * Fixed auto-complete error. * Combined copy instructions into one layer. * Added logging func for storages. * Introduced logging func for errors too. * Missed an error message * Moved config back to main. Optimized prune stats handling. * Move stats back to main package * Code doc stuff * Apply changes from #136 * Replace name field with function. * Changed receiver names from stg to b. * Renamed LogFuncDef to Log * Removed redundant package name. * Renamed storagePool to storages. * Simplified creation of new storage backend. * Added initialization for storage stats map. * Invert .dockerignore patterns. * Fix package typo
2022-08-18 08:52:09 +02:00
defer outFile.Close()
_, name := path.Split(s.file)
dst, err := openpgp.SymmetricallyEncrypt(outFile, []byte(s.c.GpgPassphrase), &openpgp.FileHints{
FileName: name,
}, nil)
if err != nil {
return fmt.Errorf("encryptArchive: error encrypting backup file: %w", err)
}
Added abstract helper interface for all storage backends (#135) * Added abstract helper interface and implemented it for all storage backends * Moved storage client initializations also to helper classes * Fixed ssh init issue * Moved script parameter to helper struct to simplify script init. * Created sub modules. Enhanced abstract implementation. * Fixed config issue * Fixed declaration issues. Added config to interface. * Added StorageProviders to unify all backends. * Cleanup, optimizations, comments. * Applied discussed changes. See description. Moved modules to internal packages. Replaced StoragePool with slice. Moved conditional for init of storage backends back to script. * Fix docker build issue * Fixed accidentally removed local copy condition. * Delete .gitignore * Renaming/changes according to review Renamed Init functions and interface. Replaced config object with specific config values. Init func returns interface instead of struct. Removed custom import names where possible. * Fixed auto-complete error. * Combined copy instructions into one layer. * Added logging func for storages. * Introduced logging func for errors too. * Missed an error message * Moved config back to main. Optimized prune stats handling. * Move stats back to main package * Code doc stuff * Apply changes from #136 * Replace name field with function. * Changed receiver names from stg to b. * Renamed LogFuncDef to Log * Removed redundant package name. * Renamed storagePool to storages. * Simplified creation of new storage backend. * Added initialization for storage stats map. * Invert .dockerignore patterns. * Fix package typo
2022-08-18 08:52:09 +02:00
defer dst.Close()
src, err := os.Open(s.file)
if err != nil {
return fmt.Errorf("encryptArchive: error opening backup file `%s`: %w", s.file, err)
}
if _, err := io.Copy(dst, src); err != nil {
return fmt.Errorf("encryptArchive: error writing ciphertext to file: %w", err)
}
s.file = gpgFile
s.logger.Info(
fmt.Sprintf("Encrypted backup using given passphrase, saving as `%s`.", s.file),
)
return nil
}
// copyArchive makes sure the backup file is copied to both local and remote locations
// as per the given configuration.
func (s *script) copyArchive() error {
_, name := path.Split(s.file)
if stat, err := os.Stat(s.file); err != nil {
return fmt.Errorf("copyArchive: unable to stat backup file: %w", err)
} else {
size := stat.Size()
s.stats.BackupFile = BackupFileStats{
Size: uint64(size),
Name: name,
FullPath: s.file,
}
}
eg := errgroup.Group{}
Added abstract helper interface for all storage backends (#135) * Added abstract helper interface and implemented it for all storage backends * Moved storage client initializations also to helper classes * Fixed ssh init issue * Moved script parameter to helper struct to simplify script init. * Created sub modules. Enhanced abstract implementation. * Fixed config issue * Fixed declaration issues. Added config to interface. * Added StorageProviders to unify all backends. * Cleanup, optimizations, comments. * Applied discussed changes. See description. Moved modules to internal packages. Replaced StoragePool with slice. Moved conditional for init of storage backends back to script. * Fix docker build issue * Fixed accidentally removed local copy condition. * Delete .gitignore * Renaming/changes according to review Renamed Init functions and interface. Replaced config object with specific config values. Init func returns interface instead of struct. Removed custom import names where possible. * Fixed auto-complete error. * Combined copy instructions into one layer. * Added logging func for storages. * Introduced logging func for errors too. * Missed an error message * Moved config back to main. Optimized prune stats handling. * Move stats back to main package * Code doc stuff * Apply changes from #136 * Replace name field with function. * Changed receiver names from stg to b. * Renamed LogFuncDef to Log * Removed redundant package name. * Renamed storagePool to storages. * Simplified creation of new storage backend. * Added initialization for storage stats map. * Invert .dockerignore patterns. * Fix package typo
2022-08-18 08:52:09 +02:00
for _, backend := range s.storages {
b := backend
eg.Go(func() error {
return b.Copy(s.file)
})
}
if err := eg.Wait(); err != nil {
return fmt.Errorf("copyArchive: error copying archive: %w", err)
}
return nil
}
// pruneBackups rotates away backups from local and remote storages using
// the given configuration. In case the given configuration would delete all
// backups, it does nothing instead and logs a warning.
func (s *script) pruneBackups() error {
if s.c.BackupRetentionDays < 0 {
return nil
}
deadline := time.Now().AddDate(0, 0, -int(s.c.BackupRetentionDays)).Add(s.c.BackupPruningLeeway)
eg := errgroup.Group{}
Added abstract helper interface for all storage backends (#135) * Added abstract helper interface and implemented it for all storage backends * Moved storage client initializations also to helper classes * Fixed ssh init issue * Moved script parameter to helper struct to simplify script init. * Created sub modules. Enhanced abstract implementation. * Fixed config issue * Fixed declaration issues. Added config to interface. * Added StorageProviders to unify all backends. * Cleanup, optimizations, comments. * Applied discussed changes. See description. Moved modules to internal packages. Replaced StoragePool with slice. Moved conditional for init of storage backends back to script. * Fix docker build issue * Fixed accidentally removed local copy condition. * Delete .gitignore * Renaming/changes according to review Renamed Init functions and interface. Replaced config object with specific config values. Init func returns interface instead of struct. Removed custom import names where possible. * Fixed auto-complete error. * Combined copy instructions into one layer. * Added logging func for storages. * Introduced logging func for errors too. * Missed an error message * Moved config back to main. Optimized prune stats handling. * Move stats back to main package * Code doc stuff * Apply changes from #136 * Replace name field with function. * Changed receiver names from stg to b. * Renamed LogFuncDef to Log * Removed redundant package name. * Renamed storagePool to storages. * Simplified creation of new storage backend. * Added initialization for storage stats map. * Invert .dockerignore patterns. * Fix package typo
2022-08-18 08:52:09 +02:00
for _, backend := range s.storages {
b := backend
eg.Go(func() error {
if skipPrune(b.Name(), s.c.BackupSkipBackendsFromPrune) {
s.logger.Info(
fmt.Sprintf("Skipping pruning for backend `%s`.", b.Name()),
)
return nil
}
stats, err := b.Prune(deadline, s.c.BackupPruningPrefix)
if err != nil {
return err
}
s.stats.Lock()
s.stats.Storages[b.Name()] = StorageStats{
Added abstract helper interface for all storage backends (#135) * Added abstract helper interface and implemented it for all storage backends * Moved storage client initializations also to helper classes * Fixed ssh init issue * Moved script parameter to helper struct to simplify script init. * Created sub modules. Enhanced abstract implementation. * Fixed config issue * Fixed declaration issues. Added config to interface. * Added StorageProviders to unify all backends. * Cleanup, optimizations, comments. * Applied discussed changes. See description. Moved modules to internal packages. Replaced StoragePool with slice. Moved conditional for init of storage backends back to script. * Fix docker build issue * Fixed accidentally removed local copy condition. * Delete .gitignore * Renaming/changes according to review Renamed Init functions and interface. Replaced config object with specific config values. Init func returns interface instead of struct. Removed custom import names where possible. * Fixed auto-complete error. * Combined copy instructions into one layer. * Added logging func for storages. * Introduced logging func for errors too. * Missed an error message * Moved config back to main. Optimized prune stats handling. * Move stats back to main package * Code doc stuff * Apply changes from #136 * Replace name field with function. * Changed receiver names from stg to b. * Renamed LogFuncDef to Log * Removed redundant package name. * Renamed storagePool to storages. * Simplified creation of new storage backend. * Added initialization for storage stats map. * Invert .dockerignore patterns. * Fix package typo
2022-08-18 08:52:09 +02:00
Total: stats.Total,
Pruned: stats.Pruned,
}
s.stats.Unlock()
return nil
})
}
if err := eg.Wait(); err != nil {
return fmt.Errorf("pruneBackups: error pruning backups: %w", err)
}
return nil
}
// must exits the script run prematurely in case the given error
// is non-nil.
func (s *script) must(err error) {
if err != nil {
s.logger.Error(
fmt.Sprintf("Fatal error running backup: %s", err),
)
panic(err)
}
}
// skipPrune returns true if the given backend name is contained in the
// list of skipped backends.
func skipPrune(name string, skippedBackends []string) bool {
return slices.ContainsFunc(
skippedBackends,
func(b string) bool {
return strings.EqualFold(b, name) // ignore case on both sides
},
)
}