* fix(sync): use per-cluster TLS for HTTP volume connections in filer.sync (#8965)
When filer.sync runs with -a.security and -b.security flags, only gRPC
connections received per-cluster TLS configuration. HTTP clients for
volume server reads and uploads used a global singleton with the default
security.toml, causing TLS verification failures when clusters use
different self-signed certificates.
Load per-cluster HTTPS client config from the security files and pass
dedicated HTTP clients to FilerSource (for downloads) and FilerSink
(for uploads) so each direction uses the correct cluster's certificates.
* fix(sync): address review feedback for per-cluster HTTP TLS
- Add insecure_skip_verify support to NewHttpClientWithTLS and read it
from per-cluster security config via https.client.insecure_skip_verify
- Error on partial mTLS config (cert without key or vice versa)
- Add nil-check for client parameter in DownloadFileWithClient
- Document SetUploader as init-only (same pattern as SetChunkConcurrency)
* Add -insecureSkipVerify flag and config option for filer.sync HTTPS connections
When using filer.sync between clusters with different CAs (e.g., separate
OpenShift clusters), TLS certificate verification fails with "x509:
certificate signed by unknown authority". This adds two ways to skip TLS
certificate verification:
1. CLI flag: `weed filer.sync -insecureSkipVerify ...`
2. Config option: `insecure_skip_verify = true` under [https.client] in
security.toml
Closes#8778
* Add insecure_skip_verify option for HTTPS client in security.toml
When using filer.sync between clusters with different CAs (e.g., separate
OpenShift clusters), TLS certificate verification fails. Adding
insecure_skip_verify = true under [https.client] in security.toml allows
skipping TLS certificate verification.
The option is read during global HTTP client initialization so it applies
to all HTTPS connections including filer.sync proxy reads and writes.
Closes#8778
---------
Co-authored-by: Copilot <copilot@github.com>
* Added global http client
* Added Do func for global http client
* Changed the code to use the global http client
* Fix http client in volume uploader
* Fixed pkg name
* Fixed http util funcs
* Fixed http client for bench_filer_upload
* Fixed http client for stress_filer_upload
* Fixed http client for filer_server_handlers_proxy
* Fixed http client for command_fs_merge_volumes
* Fixed http client for command_fs_merge_volumes and command_volume_fsck
* Fixed http client for s3api_server
* Added init global client for main funcs
* Rename global_client to client
* Changed:
- fixed NewHttpClient;
- added CheckIsHttpsClientEnabled func
- updated security.toml in scaffold
* Reduce the visibility of some functions in the util/http/client pkg
* Added the loadSecurityConfig function
* Use util.LoadSecurityConfiguration() in NewHttpClient func
Chris LuandGitHub
vadimartynovandGitHub