* volume: copying a remote-backed volume only needs space for the index
VolumeCopy sized its target-location check by the source .dat even when
that .dat lives in a cloud tier and only .idx/.vif land locally, so
re-replicating a tiered volume demanded the full remote size in free
disk. Require the index size instead.
* shell: volume.tier.upload keeps volume replicas
Tiering a replicated volume deleted every replica but the upload
source, leaving one server holding the only .idx and the only .vif
that knows the remote object key — losing that server orphaned the
volume even though its data sat intact in the cloud.
Replicate the uploaded .idx/.vif onto the other replica servers
instead (VolumeCopy skips the .dat for remote-backed volumes), so all
replicas serve reads from the same remote object and the volume keeps
its replica count. An already-tiered replica is preferred as the
upload source, so a rerun after a partial failure reuses the existing
remote object instead of uploading a second copy under a new key.
* shell: group tier upload locations instead of re-prepending
* rust volume: copying a remote-backed volume only needs space for the index
Mirror the Go VolumeCopy change: size the free-location check by the
source .idx when the .dat lives in a cloud tier, since only .idx/.vif
land locally.
2026-07-11 13:49:36 -07:00
Chris LuGitHubgemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* fix(volume): don't fatal on missing .idx for remote-tiered volume
A .vif left behind without its .idx (orphaned by a crashed move, partial
copy, or hand-edit) would trip glog.Fatalf in checkIdxFile and take the
whole volume server down on boot, killing every healthy volume on it
too. For remote-tiered volumes treat it as a per-volume load error so
the server can come up and the operator can clean up the stray .vif.
Refs #9331.
* fix(balance): skip remote-tiered volumes in admin balance detection
The admin/worker balance detector had no equivalent of the shell-side
guard ("does not move volume in remote storage" in
command_volume_balance.go), so it scheduled moves on remote-tiered
volumes. The "move" copies .idx/.vif to the destination and then calls
Volume.Destroy on the source, which calls backendStorage.DeleteFile —
deleting the remote object the destination's new .vif now points at.
Populate HasRemoteCopy on the metrics emitted by both the admin
maintenance scanner and the worker's master poll, then drop those
volumes at the top of Detection.
Fixes#9331.
* Apply suggestion from @gemini-code-assist[bot]
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* fix(volume): keep remote data on volume-move-driven delete
The on-source delete after a volume move (admin/worker balance and
shell volume.move) ran Volume.Destroy with no way to opt out of the
remote-object cleanup. Volume.Destroy unconditionally calls
backendStorage.DeleteFile for remote-tiered volumes, so a successful
move would copy .idx/.vif to the destination and then nuke the cloud
object the destination's new .vif was already pointing at.
Add VolumeDeleteRequest.keep_remote_data and plumb it through
Store.DeleteVolume / DiskLocation.DeleteVolume / Volume.Destroy. The
balance task and shell volume.move set it to true; the post-tier-upload
cleanup of other replicas and the over-replication trim in
volume.fix.replication also set it to true since the remote object is
still referenced. Other real-delete callers keep the default. The
delete-before-receive path in VolumeCopy also sets it: the inbound copy
carries a .vif that may reference the same cloud object as the
existing volume.
Refs #9331.
* test(storage): in-process remote-tier integration tests
Cover the four operations the user is most likely to run against a
cloud-tiered volume — balance/move, vacuum, EC encode, EC decode — by
registering a local-disk-backed BackendStorage as the "remote" tier and
exercising the real Volume / DiskLocation / EC encoder code paths.
Locks in:
- Destroy(keepRemoteData=true) preserves the remote object (move case)
- Destroy(keepRemoteData=false) deletes it (real-delete case)
- Vacuum/compact on a remote-tier volume never deletes the remote object
- EC encode requires the local .dat (callers must download first)
- EC encode + rebuild round-trips after a tier-down
Tests run in-process and finish in under a second total — no cluster,
binary, or external storage required.
* fix(rust-volume): keep remote data on volume-move-driven delete
Mirror the Go fix in seaweed-volume: plumb keep_remote_data through
grpc volume_delete → Store.delete_volume → DiskLocation.delete_volume
→ Volume.destroy, and skip the s3-tier delete_file call when the flag
is set. The pre-receive cleanup in volume_copy passes true for the
same reason as the Go side: the inbound copy carries a .vif that may
reference the same cloud object as the existing volume.
The Rust loader already warns rather than fataling on a stray .vif
without an .idx (volume.rs load_index_inmemory / load_index_redb), so
no counterpart to the Go fatal-on-missing-idx fix is needed.
Refs #9331.
* fix(volume): preserve remote tier on IO-error eviction; fix EC test target
Two review nits:
- Store.MaybeAddVolumes' periodic cleanup pass deleted IO-errored
volumes with keepRemoteData=false, so a transient local fault on a
remote-tiered volume would also nuke the cloud object. Track the
delete reason via a parallel slice and pass keepRemoteData=v.HasRemoteFile()
for IO-error evictions; TTL-expired evictions still pass false.
- TestRemoteTier_ECEncodeDecode_AfterDownload deleted shards 0..3 but
called them "parity" — by the klauspost/reedsolomon convention shards
0..DataShardsCount-1 are data and DataShardsCount..TotalShardsCount-1
are parity. Switch the loop to delete the parity range so the
intent matches the indices.
---------
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* fix command_volume_tier_upload bug: Avoid deleting volumes under the same collection
* simplify a bit
---------
Co-authored-by: hzxialei <hzxialei@corp.netease.com>
Co-authored-by: chrislu <chris.lu@gmail.com>
Chris LuandGitHub
NyaMistyandGitHub
Max DenushevandGitHub
chrislu