1 Commits

Include renames

Author	SHA1	Message	Date
Chris LuandGitHub	41d6c821ba	feat(topology): report empty disks (per-disk type + capacity in heartbeat) (#10166) ... * fix(topology): keep physical disk 0 distinct in SplitByPhysicalDisk DiskId 0 doubles as the first physical disk (Locations[0]) and the protobuf "unset" default. SplitByPhysicalDisk folded every DiskId-0 record onto the aggregate DiskId whenever that was non-zero, so on a multi-disk node the first disk's volumes merged into whichever disk held volumes[0]: the node reported one fewer disk, the sibling showed ~2x volumes, and per-disk max was smeared across the survivors. This surfaced as cluster.status and volume.list undercounting disks. Only treat 0 as unset when no record carries a non-zero DiskId; with a mix, 0 is a real disk and keeps its own entry. * fix(admin): resolve physical disk 0 in active-topology indexes rebuildIndexes re-derived each volume/EC record's physical disk id with the same "DiskId 0 means unset" heuristic SplitByPhysicalDisk used, so the two agreed only by sharing the bug. Now that SplitByPhysicalDisk keeps disk 0 distinct, the duplicated heuristic would fold disk-0 records onto a sibling while at.disks kept them on disk 0; GetVolumeLocations and GetECShardLocations then matched no record and silently dropped every volume and EC shard on the first disk, starving balance and EC tasks. Build the indexes from the same SplitByPhysicalDisk reconstruction that builds at.disks, so the keys always resolve. One source of truth instead of a parallel normalize. * fix(ec): allow physical disk 0 as preferred EC shard target pickBestDiskOnNode gated its result on bestDiskId != 0, but 0 is both a valid physical disk and the uint32 zero value, so a best-scoring disk 0 was discarded and the non-matching fallback returned instead. Gate on bestScore. * test(admin): cover EC-shard index resolution for physical disk 0 rebuildIndexes builds ecShardIndex the same way as volumeIndex; pin the EC path too so a shard on disk 0 keeps resolving via GetECShardLocations. * proto: per-disk type/capacity in DiskTag, DiskInfo.physical_disks DiskTag gains type + max_volume_count so the heartbeat can describe every physical disk, including ones holding no volumes or EC shards. DiskInfo gains physical_disks so the master can hand the full per-type disk set to per-physical-disk consumers. * feat(volume): report each physical disk's type and capacity CollectHeartbeat fills DiskTag.type and the per-disk effective max for every location, so the master can account for disks that hold no volumes or EC shards yet. Rust heartbeat mirrors it. * feat(master): surface empty disks in the per-physical-disk view The master records each disk's type and max from DiskTags and lists them on DiskInfo.physical_disks per type, including disks with no volumes or EC shards. SplitByPhysicalDisk enumerates that full set and gives each disk its exact max, so cluster.status, volume.list and the admin topology count and can target empty disks. Without physical_disks the even-split fallback is unchanged. * fix(master): clamp per-disk free at zero for over-allocated disks In the exact-max path FreeVolumeCount could go negative when a disk holds more volumes than its max; a negative would reduce the node's summed free and block placement on healthy disks. Clamp at 0. * fix(master): rebuild disk tags fresh each heartbeat DiskTags is the full authoritative per-disk list every heartbeat, so rebuild dn.diskTags from scratch like dn.diskBackends; merging left stale entries for removed disks. * fix(master): keep zero-capacity disks in physical_disks A disk reporting max 0 (an unavailable disk) is a valid physical disk, not a signal to drop it. List every disk of the type, but only emit physical_disks when the node reports real per-disk capacity, so an older server sending all zeros still falls back to the aggregate split. * test(volume): cover disk-space-low per-disk max in heartbeat Assert DiskTag.max_volume_count follows the used-slots override when a location is low on space, matching the per-type max_volume_counts. * chore: trim comments on the empty-disk change Drop narration; keep only the non-obvious why (disk-0 sentinel, exact-max free clamp, EC slots not subtracted, all-zeros fallback). * refactor(master): merge per-disk tags and capacity into one map diskTags and diskBackends were parallel maps keyed by the same DiskId and filled together from DiskTags. Fold them into one diskMetas map of {tags, type, max}. * refactor(proto): per-disk max as a map keyed by disk id physical_disks was a repeated {disk_id, max_volume_count} whose fields duplicated DiskInfo's own disk_id/max_volume_count. A map<uint32,int64> keyed by disk id expresses "max per disk" directly, drops the extra PhysicalDiskInfo message, and the consumer reads it as the disk set. * docs(proto): note DiskInfo.disk_id's two meanings Identity on a per-physical-disk DiskInfo (from SplitByPhysicalDisk), representative fallback on the type-keyed aggregate.	2026-06-30 18:45:44 -07:00