mirror of
https://github.com/seaweedfs/seaweedfs.git
synced 2026-08-16 04:06:44 +00:00
Chris LuandGitHub
567052bfb6
* pb: ask the master what each collection holds Callers tracking usage were sent every volume in the cluster to add up themselves, which is the master's largest single allocation. * topology: summarise what each collection holds One pass over the topology, allocating per collection rather than per volume. Regular volumes count once each for logical totals and once per replica for physical, taken from the lookup index, which is already keyed by volume and so needs no set of seen ids. Ec shards are node-local so their sizes sum, while the file and delete counts describe the volume and resolve once every holder has been seen. Replicas of one volume disagree while a write is landing or a heartbeat is late. Walking a full listing took whichever replica the map iteration reached first, so the answer moved between runs; this takes the largest, which is stable and never reports usage below what some replica already holds. * s3: take bucket sizes from the master's summary The bucket size metrics pulled the whole volume list once a minute and added it up, which cost the master 184.6MB of allocation and 17.8MB on the wire for six numbers per collection. VolumeList over 550k volumes 184.6 MB allocated, 17.8 MB on the wire CollectionStatistics 176 bytes allocated, 47 bytes on the wire The aggregation moves to the master with it, so the cases the removed tests covered are now asserted against it directly. * topology: count the replica holding the most live data Quotas are enforced on size less deletions, and the replica with the biggest raw size can be the one that has deleted the most. Counting it reported a bucket smaller than it is and would leave one writable over its quota, which is the opposite of what picking the largest was meant to guarantee. * topology: cap a volume's deletions at what it holds Live usage is read as a collection's size less its deletions, so a volume reporting more deleted bytes than it has cancels live bytes belonging to other volumes in the same bucket and reports it smaller than it is. Replica selection already floored that volume's own live size at zero; the totals have to agree with it.