mirror of
https://github.com/versity/scoutfs.git
synced 2026-09-08 09:07:30 +00:00
Zach Brown
1270553f1f
Initially items were stored in memory with an rbtree. That let us build up the API above items without worrying about their storage. That gave us dirty items in memory and we could start working on writing them to and reading them from the log segment blocks. Now that we have the code on either side we can get rid of the item cache in between. It had some nice properties but it's fundamentally duplicating the item storage in cached log segment blocks. We'd also have to teach it to differentiate between negative cache entries and missing entries that need to be filled from blocks. And the giant item index becomes a bottleneck. We have to index items in log segments anyway so we rewrite the item APIs to read and write the items in the log segments directly. Creation writes to dirty blocks in memory and reading and iteration walk through the cached blocks in the buffer cache. I've tried to comment the files and functions appropriately so most of the commentary for the new methods is in the body of the commit. The overall theme is making it relatively efficient to operate on individual items in log segments. Previously we could only walk all the items in an existing segment or write all the dirty items to a new segment. Now we have bloom filters and sorted item headers to let us test for the presence of an item's key with progressively more expensive methods. We hold on to a dirty segment and fill it as we create new items. This needs more fleshing out and testing but this is a solid first pass and it passes our existing tests. Signed-off-by: Zach Brown <zab@versity.com>