mirror of
https://github.com/google/nomulus
synced 2026-08-16 04:06:09 +00:00
gbrodmanandGitHub
a8f01d590d
Currently, we process (repoId, revisionId) pairs for DomainHistory and HostHistory individually -- they may be farmed out to worker nodes in parallel, but each EppResource uses a separate transaction and a separate read, which doesn't scale well when there are lots of domains/hosts. So as a result, we should batch them up so we can load (by default) 500 per transaction at a time. We don't want to batch-load the domains/hosts at the same time that we retrieve the most recent history entry for each type -- this would mean passing relatively large objects across pipeline steps. Instead, we keep passing the KV<String, Long> and batch retrievals. This isn't necessarily much faster (due to having to wait on batching) but there'll be less load on the DB. Self-scan D.2 number 5