mirror of
https://github.com/scylladb/scylladb.git
synced 2026-04-24 10:30:38 +00:00
52f92b886b
This patch fixes a bug which caused sporadic failures of the Alternator test - test_streams.py::test_streams_last_result. The GetRecords operation reads from an Alternator Streams shard and then returns an "iterator" from where to continue reading next time. Because we obviously don't want to read the same change again, we "incremented" the current position, to start at the incremented position on the next read. Unfortunately, the implementation of the increment() function wasn't quite right. The position in the CDC log is a timeuuid, which has a really bizarre comparison function (see compare_visitor in types.cc). In particular the least-sigificant bytes of the UUID are compared as *signed* bytes. This means that if the last byte of the UUID was 127, and increment() increased it to 128, and this was wrong because the comparison function later deemed that as a signed byte, where 128 is lower than 127, not higher! The result was that with 1/256 probability (whenever the last byte of the position was 127) we would return an item twice. This was reproduced (with 1/256 probability) by the test test_streams_last_result, as reported in issue #7004. The fix in this patch is to drop the increment() and replace it by a flag whether an iterator is inclusive of the threshold (>=) or exclusive (>). The internal representation of the iterator has a boolean flag "inclusive", and the string representation uses the prefixes "I" or "i" to indicate an inclusive or exclusive range, respectively - whereas before this patch we always used the prefix "I". Although increment() could have been fixed to work correctly, the result would have been ugly because of the weirdness of the timeuuid comparison function. increment() would also require extensive new unit-tests: we were lucky that the high-level functional tests caught a 1 in 256 error, but they would not have caught rarer errors (e.g., 1 in 2^32). Furthermore, I am looking at Alternator as the first "user" of CDC, and seeing how complicated and error-prone increment() is, we should not recommend to users to use this technique - they should use exclusive (>) range queries instead. Fixes #7004. Signed-off-by: Nadav Har'El <nyh@scylladb.com> Message-Id: <20200901102718.435227-1-nyh@scylladb.com>