A vector search operation in Alternator (VectorSearch option to Query)
returns items sorted by decreasing similarity to the searched vector.
Although the items are sorted by decreasing similarity scores, before
this patch the user had no way to see the values of these scores.
This patch adds a new VectorSearch option, `ReturnScores`. This option
defaults to `NONE`. But if set to `SIMILARITY`, the query will return
an array `Scores` with the same length as `Items`, which gives the
similarity score for each item.
As usual, this patch includes the implementation, the documentation,
and tests for the new feature.
Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Today in Alternator vector search, vectors are presented to the API as
lists of numbers. I.e., in JSON a vector is sent in requests and responses
as:
{"L": [{"N": "3.14159"}, {"N":" "6.7"}}
This format is verbose and inefficient for long vectors. Even worse,
because the "N" number format has precision guarantees in DynamoDB,
we cannot optimize the storage of such vectors by, for example, storing
the numbers as 32-bit floats. We actually store these vectors as JSON,
exactly as shown above.
So in this patch we introduce a new DynamoDB type, "FLOAT32VECTOR", for
vectors. The above vector will look like this in JSON:
{"FLOAT32VECTOR": [3.14159, 6.7]}
Note that each number is an unquoted JSON number, not a JSON string.
Importantly, the definition of the "FLOAT32VECTOR" type specifies that
components of the vector only have 32-bit precision. This means that
Scylla may store internally these vectors as lists of 32-bit floats -
not as a JSON. And indeed, this patch includes this optimization:
Top-level vector attributes are now encoded in an optimized way,
as a byte 5 (alternator_type::FLOAT32VECTOR) followed by the elements
of the vector, just 4 bytes each (the 4-byte big-endian IEEE 754
representation of each floating-point component).
This patch also includes documentation, and extensive tests that the
new "FLOAT32VECTOR" type works (which also serves as an example how to
use it in the boto3 SDK), that it is indeed encoded internally as 32-bit
floats and not wasteful JSON strings, and that vector search on such items
work. The last thing requires cooperation from the vector store, of
course - it needs to be able to understand the new optimized encoding
of vector attributes in addition to the old unoptimized one.
Note that the old unoptimized ("list of numbers") vectors are still
supported. Although not recommended for general use, some users might
still want to use the unoptimized type if they have pre-existing data
created on DynamoDB or Alternator without vector search in mind, and
the vectors already exist as lists of numbers.
Although this is less important, the new vector type "FLOAT32VECTOR"
is also allowed in a Query's QueryVector.
Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Before this patch, we did not have any special metrics for vector search
in Alternator. We have had count of "Query" operations, but there was no
distinction between "standard" queries - of a base table or GSI/LSI -
and vector-search queries.
This patch adds four new metrics:
* vector_search_query - counting how many Query requests are actually
vector searches.
* vector_search_query_returned_items - counting how many items were
returned by vector searches.
* vector_search_query_items_from_vs - counting how many results were
retrieved from the vector-store backend.
* vector_search_query_items_from_base_table - counting how many items
were read from the base table during vector-search queries. Some
vector search queries using SELECT=ALL_PROJECTED_ATTRIBUTES or COUNT
are optimized to not need to read items from the base table.
This patch also includes documentation for the new four metrics, and
tests that they count what we want them to count.
Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Non-finite numbers (Inf, NaN) don't make sense in vector search, and
also not allowed in the DynamoDB API as numbers. But the parsing code
in Query's QueryVector accepted "Inf" and "NaN" and then failed to
send the request to the vector store, resulting in a strange error
message. Let's fix it in the parsing code.
We have a test (test_query_vectorsearch_queryvector_bad_number_string)
that verifies this fix.
Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Before this patch, if we attempt a Query with IndexName is a vector index
but forget a "VectorSearch" parameter, the error is misleading: The code
expects a GSI or LSI, and when it can't find a GSI or LSI with that name,
it reports that the index is missing. But this is not helpful. So in this
patch we produce a more helpful message: That the index does exist, and
is a vector index, so a "VectorSearch" parameter is mandatory and is
missing.
The per-table metrics for Query were not incremented for the
vector variant of the Query operations, only the global metrics were
incremented. This patch fixes this oversight, and add a test that
reproduces it (the new test fails before this patch, and passes after).
De-duplicate some code introduced in earlier patches, such a two
nearly-identical loops over the indexes (one to check if there is a
vector index, the second to get its dimensions), and two nearly-
identical chunks of code to get the item contents when there is or
there isn't a clustering key.
There should be no functional changes in this patch.
Signed-off-by: Nadav Har'El <nyh@scylladb.com>
In earlier patches, when Query'ing a vector index, we set the default
Select to ALL_ATTRIBUTES. However, according to the DynamoDB documentation
for Query,
"If neither Select nor ProjectionExpression are specified, DynamoDB
defaults to ALL_ATTRIBUTES when accessing a table, and
ALL_PROJECTED_ATTRIBUTES when accessing an index."
This default should also apply to vector index, so this patch fixes this.
The new behavior is not only more compatible with DynamoDB, it is also
much more efficient by default, as ALL_PROJECTED_ATTRIBUTES does not need
to read from the base table - it returns the results that the vector store
returned. Of course, if the user needs the more efficient ALL_ATTRIBUTES
this option is still available - it's just no longer the default.
Signed-off-by: Nadav Har'El <nyh@scylladb.com>
This patch continues the effort to split the huge executor.cc (5000
lines before this patch) even more.
In this patch we introduce a new source file, executor_util.cc, for
various utility functions that are used for many different operations
and therefore are useful to have in a header file. These utility
functions will now be in executor_util.cc and executor_util.hh -
instead of executor.cc and executor.hh.
Various source files, including executor.cc, the executor_read.cc
introduced in the previous patch, as well as older source files like
as streams.cc, ttl.cc and serialization.cc, use the new header file.
This patch removes over 700 lines of code from executor.cc, and
also removes a large amount of utility functions declerations from
executor.hh. Originally, executor.hh was meant to be about the
interface that the Alternator server needs to *execute* the different
DynamoDB API operations - and after this patch it returns closer to
this original goal.
Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Already six years ago, in #5783, we noticed that alternator/executor.cc
has grown too large. The previous patches added hundreds of more lines
to it to implement vector search, and it reached a whopping 7,000 lines
of code. This is too much.
This patch splits from executor.cc two major chunks:
1. The implementation of **read** requests - GetItem, BatchGetItem,
Query (base table, GSI/LSI, and vector-search), and Scan - was
moved to a new source file alternator/executor_read.cc.
The new file has 2,000 lines.
2. Moved 250 lines of template functions dealing with attribute paths
and maps of them to a new header file, attribute_path.hh.
These utilities are used for many different operations - various
read operations use them for ProjectionExpression, and UpdateItem
uses them for modifications to nested attributes, so we need the
new header file from both executor.cc and executor_read.cc
The remaining executor.cc is still pretty big, 5,000 lines, and
contains write operations (PutItem, UpdateItem, DeleteItem,
BatchWriteItem) as well as various table and other operations, and
also many utility functions used by many types of operations, so
we can later continue this refactoring effort.
Refs #5783
Signed-off-by: Nadav Har'El <nyh@scylladb.com>
