A vector search operation in Alternator (VectorSearch option to Query)
returns items sorted by decreasing similarity to the searched vector.
Although the items are sorted by decreasing similarity scores, before
this patch the user had no way to see the values of these scores.
This patch adds a new VectorSearch option, `ReturnScores`. This option
defaults to `NONE`. But if set to `SIMILARITY`, the query will return
an array `Scores` with the same length as `Items`, which gives the
similarity score for each item.
As usual, this patch includes the implementation, the documentation,
and tests for the new feature.
Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Today in Alternator vector search, vectors are presented to the API as
lists of numbers. I.e., in JSON a vector is sent in requests and responses
as:
{"L": [{"N": "3.14159"}, {"N":" "6.7"}}
This format is verbose and inefficient for long vectors. Even worse,
because the "N" number format has precision guarantees in DynamoDB,
we cannot optimize the storage of such vectors by, for example, storing
the numbers as 32-bit floats. We actually store these vectors as JSON,
exactly as shown above.
So in this patch we introduce a new DynamoDB type, "FLOAT32VECTOR", for
vectors. The above vector will look like this in JSON:
{"FLOAT32VECTOR": [3.14159, 6.7]}
Note that each number is an unquoted JSON number, not a JSON string.
Importantly, the definition of the "FLOAT32VECTOR" type specifies that
components of the vector only have 32-bit precision. This means that
Scylla may store internally these vectors as lists of 32-bit floats -
not as a JSON. And indeed, this patch includes this optimization:
Top-level vector attributes are now encoded in an optimized way,
as a byte 5 (alternator_type::FLOAT32VECTOR) followed by the elements
of the vector, just 4 bytes each (the 4-byte big-endian IEEE 754
representation of each floating-point component).
This patch also includes documentation, and extensive tests that the
new "FLOAT32VECTOR" type works (which also serves as an example how to
use it in the boto3 SDK), that it is indeed encoded internally as 32-bit
floats and not wasteful JSON strings, and that vector search on such items
work. The last thing requires cooperation from the vector store, of
course - it needs to be able to understand the new optimized encoding
of vector attributes in addition to the old unoptimized one.
Note that the old unoptimized ("list of numbers") vectors are still
supported. Although not recommended for general use, some users might
still want to use the unoptimized type if they have pre-existing data
created on DynamoDB or Alternator without vector search in mind, and
the vectors already exist as lists of numbers.
Although this is less important, the new vector type "FLOAT32VECTOR"
is also allowed in a Query's QueryVector.
Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Before this patch, vector search always used the COSINE similarity
function. In this patch we add the ability to choose a different
similarity function when creating a new vector index (with CreateTable
or UpdateTable) by using the SimilarityFunction option. We still default
to "COSINE" if SimilarityFunction isn't specified.
Allowed similarity functions are COSINE, DOT_PRODUCT, and EUCLIDEAN.
DescribeTable can also retrieve a vector index's SimilarityFunction.
As usual, this patch also includes documentation for the new feature,
and tests. Some of the tests can run without a vector store - verifying
the API syntax and which similarity function is supported - but we
also add tests that require the vector store and check that the different
similarity functions actually sort the nearest items in the expected
order.
Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Before this patch, we did not have any special metrics for vector search
in Alternator. We have had count of "Query" operations, but there was no
distinction between "standard" queries - of a base table or GSI/LSI -
and vector-search queries.
This patch adds four new metrics:
* vector_search_query - counting how many Query requests are actually
vector searches.
* vector_search_query_returned_items - counting how many items were
returned by vector searches.
* vector_search_query_items_from_vs - counting how many results were
retrieved from the vector-store backend.
* vector_search_query_items_from_base_table - counting how many items
were read from the base table during vector-search queries. Some
vector search queries using SELECT=ALL_PROJECTED_ATTRIBUTES or COUNT
are optimized to not need to read items from the base table.
This patch also includes documentation for the new four metrics, and
tests that they count what we want them to count.
Signed-off-by: Nadav Har'El <nyh@scylladb.com>
This patch adds a new document, docs/alternator/vector-search.md, on the
new vector search feature in Alternator. It introduces this feature, and
the DynamoDB APIs that we extended to support it.
Signed-off-by: Nadav Har'El <nyh@scylladb.com>
