Commit Graph

19272 Commits

Author SHA1 Message Date
Nadav Har'El
3600a096be alternator: fix JSON in DescribeTable response
The structure's name in DescribeTable's output is supposed to be called
"Table", not "TableDescription". Putting in the wrong place caused the
driver's table creation waiters to fail.

Signed-off-by: Nadav Har'El <nyh@scylladb.com>
2019-08-19 15:48:16 +03:00
Nadav Har'El
9fbf601f44 alternator: validate table name in CreateTable
validate table name in CreateTable, and if it doesn't fit DynamoDB's
requirement, return the appropriate error as drivers expect.

With this patch, test_table.py::test_create_table_unsupported_names
now passes (albeit with a one minute pause - this a bug with keep-alive
support...).

Signed-off-by: Nadav Har'El <nyh@scylladb.com>
2019-08-19 15:48:16 +03:00
Nadav Har'El
7b400913fc alternator-test: test_create_table_unsupported_names minor fix
Check the expected error message to contain just ValidationException
instead of an overly specific text message from DynamoDB, so we aren't
so constraint in our own messages' wording.

Signed-off-by: Nadav Har'El <nyh@scylladb.com>
2019-08-19 15:48:16 +03:00
Nadav Har'El
623fd5d10d alternator-test: test for creating table with very long name
Dynamo allows tables names up to 255 characters, but when this is tested on
Alternator, the results are disasterous: mkdir with such a long directory
name fails, Scylla considers this an unrecoverable "I/O error", and exits
the server.

Signed-off-by: Nadav Har'El <nyh@scylladb.com>
2019-08-19 15:48:16 +03:00
Nadav Har'El
8c9e88f77d test-table: test DescribeTable on non-existent table
Signed-off-by: Nadav Har'El <nyh@scylladb.com>
2019-08-19 15:48:16 +03:00
Nadav Har'El
4f3b0af6d0 Add "--local" option to run test against local Scylla installation
For example "pytest --local test_item.py"

Signed-off-by: Nadav Har'El <nyh@scylladb.com>
2019-08-19 15:48:16 +03:00
Nadav Har'El
830805e608 test_item.py: basic string put and get test
Signed-off-by: Nadav Har'El <nyh@scylladb.com>
2019-08-19 15:48:16 +03:00
Nadav Har'El
b87057eb08 test_table fixture: be quicker to realize table was created.
Signed-off-by: Nadav Har'El <nyh@scylladb.com>
2019-08-19 15:48:16 +03:00
Nadav Har'El
53ac8dfb0a test_table fixture: automatically delete
Automatically delete the test table when the test ends.

Signed-off-by: Nadav Har'El <nyh@scylladb.com>
2019-08-19 15:48:16 +03:00
Nadav Har'El
8a19ae8e39 test_item.py: start testing CRUD operations
Signed-off-by: Nadav Har'El <nyh@scylladb.com>
2019-08-19 15:48:16 +03:00
Nadav Har'El
41775a6e5d Start to use "test fixtures"
Start to use "test fixtures" defined in conftest.py: The connection to
the DynamoDB API, and also temporary tables, can be reused between multiple
tests.

Signed-off-by: Nadav Har'El <nyh@scylladb.com>
2019-08-19 15:48:16 +03:00
Nadav Har'El
7adde41f55 Add some table tests and README
Signed-off-by: Nadav Har'El <nyh@scylladb.com>
2019-08-19 15:48:16 +03:00
Nadav Har'El
c926f1bb53 alternator: very initial implementation of DescribeTable
This initial implementation is enough to pass a test of getting a
failure for a non-existant table -
test_table.py::test_describe_table_non_existent_table
and to recognize an existing table. But it's still missing a lot
of fields for an existing table (among others, the schema).

Signed-off-by: Nadav Har'El <nyh@scylladb.com>
2019-08-19 15:48:16 +03:00
Nadav Har'El
e6ff060944 alternator: errors should be output from server as Dynamo drivers expect
Exceptions from the handlers need to be output in a certain way - as
a JSON with specific fields - as DynamoDB drivers expect them to be.
If a handler throws an alternator::api_error with these specific fields,
they are output, but any other exception is converted into the same
format as an "Internal Error".

After this patch, executor code can throw an alternator::api_error and
the client will receive this error in the right format.

Signed-off-by: Nadav Har'El <nyh@scylladb.com>
2019-08-19 15:48:16 +03:00
Nadav Har'El
b5fa4ce3f3 alternator: add alternator::api_error exception type
DynamoDB error messages are returned in JSON format and expect specific
information: Some HTTP error code (often but not always 400), a string
error "type" and a user-readable message. Code that wants to return
user-visible exceptions should use this type, and in the next patch we
will translate it to the appropriate JSON string.

Signed-off-by: Nadav Har'El <nyh@scylladb.com>
2019-08-19 15:48:16 +03:00
Nadav Har'El
6e697e4cca alternator: table creation time is in seconds
The "Timestamp" type returned for CreationDateTime can be one of several
things but if it is a number, it is supposed to be the time in *seconds*
since the epoch - not in milliseconds. Returning milliseconds as we
wrongly did causes boto3 (AWS's Python driver) to throw a parse exception
on this response.

Signed-off-by: Nadav Har'El <nyh@scylladb.com>
2019-08-19 15:48:16 +03:00
Nadav Har'El
b2b7ae4e41 alternator: require alternator-port configuration
Until now, we always opened the Alternator port along with Scylla's
regular ports (CQL etc.). This should really be made optional.

With this patch, by default Alternator does NOT start and does not
open a port. Run Scylla with --alternator-port=8000 to open an Alternator
API port on port 8000, as was the default until now. It's also possible
to set this in scylla.yaml.

Signed-off-by: Nadav Har'El <nyh@scylladb.com>
2019-08-19 15:48:16 +03:00
Piotr Sarna
fca1e655b4 alternator: add minimal HTTP interface
The interface works on port 8000 by default and provides
the most basic alternator operations - it's an incomplete
set without validation, meant to allow testing as early as possible.
2019-08-19 15:46:47 +03:00
Avi Kivity
20aed3398d Merge "Simplify types" from Rafael
"
This is hopefully the last large refactoring on the way of UDF.

In UDF we have to convert internal types to Lua and back. Currently
almost all our types and hidden in types.cc and expose functionality
via virtual functions. While it should be possible to add a
convert_{to|from}_lua virtual functions, that seems like a bad design.

In compilers, the type definition is normally public and different
passes know how to reason about each type. The alias analysis knows
about int and floats, not the other way around.

This patch series is inspired by both the LLVM RTTI
(https://www.llvm.org/docs/HowToSetUpLLVMStyleRTTI.html) and
std::variant.

The series makes the types public, adds a visit function and converts
the various virtual methods to just use visit. As a small example of
why this is useful, it then moves a bit of cql3 and json specific
logic out of types.cc and types.hh. In a similar way, the UDF code
will be able to used visit to convert objects to Lua.

In comparison with the previous versions, this series doesn't require the intermediate step of converting void* to data_value& in a few member functions.

This version also has fewer double dispatches I a am fairly confident has all the tools for avoiding all double dispatches.
"

* 'simplify-types-v3' of https://github.com/espindola/scylla: (80 commits)
  types: Move abstract_type visit to a header
  types: Move uuid_type_impl to a header
  types: Move inet_addr_type_impl to a header
  types: Move varint_type_impl to a header
  types: Move timeuuid_type_impl to a header
  types: Move date_type_impl to a header
  types: Move bytes_type_impl to a header
  types: Move utf8_type_impl to a header
  types: Move ascii_type_impl to a header
  types: Move string_type_impl to a header
  types: Move time_type_impl to a header
  types: Move simple_date_type_impl to a header
  types: Move timestamp_type_impl to a header
  types: Move duration_type_impl to a header
  types: Move decimal_type_impl to a header
  types: Move floating point types  to a header
  types: Move boolean_type_impl to a header
  types: Move integer types to a header
  types: Move integer_type_impl to a header
  types: Move simple_type_impl to a header
  ...
2019-08-18 19:04:05 +03:00
Takuya ASADA
f574112301 dist/debian: handle --dist correctly
On ac9b115, it mistakenly ignores --dist option.
It should set 'housekeeping' template variable to 'enable'.

Fixes #4857

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20190816120127.14099-1-syuu@scylladb.com>
2019-08-18 15:00:33 +03:00
Avi Kivity
14d40cc659 Update seastar submodule
* seastar fe2b5b0c6...afc5bbf51 (4):
  > Merge "handle discarded futures or suppress warning" from Benny
  > Remove variadic futures from the Seastar implementation
  > Revert "Merge "handle discarded futures or suppress warning" from Benny"
  > io_queue: Forward declare smp class
2019-08-17 12:18:18 +03:00
Rafael Ávila de Espíndola
7f0a434cfa types: Move abstract_type visit to a header
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 16:25:43 -07:00
Rafael Ávila de Espíndola
dccefd1ddb types: Move uuid_type_impl to a header
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 16:25:43 -07:00
Rafael Ávila de Espíndola
038728a381 types: Move inet_addr_type_impl to a header
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 16:25:43 -07:00
Rafael Ávila de Espíndola
1966416cb3 types: Move varint_type_impl to a header
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 16:25:43 -07:00
Rafael Ávila de Espíndola
9229f99c86 types: Move timeuuid_type_impl to a header
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 16:25:43 -07:00
Rafael Ávila de Espíndola
993f132619 types: Move date_type_impl to a header
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 16:25:43 -07:00
Rafael Ávila de Espíndola
a299ed3b9b types: Move bytes_type_impl to a header
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 16:25:43 -07:00
Rafael Ávila de Espíndola
09ac2a1bc6 types: Move utf8_type_impl to a header
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 16:25:43 -07:00
Rafael Ávila de Espíndola
da472a65ec types: Move ascii_type_impl to a header
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 16:25:43 -07:00
Rafael Ávila de Espíndola
b98bac65b0 types: Move string_type_impl to a header
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 16:25:43 -07:00
Rafael Ávila de Espíndola
3e5b1e2630 types: Move time_type_impl to a header
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 16:25:43 -07:00
Rafael Ávila de Espíndola
909df932ac types: Move simple_date_type_impl to a header
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 16:25:43 -07:00
Rafael Ávila de Espíndola
8f3bebb6e8 types: Move timestamp_type_impl to a header
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 16:25:43 -07:00
Rafael Ávila de Espíndola
3260153d35 types: Move duration_type_impl to a header
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 16:25:42 -07:00
Rafael Ávila de Espíndola
2f6a26b1c1 types: Move decimal_type_impl to a header
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 16:25:42 -07:00
Rafael Ávila de Espíndola
480ca52b59 types: Move floating point types to a header
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 16:25:42 -07:00
Rafael Ávila de Espíndola
6a4ec7488e types: Move boolean_type_impl to a header
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 16:25:42 -07:00
Rafael Ávila de Espíndola
404b26d3fa types: Move integer types to a header
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 16:25:42 -07:00
Rafael Ávila de Espíndola
bd3e725605 types: Move integer_type_impl to a header
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 16:25:42 -07:00
Rafael Ávila de Espíndola
03aca28dc5 types: Move simple_type_impl to a header
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 16:25:42 -07:00
Rafael Ávila de Espíndola
e8ba37fa5a types: Move counter_type_impl to a header
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 16:25:42 -07:00
Rafael Ávila de Espíndola
cb03c79a48 types: Move empty_type_impl to a header
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 16:25:42 -07:00
Rafael Ávila de Espíndola
1cb7127bf3 types: Make abstract_type::serialize a static helper
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 16:25:42 -07:00
Rafael Ávila de Espíndola
b175657ee7 types: Devirtualize abstract_type::validate
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 16:25:42 -07:00
Rafael Ávila de Espíndola
bf96f1111c types: Make abstract_type::serialized_size a static helper
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 16:25:41 -07:00
Rafael Ávila de Espíndola
6831e05471 types: Move functions that use abstract_type::serialized_size out of line
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 10:02:00 -07:00
Rafael Ávila de Espíndola
047e34a31d types: Remove serialize_value
It is no longer needed.

Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 10:02:00 -07:00
Rafael Ávila de Espíndola
1e0663c56c types: Devirtualize abstract_type::from_string
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 10:02:00 -07:00
Rafael Ávila de Espíndola
68b26047cc types: Devirtualize abstract_type::serialize
Signed-off-by: Rafael Ávila de Espíndola <espindola@scylladb.com>
2019-08-14 10:02:00 -07:00