Commit Graph

5270 Commits

Author SHA1 Message Date
Asias He
efbdf428fc gossip: Remove commented code of resetVersion and destroyConnectionPool
Our messaging service is completely different. No need to reset version
or destroy connection pool.
2015-07-31 10:43:39 +08:00
Asias He
43c7ff217d gossip: Kill two warn unimplemented
It is implemented now.
2015-07-31 10:43:39 +08:00
Tomasz Grabiec
c68c37ae2f Revert "schema: support dropped columns"
This reverts commit 9887bc5f5f.

Breaks "CREATE TABLE" statement.
2015-07-30 10:56:01 +02:00
Avi Kivity
3a69561061 gdb: add 'scylla memory' command
Useful for analyzing memory leaks:

(gdb) scylla memory
objsz spansz    usedobj       memory  wst%
    1   4096          0            0   0.0
    1   4096          0            0   0.0
    1   4096          0            0   0.0
    1   4096          0            0   0.0
    2   4096          0            0   0.0
    2   4096          0            0   0.0
    3   4096          0            0   0.0
    3   4096          0            0   0.0
    4   4096          0            0   0.0
    5   4096          0            0   0.0
    6   4096          0            0   0.0
    7   4096          0            0   0.0
    8   4096        422         8192  58.8
   10   4096          0            0   0.0
   12   4096          0            0   0.0
   14   4096          0         8192  99.8
   16   4096        409         8192  20.1
   20   4096          0            0   0.0
   24   4096          0            0   0.0
   28   4096        653        24576  25.4
   32   4096       1069        36864   7.2
   40   4096          0            0   0.0
   48   4096          0            0   0.0
   56   4096       1228        73728   6.5
   64   4096        616        40960   3.8
   80   4096          0            0   0.0
   96   4096          0            0   0.0
  112   4096        919       110592   5.4
  128   4096        141        24576  26.6
  160   8192          0            0   0.0
  192   8192          0            0   0.0
  224   8192        128        49152  40.1
  256   8192         72        40960  55.0
  320  16384          0            0   0.0
  384  16384          0            0   0.0
  448  16384         60        49152  43.8
  512  16384         29        49152  69.8
  640  32768          0            0   0.0
  768  32768          0            0   0.0
  896  32768         47        98304  55.6
 1024  32768          5        98304  94.8
 1280  65536          0            0   0.0
 1536  65536          0            0   0.0
 1792  65536         31       196608  70.2
 2048  65536          8       196608  91.7
 2560 131072          0            0   0.0
 3072 131072          0            0   0.0
 3584 131072         34       393216  67.4
 4096 131072          0       393216 100.0
 5120 262144          0            0   0.0
 6144 262144          0            0   0.0
 7168 262144         33       786432  68.4
 8192 262144          4       786432  95.8
10240 524288          0            0   0.0
12288 524288          0            0   0.0
14336 524288         33      1572864  68.4
16384 524288          0      1572864 100.0
2015-07-30 10:56:01 +02:00
Pekka Enberg
1626ec1986 log: Show log level when logging to stdout
Fixes #38.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-07-30 10:56:01 +02:00
Avi Kivity
54af3e2d27 messaging_service: stop rpc client before removing it
Instead of just destroying the client, stop() it first and wait for that
to complete.  Otherwise any connect in progress, or any requests in progress,
will access a destroyed object (in practice it will fail earlier when
the _stopped promise is destroyed without having been fulfilled).

Depends on rpc client stop patch.
2015-07-30 11:13:30 +03:00
Avi Kivity
e93bdbb4f0 Merge seastar upstream
* seastar 98c6bba...572ec2d (2):
  > rpc: implement client::stop()
  > dpdk: i40e_should_linearize(): Remove extra check outside the loop
2015-07-30 11:12:49 +03:00
Avi Kivity
40c51733d0 Merge "Implement missing fields for legacy_schema_tables" from Glauber
"Issue #8 mentions the fact that we are missing some fields in legacy_schema_tables.

After this patchset, the only one missing is the subcomparator. That one only exists
for super tables - which we don't support - and will be null otherwise: so we don't
really need it."
2015-07-30 08:24:05 +03:00
Glauber Costa
9887bc5f5f schema: support dropped columns
Because we don't support alter table, we won't have anyone actually manipulating this.
But we will set up so it appears properly as an empty map in queries.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-29 18:44:59 -04:00
Glauber Costa
f49eb92880 schema: speculative retry
When added to the schema and handled by legacy_schema_tables.cc, will then
appear correctly in the respective system tables.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-29 18:44:59 -04:00
Glauber Costa
438a3f8619 storage_proxy: move speculative_retry type to schema
It will be stored in the schema, so move there, where it belongs.  We'll need
to do more than just store a type, so provide a class that encapsulates it.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-29 18:44:59 -04:00
Glauber Costa
01e5fcad5e schema: memtable flush period
currently set to 0, which is Origin's default

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-29 18:44:59 -04:00
Avi Kivity
dce642f472 commitlog: fix use-after-free of file during close
Reviewed-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-07-29 20:36:03 +03:00
Raphael S. Carvalho
d791438a43 db: enable automatic compaction by default
So far, automatic compaction was disabled, but now that we support
size-tiered strategy, the default compaction strategy algorithm,
we could definitely enable automatic compaction by default.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2015-07-29 19:02:16 +03:00
Avi Kivity
c93bb0d16c Merge "retry policy for compaction" from Raphael 2015-07-29 11:24:23 +03:00
Avi Kivity
f749c98748 Merge "gossip: shadow round fix" from Asias 2015-07-29 11:10:41 +03:00
Asias He
f326704240 gossip: Drop unused code 2015-07-29 15:45:46 +08:00
Asias He
04e1cbfac7 storage_service: Enable check_for_endpoint_collision
We do a shadow gossip round to check if the ip address is used by
another node.
2015-07-29 15:41:52 +08:00
Asias He
782b78a4c7 gossip: Introduce handle_ack_msg helper
It is shared by shadow-round and non-shadow-round.
2015-07-29 15:16:01 +08:00
Asias He
2ab3033d5d gossip: Fix do_shadow_round
Add the sleep logic.
2015-07-29 15:16:01 +08:00
Asias He
d38deef499 gossip: Kill a FIMXE for knows_version 2015-07-29 11:25:38 +08:00
Asias He
471a964d8e gossip: Kill a FIXME for clone_with_higher_version 2015-07-29 11:15:52 +08:00
Asias He
d799d1aa8f gossip: Implement the missing sleep logic 2015-07-29 11:13:35 +08:00
Asias He
1a6f7cf2aa gossip: Fix assassinate
- Use seastar::async to simplify the sleep logic.
- Futurize assassinate so that future is ready when assassination finishes.
2015-07-29 10:57:23 +08:00
Asias He
acd6e6268f gossip: Fix storage_service_value_factory
We have value_factory in storage_service now. Use it.
2015-07-29 10:06:43 +08:00
Asias He
33d3fcf7db gossip: Add set_last_processed_message_at() helper
Set time_point to now by default.
2015-07-29 09:54:29 +08:00
Asias He
65232edfe8 gossip: Move swagger API to source file 2015-07-29 09:49:28 +08:00
Raphael S. Carvalho
5a70c8c8f4 db: implement retry policy for compaction
Currently, compaction will no longer happen for a column family which
a compaction failed for some unexpected reason.
We want to implement a retry policy that will sleep for a while until
the next compaction attempt. This patch implements retry policy for
compaction using exponential_backoff_retry.
With exponential_backoff_retry, the sleep time grows exponentially
with the number of retries until the maximum sleep time is reached.
For compaction specifically, the base sleep time will be 5 seconds and
the maximum sleeping time will be 300 seconds, i.e. 5 minutes.
If compaction succeeded after a retry, the sleep time will be reset to
the base sleep time.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2015-07-28 18:24:04 -03:00
Raphael S. Carvalho
f71932f37f utils: add exponential_backoff_retry
This is a retry mechanism that exponentially increases sleep time
between retries.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2015-07-28 17:49:45 -03:00
Avi Kivity
f6e17ecab2 Merge seastar upstream
* seastar ce6c5c8...98c6bba (2):
  > dpdk: Linearize too fragmented xmit packet for i40e NICs
  > dpdk: get rid of backward compatibility code
2015-07-28 22:20:35 +03:00
Raphael S. Carvalho
c9fdc7dc5d compaction: get rid of invalid FIXME comment
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2015-07-28 19:22:26 +03:00
Avi Kivity
71579001ee Merge seastar upstream
* seastar 495822f...ce6c5c8 (2):
  > memory: count small_pool spans in use
  > build: disable dpdk kernel-mode drivers
2015-07-28 18:35:24 +03:00
Avi Kivity
b78621c5c6 Merge seastar upstream
* seastar 581f262...495822f (1):
  > core: fix close-after-free in recursive mkdir

Fixes #51
2015-07-28 17:12:11 +03:00
Pekka Enberg
804f5d98f1 version.hh: Pretend to be Cassandra 2.1.8
We are targeting the 2.1.x series so advertise that we're Cassandra
2.1.8 which is the latest stable release.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-07-28 16:58:46 +03:00
Asias He
079ea13e67 main: Add init_storage_service helper
Simplifies deinit call.
2015-07-28 15:21:08 +03:00
Avi Kivity
1df8b0471b Merge 2015-07-28 14:59:51 +03:00
Asias He
0ce3d89a85 gossip: Switch to use chrono for time operation
This is a long-awaited cleanup. Gossiper code runs every second, it is
not performance sensitive, so it does not make much sense to stick to
lowres db_clock, use high_resolution_clock instead.
2015-07-28 14:59:44 +03:00
Avi Kivity
fa13050f64 tests: deinit storage_service in cql_tests_env 2015-07-28 14:57:57 +03:00
Avi Kivity
f9d1c6b79d Merge "Improve CQL transport error handling" from Pekka
"This series converts some user-triggerable CQL transport layer
assertions to exceptions that are converted to proper CQL error
messages."
2015-07-28 14:29:13 +03:00
Pekka Enberg
1ae8522e9e transport/server: Fix unknown consistency level error handling
If we receive an unknown consistency level, fail with PROTOCOL_ERROR. If
we're trying to send out an unknown consistency level, fail with
SERVER_ERROR.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-07-28 14:02:40 +03:00
Pekka Enberg
c15de0257d transport/server: Fail with PROTOCOL_ERROR on unknown frame version
If we don't recognize a frame version, throw a protocol_exception like
Origin does rather than killing the process.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-07-28 13:59:11 +03:00
Pekka Enberg
d22adbf878 transport/server: Fail gracefully on unknown change and target type
If the switch statements don't match to change or target type
enumerations, throw a invalid_argument exception that's translated to
proper SERVER_ERROR. The error is not triggerable in practice as
change_type and target_type are enumerations. However, the compiler is
unhappy if we don't have them so lets just clean them up.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-07-28 13:55:26 +03:00
Pekka Enberg
e2692a9d08 transport/server: Fail gracefully if client asks for frame compression
Throw a runtime_exception which is translated to proper SERVER_ERROR
message rather than killing the process with assert().

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-07-28 13:46:57 +03:00
Pekka Enberg
d0826bc10d transport/server: Fix unknown opcode error handling
If we encounter an unknown opcode, throw a protocol exception rather
than killing the process with assert().

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-07-28 13:44:37 +03:00
Pekka Enberg
980dc910d9 exceptions: Move protocol_exception to exceptions.hh
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-07-28 13:44:20 +03:00
Avi Kivity
057cc804dc Merge "Clean up service startup code" from Asias
"With the seastar patch below, if we have this code

  engine().at_exit([] { return f1() });
  engine().at_exit([] { return f2() });

We can make sure f2 is called before f1, like

   f2.then([] {
       reutrn f1();
   })

We can use this feature to simplfy deinit code a lot. If we have the correct
init order, the deinit order will be correct if we always call
engine().at_exit() when we do init."
2015-07-28 13:24:21 +03:00
Asias He
e6438a701f main: Print ip address info for CQL and Thrift server
CQL server listening on 127.0.0.3:9042 ...
Thrift server listening on 127.0.0.3:9160 ...
Seastar HTTP server listening on 127.0.0.3:10000 ...
2015-07-28 18:20:45 +08:00
Asias He
15001ef1de main: Introduce api-address option
It is default to rpc-address.
2015-07-28 18:20:45 +08:00
Asias He
4839d385dc main: Cleanup snitch storage_service and db init 2015-07-28 18:20:45 +08:00
Asias He
a2b54fc757 main: Introduce init.cc to cleanup service startup code
This patch introduce init.cc file which hosts all the initialization
code. The benefits are 1) we can share initialization code with tests
code. 2) all the service startup dependency / order code is in one
single place instead of everywhere.
2015-07-28 18:20:45 +08:00