Commit Graph

5949 Commits

Author SHA1 Message Date
Asias He
33db0995b9 api/storage_service: Add get_release_version
$ curl -X GET --header "Accept: application/json"
"http://127.0.0.1:10000/storage_service/release_version"

"2.1.8"
2015-08-26 06:51:47 +08:00
Asias He
620a5ae5b6 storage_service: Implement get_schema_version 2015-08-26 06:51:47 +08:00
Asias He
9172578f78 storage_service: Implement get_release_version 2015-08-26 06:51:47 +08:00
Avi Kivity
e6965c520d Merge "Adding the ownership suport to storage_service" from Amnon
"This series adds the missing code from origin to support this functionality.
While doing so, some method where changed to be const when it was more
appropriate and a few const version of methods where added when the two
variation was required."
2015-08-25 20:13:33 +03:00
Avi Kivity
6e3cf26c73 Merge "improve mutation related errors logging" from Vlad
Fixes #30.
2015-08-25 19:44:25 +03:00
Amnon Heiman
c92bd9b121 API: Adding the ownership implmentation to storage_service
This adds the ownwership method implementation to the storage_service
API. After the patch the following url will be supported:

GET /storage_service/ownership/{keyspace}
GET /storage_service/ownership/

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-08-25 19:39:14 +03:00
Amnon Heiman
2c5716dac3 API: storage_service Add the swagger definition for ownership
This adds the API for get_effective_ownership and
get_ownership in storage_service.

It is based on the StorageServiceMBean definition.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-08-25 19:39:13 +03:00
Amnon Heiman
fd20f167e7 storage_service: get_ranges_for_endpoint, get_ownership and
effective_ownership

This patch adds the implementation for get_ranges_for_endpoint,
get_ownership and effective_ownership based on origin implementation.

The methods are used by the API.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-08-25 19:39:13 +03:00
Amnon Heiman
b5ceef451e keyspace: Add the get_non_system_keyspaces and expose the replication
This patch adds the get_non_system_keyspaces that found in origin and
expose the replication strategy. With the get_replication_strategy
method.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-08-25 19:39:13 +03:00
Amnon Heiman
6d875f1ec1 token_metadata: Add const when applicable
This patch adds a const version for get_datacenter_endpoints and
get_topology.

It modified the token iterator to use a const version of token_metadata
and it make first_token, first_token_index, tokens_end and ring_range to
be a const method.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-08-25 19:39:13 +03:00
Amnon Heiman
3d27fa442b strategy: make some of the function const
This patch adds const for some of the methods that are not actually
changing the object content.

This is useful when using a const reference to the object.

calculate_natural_endpoints and has_sufficient_replicas where set as
const. The override was added where it was missing.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-08-25 19:39:13 +03:00
Amnon Heiman
5e03524d9d murmur3_partitioner: add describe_ownership from origin
The code was taken from origin, where instead of BigInteger, uint64_t was used.

The function returns the part that each token is responsible for, the sum of
all is raughly 1.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-08-25 19:39:08 +03:00
Vlad Zolotarov
08e7736f0b database::find_column_family(): init the exception with the readable message
Make the exceptions created inside database::find_column_family() return
a readable message from their what() method.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-08-25 18:00:19 +03:00
Vlad Zolotarov
eafa491009 storage_proxy: log all exceptions thrown in the mutation verb handler flow
Catch and log the exceptions thrown in the mutation receiver flow for better
debugability.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>

New in v2:
   - Don't dowithicate the ready future - call make_ready_future<>().then(...)
     directly.
   - Fix and unify the messages printed when the exception is caught.
2015-08-25 18:00:16 +03:00
Tomasz Grabiec
2bfb138910 Fix typos 2015-08-25 17:07:35 +03:00
Tomasz Grabiec
f2713561f5 row_cache: Avoid copy when moving whole entry from memtable to cache 2015-08-25 17:07:34 +03:00
Tomasz Grabiec
cb72d02c98 row_cache: Rename underlying_negative to presence_checker 2015-08-25 17:07:33 +03:00
Avi Kivity
b22a598efb mutation_reader: make noncopyable
Many mutation_reader implementations capture 'this', which, if copied,
becomes invalid.  Protect against this error my making mutation_reader
a non-copyable object.

Fix inadvertant copied around the code base.
2015-08-25 15:49:08 +03:00
Avi Kivity
5cd8a66291 compaction: avoid use-after-free
Compaction moves output_writer and done out of variables which are then
reused, causing use-after-free.  In addition it fails to protect the variables
for the end of the loop.

Fix by avoiding the moves and adding an additional capture.
2015-08-25 15:49:08 +03:00
Amnon Heiman
f55c168ae7 API: Add column family mean row size
This adds the column family mean row size in the per column family and
the total version. I uses the ratio_helper class to calculate the mean
over all the shrades.
2015-08-25 15:48:25 +03:00
Amnon Heiman
7dc098ed3d estimated histogram mean method should be const
The mean method does not change the object and should be const.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-08-25 15:48:24 +03:00
Amnon Heiman
db30a588b2 API: Break the async repair into two operations
This distinguish between the async repair that starts the repair, that
will now be a POST request and the method that check on the command
progress that will now be a GET command.

After the change each operation would get the parameters that it needs.

The GET will return an enum based on the repair_status.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-08-25 15:45:30 +03:00
Tomasz Grabiec
4c030bb597 api: Fix return type for /lsa/compact 2015-08-25 15:29:10 +03:00
Gleb Natapov
8599e9d84f protect messaging_service destruction by a gate object
Pointer to messageing_service object is stored in each request
continuation, so the object destruction should not happen while any of
these continuations is scheduled. Use gate object to ensure that.
2015-08-25 15:28:58 +03:00
Avi Kivity
979b7116d0 Merge seastar upstream
* seastar 5efee51...23f4fae (2):
  > stream: use a smaller stick for failed producers
  > stream: properly set exception
2015-08-25 15:24:49 +03:00
Avi Kivity
ab113731e8 Merge seastar upstream
* seastar 6444208...5efee51 (1):
  > json2code: Add enum support in return type
2015-08-25 14:32:46 +03:00
Avi Kivity
185a94135b Merge "export bloom filter statistics through management API" from Glauber
"This patchset implements the functions requested in Issue 95.
They are now made available through the management interface."

Fixes #95.
2015-08-25 08:57:30 +03:00
Glauber Costa
ca2d058520 api/column family: bloom filter file size
Export information about on-disk space used by bloom filters.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-08-24 20:14:31 -05:00
Glauber Costa
3dc135c380 api/column family: bloom filter ratios
Just like the simple statistics, but composed derived from them.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-08-24 20:14:28 -05:00
Glauber Costa
c094ba22c8 api/column family: bloom filter statistics
This patch uses the now existing infrastructure to expose statistics about the bloom
filters hit/miss rates.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-08-24 19:18:54 -05:00
Glauber Costa
2bfc2697c1 sstables: add method to grab filter size
This is one of the statistics we need to export

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-08-24 19:18:54 -05:00
Glauber Costa
7f04c1bf9b sstables: simplify filter tracker
The current filter tracker uses a distributed mechanism, even though the values
for all CPUs but one are usually at zero. This is because - I wrongly assumed -
that when using legacy sstables, the same sstable would be serving keys for
multiple shards, leading to the map reduce being a necessary operation in this
case.

However, Avi currently point out that:

"It is and it isn't [the case].  Yes the sstable will be loaded on multiple cores, but
each core will have its own independent sstable object (only the files on disk
are shared).

So to aggregate statistics on such a shared sstables, you have to match them by
name (and the sharded<filter_tracker> is useless)."

Avi is correct in his remarks. The code will hereby be simplified by keeping
local counters only, and the map reduce operation will happen at a higher
level.

Also, because the users of the get methods will go through the sstable, we can
actually just move them there. With that we can leave the counters private to
the external world in the filter itself.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-08-24 19:17:31 -05:00
Glauber Costa
df56020d58 filter: initialize all statistics
We are currently initializing some of the filter statistics. That can lead to
bogus values.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-08-24 17:10:57 -05:00
Avi Kivity
0e089d2a93 Merge 2015-08-24 19:16:42 +03:00
Gleb Natapov
bfa1ec31b9 drop explicit copying of captured parameters in mutation sending code
Now they are copied implicitly during call to messaging_service::send_mutation()
2015-08-24 19:16:37 +03:00
Avi Kivity
0617aecb62 lsa: downgrade "no compactible pool" warning to trace
It's a fairly standard condition.
2015-08-24 17:26:48 +02:00
Avi Kivity
4390be3956 Rename 'negative_mutation_reader' to 'partition_presence_checker'
Suggested by Tomek.
2015-08-24 18:03:22 +03:00
Avi Kivity
76703cceab Merge seastar upstream
* seastar cfa8db0...6444208 (1):
  > file: don't be noisy during close-in-destructor
2015-08-24 16:06:44 +03:00
Avi Kivity
3c29f255fb Merge seastar upstream
* seastar e359379...cfa8db0 (6):
  > rpc: test rcp timeout
  > rpc: add timeout support
  > tests: add (another) allocator test
  > memory: try harder to allocate large blocks
  > memory: avoid allocation in realloc() if shrinking object size
  > Update dpdk submodule
2015-08-24 15:24:57 +03:00
Avi Kivity
e222d2a463 Merge "storage_service update: fix bootstrap state" from Asias
"The bootstrap state is now correct with cqlsh query."
2015-08-24 14:24:29 +03:00
Asias He
0d047e86d7 storage_service: Kill one FIXME in join_token_ring
string to token function is now available.
2015-08-24 18:54:42 +08:00
Asias He
ce0435c105 storage_service: Switch to use get0 instead of std::get
It is simpler.
2015-08-24 18:54:42 +08:00
Asias He
1deaef1cc5 storage_service: Use get() inside set_tokens
set_tokens always runs inside a seastar thread. Use get() instead
returning a future. Preparation for calling get_local_tokens.
2015-08-24 18:54:42 +08:00
Asias He
7c4703cebf storage_service: Enable get_local_tokens
Will be used in set_tokens.
2015-08-24 18:54:42 +08:00
Asias He
dfae224cb4 storage_service: Update license for storage_service.cc
This file contains code converted from Origin.
2015-08-24 18:54:42 +08:00
Asias He
cd1c902cf9 storage_service: Call boot_strapper::bootstrap
Now that boot_strapper::bootstrap is available, use it. It will set
_is_bootstrap_mode set to false, so now we can enable the
assert(!_is_bootstrap_mode) follows the call to bootstrap.
2015-08-24 18:54:42 +08:00
Asias He
22ee468428 db/system_keyspace: Fix set_bootstrap_state
We set status to COMPLETED in join_token_ring

   set_bootstrap_state(db::system_keyspace::bootstrap_state::COMPLETED)

but

   cqlsh 127.0.0.$i -e "SELECT * from system.local;"

shows

    bootstrapped -> IN_PROGRESS

The static sstring state_name is the bad boy.
2015-08-24 18:54:42 +08:00
Asias He
126fc5869c dht/boot_strapper: Move code to source file
get_bootstrap_tokens and get_random_tokens are moved.
2015-08-24 18:54:42 +08:00
Asias He
26861ddc29 dht/boot_strapper: Use unordered_set for tokens
unordered_set is used everywhere for tokens. This makes it is easier to
construct a boot_strapper object in storage_service::bootstrap where
unordered_set is used for tokens.
2015-08-24 18:54:42 +08:00
Asias He
2ebd08cb42 dht/boot_strapper: Partially implement bootstrap 2015-08-24 18:54:42 +08:00