Commit Graph

3843 Commits

Author SHA1 Message Date
Avi Kivity
c209bbfee3 Merge "messaging service API" from Amnon
"This patch series adds counters to the rpc clients then it expose them in the
messaging service with a rest API, it was tested with the messaging_service
test app that was modified to run the API."
2015-06-16 15:23:30 +03:00
Amnon Heiman
0ffea496ac Adding the http server to the messaging_service test
To test the messaging service API it needs to be included in the
messaging service test.

To test it, start the server with --stay-alive true
then you can use the API to get the messges information:
http://localhost:10001/messaging_service/messages/exception
will return the number of exception per connection
http://localhost:10001/messaging_service/command/completed
will return the number of completed command per connection.

Note that because the two servers are running on the same machine, to
prevent port conflict, the server port will be increment by one (i.e
10001)

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-06-16 14:40:08 +03:00
Amnon Heiman
a928c7422d Add the messaging_service API implementation
This API gother information about messages passing in the system, in
this patch the following API will be supported:

/messaging_service/messages/replied
/messaging_service/messages/sent
/messaging_service/messages/pending
/messaging_service/messages/exception
/messaging_service/messages/respond_pending
/messaging_service/messages/dropped

The swagger defintion can be retrieved from:
/api-doc/messaging_service
2015-06-16 14:40:08 +03:00
Amnon Heiman
25188ed28e API: Adding a helper function to sum a map
In a typical scenario the API collect values from multiple distributed
instances. Sometimes it is needed to reduce multiple maps by merging
their keys and accumulating their values.

This is a helper function that can be used in map_reduce to perform maps
sum.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-06-16 14:40:08 +03:00
Amnon Heiman
04de4382f3 API Doc: Adding the messaging_service swagger definition
This adds the messaging_service API definition it will expose the
messaging_service metrics.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-06-16 14:39:51 +03:00
Amnon Heiman
896f562de7 Adding dropped messages counter to messaging_service
This adds a drop messages counter per verb type to the messaging
service. It will be used by the API to return the number of dropped
messages.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-06-16 14:35:21 +03:00
Amnon Heiman
b236e85961 Add a foreach_client method to the messaging service
The messaging service holds a table of clients which the API needs
information from. This adds a foreach_client method that recieve a
functions and itererate over all the clients calling the given function
on each of them.

This implementation support the current table that holds unique ptr.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-06-16 14:35:21 +03:00
Pekka Enberg
2d9397de58 db/legacy_schema_tables: Store columns to system tables
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-06-16 12:20:20 +02:00
Pekka Enberg
2776b5e657 db/legacy_schema_tables: Add missing attributes in add_table_to_schema_mutation()
Add attributes that we didn't support when the code was first
translated. Please note that we still don't cover all of the so somebody
needs to revisit this later.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-06-16 12:20:20 +02:00
Avi Kivity
78c2fc27f6 Merge "storage service updates" from Asias 2015-06-16 12:52:45 +03:00
Shlomi Livne
954c697958 dht: Update name of Murmur3Partitioner to align with org.apache.cassandra.dht.Murmur3Partitioner
In sstables the paritioner name is store for validation. To allow Origin
to process our files we need to comply with Origin's paritioner name or
else Origin's SSTableReader::open fails on paritioner comparison.
2015-06-16 12:20:04 +03:00
Nadav Har'El
49efd2a9ed build: strip all tests
[I'm sending this patch to Urchin, but it will also apply (with different
description, to Seastar]

This patch saves about 12 GB of disk space in Urchin's build directory.

Our debugging information is absurdly huge, and added to every single
executable we generate. This is most noticable in tests - we currently
have about 60 test executables, and each one contains a copy of all
this debug information.

This patch creates all test executables - whether debug or release mode -
stripped. When a user encounters a failing test he wants debug information
for (for gdb or the sanitizer), he can trivially relink it unstripped,
with a command like:

	ninja build/debug/tests/urchin/sstable_test_g

(note the added "_g"). This links the already existing object files
(which still have their debug information), which takes a few seconds.

On my machine, this patch reduces the Urchin build directory from about
22 GB to less than 10 GB. The build/release/tests directory drops from
6.8 GB to just 0.6 GB! The build/debug/tests directory is still huge (6 GB)
and could be improved further, but is also less than half of what it used
to be.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-06-16 11:41:26 +03:00
Asias He
2682d442aa storage_service: Add stop() needed by distributed<> 2015-06-16 15:45:15 +08:00
Asias He
7b6ab5aaa1 storage_service: Implement gossip_snitch_info
Now we have rack and datacenter info injected into gossip.

{ 3 : Value(rack1,9) }  { 4 : Value(datacenter1,11) }
2015-06-16 15:44:30 +08:00
Asias He
13f2292596 storage_service: Use fb_utilities::get_broadcast_address 2015-06-16 15:08:44 +08:00
Avi Kivity
743b6efd54 Merge "initial mutation clustering" from Gleb 2015-06-15 13:25:01 +03:00
Gleb Natapov
969134280a initial mutation clustering code 2015-06-15 12:53:10 +03:00
Gleb Natapov
7cee40bff2 add frozen_mutation serializer 2015-06-15 12:51:09 +03:00
Gleb Natapov
8fd7c45d4c make get_replication_factor a virtual function of abstract_replication_strategy 2015-06-15 12:51:09 +03:00
Gleb Natapov
5f3f8e641f add token_metadata::pending_enpoints_for() stub function 2015-06-15 12:51:09 +03:00
Gleb Natapov
a4c6b4a53f drop frozen_mutation::from_bytes() in favor of explicit constructor 2015-06-15 12:51:09 +03:00
Gleb Natapov
0657396fcb message: add generic std::vector serializer 2015-06-15 12:51:09 +03:00
Gleb Natapov
2d409250f2 remove ad-hoc token_metadata creation 2015-06-15 12:51:09 +03:00
Gleb Natapov
c418f9496b fix cql_query_test to initialize service that will be needed by clustering 2015-06-15 12:51:09 +03:00
Gleb Natapov
c500823d35 move init_storage_service out of main.cc 2015-06-15 12:51:04 +03:00
Pekka Enberg
e12cbb1836 db/legacy_schema_tables: Use async() for merge_schema()
Convert the merge_schema() function to use async() to make the code
easier to read and write.

Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-06-15 12:50:07 +03:00
Gleb Natapov
238a9d5d30 move init_messaging_service from main 2015-06-15 12:48:00 +03:00
Asias He
d2a9ea7ca6 storage_service: Fix Unable to contact any seeds
Sleep before do bootstrap. This code was not converted from Origin.

With this we can start multiple nodes simultaneously.

./build/release/seastar -c 1 -m 128M --rpc-address 127.0.0.1
--listen-address 127.0.0.1 --seed-provider-parameters 127.0.0.1
--datadir `pwd`/tmp/1 --commitlog-directory `pwd`/tmp/1 2>&1 | tee
/tmp/out1 &

./build/release/seastar -c 1 -m 128M --rpc-address 127.0.0.2
--listen-address 127.0.0.2 --seed-provider-parameters 127.0.0.1
--datadir `pwd`/tmp/2 --commitlog-directory `pwd`/tmp/2 2>&1 | tee
/tmp/out2 &

./build/release/seastar -c 1 -m 128M --rpc-address 127.0.0.3
--listen-address 127.0.0.3 --seed-provider-parameters 127.0.0.1
--datadir `pwd`/tmp/3 --commitlog-directory `pwd`/tmp/3 2>&1 | tee
/tmp/out3 &

./build/release/seastar -c 1 -m 128M --rpc-address 127.0.0.4
--listen-address 127.0.0.4 --seed-provider-parameters 127.0.0.1
--datadir `pwd`/tmp/4 --commitlog-directory `pwd`/tmp/4 2>&1 | tee
/tmp/out4
2015-06-15 12:30:02 +03:00
Avi Kivity
ca6e4e7d10 Merge seastar upstream 2015-06-15 11:45:00 +03:00
Avi Kivity
a4c711afd4 future: fix future<>::get0()
Empty tuples don't have a first element.
2015-06-15 11:43:38 +03:00
Avi Kivity
ddaaa315c8 future: add a get0() helper to get the first/only member of the result tuple
Reviewed-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-06-15 11:29:02 +03:00
Avi Kivity
ba24552efe Merge branch 'master' of github.com:cloudius-systems/urchin into db 2015-06-15 11:00:47 +03:00
Avi Kivity
446731cf88 Merge "column family API"
Column family API, from Amnon.
2015-06-15 10:50:23 +03:00
Tomasz Grabiec
c53ed98731 Merge branch 'shlomi/enable_cluster_via_config' from seatar-dev.git 2015-06-15 09:22:41 +02:00
Asias He
1d0b78d80f gossip: Fix capture by ref on a stack variable 2015-06-15 10:08:17 +03:00
Avi Kivity
053566b3e9 deleter: document 2015-06-15 08:47:44 +03:00
Avi Kivity
1769cb4520 temporary_buffer: document 2015-06-14 23:52:25 +03:00
Avi Kivity
5be417175e Merge "snitch creation"
From Vlad:

"Currently database always created a SimpleSnitch and ignores the corresponding parameter
provided by the user. This series fixes this situation:
   - Changes the snitch creation interface to comply the Java-like interface that
     has already been used in a topology_strategy classes family.
   - Fix all the places where a SimpleSnitch has been created ignoring the user configuration."
2015-06-14 17:59:15 +03:00
Avi Kivity
06031ea273 Merge seastar upstream 2015-06-14 17:47:32 +03:00
Avi Kivity
25420a6fdf core: add support for --cpuset command line option
Syntax: [cpu-]cpu(,[cpu-]cpu=)...
Default: all processors
2015-06-14 16:11:31 +03:00
Avi Kivity
f85a2b48bb resource: support using only a subset of a machine's processors
This is useful for running multiple seastar applications on the same
machine, for testing purposes.
2015-06-14 16:10:21 +03:00
Gleb Natapov
50b18a56cd test: add semaphore test 2015-06-14 16:03:46 +03:00
Gleb Natapov
361db498d1 semaphore: add wait() with timeout support 2015-06-14 16:02:19 +03:00
Gleb Natapov
f19ba7c334 Add timer move constructor 2015-06-14 16:02:18 +03:00
Gleb Natapov
1b2bf57a2b move timer out of reactor.hh to its own header 2015-06-14 16:02:16 +03:00
Vlad Zolotarov
2f14c53f4e gossiping_property_file_snitch: use a logger from i_endpoint_snitch
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-06-14 15:31:58 +03:00
Vlad Zolotarov
e045d8465c db: use snitch name from the configuration file
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-06-14 15:31:58 +03:00
Vlad Zolotarov
03ffaea768 locator: introduce i_endpoint_snitch::create_snitch()
- Kill make_snitch().
   - i_endpoint_snitch::create_snitch() uses the utilities from class_registrator.hh.

Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-06-14 15:31:49 +03:00
Shlomi Livne
39429e2481 Alignment
Signed-off-by: Shlomi Livne <shlomi@cloudius-systems.com>
2015-06-14 12:19:11 +03:00
Shlomi Livne
edfb4fc671 Add support for read seeds from configuration
Moved setting of configuration variables after the configuration file
has been read.

Updated the code parsing seeds to comply with configuration file format
- seeds: <ip1>,<ip2>,<ip3>

Signed-off-by: Shlomi Livne <shlomi@cloudius-systems.com>
2015-06-14 11:48:03 +03:00