Commit Graph

3566 Commits

Author SHA1 Message Date
Avi Kivity
8bbfa540e5 doc: fix typo in do_until() 2015-06-02 11:22:41 +02:00
Avi Kivity
d91875d20a doc: low-level memory management 2015-06-02 11:22:41 +02:00
Avi Kivity
a77c8905cd doc: disambigute class future from the future documentation module 2015-06-02 11:22:40 +02:00
Avi Kivity
cb0cbff0c8 future: document more future-util functions 2015-06-02 11:22:40 +02:00
Amnon Heiman
7a4a9f37b1 http: Support file transformers
This runs the file transformer on a file before returning the result.

This is used for templating support.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-06-02 11:22:40 +02:00
Avi Kivity
a09619503e future: fix documentation errors
Noted by Nadav.
2015-06-02 11:22:40 +02:00
Avi Kivity
0877a666c4 doc: add main doxygen page 2015-06-02 11:22:40 +02:00
Avi Kivity
3b7f6f7704 future: user-level documentation 2015-06-02 11:22:39 +02:00
Avi Kivity
ad05d8e6a0 future: separate task class into its own header
tasks are a lower-level concept than future/promise, so move them to their
own header.
2015-06-02 11:22:39 +02:00
Gleb Natapov
ce89b77bba tests: add missing header 2015-06-02 11:22:39 +02:00
Avi Kivity
db46ced43c cql3: fix shared_ptr misuse in select_statement
A shared_ptr is mutable, so it must be thread_local, not static.
2015-06-01 17:34:00 +02:00
Avi Kivity
750543fc04 cql3: fix shared_ptr misuse in modification_statement
A shared_ptr is mutable, so it must be thread_local, not static.
2015-06-01 17:31:57 +02:00
Avi Kivity
b261bc9f42 cql3: fix shared_ptr misuse in cql3::constants
Global shared_ptrs are mutable, so make them shared_local.
2015-06-01 17:18:46 +02:00
Avi Kivity
b2ca316516 storage_proxy: fix shared_ptr misuse in query_local()
Instead of using do_with(), we open-code it and do it badly, dropping
the reference count on the remote shard.

Fix by dropping the reference count on the local core.
2015-06-01 17:18:35 +02:00
Calle Wilund
0729580f84 Separate replay_position into its own header
Less include bloat...

Signed-off-by: Calle Wilund <calle@cloudius-systems.com>
2015-06-01 14:17:43 +03:00
Gleb Natapov
fc176415f1 remove leftovers from db/serializer.hh 2015-06-01 09:31:58 +02:00
Avi Kivity
34f18d1847 Merge "token bootstrap"
Token bootstrap, from Asias.
2015-06-01 09:55:24 +03:00
Asias He
05939af7c7 token_metadata: Move update_host_id to source file 2015-06-01 11:33:11 +08:00
Asias He
40bf00bdb1 storage_service: Move some code to source file 2015-06-01 11:24:39 +08:00
Asias He
4c23121b47 token_metadata: Add helper to print debug info for _token_metadata 2015-06-01 11:24:39 +08:00
Asias He
7d39ca0b4a storage_service: Insert local host_id to _token_metadata.
This is how host_id is generated for each and every node.
We will learn other host_id of other node through gossip.
2015-06-01 11:24:39 +08:00
Asias He
3b4c233f55 system_keyspace: Add get_local_host_id and set_local_host_id 2015-06-01 11:24:39 +08:00
Asias He
6bffee5e11 token_metadata: Add update_host_id
Use std::unordered_map instead of boost:bimap. std::unordered_map is
much much easier to use. It is a premature optimization to user bimap.
We can iterate the map to check if host_id is unique. Modification of
host_id is not a frequent or performance sensitive operation anyway.
2015-06-01 11:24:38 +08:00
Asias He
e908c6a7ed UUID: Add default constructor
Needed by std::map operator[].
2015-06-01 11:24:38 +08:00
Asias He
291690d745 storage_service: Rename getRingDelay to get_ring_delay 2015-06-01 11:24:38 +08:00
Asias He
0acf306ccf storage_service: Implement on_change 2015-06-01 11:24:38 +08:00
Asias He
4829965496 storage_service: Enable handleStateBootstrap and friends
They will be used in on_change functions.
2015-06-01 11:24:38 +08:00
Asias He
904e584c14 storage_service: Implement on_join 2015-06-01 11:24:38 +08:00
Asias He
9f5093230c locator: Import TokenMetadata.java
We still need a lot of functions in Origin. Import the complete
TokenMetadata.java and convert as needed. For example, storage_service
will need updateHostId, addBootstrapTokens and more.
2015-06-01 11:24:38 +08:00
Asias He
8be4226a36 storage_service: Update _token_metadata with update_normal_token 2015-06-01 11:24:38 +08:00
Asias He
74fcb3d9aa storage_service: Add get_broadcast_address helper 2015-06-01 11:24:38 +08:00
Asias He
cbbaecf85c storage_service: Drop messaging_service handler registration code
We register the messaging_service handler within the subsystem who uses
it, e.g., we register gossip related in the gossiper class.
2015-06-01 11:24:38 +08:00
Asias He
3dc03738bc storage_service: Add debug info in gossip state change callback
Now, we can verify that get_tokens_for(ip) get the correct tokens
through gossip.

Start the seed node first:
$./seastar --listen-address 127.0.0.1 --seed-provider-parameters 127.0.0.1

Then start a normal node:
$./seastar --listen-address 127.0.0.2 --seed-provider-parameters 127.0.0.1

On the second node:

SS::on_join endpoint=127.0.0.1
endpoint=127.0.0.1,
tokens_string=8768e8a2d360bce8;d99d50eae7e514a2;f937ba030e4f1fa2
t=87 68 e8 a2 d3 60 bc e8
t=d9 9d 50 ea e7 e5 14 a2
t=f9 37 ba 03 0e 4f 1f a2

ep=127.0.0.1, eps=EndpointState: HeartBeatState = generation =
1432868747, version = 52, AppStateMap =  { 0 : Value(BOOT,TOKENS,11) }
{ 5 : Value(urchin_1_0,4) }  { 8 : Value(,3) }  { 11 : Value(ms_1_0,1) }
{ 12 : Value(9a6e8c60-05af-11e5-b9b8-000000000000,2) }  { 13 :
Value(8768e8a2d360bce8;d99d50eae7e514a2;f937ba030e4f1fa2,9) }
2015-06-01 11:24:38 +08:00
Asias He
e1b9b2e09a storage_service: Set tokens on startup 2015-06-01 11:24:38 +08:00
Asias He
03b526f06b storage_service: Implement get_tokens_for 2015-06-01 11:24:38 +08:00
Asias He
322b27c12b storage_service: Implement set_tokens 2015-06-01 11:24:38 +08:00
Asias He
d28503374d storage_service: Add debug print helper 2015-06-01 11:24:38 +08:00
Asias He
8d7aff89e3 storage_service: Reduce default num_tokens to 3
The default value 256 in Origin is too big for debug. We can set it back
when storage_service is maturer.
2015-06-01 11:24:38 +08:00
Asias He
1e289018ea gossip: Implement versioned_value for tokens
Tokens are store in versioned_value hex string split by semicolon, e.g.:

9f6fd6dd5149e39c;59068b2415190651;63684ccb1b73c1e3
2015-06-01 11:24:38 +08:00
Asias He
d026ca6852 token: Add constructor token(kind, bytes) 2015-06-01 11:24:38 +08:00
Asias He
487768732f storage_service: Rename bootstrapTokens to _bootstrap_tokens 2015-06-01 11:24:38 +08:00
Asias He
e6f5618d97 storage_service: Rename tokenMetadata to _token_metadata 2015-06-01 11:24:37 +08:00
Amnon Heiman
bdb2a7ff47 api: Remove empty parameter from storage_service.json
Empty parameter definition is not accepted by the swagger UI, instead,
the parameter list itself should be empty.

Signed-off-by: Amnon Heiman <amnon@cloudius-systems.com>
2015-05-31 18:17:29 +03:00
Avi Kivity
ff8769fdf7 Merge "Extend storage_service API support"
From Amnon:

"This seriese adds functionality to the storage_service API. It focus on tokens
information, that are partially supported in the storage_service and the call
to the API will return an empty list."
2015-05-31 17:57:30 +03:00
Gleb Natapov
699a47342d message: drop tuple<int, long> serializer
It is used only in test and even there it is not needed.
2015-05-31 17:53:51 +03:00
Vlad Zolotarov
81c9b327c6 compress: Use file::dma_read_exactly() instead of a local version of it.
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
2015-05-31 11:21:56 +03:00
Nadav Har'El
33270efc39 sstables: make consume_row_end() a future
After commit 3ae81e68a0, we already support
in input_stream::consume() the possibility of the consumer blocking by
returning a future. But the code for sstable consumption had now way to
use this capability. This patch adds a future<> return code for
consume_row_end(), allowing the consumer to pause after reading each
sstable row (but not, currently, after each cell in the row).

We also need to use this capability in read_range_rows(), which wrongly
ignored the future<> returned by the "walker" function - now this future<>
is returned to the sstable reader, and causes it to pause reading until
the future is fulfilled.

Signed-off-by: Nadav Har'El <nyh@cloudius-systems.com>
2015-05-28 18:53:32 +03:00
Glauber Costa
c02e3fc16a tests:: fix sstable test files removal for good
The current scheme of creating temporary files inside the current directory is
very fragile.

First, whenever we start writing new files, the test breaks if we forget to
include it in the list of removed files (likely).

Second, because we have to manually delete the files after we're done with
them, we can either forget, or if the test crashes, the final condition may not
run.

Instead of doing that, let's move the files to a separate directory. That makes
it a lot easier to make sure we're deleting all the relevant files, because we
can just wipe out the whole directory.

All tests are then wrapped in a function that makes sure that the proper setup
is in place.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-28 18:03:37 +03:00
Avi Kivity
7e4d8c3796 Merge seastar upstream 2015-05-28 18:01:10 +03:00
Glauber Costa
b77605cdba reactor: call remove instead of unlink
remove() is a convenient glibc wrapper that will unlink() a file and rmdir() a
directory. It will allow us to operate on both without creating a separate
function for a directory.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-05-28 17:56:42 +03:00