Commit Graph

53948 Commits

Author SHA1 Message Date
Pekka Enberg
e8d110a8e3 db/legacy_schema_tables: Convert calculateSchemaDigest() to C++
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-07-16 14:53:31 +03:00
Pekka Enberg
2520112bda db/system_keyspace: Add query_mutations() function
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-07-16 14:53:30 +03:00
Pekka Enberg
440157b07b db/system_keyspace: Use database::get_version() instead of random UUID
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-07-16 14:53:30 +03:00
Pekka Enberg
81cddec777 database: Add versioning support
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-07-16 14:53:30 +03:00
Pekka Enberg
c003f89484 utils/UUID_gen: Add bytes_view variant of get_name_UUID()
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
2015-07-16 14:53:30 +03:00
Avi Kivity
0b9eecc2d1 Merge branch 'master' of github.com:cloudius-systems/urchin into db 2015-07-16 13:10:49 +03:00
Avi Kivity
c74e36c30e Merge branch 'master' of github.com:cloudius-systems/urchin into db
Conflicts:
	message/messaging_service.cc
	message/messaging_service.hh
2015-07-16 12:51:19 +03:00
Tomasz Grabiec
3e69a0b137 Merge tag 'avi/syslog/v1' from seastar-dev.git
syslogd supprt from Avi:

Allow log output to go to syslog, from where it can be shipped to journald/
some other machine/splunk/whatever.  Disabled by default, exposed via config.
2015-07-16 12:45:45 +03:00
Avi Kivity
8af2975473 Merge "Improve messaging_service build time, part 4" from Asias 2015-07-16 12:33:27 +03:00
Avi Kivity
1d4805236b messaging_service: don't include config.hh in .hh
config.hh changes rapidly, so don't force lots of recompiles by including it.

Need to place seed_provider_type in namespace scope, so we can forward
declare it for that.
2015-07-16 12:26:02 +03:00
Asias He
c75e89da88 messaging_service: Farewell rpc.hh 2015-07-16 17:23:26 +08:00
Asias He
48c895aa18 messaging_service: Move rpc_protocol_wrapper and friends to .cc file 2015-07-16 17:23:26 +08:00
Asias He
244b9289c6 api/messaging_service: Use get_stats
Hide rpc::protocol<serializer, messaging_verb>::client from it.
2015-07-16 17:23:26 +08:00
Asias He
70236dbfa6 messaging_service: Add shard_info::get_stats helper
Will be used in api/messaging_service.cc.
2015-07-16 17:23:26 +08:00
Asias He
8c2cd037a2 messaging_service: Include rpc_types.hh
so that when we remove rpc.hh, types like rpc::closed_error will be
available.
2015-07-16 17:23:26 +08:00
Asias He
8bc59fb0b9 messaging_service: Move register_handler and send_message to .cc file 2015-07-16 17:23:26 +08:00
Asias He
14526e31b0 tests: Disable tests/urchin/message.cc for now
The interface of ms.send_message() will be gone shortly. We need to
write the tests using the new interface. Let's postpone it a bit.
2015-07-16 17:23:26 +08:00
Asias He
b07ae77585 messaging_service: Add wrapper for READ_DIGEST verb 2015-07-16 17:23:26 +08:00
Asias He
c19f5edefe messaging_service: Add wrapper for READ_MUTATION_DATA verb 2015-07-16 17:23:26 +08:00
Asias He
85c6dc0b7e messaging_service: Add wrapper for READ_DATA verb 2015-07-16 17:23:26 +08:00
Asias He
0707c15000 messaging_service: Add wrapper for MUTATION_DONE verb 2015-07-16 17:23:26 +08:00
Asias He
196c7d6f2b messaging_service: Add wrapper for MUTATION verb 2015-07-16 17:23:22 +08:00
Asias He
292ddd68c2 streaming: Delay adding wrappers for some stream verbs
It might be changed or be dropped since the upcoming change of streaming
code. It is not used at the moment.
2015-07-16 17:22:53 +08:00
Asias He
857fa5ccbb messaging_service: Add wrapper for STREAM_MUTATION verb 2015-07-16 17:19:51 +08:00
Asias He
1f6a76f2a0 messaging_service: Add wrapper for PREPARE_MESSAGE verb 2015-07-16 17:19:51 +08:00
Asias He
49b80535d9 messaging_service: Add wrapper for DEFINITIONS_UPDATE verb 2015-07-16 17:19:51 +08:00
Asias He
3e9d88e67c migration_manager: Do not use make_lw_shared in push_schema_mutation
The code does:

    auto fm = make_lw_shared<std::vector<frozen_mutation>>(schema.begin(), schema.end());
    return net::get_local_messaging_service().send_message_oneway(net::messaging_verb::DEFINITIONS_UPDATE,
         std::move(id), std::move(fm));

    ms.register_handler(net::messaging_verb::DEFINITIONS_UPDATE, [this] (std::vector<frozen_mutation> m) {
    ...
    }

We should not send a lw_shared_ptr, but std::vector<frozen_mutation>.

However, from Gleb:

   Well, this is not a bug, this is really cool (and to be honest
   unintended) feature of RPC. It is smart enough to detect that it is a
   smart pointer to an object and dereference it. But in this particular
   case there is not justification to use shared_ptr in the first place.

So, drop the lw_shared_ptr anyway.
2015-07-16 17:19:51 +08:00
Asias He
47503d0eaf messaging_service: Add wrapper for verbs used by gossip
Tested with tests/urchin/gossip.cc.
2015-07-16 17:19:51 +08:00
Asias He
ec0c1187ff messaging_service: Add out of line destructor 2015-07-16 17:19:50 +08:00
Raphael S. Carvalho
719898d0e5 introduce automatic compaction
As the name implies, this patch introduces the concept of automatic
compaction for sstables.

Compaction task is triggered whenever a new sstable is written.
Concurrent compaction on the same column family isn't supported, so
compaction may be postponed if there is an ongoing compression.
In addition, seastar::gate is used both to prevent a new compaction
from starting and to wait for an ongoing compaction to finish, when
the system is asked for a shutdown.

This patch also introduces an abstract class for compaction strategy,
which is really useful for supporting multiple strategies.
Currently, null and major compaction strategies are supported.
As the name implies, null compaction strategy does nothing.
Major compaction strategy is about compacting all sstables into one.
This strategy may end up being helpful when adding support to major
compaction via nodetool.

Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2015-07-16 12:00:12 +03:00
Avi Kivity
6257ef1d83 Merge "Add mutation::live_row_count()" from Tomasz 2015-07-16 10:32:11 +03:00
Avi Kivity
c6f889db03 Merge seastar upstream 2015-07-16 10:10:34 +03:00
Avi Kivity
2878bc887d Merge "Add rpc/rpc_types.hh" from Asias 2015-07-16 10:04:32 +03:00
Asias He
624388ae4b rpc: Move stats to rpc_types.hh
It will be needed by user does not want to include rpc.hh
2015-07-16 14:55:51 +08:00
Asias He
6ad04ff5e8 rpc: Introduce rpc/rpc_types.hh
Include rpc_types.hh instead of rpc.hh when no_wait_type and friends are
needed.
2015-07-16 14:55:50 +08:00
Raphael S. Carvalho
f7a1a5618b sstables: add missing #include guard
Signed-off-by: Raphael S. Carvalho <raphaelsc@cloudius-systems.com>
2015-07-16 07:25:56 +03:00
Tomasz Grabiec
c304b83e8d messaging_service: Fix read_serializable()
The "v" parameter points to uninitialized object in RPC deserializer,
but db::serializer<>::read() variant with output parameter assumes it
is initialized.

Spotted by Gleb. Should fix #21.
2015-07-16 06:50:28 +03:00
Tomasz Grabiec
e065480341 tests: Add test for mutation::live_row_count() 2015-07-15 18:56:10 +02:00
Tomasz Grabiec
33ca07af33 mutation: Introduce live_row_count() 2015-07-15 18:56:10 +02:00
Tomasz Grabiec
446ba5bf27 mutation_partition: Extract row liveness checking logic to deletable_row::is_live() 2015-07-15 18:38:12 +02:00
Avi Kivity
f256243ebd memory: fix debug build break (statistics) 2015-07-15 18:17:49 +03:00
Avi Kivity
248aa4f55e reactor: wire up free memory and reclaim counters to collected 2015-07-15 18:09:13 +03:00
Avi Kivity
a597d37bc1 memory: add statistics for reclaim operations and free memory 2015-07-15 18:08:54 +03:00
Avi Kivity
3d3da0ceb0 Merge "No longer pass db naked pointer to multiple CF methods" from Glauber
"Cleanups to avoid passing pointers around some CF methods."
2015-07-15 18:03:01 +03:00
Glauber Costa
04c0fbcb8c remove calls to seal_active_memtable
It should not be called directly: externall callers should be calling flush()
instead.

To be sure it doesn't happen again, make seal_active_memtable private.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-15 10:24:20 -04:00
Glauber Costa
9c464aff9b database: clean up various APIs
In much of our column_families APIs, we need to pass a pointer to the database.
The only reason we do that, is so we can properly handle the commit log entries
after we seal the current memtables into sstables.

Now that we store a pointer to the commit log in the CF itself at the time it
is created, we no longer have to do it. As a result, the APIs are a lot
cleaner, with no gratuitous parameters.

My motivation for this was the flush method, but as a result, apply() also gets
cleaner.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-15 10:24:20 -04:00
Glauber Costa
ad46daa6aa column_family: add the commitlog as a parameter
When we create a column family, we can pass as an extra parameter, the
commitlog - or lack thereof. Because the commitlog is optional to begin with -
it won't exist if we don't call init_commitlog, we can have this to be empty
meaning no commit log.

The creation of a column family should be always done through
add_column_family. And if that is the case, we have the database's commitlog
right there and can get the pointer through the db. Only tests are not creating
the column family this way, and for them, it is fine.

We want to do that, because some column family operations will use the commit log.
Right now, they are forcing us to add parameters to APIs that would be much cleaner
without it. So while separation is good, this level of coupling is a net win as it
allows us to clean up some visible APIs.

Signed-off-by: Glauber Costa <glommer@cloudius-systems.com>
2015-07-15 10:24:20 -04:00
Avi Kivity
f8a7c5bd2b main: allow configuring log destination (stdout and/or syslog) 2015-07-15 16:21:12 +03:00
Avi Kivity
91d8d46709 logger: support for syslog()
Note that while syslog() may block, we still call it synchronously via
the reactor thread.  Anything else would make logging a nightmare.
2015-07-15 16:20:37 +03:00
Avi Kivity
5f88707d66 Merge "API: Replacing empty string to void" from Amnon 2015-07-15 14:52:15 +03:00