Commit Graph

13 Commits

Author SHA1 Message Date
Piotr Sarna
5d76a635ca distributed_loader: migrate flush_upload_dir to thread
Flushing upload dir code suffers from overcomplication,
so in order to make it a little bit simpler, it's moved
to threaded context.

Refs #4118

Message-Id: <232cca077bae7116cfa87de9c9b4ba60efc2a01d.1548077720.git.sarna@scylladb.com>
2019-01-21 15:48:17 +02:00
Tomasz Grabiec
d7c701d2d1 Merge "Type-erase gratuitous templates with functions" from Avi
Many area of the code are splattered with unneeded templates. This patchset replaces
some of them, where the template parameter is a function object, with an std::function
or noncopyable_function (with a preference towards the latter; but it is not always
possible). As the template is compiled for each instantiation (if the function
object is a lambda) while a function is compiled only once, there are significant
savings in compile time and bloat.

   text    data     bss     dec     hex filename
85160690          42120  284910 85487720        5187068 scylla.before
84824762          42120  284910 85151792        5135030 scylla.after

* https://github.com/avikivity/scylla detemplate/v2:
  api/commitlog: de-template acquire_cl_metric()
  database: de-template do_parse_schema_tables
  database: merge for_all_partitions and for_all_partitions_slow
  hints: de-template scan_for_hints_dirs()
  schema_tables: partially de-template make_map_mutation()
  distributed_loader: de-template
  tests: commitlog_test: de-template
  tests: cql_auth_query_test: de-template
  test: de-template eventually() and eventually_true()
  tests: flush_queue_test: de-template
  hint_test: de-template
  tests: mutation_fragment_test: de-template
  test: mutation_test: de-template
2019-01-21 11:32:22 +01:00
Avi Kivity
baf9480c8d distributed_loader: de-template
distributed_loader has several large templates that can be converted to normal
function with the help of noncopyable_function<>, reducing code bloat.

One of the lambdas used as an actual argument was adjusted, because the de-templated
callee only accepts functions returning a future, while the original accepted both
functions returning a future and functions returning void (similar to future::then).
2019-01-20 15:55:20 +02:00
Avi Kivity
6e6372e8d2 Revert "Merge "Type-eaese gratuitous templates with functions" from Avi"
This reverts commit 31c6a794e9, reversing
changes made to 4537ec7426. It causes bad_function_calls
in some situations:

INFO  2019-01-20 01:41:12,164 [shard 0] database - Keyspace system: Reading CF sstable_activity id=5a1ff267-ace0-3f12-8563-cfae6103c65e version=d69820df-9d03-3cd0-91b0-c078c030b708
INFO  2019-01-20 01:41:13,952 [shard 0] legacy_schema_migrator - Moving 0 keyspaces from legacy schema tables to the new schema keyspace (system_schema)
INFO  2019-01-20 01:41:13,958 [shard 0] legacy_schema_migrator - Dropping legacy schema tables
INFO  2019-01-20 01:41:14,702 [shard 0] legacy_schema_migrator - Completed migration of legacy schema tables
ERROR 2019-01-20 01:41:14,999 [shard 0] seastar - Exiting on unhandled exception: std::bad_function_call (bad_function_call)
2019-01-20 11:32:14 +02:00
Piotr Sarna
3d65eb5d4a distributed_loader: restore indentation 2019-01-18 10:59:37 +01:00
Piotr Sarna
e50e9b5150 distributed_loader: restore always mutating to level 0
When introducing view update generation path for sstables
in /upload directory, mutating these sstables was moved
to regular path only. It was wrong, because sstables that
need view updates generated from them may still need
to be downgraded to LCS level 0, so they won't disrupt
LCS assumptions after being loaded.

Reported-by: Nadav Har'El <nyh@scylladb.com>
2019-01-18 10:35:20 +01:00
Avi Kivity
b6239134c2 distributed_loader: de-template
distributed_loader has several large templates that can be converted to normal
function with the help of noncopyable_function<>, reducing code bloat.
2019-01-17 18:56:22 +02:00
Piotr Sarna
0eb703dc80 all: rename view_update_from_staging_generator
The new name, view_update_generator, is both more concise
and correct, since we now generate from directories
other than "/staging".
2019-01-15 17:31:47 +01:00
Piotr Sarna
a5d24e40e0 distributed_loader: fix indentation
Bad indentation was introduced in the previous commit.
2019-01-15 17:31:37 +01:00
Piotr Sarna
13c8c84045 service: add generating view updates from uploaded sstables
SSTables loaded to the system via /upload dir may sometimes be needed
to generate view updates from them (if their table has accompanying
views).

Fixes #4047
2019-01-15 17:31:37 +01:00
Piotr Sarna
76616f6803 distributed_loader: use proper directory for opening SSTable
Previous implementation assumes that each SSTable resides directly
in table::datadir directory, while what should actually be used
is directory path from SSTable descriptor.
This patch prevents a regression when adding staging sstables support
for upload/ dir.
2019-01-15 16:47:01 +01:00
Duarte Nunes
b851cb1a9a distributed_loader: Forbid uploading MV sstables
Instead suggest that the views be re-created.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <20190103142933.35354-1-duarte@scylladb.com>
2019-01-03 16:31:20 +02:00
Avi Kivity
c180a18dbb Distribute distributed_loader into its own header and source files
distributed_loader is a sizeable fraction of database.cc, so moving it
out reduces compile time and improves readability.
Message-Id: <20181230200926.15074-1-avi@scylladb.com>
2018-12-31 14:27:27 +02:00