Commit Graph

125 Commits

Author SHA1 Message Date
Amnon Heiman
75c53e4f24 scylla-housekeeping: rename configuration file from conf to cfg
Files with a conf extension are run by the scylla_prepare on the AMI.
The scylla-housekeeping configuration file is not a bash script and
should not be run.

This patch changes its extension to cfg which is more python like.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1470896759-22651-2-git-send-email-amnon@scylladb.com>
(cherry picked from commit 5a4fc9c503)
2016-08-11 14:45:11 +03:00
Avi Kivity
e2c9feafa3 Merge "Add configuration file to scylla-housekeeping" from Amnon
"The series adds an optional configuration file to the scylla-housekeeping. The
file act as a way to prevent the scylla-housekeeping to run. A missing
configuration file, will make the scylla-housekeeping immediately.

The series adds a flag to the build_rpm that differentiate between public
distributions that would contain the configuration file and private
distributions that will not contain it which will cause the setup script to
create it."

(cherry picked from commit da4d33802e)
2016-08-10 13:34:04 +03:00
Takuya ASADA
aecda01f8e dist/common/scripts: disable coredump compression by default, add an argument to enable compression on scylla_coredump_setup
On large memory machine compression takes too long, so disable it by default.
Also provide a way to enable it again.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1469706934-6280-1-git-send-email-syuu@scylladb.com>
(cherry picked from commit 89b790358e)
2016-08-05 09:47:17 +03:00
Amnon Heiman
d096a6762a scylla_setup: Ask if to start scylla-housekeeping
The scylla-server.service will try to start the scylla-housekeeping.

This patch adds a question to the scylla_setup if to enable the version
check, if the answer is no, the scylla-housekeeping will be masked.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1468741129-1977-1-git-send-email-amnon@scylladb.com>
2016-07-17 17:57:12 +03:00
Amnon Heiman
610fe274fd services: Make scylla-jmx service depends on scylla-server
The scylla-jmx no longer shutdown itself. A better setup would be that
the it would be started when the scylla-server starts and that it would
shutdown when the scylla-server shutdown.

This patch do the scylla-server part of the change.

The scylla-server definition would Want the scylla-jmx.service so there
is no need to enable the scylla-jmx.service.

A patch to the scylla-jmx would cause it to shutdown when the scylla-jmx
shutsdown.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1467184502-4358-1-git-send-email-amnon@scylladb.com>
2016-06-29 11:36:04 +03:00
Benoît Canet
e37b18b231 scylla_ntp_setup: Define an ntp server on ubuntu if there is none
The pool directive from ntp.conf is not recognized by ntpdate.
Strip it and put the ubuntu server in place.

Fixes: #1345

Signed-of-by: Benoît Canet <benoit@scylladb.com>
Message-Id: <1466607457-14029-1-git-send-email-benoit@scylladb.com>
2016-06-23 12:40:13 +03:00
Tzach Livyatan
3fa7bb1292 scylla_setup: Ignore case in prompt responses
Fix #1376 by converting each response to lowercase.

Signed-off-by: Tzach Livyatan <tzach@scylladb.com>
Message-Id: <1466672539-5625-1-git-send-email-tzach@scylladb.com>
2016-06-23 12:08:26 +03:00
Benoît Canet
8e4dee0bd1 scylla_setup: Hide /dev/loop*
The user probably don't want to use
/dev/loop* as RAID devices.

Fixes: #1259

Signed-of-by: Benoît Canet <benoit@scylladb.com>
Message-Id: <1466520602-7888-1-git-send-email-benoit@scylladb.com>
2016-06-21 19:27:40 +03:00
Tzach Livyatan
27b99f47e8 scylla_setup: improve the wording of disk setup phase.
Fix #1197 by adding XFS related info to the interactive prompt

Signed-off-by: Tzach Livyatan <tzach@scylladb.com>
Message-Id: <1466504625-28926-1-git-send-email-tzach@scylladb.com>
2016-06-21 19:26:31 +03:00
Lucas Meneghel Rodrigues
ae622b0c08 dist/common/scripts/scylla_kernel_check: Update messages
Small grammar tweaks to the script's output messages.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@scylladb.com>
Message-Id: <1466205496-3885-3-git-send-email-lmr@scylladb.com>
2016-06-19 19:28:58 +03:00
Lucas Meneghel Rodrigues
aacf7eb2ae dist/common/scripts/scylla_kernel_check: Fix conditional statement
Since most of the time people are running scylla_setup on
a fully upgraded ubuntu 14.04 box, we rarely reach that
code path, but once we do we end up with an error. Let's
fix that.

Signed-off-by: Lucas Meneghel Rodrigues <lmr@scylladb.com>
Message-Id: <1466205496-3885-2-git-send-email-lmr@scylladb.com>
2016-06-19 19:28:56 +03:00
Benoît Canet
c133748a24 scylla_setup: Fix RAID device enumeration
Commit f42673ed1e ("scylla_setup: Hide
busy block devices from RAID0 configuration") wasn't enumerating
anything.  Additionally it listed from /dev/ and not /dev/dm which broke
the tests conditions.

This one uses blkid instead of /proc/partitions.

A follow up patch will be required to mask encrypted devices.

Signed-of-by: Benoît Canet <benoit@scylladb.com>
Message-Id: <1466059657-12377-1-git-send-email-benoit@scylladb.com>
2016-06-16 09:52:25 +03:00
Benoît Canet
0cf8144485 scylla_setup: Propose defaults values when judicious
Also takes care of explaining the options.

Fixes #1031

Signed-of-by: Benoît Canet <benoit@scylladb.com>
Message-Id: <1466011848-11054-1-git-send-email-benoit@scylladb.com>
2016-06-15 20:33:55 +03:00
Benoît Canet
263a55c0da scylla_setup: Inform the user that he can skip any step
Fixes: #1188

Signed-of-by: Benoît Canet <benoit@scylladb.com>
Message-Id: <1466001423-9547-3-git-send-email-benoit@scylladb.com>
2016-06-15 19:38:23 +03:00
Benoît Canet
f42673ed1e scylla_setup: Hide busy block devices from RAID0 configuration
This patch look in /proc/mount for the device name so
the device or it's subdevices will be excluded from the availables
RAID0 targets. It does the same with physical volume from device
mapper.

Fixes #1189
Message-Id: <1466001423-9547-4-git-send-email-benoit@scylladb.com>
2016-06-15 19:36:11 +03:00
Shlomi Livne
ac6f2b5c13 dist/common: Update scylla_io_setup to use settings done in cpuset.conf
scylla_io_setup is searching for --smp and --cpuset setting in
SCYLLA_ARGS. We have moved the settings of this args into
/etc/scylla.d/cpuset.conf and they are set by scylla_cpuset_setup into
CPUSET.

Fixes: #1327

Signed-off-by: Shlomi Livne <shlomi@scylladb.com>
Message-Id: <2735e3abdd63d245ec96cfa1e65f766b1c12132e.1465508701.git.shlomi@scylladb.com>
2016-06-10 09:37:44 +03:00
Takuya ASADA
03a762bb0b dist/common/scripts: Ask to set SET_NIC=yes on scylla_setup interactive prompt
We supported SET_NIC on non-AMI environment, so ask user to use it on scylla_setup interactive prompt.
2016-05-19 06:26:23 +09:00
Takuya ASADA
898243929f dist/common/scripts: specify queue settings for posix_net_conf.sh on scylla_prepare
posix_net_conf.sh wants -sq/-mq options, so detect number of queues and specify the option in scylla_prepare.
2016-05-19 06:26:23 +09:00
Takuya ASADA
f84b7b094f dist/common/scripts: drop special condition to enable SET_NIC on AMI, do this on AMI installation script
Remove special case of SET_NIC in AMI, do this in scylla-ami-setup.service.
2016-05-19 06:25:41 +09:00
Takuya ASADA
49cdd0b786 dist: move '--cpuset' and '--smp' configuration to scylla_cpuset_setup / cpuset.conf
These parameters are only required for AMI, not for non-AMI environment which want to enable SET_NIC, so split them to indivisual script / conf file, call it from AMI install script.
2016-05-19 06:25:28 +09:00
Takuya ASADA
46fa80a5a6 dist/common/scripts: replace IFNAME variable when --nic specified to scylla_sysconfig_setup
scylla_sysconfig_setup has bug that it not replaces IFNAME variable, so fixed.
2016-05-19 06:25:15 +09:00
Takuya ASADA
4972a72380 dist: drop 'sudo -E' and SETENV for security reason, source envfile from scripts
As Nadav pointed out, SETENV and sudo -E might be causes security hole:
https://github.com/scylladb/scylla/issues/1028#issuecomment-196202171
So drop them now, sourcing envfiles from scylla_prepare / scylla_stop scripts
instead.

Also on "[PATCH] ubuntu: Fix the init script variable sourcing" thread
we have problem to passing variables from envfiles to scylla_prepare /
scylla_stop on Ubuntu, it seems better to sourcing from these scripts.

Additionally, this fixes #1249

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1462989906-30062-1-git-send-email-syuu@scylladb.com>
2016-05-17 10:31:03 +03:00
Takuya ASADA
80037aa95b dist/common/scripts: don't proceed to run scylla_raid_setup when disks not selected, on interactive RAID setup
When disks not selected, run disk select prompt again.
Fixes #1260

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1463388933-3640-1-git-send-email-syuu@scylladb.com>
2016-05-16 13:45:17 +03:00
Takuya ASADA
c38b5fbb3d dist/common/scripts: On scylla_io_setup, run iotune on correct data directory which specified on scylla.yaml
Currently scylla_io_setup hardcoded to run iotune on /var/lib/scylla, but user may change data directory by modifying scylla.yaml, and it may on different block device.
So use scylla_config_get.py to get configuration from scylla.yaml, passes it to iotune.

Fixes #1167

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1462955824-21983-2-git-send-email-syuu@scylladb.com>
2016-05-11 13:02:25 +03:00
Takuya ASADA
53820393da dist/common/scripts: add scylla.yaml parser for scripts
To parse scylla.yaml, scylla_config_get.py is added.
It can be use like 'scylla_config_get.py [key name]' from shell script, or command line.
This is needed for scylla_io_setup, to get 'data_file_directories' from shellscript.
Currently it does not supported to specify key name of nested data structure, but enough for scyll_io_setup.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1462955824-21983-1-git-send-email-syuu@scylladb.com>
2016-05-11 13:02:23 +03:00
Takuya ASADA
122330a5eb dist/common/scripts: add interactive prompt for package installation check, also check scylla-tools installed
Currently scylla_setup is unusable when user does not want to install scylla-jmx because it checks package unconditionally, but some users (or developers) does not want to install it, so let's ask to skip check or not on interactive prompt.

Also, scylla-tools package should installed for most of the case, added check code for the package.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1460662354-10221-1-git-send-email-syuu@scylladb.com>
2016-05-01 14:50:50 +03:00
Takuya ASADA
3ec47fbcf0 dist/ubuntu: unofficial support Debian 8.4
Unofficial support for Debian 8.4.
Now we supported both ubuntu and debian, but keep directory name as 'dist/ubuntu' for now.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1461006868-28273-1-git-send-email-syuu@scylladb.com>
2016-04-27 15:39:20 +03:00
Takuya ASADA
eb9bd3ee21 dist/common/scripts: show knowledge base URL when kernel is too old
To explain why this kernel is not supported, we need to show kb URL here.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1461644708-32078-1-git-send-email-syuu@scylladb.com>
2016-04-26 14:43:10 +03:00
Takuya ASADA
05ac4bb99d dist/common/scripts: notice restart required after changing bootparameters
Fixes #1115

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1459330851-32470-1-git-send-email-syuu@scylladb.com>
2016-04-26 14:41:49 +03:00
Takuya ASADA
c657a431dc dist/common/scripts: Fix incorrect order to run scylla_sysconfig_setup on scylla_setup
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1461174517-12441-2-git-send-email-syuu@scylladb.com>
2016-04-25 11:09:49 +03:00
Takuya ASADA
9a99231f6b dist/common/scripts: On scylla_setup, skip showing 'lo' interface on sysconfig prompt
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1461174517-12441-1-git-send-email-syuu@scylladb.com>
2016-04-25 11:09:48 +03:00
Takuya ASADA
611b0a3400 dist/common/scripts: Add kernel version check
Check kernel version at beginning of scylla_setup, show error when kernel is too old.
Use iotune --fs-check to check kernel.

Fixes #1116

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1459886738-10882-1-git-send-email-syuu@scylladb.com>
2016-04-24 17:47:13 +03:00
Benoît Canet
e17795d2dd scylla_dev_mode_setup: Unify --developer-mode prompt and parsing
Fixes: #1194

Signed-of-by: Benoît Canet <benoit@scylladb.com>
Message-Id: <1461002978-5379-2-git-send-email-benoit@scylladb.com>
2016-04-19 09:38:03 +03:00
Takuya ASADA
f98997120a dist: #!/bin/bash for all scripts
We choosed #!/bin/sh for shebang when we started to implement installer scripts, not bash.
After we started to work on Ubuntu, we found that we mistakenly used bash syntax on AMI script, it caused error since /bin/sh is dash on Ubuntu.
So we changed shebang to /bin/bash for the script, from that time we have both sh scripts and bash scripts.
(2f39e2e269)
If we use bash syntax on sh scripts, it won't work on Ubuntu but works on Fedora/CentOS, could be very easy to confusing.
So switch all scripts to #!/bin/bash. It will much safer.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1460594643-30666-1-git-send-email-syuu@scylladb.com>
2016-04-14 12:01:28 +03:00
Benoît Canet
7c99ecf16f scylla_setup: Check if scylla-jmx is installed
Signed-of-by: Benoît Canet <benoit@scylladb.com>

Fixes #1107
Message-Id: <1460045692-815-1-git-send-email-benoit@scylladb.com>
2016-04-08 09:03:38 +03:00
Takuya ASADA
73fa36b416 dist/common/scripts: update SET_NIC when --setup-nic passed to scylla_sysconfig_setup
scylla_sysconfig_setup mistakenly ignores --setup-nic argument.
Fixes #1132

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1459285500-22185-1-git-send-email-syuu@scylladb.com>
2016-03-30 11:07:33 +03:00
Takuya ASADA
c1277bacb4 dist/common/scripts: prevent misinterpret blank input as '/dev/', show error when inputted device path is not found
Fixes #1110

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1459267786-19123-1-git-send-email-syuu@scylladb.com>
2016-03-29 19:18:51 +03:00
Takuya ASADA
201b0c6ab3 dist: re-enable clocksource=tsc on AMI
clocksource=tsc on boot parameter mistakenly dropped on b3c85aea89, need to re-enable.

[ penberg: Manual backport of commit 050fb911d5 to 1.0. ]
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1459180643-4389-1-git-send-email-syuu@scylladb.com>

(cherry picked from commit 80242ff443)
2016-03-29 10:17:41 +03:00
Pekka Enberg
227daecba6 Revert "dist: move setup scripts to /usr/sbin"
This reverts commit 989357189a because it
broke our Jenkins packaging jobs.
2016-03-29 10:17:05 +03:00
Takuya ASADA
989357189a dist: move setup scripts to /usr/sbin
Since these scripts are user command, should be on $PATH.

Fixes #1092

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1458860407-25269-1-git-send-email-syuu@scylladb.com>
2016-03-25 11:50:13 +03:00
Shlomi Livne
d3a91e737b fix a collision betwen --ami command line param and env
sysconfig scylla-server includes an AMI, the script also used an AMI
variable fix this by renaming the script variable

6a18634f9f introduced this issue since it
started imported the sysconfig scylla-server

Signed-off-by: Shlomi Livne <shlomi@scylladb.com>
Message-Id: <0bc472bb885db2f43702907e3e40d871f1385972.1458767984.git.shlomi@scylladb.com>
2016-03-24 08:14:41 +02:00
Shlomi Livne
6a18634f9f scylla_io_setup import scylla-server env args
scylla_io_seup requires the scylla-server env to be setup to run
correctly. previously scylla_io_setup was encapsulated in
scylla-io.service that assured this.

extracting CPUSET,SMP from SCYLLA_ARGS as CPUSET is needed for invoking
io_tune

Signed-off-by: Shlomi Livne <shlomi@scylladb.com>
Message-Id: <d49af9cb54ae327c38e451ff76fe0322e64a5f00.1458747527.git.shlomi@scylladb.com>
2016-03-23 17:54:06 +02:00
Shlomi Livne
4ecc37111f dist/ami: Use the actual number of disks instead of AWS meta service
We have seen in some cases that when using the boto api to start
instances the aws metadata service
http://169.254.169.254/latest/meta-data/block-device-mapping/ returns
incorrect number of disks - workaround that by checking the actual
number of disks using lsblk

Adding a validation at the end verifying that after all computations the
NR_IO_QUEUES will not be greater then the number of shards (we had an
issue with i2.8x)

Fixes: #1062

Signed-off-by: Shlomi Livne <shlomi@scylladb.com>
Message-Id: <54c51cd94dd30577a3fe23aef3ce916c01e05504.1458721659.git.shlomi@scylladb.com>
2016-03-23 10:47:08 +02:00
Takuya ASADA
dac2bc3055 dist: on scylla_io_setup, SMP and CPUSET should be empty when the parameter not present
Fixes #1060

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1458659928-2050-1-git-send-email-syuu@scylladb.com>
2016-03-22 17:49:06 +02:00
Takuya ASADA
9889712d43 dist: remove scylla-io-setup.service and make it standalone script 2016-03-22 17:45:58 +02:00
Takuya ASADA
2cedab07f2 dist: on scylla_io_setup print out message both for stdout and syslog 2016-03-22 17:45:58 +02:00
Takuya ASADA
83112551bb dist: introduce dev-mode.conf and scylla_dev_mode_setup 2016-03-22 17:45:58 +02:00
Takuya ASADA
6b2a8a2f70 dist: enable collectd on scylla_setup by default, to make scyllatop usable
Fixes #1037

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1458324769-9152-1-git-send-email-syuu@scylladb.com>
2016-03-22 15:02:18 +02:00
Takuya ASADA
6edd909b00 dist: stop using '-p' option on lsblk since Ubuntu doesn't supported it
On scylla_setup interactive mode we are using lsblk to list up candidate
block devices for RAID, and -p option is to print full device paths.

Since Ubuntu 14.04LTS version of lsblk doesn't supported this option, we
need to use non-full path name and complete paths before passes it to
scylla_raid_setup.

Fixes #1030

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1458325411-9870-1-git-send-email-syuu@scylladb.com>
2016-03-21 14:54:36 +02:00
Avi Kivity
e1e4766cc6 Merge "Ubuntu based AMI support" from Takuya
"This provides Ubuntu based AMI support.
With this patchset, you will able to run build_ami.sh on Ubuntu 14.04LTS."
2016-03-20 20:40:21 +02:00