Commit Graph

406 Commits

Author SHA1 Message Date
Takuya ASADA
caab57bb24 dist/redhat/centos_dep: disable go and ada language on scylla-gcc package, since ScyllaDB never use them
centos-master jenkins job failed at building libgo, but we don't need go language, so let's disable it on scylla-gcc package.
Also we never use ada, disable it too.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1468166660-23323-1-git-send-email-syuu@scylladb.com>
(cherry picked from commit d2caa486ba)
2016-07-19 11:03:42 +03:00
Pekka Enberg
d3ffa00eb2 systemd: Use PermissionsStartOnly instead of running sudo
Use the PermissionsStartOnly systemd option to apply the permission
related configurations only to the start command. This allows us to stop
using "sudo" for ExecStartPre and ExecStopPost hooks and drop the
"requiretty" /etc/sudoers hack from Scylla's RPM.

Tested-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1466407587-31734-1-git-send-email-penberg@scylladb.com>
(cherry picked from commit 1d5f7be447)
2016-06-21 08:49:17 +03:00
Avi Kivity
c6a9844dfe dist: fix scylla-kernel-conf postinstall scriptlet failure
Because we build on CentOS 7, which does not have the %sysctl_apply macro,
the macro is not expanded, and therefore executed incorrectly even on 7.2,
which does.

Fix by expanding the macro manually.

Fixes #1360.
Message-Id: <1466250006-19476-1-git-send-email-avi@scylladb.com>

(cherry picked from commit 07045ffd7c)
2016-06-20 09:37:06 +03:00
Pekka Enberg
7b53e969d2 dist/docker: Use Scylla 1.2 RPM repository 2016-06-15 19:50:02 +03:00
Shlomi Livne
3688542323 dist/common: Update scylla_io_setup to use settings done in cpuset.conf
scylla_io_setup is searching for --smp and --cpuset setting in
SCYLLA_ARGS. We have moved the settings of this args into
/etc/scylla.d/cpuset.conf and they are set by scylla_cpuset_setup into
CPUSET.

Fixes: #1327

Signed-off-by: Shlomi Livne <shlomi@scylladb.com>
Message-Id: <2735e3abdd63d245ec96cfa1e65f766b1c12132e.1465508701.git.shlomi@scylladb.com>
(cherry picked from commit ac6f2b5c13)
2016-06-10 09:38:17 +03:00
Pekka Enberg
5e0b113732 Update scylla-ami submodule
* dist/ami/files/scylla-ami 72ae258...863cc45 (3):
  > Move --cpuset/--smp parameter settings from scylla_sysconfig_setup to scylla_ami_setup
  > convert scylla_install_ami to bash script
  > 'sh -x -e' is not valid since all scripts converted to bash script, so remove them
2016-06-06 13:38:53 +03:00
Avi Kivity
e515933c70 dist: tune scheduler for lower latency
Scylla-jmx and collectd can preempt scylla and induce long latencies.  Tune
the scheduler to provide lower latencies.

Since when the support processes are not running we normally do not context
switch (one thread per core, remember?), there should be no effect on
throughput.

The tunings are provided in a separate package, which can be uninstalled
if the server is shared with other applications which are negatively
affected by the tuning.

Fixes #1218.
Message-Id: <1464529625-12825-1-git-send-email-avi@scylladb.com>
2016-05-30 08:42:19 +03:00
Yoav Kleinberger
e580ac5dae docker: fix Ubuntu Dockerfile
one needs to update the repository info before one can install packages.
Fixes issue #1296.

Signed-off-by: Yoav Kleinberger <yoav@scylladb.com>
Message-Id: <a906e76d584baff5988cb31a4003de27455e0741.1464529740.git.yoav@scylladb.com>
2016-05-29 17:00:25 +03:00
Pekka Enberg
b7e79b72d5 Merge "Introduce SET_NIC for non-AMI environment" from Takuya
"This patchset provides a way to enable SET_NIC(posix_net_conf.sh) on
 non-AMI environment.
 Also support -mq option of the script.
 This also contains number of bug fixes of scripts.

 Fixes #1192"
2016-05-26 13:37:06 +03:00
Yoav Kleinberger
26c0d86401 tools/scyllatop: improved user interface: scrollable views
NOTE: scyllatop now requires the urwid library

previously, if there were more metrics that lines in the terminal
window, the user could not see some of the metrics.  Now the user can
scroll.

As an added bonus, the program will not crash when the window size
changes.

Signed-off-by: Yoav Kleinberger <yoav@scylladb.com>
Message-Id: <1464098832-5755-1-git-send-email-yoav@scylladb.com>
2016-05-26 13:36:28 +03:00
Takuya ASADA
53cebb4a5e dist/ubuntu: don't rebuild dependency packages by default
Same as CentOS, do not build dependencies by default, install binary packages from our repository.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1464023451-21436-1-git-send-email-syuu@scylladb.com>
2016-05-24 14:10:59 +03:00
Takuya ASADA
5d5d525a14 dist/ubuntu: fix incorrect dependency package name
PyYAML is CentOS/RHEL/Fedora package name, python-yaml is correct one.

Fixes #1279

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1463987823-22837-1-git-send-email-syuu@scylladb.com>
2016-05-23 10:21:29 +03:00
Pekka Enberg
8a7197e390 dist/docker: Fetch RPM repository from Scylla web site
Fix the hard-coded Scylla RPM repository by downloading it from Scylla
web site. This makes it easier to switch between different versions.

Message-Id: <1463981271-25231-1-git-send-email-penberg@scylladb.com>
2016-05-23 09:45:41 +03:00
Avi Kivity
5e5317b228 dist: add build dependencies for sctp
Required by new seastar
2016-05-22 19:10:25 +03:00
Takuya ASADA
03a762bb0b dist/common/scripts: Ask to set SET_NIC=yes on scylla_setup interactive prompt
We supported SET_NIC on non-AMI environment, so ask user to use it on scylla_setup interactive prompt.
2016-05-19 06:26:23 +09:00
Takuya ASADA
88fde0a91e dist/ami: fix dependency unresolved error on AMI build script with local package, by adding scylla-conf package
Since we added scylla-conf package, we cannot install scylla-server/-tools without the package, because of this --localrpm is failing.
So copy scylla-conf package to AMI, and install it to fix the problem.
2016-05-19 06:26:23 +09:00
Takuya ASADA
898243929f dist/common/scripts: specify queue settings for posix_net_conf.sh on scylla_prepare
posix_net_conf.sh wants -sq/-mq options, so detect number of queues and specify the option in scylla_prepare.
2016-05-19 06:26:23 +09:00
Takuya ASADA
f84b7b094f dist/common/scripts: drop special condition to enable SET_NIC on AMI, do this on AMI installation script
Remove special case of SET_NIC in AMI, do this in scylla-ami-setup.service.
2016-05-19 06:25:41 +09:00
Takuya ASADA
49cdd0b786 dist: move '--cpuset' and '--smp' configuration to scylla_cpuset_setup / cpuset.conf
These parameters are only required for AMI, not for non-AMI environment which want to enable SET_NIC, so split them to indivisual script / conf file, call it from AMI install script.
2016-05-19 06:25:28 +09:00
Takuya ASADA
46fa80a5a6 dist/common/scripts: replace IFNAME variable when --nic specified to scylla_sysconfig_setup
scylla_sysconfig_setup has bug that it not replaces IFNAME variable, so fixed.
2016-05-19 06:25:15 +09:00
Takuya ASADA
4972a72380 dist: drop 'sudo -E' and SETENV for security reason, source envfile from scripts
As Nadav pointed out, SETENV and sudo -E might be causes security hole:
https://github.com/scylladb/scylla/issues/1028#issuecomment-196202171
So drop them now, sourcing envfiles from scylla_prepare / scylla_stop scripts
instead.

Also on "[PATCH] ubuntu: Fix the init script variable sourcing" thread
we have problem to passing variables from envfiles to scylla_prepare /
scylla_stop on Ubuntu, it seems better to sourcing from these scripts.

Additionally, this fixes #1249

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1462989906-30062-1-git-send-email-syuu@scylladb.com>
2016-05-17 10:31:03 +03:00
Takuya ASADA
80037aa95b dist/common/scripts: don't proceed to run scylla_raid_setup when disks not selected, on interactive RAID setup
When disks not selected, run disk select prompt again.
Fixes #1260

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1463388933-3640-1-git-send-email-syuu@scylladb.com>
2016-05-16 13:45:17 +03:00
Takuya ASADA
8503600e30 dist/common/systemd: drop hardcoded path
Stop using /var/lib/scylla, use $SCYLLA_HOME instead.
systemd seems does not extract variables on Environment="HOME=$SCYLLA_HOME", but both CentOS/Ubuntu able to run scylla-server without $HOME, so dropped it.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1462977871-26632-1-git-send-email-syuu@scylladb.com>
2016-05-11 17:53:53 +03:00
Takuya ASADA
c38b5fbb3d dist/common/scripts: On scylla_io_setup, run iotune on correct data directory which specified on scylla.yaml
Currently scylla_io_setup hardcoded to run iotune on /var/lib/scylla, but user may change data directory by modifying scylla.yaml, and it may on different block device.
So use scylla_config_get.py to get configuration from scylla.yaml, passes it to iotune.

Fixes #1167

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1462955824-21983-2-git-send-email-syuu@scylladb.com>
2016-05-11 13:02:25 +03:00
Takuya ASADA
53820393da dist/common/scripts: add scylla.yaml parser for scripts
To parse scylla.yaml, scylla_config_get.py is added.
It can be use like 'scylla_config_get.py [key name]' from shell script, or command line.
This is needed for scylla_io_setup, to get 'data_file_directories' from shellscript.
Currently it does not supported to specify key name of nested data structure, but enough for scyll_io_setup.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1462955824-21983-1-git-send-email-syuu@scylladb.com>
2016-05-11 13:02:23 +03:00
Takuya ASADA
a73924b4e0 dist/ubuntu/dep: introduce scylla-gdb-7.11 for Ubuntu 14.04LTS
Introduce scylla-gdb-7.11 for Ubuntu 14.04LTS, to get better support of recent version of g++ on gdb.

Fixes #969

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1462825880-20866-3-git-send-email-syuu@scylladb.com>
2016-05-10 17:53:32 +03:00
Takuya ASADA
9ff2efb28b dist/common/dep: add Ubuntu support for scylla-env
Since Ubuntu 14.04LTS needs scylla-gdb package which install to /opt/scylladb, we need to port scylla-env package to Ubuntu as well.
This change introduces scylla-env package to Ubuntu 14.04LTS.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1462825880-20866-2-git-send-email-syuu@scylladb.com>
2016-05-10 17:53:32 +03:00
Takuya ASADA
43cc77d1b8 dist/redhat/centos_dep: move scylla-env to dist/common to share with Ubuntu
Since Ubuntu 14.04LTS needs scylla-gdb package which install to /opt/scylladb, we need to port scylla-env package to Ubuntu as well.
To do it, share the package directory on dist/common/dep at first.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1462825880-20866-1-git-send-email-syuu@scylladb.com>
2016-05-10 17:53:31 +03:00
Takuya ASADA
f7d41ba07a dist: Extract scylla.yaml and create metapackage
This patch create a scylla-conf package containing
scylla.yaml and a scylla package acting as a metapackage.

Fixes #421

Signed-off-by: Benoît Canet <benoit@scylladb.com>
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1462280987-26909-1-git-send-email-syuu@scylladb.com>
2016-05-09 11:23:28 +03:00
Takuya ASADA
122330a5eb dist/common/scripts: add interactive prompt for package installation check, also check scylla-tools installed
Currently scylla_setup is unusable when user does not want to install scylla-jmx because it checks package unconditionally, but some users (or developers) does not want to install it, so let's ask to skip check or not on interactive prompt.

Also, scylla-tools package should installed for most of the case, added check code for the package.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1460662354-10221-1-git-send-email-syuu@scylladb.com>
2016-05-01 14:50:50 +03:00
Takuya ASADA
cc74b6ff5f dist/ubuntu: move lines from rules to .install/.dirs/.docs
To simplify build script, and make it easier spliting two packages,
use .install/.dirs/.docs instead of rules.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1461960695-30647-1-git-send-email-syuu@scylladb.com>
2016-05-01 10:16:35 +03:00
Avi Kivity
434db0bc8b Update scylla-ami submodule
* dist/ami/files/scylla-ami 7019088...72ae258 (1):
  > Add --repo option to scylla_install_ami to construct AMI with custom repository URL
2016-04-28 16:41:30 +03:00
Takuya ASADA
6723978891 dist/ami: Add --repo option to build_ami.sh to construct AMI with custom repository URL
To build AMI from specified build of .rpm, custom repo URL option is required.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1461849370-11963-1-git-send-email-syuu@scylladb.com>
2016-04-28 16:40:49 +03:00
Takuya ASADA
3ec47fbcf0 dist/ubuntu: unofficial support Debian 8.4
Unofficial support for Debian 8.4.
Now we supported both ubuntu and debian, but keep directory name as 'dist/ubuntu' for now.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1461006868-28273-1-git-send-email-syuu@scylladb.com>
2016-04-27 15:39:20 +03:00
Takuya ASADA
1cfde50102 dist/ubuntu: support 16.04
Drop 'unsupported release' message on 16.04.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-04-27 18:06:59 +09:00
Takuya ASADA
988b7bcd3d dist/ubuntu: don't use ubuntu-toolchain-r/test ppa repo on recent versions of Ubuntu, since it has newer g++
On Ubuntu 15.04 and newer, official g++ package is >= g++-4.9.
So we don't need to use development repository, just use official package.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-04-27 18:06:59 +09:00
Takuya ASADA
fa0b90b727 dist/ubuntu: add dependency for libsystemd-dev to handle startup correctly on recent versions of Ubuntu
To handle scylla startup correctly on systemd versions of Ubuntu, scylla requires to build with libsystemd-dev.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-04-27 18:06:59 +09:00
Takuya ASADA
eae881ff70 dist/ubuntu: skip dh_installinit --upstart-only on recent versions of Ubuntu
Since 16.04LTS does not support this argument anymore, drop it on recent version of Ubuntu which does not uses Upstart.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-04-27 18:06:59 +09:00
Takuya ASADA
d5efa02eab dist/ubuntu/dep: Drop python-support on Ubuntu 16.04
Ubuntu 16.04 seems dropped python-support, so remove it from thrift package.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-04-27 18:06:59 +09:00
Takuya ASADA
e733c2aae8 dist/ubuntu/dep: use distribution's thrift-compiler-0.9.1 on newer versions of Ubuntu
Use distribution's thrift if version > 14.04LTS.
14.04LTS doesn't have thrift-compiler-0.9.1, use our version.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-04-27 18:06:59 +09:00
Avi Kivity
454512a272 dist/redhat: package scylla_kernel_check
Can't build rpm without this.
Message-Id: <1461683947-30356-1-git-send-email-avi@scylladb.com>
2016-04-27 08:38:48 +03:00
Takuya ASADA
eb9bd3ee21 dist/common/scripts: show knowledge base URL when kernel is too old
To explain why this kernel is not supported, we need to show kb URL here.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1461644708-32078-1-git-send-email-syuu@scylladb.com>
2016-04-26 14:43:10 +03:00
Takuya ASADA
05ac4bb99d dist/common/scripts: notice restart required after changing bootparameters
Fixes #1115

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1459330851-32470-1-git-send-email-syuu@scylladb.com>
2016-04-26 14:41:49 +03:00
Takuya ASADA
e7f438eeae dist/ubuntu: Drop dependency to libthrift0, link it statically
Drop dependency to libthrift0 on installation time, link libthrift statically.
With this fix, we don't need to distribute libthrift0 deb package anymore to install scylla-server binary package.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1461594460-2403-2-git-send-email-syuu@scylladb.com>
2016-04-25 17:44:46 +03:00
Takuya ASADA
c657a431dc dist/common/scripts: Fix incorrect order to run scylla_sysconfig_setup on scylla_setup
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1461174517-12441-2-git-send-email-syuu@scylladb.com>
2016-04-25 11:09:49 +03:00
Takuya ASADA
9a99231f6b dist/common/scripts: On scylla_setup, skip showing 'lo' interface on sysconfig prompt
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1461174517-12441-1-git-send-email-syuu@scylladb.com>
2016-04-25 11:09:48 +03:00
Takuya ASADA
611b0a3400 dist/common/scripts: Add kernel version check
Check kernel version at beginning of scylla_setup, show error when kernel is too old.
Use iotune --fs-check to check kernel.

Fixes #1116

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1459886738-10882-1-git-send-email-syuu@scylladb.com>
2016-04-24 17:47:13 +03:00
Benoît Canet
e17795d2dd scylla_dev_mode_setup: Unify --developer-mode prompt and parsing
Fixes: #1194

Signed-of-by: Benoît Canet <benoit@scylladb.com>
Message-Id: <1461002978-5379-2-git-send-email-benoit@scylladb.com>
2016-04-19 09:38:03 +03:00
Takuya ASADA
2eb91421eb dist/ami: Show correct login message when scylla-ami-setup.service is still running
While scylla-ami-setup.service is running, login message says "run systemctl status scylla-server" to see status, but it actually never launched yet.

This patch fixes the message to notice RAID construction is running, and 'systemctl status scylla-ami-setup' is the correct way to see status.

Fixes #1035

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1460660628-10103-2-git-send-email-syuu@scylladb.com>
2016-04-18 15:22:02 +03:00
Takuya ASADA
07a6057c03 dist/ami: fix incorrect service name on .bash_profile
Ubuntu's service name on .bash_profile is incorrect, fix it.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1460660628-10103-1-git-send-email-syuu@scylladb.com>
2016-04-18 15:21:48 +03:00