Commit Graph

164 Commits

Author SHA1 Message Date
Amnon Heiman
75c53e4f24 scylla-housekeeping: rename configuration file from conf to cfg
Files with a conf extension are run by the scylla_prepare on the AMI.
The scylla-housekeeping configuration file is not a bash script and
should not be run.

This patch changes its extension to cfg which is more python like.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1470896759-22651-2-git-send-email-amnon@scylladb.com>
(cherry picked from commit 5a4fc9c503)
2016-08-11 14:45:11 +03:00
Avi Kivity
e2c9feafa3 Merge "Add configuration file to scylla-housekeeping" from Amnon
"The series adds an optional configuration file to the scylla-housekeeping. The
file act as a way to prevent the scylla-housekeeping to run. A missing
configuration file, will make the scylla-housekeeping immediately.

The series adds a flag to the build_rpm that differentiate between public
distributions that would contain the configuration file and private
distributions that will not contain it which will cause the setup script to
create it."

(cherry picked from commit da4d33802e)
2016-08-10 13:34:04 +03:00
Avi Kivity
f4dea17c19 Merge "housekeeping: Switch to pytho2 and handle version" from Amnon
This series handle two issues:
* Moving to python2, though python3 is supported, there are modules that we
  need that are not rpm installable, python3 would wait when it will be more
  mature.

* Check version should send the current version when it check for a new one and
  a simple string compare is wrong.

(cherry picked from commit ec62f0d321)
2016-08-10 13:31:50 +03:00
Amnon Heiman
0eba2b8fd5 scylla.spec.in: Pack the scylla-housekeeping service
This change pack and install the scylla-housekeeping service under
redhat like systems.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-07-12 12:32:48 +03:00
Takuya ASADA
d2caa486ba dist/redhat/centos_dep: disable go and ada language on scylla-gcc package, since ScyllaDB never use them
centos-master jenkins job failed at building libgo, but we don't need go language, so let's disable it on scylla-gcc package.
Also we never use ada, disable it too.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1468166660-23323-1-git-send-email-syuu@scylladb.com>
2016-07-10 19:12:52 +03:00
Takuya ASADA
73ba4ac337 dist: drop sudoers.d from .rpm, since systemd moved to PermissionsStartOnly
Since systemd moved to PermissionsStartOnly, only upstart uses sudoers.
So move common/sudoers.d to dist/ubuntu, drop them from .rpm.
Also, Ubuntu 15.10/16.04 does not requires sudoers since these are uses systemd.
So copy sudoers only for 14.04.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1466536491-9860-1-git-send-email-syuu@scylladb.com>
2016-06-21 22:59:18 +03:00
Pekka Enberg
1d5f7be447 systemd: Use PermissionsStartOnly instead of running sudo
Use the PermissionsStartOnly systemd option to apply the permission
related configurations only to the start command. This allows us to stop
using "sudo" for ExecStartPre and ExecStopPost hooks and drop the
"requiretty" /etc/sudoers hack from Scylla's RPM.

Tested-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1466407587-31734-1-git-send-email-penberg@scylladb.com>
2016-06-20 11:53:24 +03:00
Avi Kivity
07045ffd7c dist: fix scylla-kernel-conf postinstall scriptlet failure
Because we build on CentOS 7, which does not have the %sysctl_apply macro,
the macro is not expanded, and therefore executed incorrectly even on 7.2,
which does.

Fix by expanding the macro manually.

Fixes #1360.
Message-Id: <1466250006-19476-1-git-send-email-avi@scylladb.com>
2016-06-20 09:36:39 +03:00
Avi Kivity
e515933c70 dist: tune scheduler for lower latency
Scylla-jmx and collectd can preempt scylla and induce long latencies.  Tune
the scheduler to provide lower latencies.

Since when the support processes are not running we normally do not context
switch (one thread per core, remember?), there should be no effect on
throughput.

The tunings are provided in a separate package, which can be uninstalled
if the server is shared with other applications which are negatively
affected by the tuning.

Fixes #1218.
Message-Id: <1464529625-12825-1-git-send-email-avi@scylladb.com>
2016-05-30 08:42:19 +03:00
Pekka Enberg
b7e79b72d5 Merge "Introduce SET_NIC for non-AMI environment" from Takuya
"This patchset provides a way to enable SET_NIC(posix_net_conf.sh) on
 non-AMI environment.
 Also support -mq option of the script.
 This also contains number of bug fixes of scripts.

 Fixes #1192"
2016-05-26 13:37:06 +03:00
Yoav Kleinberger
26c0d86401 tools/scyllatop: improved user interface: scrollable views
NOTE: scyllatop now requires the urwid library

previously, if there were more metrics that lines in the terminal
window, the user could not see some of the metrics.  Now the user can
scroll.

As an added bonus, the program will not crash when the window size
changes.

Signed-off-by: Yoav Kleinberger <yoav@scylladb.com>
Message-Id: <1464098832-5755-1-git-send-email-yoav@scylladb.com>
2016-05-26 13:36:28 +03:00
Avi Kivity
5e5317b228 dist: add build dependencies for sctp
Required by new seastar
2016-05-22 19:10:25 +03:00
Takuya ASADA
49cdd0b786 dist: move '--cpuset' and '--smp' configuration to scylla_cpuset_setup / cpuset.conf
These parameters are only required for AMI, not for non-AMI environment which want to enable SET_NIC, so split them to indivisual script / conf file, call it from AMI install script.
2016-05-19 06:25:28 +09:00
Takuya ASADA
53820393da dist/common/scripts: add scylla.yaml parser for scripts
To parse scylla.yaml, scylla_config_get.py is added.
It can be use like 'scylla_config_get.py [key name]' from shell script, or command line.
This is needed for scylla_io_setup, to get 'data_file_directories' from shellscript.
Currently it does not supported to specify key name of nested data structure, but enough for scyll_io_setup.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1462955824-21983-1-git-send-email-syuu@scylladb.com>
2016-05-11 13:02:23 +03:00
Takuya ASADA
9ff2efb28b dist/common/dep: add Ubuntu support for scylla-env
Since Ubuntu 14.04LTS needs scylla-gdb package which install to /opt/scylladb, we need to port scylla-env package to Ubuntu as well.
This change introduces scylla-env package to Ubuntu 14.04LTS.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1462825880-20866-2-git-send-email-syuu@scylladb.com>
2016-05-10 17:53:32 +03:00
Takuya ASADA
43cc77d1b8 dist/redhat/centos_dep: move scylla-env to dist/common to share with Ubuntu
Since Ubuntu 14.04LTS needs scylla-gdb package which install to /opt/scylladb, we need to port scylla-env package to Ubuntu as well.
To do it, share the package directory on dist/common/dep at first.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1462825880-20866-1-git-send-email-syuu@scylladb.com>
2016-05-10 17:53:31 +03:00
Takuya ASADA
f7d41ba07a dist: Extract scylla.yaml and create metapackage
This patch create a scylla-conf package containing
scylla.yaml and a scylla package acting as a metapackage.

Fixes #421

Signed-off-by: Benoît Canet <benoit@scylladb.com>
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1462280987-26909-1-git-send-email-syuu@scylladb.com>
2016-05-09 11:23:28 +03:00
Avi Kivity
454512a272 dist/redhat: package scylla_kernel_check
Can't build rpm without this.
Message-Id: <1461683947-30356-1-git-send-email-avi@scylladb.com>
2016-04-27 08:38:48 +03:00
Takuya ASADA
f98997120a dist: #!/bin/bash for all scripts
We choosed #!/bin/sh for shebang when we started to implement installer scripts, not bash.
After we started to work on Ubuntu, we found that we mistakenly used bash syntax on AMI script, it caused error since /bin/sh is dash on Ubuntu.
So we changed shebang to /bin/bash for the script, from that time we have both sh scripts and bash scripts.
(2f39e2e269)
If we use bash syntax on sh scripts, it won't work on Ubuntu but works on Fedora/CentOS, could be very easy to confusing.
So switch all scripts to #!/bin/bash. It will much safer.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1460594643-30666-1-git-send-email-syuu@scylladb.com>
2016-04-14 12:01:28 +03:00
Takuya ASADA
1eebe8bce1 dist: Support systemd for Ubuntu 15.10
To share systemd unit file between Fedora/CentOS and Ubuntu, generate
systemd unit file on building time since Fedora/CentOS and Ubuntu has
sysconfdir on different place.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1459779957-11007-1-git-send-email-syuu@scylladb.com>
2016-04-12 14:39:26 +03:00
Takuya ASADA
cad5edc53b dist: fix build error at copy symlinks
Both build_rpm.sh and build_deb.sh will fail with "cannot stat 'xxx': No such file or directory" when scylla-server package is not installed, need to prevent it by --no-dereference option of cp.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1459523585-9108-1-git-send-email-syuu@scylladb.com>
2016-04-03 12:49:55 +03:00
Takuya ASADA
d59c1c7648 dist/redhat: drop very old %pre script
These lines are needed for very old version of scylla, not for 1.0.
Can be removed now.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1459177601-20269-1-git-send-email-syuu@scylladb.com>
2016-04-01 09:41:18 +03:00
Takuya ASADA
58fb7000b1 dist: add setup scripts symlink to /usr/sbin
Instead of moving script to /usr/sbin, create symlink from /usr/lib/scylla/scylla_*setup to /usr/sbin/

Fixes #1092

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1459324684-31364-1-git-send-email-syuu@scylladb.com>
2016-03-30 11:04:41 +03:00
Takuya ASADA
42ce77a3b7 dist/redhat: prevent 'yum: command not found' on some Fedora environment
On some Fedora environments such as Fedora official AMI, dnf-yum package
is not installed by default, causes command not found error when we run
our setup scripts.

To prevent this, we need to add dnf-yum to scylla-server package
dependency.

Fixes #1106

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1459099744-23068-1-git-send-email-syuu@scylladb.com>
2016-03-29 11:29:09 +03:00
Pekka Enberg
227daecba6 Revert "dist: move setup scripts to /usr/sbin"
This reverts commit 989357189a because it
broke our Jenkins packaging jobs.
2016-03-29 10:17:05 +03:00
Takuya ASADA
989357189a dist: move setup scripts to /usr/sbin
Since these scripts are user command, should be on $PATH.

Fixes #1092

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1458860407-25269-1-git-send-email-syuu@scylladb.com>
2016-03-25 11:50:13 +03:00
Takuya ASADA
50db64de33 dist: drop -j2 option on .spec, make build_rpm.sh able to specify -j option
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1458678665-30273-1-git-send-email-syuu@scylladb.com>
2016-03-23 13:32:14 +02:00
Takuya ASADA
9889712d43 dist: remove scylla-io-setup.service and make it standalone script 2016-03-22 17:45:58 +02:00
Takuya ASADA
83112551bb dist: introduce dev-mode.conf and scylla_dev_mode_setup 2016-03-22 17:45:58 +02:00
Takuya ASADA
0ab3d0fd52 dist: use SEASTAR_IO instead of SCYLLA_IO
sync with iotune, fixes #1010

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1457530910-1273-1-git-send-email-syuu@scylladb.com>
2016-03-09 15:45:34 +02:00
Takuya ASADA
9ee14abf24 dist: export sysconfig for scylla-io-setup.service
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-03-07 18:13:30 +09:00
Takuya ASADA
e8a107de43 dist: install example io.conf by default
Prevent error on 'EnvironmentFile=/etc/scylla.d/*.conf'.
Parameters are commented out, and the file will replace when scylla starts, by scylla-io-setup.service.
2016-02-29 20:12:54 +09:00
Takuya ASADA
a281b10210 dist: run scylla-io-setup.service before scylla-server.service on anycase
We needed "systemctl enable scylla-io-setup.service; systemctl start scylla-server.service" to launch scylla-io-setup.service before scylla-server.service.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-02-29 18:59:10 +09:00
Takuya ASADA
11a616d4d6 dist: add scyllaio-setup.service for %systemd_post and %systemd_preun on .rpm
This is needed for initialize the service correctly

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-02-29 18:59:10 +09:00
Takuya ASADA
aa3f6ad462 dist: add scyllatop on .rpm/.deb
Fixes #933

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1456420768-15921-1-git-send-email-syuu@scylladb.com>
2016-02-25 19:24:11 +02:00
Takuya ASADA
b250a3b116 dist: Add collectd configuration support on .rpm/.deb
Depends on collectd, add /etc/collectd.d/scylla.conf on scylla-server package installation.
Fixes #946

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1456336200-11876-1-git-send-email-syuu@scylladb.com>
2016-02-25 10:35:47 +02:00
Takuya ASADA
a9926f1cea dist: introduce scylla-io-setup.service to setup io parameters on first startup
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-02-24 03:33:03 +09:00
Avi Kivity
889bf1aef2 dist: build only scylla and iotune binaries, not all the tests 2016-02-20 18:45:42 +02:00
Takuya ASADA
ca9f7bf72e dist: add iotune on .rpm/.deb packages
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1455929805-28987-1-git-send-email-syuu@scylladb.com>
2016-02-20 18:44:47 +02:00
Avi Kivity
1b49c0ce19 dist: Restart scylla on abnormal termination
Restarting the service can recover from a transient failure.

Fixes #904 (on systemd systems only)
Message-Id: <1455441103-11963-1-git-send-email-avi@scylladb.com>
2016-02-17 21:30:40 +01:00
Takuya ASADA
6b93505952 dist: long options for build_rpm.sh
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-02-17 07:34:08 +09:00
Takuya ASADA
dc9012d5a4 dist: move selinux setup code to scylla_selinux_setup
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-02-17 07:33:01 +09:00
Shlomi Livne
f938e1d303 dist: start scylla with SCYLLA_IO
Signed-off-by: Shlomi Livne <shlomi@scylladb.com>
Message-Id: <d93a7b41a285fcde796c5681479a328f1efac0c3.1455188901.git.shlomi@scylladb.com>
2016-02-11 17:01:03 +02:00
Takuya ASADA
f33656ef03 dist: eliminate startup script
Fixes #373

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1453910311-24928-1-git-send-email-syuu@scylladb.com>
2016-01-27 18:45:35 +02:00
Avi Kivity
c75d1c4eeb Merge "Ubuntu 'expect stop' related fixes" from Takuya 2016-01-27 17:00:23 +02:00
Takuya ASADA
89f0fc89b4 dist: set ulimit in upstart job
Upstart job able to specify ulimit like systemd, so drop ubuntu's scylla_run and merge with redhat one.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-01-27 23:29:52 +09:00
Takuya ASADA
559f913494 dist: use nightly for prebuilt 3rdparty packages (CentOS)
Developers probably wants to use latest dependency packages, so switch to nightly.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1453904521-2716-1-git-send-email-syuu@scylladb.com>
2016-01-27 16:24:49 +02:00
Takuya ASADA
e9fdb426b6 dist: add pyparsing on CentOS build time dependency CentOS
Porting pyparsing from Fedora23, build it for python34 which provided by epel.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1453720780-21105-1-git-send-email-syuu@scylladb.com>
2016-01-25 13:26:58 +02:00
Takuya ASADA
67d2aa677e dist: add pyparsing on Fedora build time dependency
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1453715594-32675-1-git-send-email-syuu@scylladb.com>
2016-01-25 11:59:32 +02:00
Takuya ASADA
78d107ccaa dist: add missing dependencies for scylla-gdb
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1453715339-32296-1-git-send-email-syuu@scylladb.com>
2016-01-25 11:59:31 +02:00