Commit Graph

104 Commits

Author SHA1 Message Date
Amnon Heiman
84f7d9a49c build_deb: Add dist flag
The dist flag mark the debian package as distributed package.
As such the housekeeping configuration file will be included in the
package and will not need to be created by the scylla_setup.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1470907208-502-2-git-send-email-amnon@scylladb.com>
(cherry picked from commit a24941cc5f)
2016-08-11 12:25:28 +03:00
Avi Kivity
f4dea17c19 Merge "housekeeping: Switch to pytho2 and handle version" from Amnon
This series handle two issues:
* Moving to python2, though python3 is supported, there are modules that we
  need that are not rpm installable, python3 would wait when it will be more
  mature.

* Check version should send the current version when it check for a new one and
  a simple string compare is wrong.

(cherry picked from commit ec62f0d321)
2016-08-10 13:31:50 +03:00
Amnon Heiman
260761f2dd rules.in: Add the scylla-timer to ubuntu
This adds a rule to install the scylla-timer as part of the ubuntu
package.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-07-14 12:46:47 +03:00
Amnon Heiman
3be9ab38e2 ubuntu.in: Add dependency to python3-requests
The check version script uses the python requests package, this add the
dependency to the ubuntu package.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-07-14 12:46:47 +03:00
Amnon Heiman
3b9db378ac scylla-server.install.in: Pack the scyll-housekeeping on ubuntu
This adds the scylla-housekeeping to the ubuntu packging.
2016-07-14 12:46:47 +03:00
Amnon Heiman
948140bec3 Adding a timer service for ubuntu scylla-housekeeping
Ununtu 14.4 upstart does not support timers for recurrent operations.
The upstart cookbook suggest a way to mimic this functionality here:
http://upstart.ubuntu.com/cookbook/#run-a-job-periodically

This patch adds a service that runs the house-keeping daily.

Setting it as a service insure that it would start and stop with
scylla-server service.
2016-07-14 12:46:39 +03:00
Takuya ASADA
73ba4ac337 dist: drop sudoers.d from .rpm, since systemd moved to PermissionsStartOnly
Since systemd moved to PermissionsStartOnly, only upstart uses sudoers.
So move common/sudoers.d to dist/ubuntu, drop them from .rpm.
Also, Ubuntu 15.10/16.04 does not requires sudoers since these are uses systemd.
So copy sudoers only for 14.04.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1466536491-9860-1-git-send-email-syuu@scylladb.com>
2016-06-21 22:59:18 +03:00
Avi Kivity
e515933c70 dist: tune scheduler for lower latency
Scylla-jmx and collectd can preempt scylla and induce long latencies.  Tune
the scheduler to provide lower latencies.

Since when the support processes are not running we normally do not context
switch (one thread per core, remember?), there should be no effect on
throughput.

The tunings are provided in a separate package, which can be uninstalled
if the server is shared with other applications which are negatively
affected by the tuning.

Fixes #1218.
Message-Id: <1464529625-12825-1-git-send-email-avi@scylladb.com>
2016-05-30 08:42:19 +03:00
Pekka Enberg
b7e79b72d5 Merge "Introduce SET_NIC for non-AMI environment" from Takuya
"This patchset provides a way to enable SET_NIC(posix_net_conf.sh) on
 non-AMI environment.
 Also support -mq option of the script.
 This also contains number of bug fixes of scripts.

 Fixes #1192"
2016-05-26 13:37:06 +03:00
Yoav Kleinberger
26c0d86401 tools/scyllatop: improved user interface: scrollable views
NOTE: scyllatop now requires the urwid library

previously, if there were more metrics that lines in the terminal
window, the user could not see some of the metrics.  Now the user can
scroll.

As an added bonus, the program will not crash when the window size
changes.

Signed-off-by: Yoav Kleinberger <yoav@scylladb.com>
Message-Id: <1464098832-5755-1-git-send-email-yoav@scylladb.com>
2016-05-26 13:36:28 +03:00
Takuya ASADA
53cebb4a5e dist/ubuntu: don't rebuild dependency packages by default
Same as CentOS, do not build dependencies by default, install binary packages from our repository.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1464023451-21436-1-git-send-email-syuu@scylladb.com>
2016-05-24 14:10:59 +03:00
Takuya ASADA
5d5d525a14 dist/ubuntu: fix incorrect dependency package name
PyYAML is CentOS/RHEL/Fedora package name, python-yaml is correct one.

Fixes #1279

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1463987823-22837-1-git-send-email-syuu@scylladb.com>
2016-05-23 10:21:29 +03:00
Avi Kivity
5e5317b228 dist: add build dependencies for sctp
Required by new seastar
2016-05-22 19:10:25 +03:00
Takuya ASADA
49cdd0b786 dist: move '--cpuset' and '--smp' configuration to scylla_cpuset_setup / cpuset.conf
These parameters are only required for AMI, not for non-AMI environment which want to enable SET_NIC, so split them to indivisual script / conf file, call it from AMI install script.
2016-05-19 06:25:28 +09:00
Takuya ASADA
4972a72380 dist: drop 'sudo -E' and SETENV for security reason, source envfile from scripts
As Nadav pointed out, SETENV and sudo -E might be causes security hole:
https://github.com/scylladb/scylla/issues/1028#issuecomment-196202171
So drop them now, sourcing envfiles from scylla_prepare / scylla_stop scripts
instead.

Also on "[PATCH] ubuntu: Fix the init script variable sourcing" thread
we have problem to passing variables from envfiles to scylla_prepare /
scylla_stop on Ubuntu, it seems better to sourcing from these scripts.

Additionally, this fixes #1249

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1462989906-30062-1-git-send-email-syuu@scylladb.com>
2016-05-17 10:31:03 +03:00
Takuya ASADA
53820393da dist/common/scripts: add scylla.yaml parser for scripts
To parse scylla.yaml, scylla_config_get.py is added.
It can be use like 'scylla_config_get.py [key name]' from shell script, or command line.
This is needed for scylla_io_setup, to get 'data_file_directories' from shellscript.
Currently it does not supported to specify key name of nested data structure, but enough for scyll_io_setup.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1462955824-21983-1-git-send-email-syuu@scylladb.com>
2016-05-11 13:02:23 +03:00
Takuya ASADA
a73924b4e0 dist/ubuntu/dep: introduce scylla-gdb-7.11 for Ubuntu 14.04LTS
Introduce scylla-gdb-7.11 for Ubuntu 14.04LTS, to get better support of recent version of g++ on gdb.

Fixes #969

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1462825880-20866-3-git-send-email-syuu@scylladb.com>
2016-05-10 17:53:32 +03:00
Takuya ASADA
9ff2efb28b dist/common/dep: add Ubuntu support for scylla-env
Since Ubuntu 14.04LTS needs scylla-gdb package which install to /opt/scylladb, we need to port scylla-env package to Ubuntu as well.
This change introduces scylla-env package to Ubuntu 14.04LTS.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1462825880-20866-2-git-send-email-syuu@scylladb.com>
2016-05-10 17:53:32 +03:00
Takuya ASADA
f7d41ba07a dist: Extract scylla.yaml and create metapackage
This patch create a scylla-conf package containing
scylla.yaml and a scylla package acting as a metapackage.

Fixes #421

Signed-off-by: Benoît Canet <benoit@scylladb.com>
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1462280987-26909-1-git-send-email-syuu@scylladb.com>
2016-05-09 11:23:28 +03:00
Takuya ASADA
cc74b6ff5f dist/ubuntu: move lines from rules to .install/.dirs/.docs
To simplify build script, and make it easier spliting two packages,
use .install/.dirs/.docs instead of rules.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1461960695-30647-1-git-send-email-syuu@scylladb.com>
2016-05-01 10:16:35 +03:00
Takuya ASADA
3ec47fbcf0 dist/ubuntu: unofficial support Debian 8.4
Unofficial support for Debian 8.4.
Now we supported both ubuntu and debian, but keep directory name as 'dist/ubuntu' for now.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1461006868-28273-1-git-send-email-syuu@scylladb.com>
2016-04-27 15:39:20 +03:00
Takuya ASADA
1cfde50102 dist/ubuntu: support 16.04
Drop 'unsupported release' message on 16.04.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-04-27 18:06:59 +09:00
Takuya ASADA
988b7bcd3d dist/ubuntu: don't use ubuntu-toolchain-r/test ppa repo on recent versions of Ubuntu, since it has newer g++
On Ubuntu 15.04 and newer, official g++ package is >= g++-4.9.
So we don't need to use development repository, just use official package.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-04-27 18:06:59 +09:00
Takuya ASADA
fa0b90b727 dist/ubuntu: add dependency for libsystemd-dev to handle startup correctly on recent versions of Ubuntu
To handle scylla startup correctly on systemd versions of Ubuntu, scylla requires to build with libsystemd-dev.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-04-27 18:06:59 +09:00
Takuya ASADA
eae881ff70 dist/ubuntu: skip dh_installinit --upstart-only on recent versions of Ubuntu
Since 16.04LTS does not support this argument anymore, drop it on recent version of Ubuntu which does not uses Upstart.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-04-27 18:06:59 +09:00
Takuya ASADA
d5efa02eab dist/ubuntu/dep: Drop python-support on Ubuntu 16.04
Ubuntu 16.04 seems dropped python-support, so remove it from thrift package.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-04-27 18:06:59 +09:00
Takuya ASADA
e733c2aae8 dist/ubuntu/dep: use distribution's thrift-compiler-0.9.1 on newer versions of Ubuntu
Use distribution's thrift if version > 14.04LTS.
14.04LTS doesn't have thrift-compiler-0.9.1, use our version.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-04-27 18:06:59 +09:00
Takuya ASADA
e7f438eeae dist/ubuntu: Drop dependency to libthrift0, link it statically
Drop dependency to libthrift0 on installation time, link libthrift statically.
With this fix, we don't need to distribute libthrift0 deb package anymore to install scylla-server binary package.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1461594460-2403-2-git-send-email-syuu@scylladb.com>
2016-04-25 17:44:46 +03:00
Takuya ASADA
f98997120a dist: #!/bin/bash for all scripts
We choosed #!/bin/sh for shebang when we started to implement installer scripts, not bash.
After we started to work on Ubuntu, we found that we mistakenly used bash syntax on AMI script, it caused error since /bin/sh is dash on Ubuntu.
So we changed shebang to /bin/bash for the script, from that time we have both sh scripts and bash scripts.
(2f39e2e269)
If we use bash syntax on sh scripts, it won't work on Ubuntu but works on Fedora/CentOS, could be very easy to confusing.
So switch all scripts to #!/bin/bash. It will much safer.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1460594643-30666-1-git-send-email-syuu@scylladb.com>
2016-04-14 12:01:28 +03:00
Takuya ASADA
1eebe8bce1 dist: Support systemd for Ubuntu 15.10
To share systemd unit file between Fedora/CentOS and Ubuntu, generate
systemd unit file on building time since Fedora/CentOS and Ubuntu has
sysconfdir on different place.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1459779957-11007-1-git-send-email-syuu@scylladb.com>
2016-04-12 14:39:26 +03:00
Takuya ASADA
ed7a3beed2 dist/ubuntu: drop unused scripts
This was used when we didn't shared scripts between CentOS/Fedora and Ubuntu, but used anymore so drop them.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1459408583-13497-1-git-send-email-syuu@scylladb.com>
2016-04-06 08:21:09 +03:00
Takuya ASADA
cad5edc53b dist: fix build error at copy symlinks
Both build_rpm.sh and build_deb.sh will fail with "cannot stat 'xxx': No such file or directory" when scylla-server package is not installed, need to prevent it by --no-dereference option of cp.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1459523585-9108-1-git-send-email-syuu@scylladb.com>
2016-04-03 12:49:55 +03:00
Takuya ASADA
58fb7000b1 dist: add setup scripts symlink to /usr/sbin
Instead of moving script to /usr/sbin, create symlink from /usr/lib/scylla/scylla_*setup to /usr/sbin/

Fixes #1092

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1459324684-31364-1-git-send-email-syuu@scylladb.com>
2016-03-30 11:04:41 +03:00
Takuya ASADA
a5bb6c4b1b dist/ubuntu: drop classical sysv init script, only support Upstart for Ubuntu 14.04LTS
Sysv init script was added just for prevent warning message on lintian,
never really used by Ubuntu users. Result of that, we often break this
script since upstart/systemd unit file frequently changed. It may
confuse users, it's better to use Upstart only, just like Fedora/CentOS.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1459177601-20269-2-git-send-email-syuu@scylladb.com>
2016-03-29 11:48:18 +03:00
Avi Kivity
adffb1c061 dist/ubuntu: improve handling of bad command line options
On a bad command line, Scylla will exit with an exit code of 2.  Mark it
as a "normal" exit, to prevent a respawn.

Fixes #1087
Message-Id: <1458827221-12833-1-git-send-email-avi@scylladb.com>
2016-03-29 11:14:45 +03:00
Avi Kivity
c1d8fb56f7 dist/ubuntu: specify kill timeout
Allow more time for commitlog flushing
Message-Id: <1458827216-12778-1-git-send-email-avi@scylladb.com>
2016-03-29 11:14:27 +03:00
Pekka Enberg
227daecba6 Revert "dist: move setup scripts to /usr/sbin"
This reverts commit 989357189a because it
broke our Jenkins packaging jobs.
2016-03-29 10:17:05 +03:00
Takuya ASADA
989357189a dist: move setup scripts to /usr/sbin
Since these scripts are user command, should be on $PATH.

Fixes #1092

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1458860407-25269-1-git-send-email-syuu@scylladb.com>
2016-03-25 11:50:13 +03:00
Benoît Canet
1594bdd5bb dist/ubuntu: Fix the init script variable sourcing
The variable sourcing was crashing the init script on ubuntu.
Fix it with the suggestion from Avi.

Signed-off-by: Benoît Canet <benoit@scylladb.com>
Message-Id: <1458685099-1160-1-git-send-email-benoit@scylladb.com>
2016-03-23 09:03:17 +02:00
Takuya ASADA
9889712d43 dist: remove scylla-io-setup.service and make it standalone script 2016-03-22 17:45:58 +02:00
Takuya ASADA
83112551bb dist: introduce dev-mode.conf and scylla_dev_mode_setup 2016-03-22 17:45:58 +02:00
Pekka Enberg
ae33e9fe76 dist/ubuntu: Use tilde for release candidate builds
The version number ordering rules are different for rpm and deb. Use
tilde ('~') for the latter to ensure a release candidate is ordered
_before_ a final version.

Message-Id: <1458627524-23030-1-git-send-email-penberg@scylladb.com>
2016-03-22 11:52:05 +02:00
Pekka Enberg
bcdd034512 dist/ubuntu: Install wget package if it's not available
The build scripts use wget so make sure it's actually installed on the
machine.

Message-Id: <1458554706-14558-1-git-send-email-penberg@scylladb.com>
2016-03-21 12:36:52 +02:00
Takuya ASADA
7828023599 dist: launch scylla_ami_setup on Ubuntu AMI
Since upstart does not have same behavior as systemd, we need to run scylla_io_setup and scylla_ami_setup in scylla-server.conf's pre-start stanza.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-03-18 05:57:50 +09:00
Takuya ASADA
95161d5db7 dist: add scylla-gdb.py on Ubuntu dbg package
Fixes #969

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1458150248-10632-1-git-send-email-syuu@scylladb.com>
2016-03-17 09:03:00 +02:00
Takuya ASADA
f1d18e9980 dist: do not auto-start scylla-server job on Ubuntu package install time
Fixes #1017

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1458122424-22889-1-git-send-email-syuu@scylladb.com>
2016-03-16 13:55:12 +02:00
Takuya ASADA
0ab3d0fd52 dist: use SEASTAR_IO instead of SCYLLA_IO
sync with iotune, fixes #1010

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1457530910-1273-1-git-send-email-syuu@scylladb.com>
2016-03-09 15:45:34 +02:00
Takuya ASADA
18a27de3c8 dist: export all entries on /etc/default/scylla-server on Ubuntu
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-03-08 18:18:30 +09:00
Takuya ASADA
c888eaac74 dist: add /etc/scylla.d/io.conf on Ubuntu
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2016-03-07 17:15:42 +09:00
Takuya ASADA
e545013e47 Revert "dist: downgrade g++ to 4.9 on Ubuntu"
This reverts commit 01bd4959ac.

Fixes #983

Conflicts:
	dist/ubuntu/build_deb.sh
	dist/ubuntu/control.in
	dist/ubuntu/rules.in

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1456996244-19889-1-git-send-email-syuu@scylladb.com>
2016-03-03 11:12:18 +02:00