Commit Graph

210 Commits

Author SHA1 Message Date
Glauber Costa
7ef8c6aaec centos: add runtime dependency for perftune script
Signed-off-by: Glauber Costa <glauber@scylladb.com>
Message-Id: <20170406172949.13264-1-glauber@scylladb.com>
2017-04-09 11:36:26 +03:00
Pekka Enberg
a9ad5cc560 dist/redhat: Add node_health_check to package 2017-04-08 08:12:57 +03:00
Glauber Costa
039f8d5994 dist/redhat: install the new scylla_util.py file
For the debian package, the files don't have to be listed individually,
but for RPMs they do.

The rpm builds are currently failing with:

error: Installed (but unpackaged) file(s) found:
  /usr/lib/scylla/scylla_util.py

Signed-off-by: Glauber Costa <glauber@scylladb.com>
Message-Id: <20170406012336.21081-1-glauber@scylladb.com>
2017-04-06 10:20:32 +03:00
Takuya ASADA
610bc31b04 dist/redhat/scylla.spec.in: stop compiling .py on rpm
rpmbuild tries to compile any *.py by default, but it causes compilation
error on python3 code when python2 is system default (CentOS, RHEL).

So skip compiling it, drop .pyc / .pyo from the package.

Fixes #2235

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1490859768-18900-1-git-send-email-syuu@scylladb.com>
2017-04-05 12:15:22 +03:00
Takuya ASADA
4262edd843 dist: use distribution standard fstrim script instead of our custom one
We recently introduced fstrim cronjob / systemd timer unit, but some of
distributions already has their own fstrim cronjob / systemd timer unit.
So let's use them when it's posssible.

Fixes #2233

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1491226727-10507-1-git-send-email-syuu@scylladb.com>
2017-04-04 16:44:45 +03:00
Pekka Enberg
57c4bed420 Revert "dist: add --options-file /etc/scylla/scylla.yaml on sysconfig"
This reverts commit 58e628eb3d. Takuya says it
does not fix issue #2236.
2017-04-03 16:33:58 +03:00
Takuya ASADA
58e628eb3d dist: add --options-file /etc/scylla/scylla.yaml on sysconfig
After RAID devices mounted to /var/lib/scylla, scylla-server doesn't able to
find ./conf/ directory since we haven't created symlinks on the volume.

Instead of creating symlink on scylla_raid_setup, let's specify scylla.yaml
path on program argument.

Fixes #2236

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1491056110-1078-1-git-send-email-syuu@scylladb.com>
2017-04-02 11:28:27 +03:00
Takuya ASADA
5e0cb39db6 dist: follow DPDK tools directory name change
Fixes #2234

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1490776630-6626-1-git-send-email-syuu@scylladb.com>
2017-03-29 11:38:39 +03:00
Vlad Zolotarov
9dd5b5762d dist: install seastar/scripts/perftune.py together with posix_net_conf.sh
posix_net_conf.sh is currently a wrapper for perftune.py script and
perftune.py has to be at the same directory as posix_net_conf.sh.


Fixes #2176.

Signed-off-by: Vlad Zolotarov <vladz@scylladb.com>
Message-Id: <1490020882-32361-1-git-send-email-vladz@scylladb.com>
2017-03-21 12:31:50 +02:00
Takuya ASADA
d016dd4b74 dist: schedule daily fstrim for data directory and commitlog directory
Schedule daily fstrim for data directory and commitlog directory, witch is
recommended by Scylla doc:
http://www.scylladb.com/doc/admin/#schedule-fstrim

Fixes #1347

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1489447472-2981-1-git-send-email-syuu@scylladb.com>
2017-03-14 11:51:53 +02:00
Takuya ASADA
6602221442 dist/redhat: enables discard on CentOS/RHEL RAID0
Since CentOS/RHEL raid module disables discard by default, we need enable it
again to use.

Fixes #2033

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1488407037-4795-1-git-send-email-syuu@scylladb.com>
2017-03-06 12:21:42 +02:00
Avi Kivity
113ed9e963 Merge seastar upstream
* seastar 28a143a...5088065 (8):
  > configure.py: switch cmake to build c-ares to do out-of-source-tree build
  > iotune: make sure help is working
  > collectd: send double correctly for gauge
  > tls: make shutdown/close do "clean" handshake shutdown in background
  > tls: Make sink/source (i.e. streams) first class channel owners
  > native-stack: Make sink/source (i.e. streams) first class channel owners
  > posix-stack: Make sink/source (i.e. streams) first class channel owners
  > Merge "Detector for tasks blocked" from Glauber

Fixes #2085.

Packaging updated to require cmake, drop libtool and automake.
2017-02-16 19:34:28 +02:00
Takuya ASADA
9c8515eeed dist/redhat: stop backporting ninja-build from Fedora, install it from EPEL instead
ninja-build-1.6.0-2.fc23.src.rpm on fedora web site deleted for some
reason, but there is ninja-build-1.7.2-2 on EPEL, so we don't need to
backport from Fedora anymore.

Fixes #2087

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1487155729-13257-1-git-send-email-syuu@scylladb.com>
2017-02-15 12:58:00 +02:00
Avi Kivity
2510b756fc dist: add build dependency on automake
Needed by seastar's c-ares.
2017-02-05 20:16:27 +02:00
Takuya ASADA
e82932b774 dist/common/systemd: introduce scylla-housekeeping restart mode
scylla-housekeeping requires to run 'restart mode' for check the version during
scylla-server restart, which wasn't called on systemd timer so added it.

Existing scylla-housekeeping.timer renamed to scylla-housekeeping-daily.timer,
since it is running 'daily mode'.

Fixes #1953

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1486180031-18093-1-git-send-email-syuu@scylladb.com>
2017-02-05 10:46:04 +02:00
Avi Kivity
4175f40da1 dist: add libtool build dependency for seastar/c-ares 2017-02-05 10:42:53 +02:00
Takuya ASADA
bce0fb3fa2 dist: add lspci on dependencies, since it used by dpdk-devbind.py
On minimum setup environment scylla_sysconfig_setup will fail because lspci command is not installed. So install it on package installation time.

Fixes #2035

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1485327435-20543-1-git-send-email-syuu@scylladb.com>
2017-01-25 10:22:57 +02:00
Amos Kong
b880bdccef dist/redhat: fix path of housekeeping.cfg
scylla-housekeeping[3857]: Config file /etc/scylla.d/housekeeping.cfg is missing, terminating

Housekeeping failed to execute for missing the config file,
the config file should be in /etc/scylla.d/.

Fixes #2020

Signed-off-by: Amos Kong <amos@scylladb.com>
Message-Id: <e63f2f8cb94410a6dca4e6193932f0079755ad47.1484724328.git.amos@scylladb.com>
2017-01-19 11:08:46 +02:00
Takuya ASADA
bee7f549a9 scylla-housekeeping: move uuid file to /var/lib/scylla-housekeeping
Since scylla-housekeeping running as scylla user, it doesn't have a permission
to create a file on /etc/scylla.d.
So introduce /var/lib/scylla-housekeeping which owns by scylla user, place uuid
file on the directory.

Fixes #2009

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1484235946-12463-1-git-send-email-syuu@scylladb.com>
2017-01-13 16:27:53 +02:00
Takuya ASADA
c07d703d0d dist/redhat/scylla.spec.in: fix typo of scylla_cpuscaling_setup
Fix packaging error

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1484191955-28006-2-git-send-email-syuu@scylladb.com>
2017-01-12 12:13:33 +02:00
Takuya ASADA
0e6df2a82e dist: follow DPDK script renaming
On DPDK 16.11 dpdk_nic_bind.py is renamed to dpdk-devbind.py, so we are
getting "file not found" both on packaging and scripts, fixed that.

Also fixed inconsistent packaging.
Since Seastar copied dpdk_nic_bind.py to its scripts/ directory, there're two
different versions of the script, .rpm/.deb packaging different one:
 dist/redhat: seastar/dpdk/tools/dpdk_nic_bind.py
 dist/ubuntu: seastar/scripts/dpdk_nic_bind.py

That's won't work because we sharing setup scripts between two
distributions, so I changed dist/ubuntu package to use DPDK one.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1484191955-28006-1-git-send-email-syuu@scylladb.com>
2017-01-12 12:13:33 +02:00
Takuya ASADA
920683a882 dist/common/scripts: add scylla_cpuscaling_setup
To setup cpu scaling governor to 'performance', add new script to do it on
scylla_setup.

Fixes #1895

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1483542216-12195-1-git-send-email-syuu@scylladb.com>
2017-01-09 09:44:41 +02:00
Takuya ASADA
43655512e1 dist/redhat: add python-setuptools on dependency since it requires for scylla-housekeeping
scylla-housekeeping breaks when python-setuptools doesn't installed, so
add it on dependency.

Fixes #1884

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1483525828-7507-1-git-send-email-syuu@scylladb.com>
2017-01-04 14:32:10 +02:00
Avi Kivity
3232d47d4f dist: remove another bc dependency
No longer used.
2017-01-01 11:13:34 +02:00
Takuya ASADA
f3e45bc9ef dist/redhat: don't try to adduser when user is already exists
Currently we get "failed adding user 'scylla'" on .rpm installation when user is already exists, we can skip it to prevent error.

Fixes #1958

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1482550075-27939-1-git-send-email-syuu@scylladb.com>
2016-12-25 11:37:25 +02:00
Takuya ASADA
c18a95cddf dist/redhat: add scylla_lib.sh to scylla.spec
Fix .rpm build error.

Fixes #1932

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1481703992-9596-1-git-send-email-syuu@scylladb.com>
2016-12-14 10:27:37 +02:00
Avi Kivity
28a5ff51cb dist: add build dependency on systemtap-sdt
Needed to newer seastar.
2016-11-29 18:49:51 +02:00
Takuya ASADA
55e5123313 dist/redhat: Support RHEL7
We supported install CentOS7 .rpm on RHEL7, but we haven't supported
building on RHEL7, since there is little difference between CentOS,
and that causes build error.

This patch fixes the error, now we can produce .rpm for RHEL7 wihout
using CentOS.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1479431134-8032-1-git-send-email-syuu@scylladb.com>
2016-11-18 11:56:05 +02:00
Takuya ASADA
3802f289f8 dist: remove bc from dependency
Since we replaced shellscript based cpuset generator with python based one,
we no longer depends to bc command.
d571123afd

So drop it from .rpm/.deb dependency.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1479152876-11020-1-git-send-email-syuu@scylladb.com>
2016-11-16 15:02:55 +02:00
Takuya ASADA
d571123afd dist/common/scripts/scylla_sysconfig_setup: stop using 'bc' command to generate cpuset parameter, use python script instead
We get error from bc command when we run the script on >34 ncpus,
to prevent the issue add a python script to generate cpuset parameter.

Fixes #1824

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1478887624-12737-1-git-send-email-syuu@scylladb.com>
2016-11-14 11:45:23 +02:00
Takuya ASADA
0f1ba1a3bb dist/redhat: remove unused dependencies
Seems like we mistakenly added unneeded packages for BuildRequires when
we created .spec file, so remove them.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1478504761-15067-1-git-send-email-syuu@scylladb.com>
2016-11-07 09:48:50 +02:00
Avi Kivity
72c2982260 dist: require scylla-boost-static for EL RPM build 2016-11-01 18:55:55 +02:00
Avi Kivity
e03ca06431 dist: fix rpm build
--static-boost is supposed to be an input to ./configure.py, not ninja.  Move
it there.
2016-10-28 08:42:26 +03:00
Avi Kivity
e995f5a3a7 dist: statically link with boost on RHEL
Reduces runtime dependencies on Scylla-provided third-party boost packages.

Message-Id: <1477552490-28961-1-git-send-email-avi@scylladb.com>
2016-10-27 12:35:12 +03:00
Avi Kivity
76628a7b0b dist: make wget quieter
wget is often used from scripts recording to logs; as it emits a log
line every second, the logs are huge and unreadable.  Make it quieter.

Message-Id: <1477558534-32718-1-git-send-email-avi@scylladb.com>
2016-10-27 12:11:26 +03:00
Avi Kivity
e261a380a9 dist: add PyYAML dependency to rpm (for blocktune) 2016-10-23 10:36:29 +03:00
Avi Kivity
db2f5e6be1 blocktune: wire up blocktune on startup
Message-Id: <1476357027-15014-3-git-send-email-avi@scylladb.com>
2016-10-20 13:24:05 +03:00
Takuya ASADA
9639cc840e dist/redhat: add missing build time dependency for libunwind
There was missing dependency for libunwind, so add it.
Fixes #1722

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1475260099-25881-1-git-send-email-syuu@scylladb.com>
2016-09-30 21:33:39 +03:00
Amnon Heiman
4e0dcb59e7 scylla.spec: package the node_exporter scripts
This patch adds the node_exporter related files to the rpm.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-09-22 09:43:53 +03:00
Takuya ASADA
647673195c dist/redhat/build_rpm.sh: add dependency for rpmbuild
Install rpmbuild when it's not installed yet.
Fixes #1651

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1473193430-14792-1-git-send-email-syuu@scylladb.com>
2016-09-14 14:57:55 +03:00
Takuya ASADA
533dc0485d dist/common/scripts/scylla_sysconfig_setup: sync cpuset parameters with rps_cpus settings when posix_net_conf.sh is enabled and NIC is single queue
On posix_net_conf.sh's single queue NIC mode (which means RPS enabled mode), we are excluded cpu0 and it's sibling from network stack processing cpus, and assigned NIC IRQ to cpu0.
So always network stack is not working on cpu0 and it's sibling, to get better performance we need to exclude these cpus from scylla too.
To do this, we need to get RPS cpu mask from posix_net_conf.sh, pass it to scylla_cpuset_setup to construct /etc/scylla.d/cpuset.conf when scylla_setup executed.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1472544875-2033-2-git-send-email-syuu@scylladb.com>
2016-08-30 16:51:16 +03:00
Amnon Heiman
91944b736e housekeeping: Add curl as a dependency
To work around an SSL problem with Python on Ubuntu 14.04, we need to
use curl. Add it as a dependency so that it's available on the host.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-08-23 18:06:13 +03:00
Takuya ASADA
aac60082ae dist/common/systemd: don't use .in for scylla-housekeeping.*, since these are not template file
.in is the name for template files witch requires to rewrite on building time, but these systemd unit files does not require rewrite, so don't name .in, reference directly from .spec.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1471607533-3821-1-git-send-email-syuu@scylladb.com>
2016-08-23 13:49:09 +03:00
Amnon Heiman
612f677283 scylla.spec: conditionally include the housekeeping.cfg in the conf package
When the housekeeping configuration name was changed from conf to cfg it
was no longer included as part of the conf rpm.

This change adds a macro that determines of if the file should be
included or not and use that marco to conditionally add the
configuration file to the rpm.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1471169042-19099-1-git-send-email-amnon@scylladb.com>
2016-08-14 13:25:59 +03:00
Amnon Heiman
5a4fc9c503 scylla-housekeeping: rename configuration file from conf to cfg
Files with a conf extension are run by the scylla_prepare on the AMI.
The scylla-housekeeping configuration file is not a bash script and
should not be run.

This patch changes its extension to cfg which is more python like.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <1470896759-22651-2-git-send-email-amnon@scylladb.com>
2016-08-11 14:44:56 +03:00
Amnon Heiman
9c4bf651ae install housekeeping.conf based on dist flag
The new dist flag difrentiate between public distribution and private
compilations.

For public distributation the housekeeping configuration file will be
installed.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-08-07 16:24:42 +03:00
Amnon Heiman
c8bcb5a8bf scylla.spec: Set the python dependencies for housekeeping
The scylla-housekeeping moved to python2, this set the python
dependencies under redhat.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-07-31 10:40:40 +03:00
Takuya ASADA
b542581d97 dist: add protobuf to dependencies, for newly added prometheus API support
Since we added prometheus API support, we need protobuf compiler and header to build seastar/scylla, so add dependencies on packaging.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1469573936-8551-1-git-send-email-syuu@scylladb.com>
2016-07-27 08:48:58 +03:00
Takuya ASADA
b5bb702b35 dist: add dependency for lspci
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1469429320-13645-1-git-send-email-syuu@scylladb.com>
2016-07-25 10:02:30 +03:00
Amnon Heiman
0eba2b8fd5 scylla.spec.in: Pack the scylla-housekeeping service
This change pack and install the scylla-housekeeping service under
redhat like systems.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2016-07-12 12:32:48 +03:00