Commit Graph

773 Commits

Author SHA1 Message Date
Takuya ASADA
1f10549056 dist/redhat: build only scylla, iotune
Since we don't package tests, we don't need to build them.
It reduces package building time.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1521066363-4859-1-git-send-email-syuu@scylladb.com>
(cherry picked from commit 1bb3531b90)
2018-03-15 10:48:36 +02:00
Takuya ASADA
c2a2560ea3 dist/debian: use 3rdparty ppa on Ubuntu 18.04
Currently Ubuntu 18.04 uses distribution provided g++ and boost, but it's easier
to maintain Scylla package to build with same version toolchain/libraries, so
switch to them.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1521075576-12064-1-git-send-email-syuu@scylladb.com>
(cherry picked from commit 945e6ec4f6)
2018-03-15 10:48:31 +02:00
Takuya ASADA
237e36a0b4 dist/ami: update CentOS base image to latest version
Since we requires updated version of systemd, we need to update CentOS base
image.

Fixes #3184

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1518118694-23770-1-git-send-email-syuu@scylladb.com>
(cherry picked from commit 69d226625a)
2018-03-15 10:47:54 +02:00
Takuya ASADA
e78c137bfc dist/redhat: switch to gcc-7.3
We have hit following bug on debug-mode binary:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=82560
Since it's fixed on gcc-7.3, we need to upgrade our gcc package.

See: https://groups.google.com/d/topic/scylladb-dev/RIdIpqMeTog/discussion
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1521064473-17906-1-git-send-email-syuu@scylladb.com>
(cherry picked from commit 856dc0a636)
2018-03-15 10:39:40 +02:00
Avi Kivity
fb99a7c902 Merge "Ubuntu/Debian build error fixes" from Takuya
* 'debian-ubuntu-build-fixes-v2' of https://github.com/syuu1228/scylla:
  dist/debian: build only scylla, iotune
  dist/debian: switch to boost-1.65
  dist/debian: switch to gcc-7.3

(cherry picked from commit bb4b1f0e91)
2018-03-14 22:51:44 +02:00
Avi Kivity
9569ba5e38 Update scylla-ami submodule
* dist/ami/files/scylla-ami 3aa87a7...5170011 (3):
  > scylla_install_ami: install enhanced networking NIC drivers
  > scylla_install_ami: set kernel-ml as default kernel
  > scylla_install_ami: fix NIC down with enhanced networking on new base AMI
2018-03-11 15:45:05 +02:00
Takuya ASADA
3229a87fee dist/debian: Drop scylla-fstrim cron job from Debian 8/9
Since we installs scylla-fstrim systemd unit files on Debian 8/9, no need to
install cron job, so drop them.

Fixes #3249

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1519950212-16231-2-git-send-email-syuu@scylladb.com>
2018-03-04 16:13:06 +02:00
Takuya ASADA
759b4de7a5 dist/debian: drop systemd unit files on Ubuntu 14.04
Ubuntu 14.04 uses upstart as init program, don't need systemd unit files,
so drop them.

Fixes #3245

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1519950212-16231-1-git-send-email-syuu@scylladb.com>
2018-03-04 16:13:05 +02:00
Takuya ASADA
101e909483 dist/debian: install scylla-housekeeping upstart script correctly on Ubuntu 14.04
Since we splited scylla-housekeeping service to two different services for systemd, we don't share same service name between systemd and upstart anymore.
So handle it independently for each distribution, try to install
/etc/init/scylla-housekeeping.conf on Ubuntu 14.04.

Fixes #3239

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1519852659-10688-1-git-send-email-syuu@scylladb.com>
2018-03-01 10:36:11 +02:00
Takuya ASADA
69e3760920 dist/redhat: support CentOS/ppc64le
Support POWER architecture on Scylla.
Since DPDK is not fully supported on POWER (no PMD supported on it yet),
disabled it for now.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20180228203048.21593-1-syuu@scylladb.com>
2018-03-01 09:59:39 +02:00
Amnon Heiman
edcfab3262 dist/docker: Add support for housekeeping
This patch takes a modified version of the Ubuntu 14.04 housekeeping
service script and uses it in Docker to validate the current version.

To disable the version validation, pass the --disable-version-check flag
when running the container.

Message-Id: <20180220161231.1630-1-amnon@scylladb.com>
2018-02-21 09:26:02 +02:00
Takuya ASADA
2c2173917c dist/common/scripts/scylla_raid_setup: skip blkdiscard when disk is not supported TRIM
Since we unconditionally running blkdiscard on disks, we may get ioctl error
message on some disks which does not support TRIM.

This can be ignore but it's bad UX, so let's skip running blkdiscard when TRIM
is not supported on the disk.

Fixes #2774

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1517992904-13838-1-git-send-email-syuu@scylladb.com>
2018-02-07 13:30:05 +02:00
Takuya ASADA
6d134c0c2b dist/redhat: block installing Scylla on older kernel
We uses AmbientCapabilities directive on systemd unit, but it does not work
on older kernel, causes following error:
"systemd[5370]: Failed at step CAPABILITIES spawning /usr/bin/scylla: Invalid argument"

It only works on kernel-3.10.0-514 == CentOS7.3 or later, block installing rpm
to prevent the error.

Fixes #3176

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1517822764-2684-1-git-send-email-syuu@scylladb.com>
2018-02-05 12:57:17 +02:00
Takuya ASADA
dc2b17b3da dist/redhat: link yaml-cpp statically
To avoid incompatibility between distribution provided libyaml-cpp, link it
statically.

Fixes #3173

Message-Id: <1517546935-15858-2-git-send-email-syuu@scylladb.com>
2018-02-03 16:34:36 +02:00
Takuya ASADA
bec2b015e3 dist/debian: link yaml-cpp statically
To avoid incompatibility between distribution provided libyaml-cpp, link it
statically.

Fixes #3164

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1517313320-10712-1-git-send-email-syuu@scylladb.com>
2018-01-30 14:22:02 +02:00
Takuya ASADA
5f835be3aa dist/common/scripts/scylla_io_setup: check data_file_directories existance before running iotune
Currently we don't check data_file_directories existance before running iotune,
therefore it's shows unclear error message.
To make the message better, check the directory existance on scylla_io_setup.

Fixes #3137

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1517200647-6347-1-git-send-email-syuu@scylladb.com>
2018-01-29 18:11:12 +02:00
Alexys Jacob
604bc40d8a dist: migrate gentoo variant setup scripts from /sbin/service to /sbin/rc-service
the 'service' binary has been removed from gentoo as per news 2017-10-13:
https://gitweb.gentoo.org/data/gentoo-news.git/plain/2017-10-13-openrc-service-binary-removal/2017-10-13-openrc-service-binary-removal.en.txt

this patch updates the scylla setup related scripts where it was used and
make use of the 'rc-service' binary instead

Signed-off-by: Alexys Jacob <ultrabug@gentoo.org>
Message-Id: <20180119161310.15435-1-ultrabug@gentoo.org>
2018-01-21 17:15:26 +02:00
Takuya ASADA
f3c8574135 dist/debian: follow gcc-7.2 package naming changes on 3rdparty repo for Debian 9
Switch to renamed gcc-7.2 package on Debian 9, too.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1516191853-2562-1-git-send-email-syuu@scylladb.com>
2018-01-17 14:38:41 +02:00
Takuya ASADA
15e266eea4 dist/debian: fix package name typo on Debian 8
Correct package name is scylla-gcc72-g++-7, not scylla-g++-7.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1516189354-5880-1-git-send-email-syuu@scylladb.com>
2018-01-17 13:45:24 +02:00
Avi Kivity
6d7d02315e dist/redhat: support nowait aio even on old distributions
Since we sometimes recommend that the user update to a newer kernel,
it's good to compile support for features that the new kernel supports.
Rather than play games with build-time dependencies, just #define
those features in. It's ugly, but better than depending on third-party
repositories and handling package conflicts.
Message-Id: <20180115143129.22190-1-avi@scylladb.com>
2018-01-17 12:13:44 +01:00
Takuya ASADA
912a14eb9b dist/debian: follow renaming of gcc-7.2 packages on Ubuntu 14.04/16.04
Now we applied our scylla-$(pkg)$(ver) style package naming on gcc-7.2,
so switch to it.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1516103292-26942-1-git-send-email-syuu@scylladb.com>
2018-01-16 13:52:05 +02:00
Mika Eloranta' via ScyllaDB development
bc1248e62a build: rpm build script --xtrace option
Enables bash "set -o xtrace" printing of full executed command lines for
debugging purposes.

Signed-off-by: Mika Eloranta <mel@aiven.io>
Message-Id: <20180113212944.86008-1-mel@aiven.io>
2018-01-14 12:32:32 +02:00
Mika Eloranta' via ScyllaDB development
7266446227 build: fix rpm build script --jobs N handling
Fixes argument misquoting at $SRPM_OPTS expansion for the mock commands
and makes the --jobs argument work as supposed.

Signed-off-by: Mika Eloranta <mel@aiven.io>
Message-Id: <20180113212904.85907-1-mel@aiven.io>
2018-01-14 12:30:19 +02:00
Takuya ASADA
b68ee98310 dist/debian: make pbuilder works on Debian 9
On Debian 9, 'pbuilder create' fails because of lack of GPG key for
3rdparty repo, so we need --allow-untrusted on 'pbuilder create' and
'pbuilder update'.

Also, apt-key adv --fetch-keys does not works correctly on it, but we can use
"curl <URL> | apt-key add -" as workaround.

Fixes #3088

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1513797714-18067-1-git-send-email-syuu@scylladb.com>
2018-01-11 15:02:05 +02:00
Takuya ASADA
420b61b466 dist/debian: follow renaming of gcc-7.2 packages on Debian 8
Now we applied our scylla-$(pkg)$(ver) style package naming on gcc-7.2,
so switch to it.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1515522920-8266-1-git-send-email-syuu@scylladb.com>
2018-01-11 15:02:04 +02:00
Avi Kivity
56801d1b8c Update scylla-ami submodule
* dist/ami/files/scylla-ami 3366c93...3aa87a7 (1):
  > Move to kernel-ml kernel stream
2018-01-10 11:58:27 +02:00
Takuya ASADA
51013f561d dist/debian: rename boost1.63 to scylla-boost163 on Debian 8
We provided "boost1.63" package for Debian 8 since we couldn't build
"scylla-boost163" package witch is available on Ubuntu14/16, but I fixed the
problem and now we have it for Debian 8 too, so switch to it.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1514220163-25985-1-git-send-email-syuu@scylladb.com>
2017-12-25 18:51:36 +02:00
Avi Kivity
8dbc6bbcdc Update scylla-ami submodule
* dist/ami/files/scylla-ami be90a3f...3366c93 (1):
  > scylla_install_ami: skip ec2_check while building AMI
2017-12-19 10:10:22 +02:00
Takuya ASADA
77fbdd487c dist/ami: Switch to official CentOS base image
We had switched our own CentOS base image since we couldn't make built AMI to
public due to base image settings, it's probably because the image provided via
AWS market place.
However, I've found an official image outside of market place, and I succeeded
making built AMI to public based on the image.
URL: https://wiki.centos.org/Cloud/AWS

Once we could able to use official image, we probably should use official one.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1513659164-28029-1-git-send-email-syuu@scylladb.com>
2017-12-19 10:07:14 +02:00
Takuya ASADA
c2e87f4677 dist/common/systemd: specify correct repo file path for housekeeping service on Ubuntu/Debian
Currently scylla-housekeeping-daily.service/-restart.service hardcoded
"--repo-files '/etc/yum.repos.d/scylla*.repo'" to specify CentOS .repo file,
but we use same .service for Ubuntu/Debian.
It doesn't work correctly, we need to specify .list file for Debian variants.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1513385159-15736-1-git-send-email-syuu@scylladb.com>
2017-12-16 22:03:25 +02:00
Amnon Heiman
bc356a3c15 scylla_setup support private repo on debian during setup
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <20170917145248.19677-1-amnon@scylladb.com>
2017-12-11 10:36:30 +02:00
Amos Kong
8fd5d27508 dist/debian: add scylla-tools-core to depends list
Signed-off-by: Amos Kong <amos@scylladb.com>
Message-Id: <db39cbda0e08e501633556ab238d816e357ad327.1512646123.git.amos@scylladb.com>
2017-12-07 13:40:10 +02:00
Amos Kong
eb3b138ee2 dist/redhat: add scylla-tools-core to requires list
Fixes #3051

Signed-off-by: Amos Kong <amos@scylladb.com>
Message-Id: <f7013a4fbc241bb4429d855671fee4b845b255cd.1512646123.git.amos@scylladb.com>
2017-12-07 13:40:08 +02:00
Takuya ASADA
aeb6ebce5a dist/debian: need apt-get update after installing GPG key for 3rdparty repo
We need apt-get update after install GPG key, otherwise we still get
unauthenticated package error on Debian package build.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1512556948-29398-1-git-send-email-syuu@scylladb.com>
2017-12-06 12:43:17 +02:00
Takuya ASADA
8f02967a3b dist/debian: install CA certificates before install repo GPG key
Since pbuilder chroot environment does not install CA certificates by default,
accessing https://download.opensuse.org will cause certificate verification
error.
So we need to install it before installing 3rdparty repo GPG key.

Also, checking existance of gpgkeys_curl is not needed, since it's always
not installed since we are running the script in clean chroot environment.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1512517001-27524-1-git-send-email-syuu@scylladb.com>
2017-12-06 08:57:01 +01:00
Takuya ASADA
b492a1e1b1 dist/redhat: fix typo on build_rpm.sh
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1512466884-18383-2-git-send-email-syuu@scylladb.com>
2017-12-05 13:40:03 +02:00
José Guilherme Vanz
5261eb7225 build_rpm.sh: command line argument not used
The command line argument `--configure-user` of the build_rpm.sh script
is used nowhere. Thus, this commit remove it all code related to
this flag.

Signed-off-by: José Guilherme Vanz <guilherme.sft@gmail.com>
Message-Id: <20171205025920.401-1-guilherme.sft@gmail.com>
2017-12-05 13:24:17 +02:00
Takuya ASADA
4ea3daede9 dist/debian: support Ubuntu 18.04LTS
Ubuntu 18.04LTS is not released yet, but it's already usable so we can prepare
for it.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2017-12-04 19:49:19 +09:00
Takuya ASADA
b4695611ed dist/debian: disable ALLOWUNTRUSTED
We have enabled secure-apt for 3rdparty repos, so we don't need ALLOWUNTRUSTED
anymore.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2017-12-04 19:49:19 +09:00
Takuya ASADA
92f8743f97 dist/debian: enable secure-apt for Debian
Enable secure-apt for Debian as well.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2017-12-04 19:49:19 +09:00
Takuya ASADA
a9fef02f9c dist/debian: enable secure-apt for Ubuntu
Our external repos are already signed repo, so let's enable secure-apt.
Seems like more recent version of Ubuntu (tested on 18.04) does not accept
skipping GPG check, so we need it anyway in near future.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2017-12-04 19:49:19 +09:00
Takuya ASADA
531c2e4e89 dist/ami: support AMI cross build
Now we can cross build our .rpm/.deb packages, so let's extend AMI build script
to support cross build, too.

Also Ubuntu 16.04 support added, since it's latest Ubuntu LTS release.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1510247204-2899-1-git-send-email-syuu@scylladb.com>
2017-12-04 12:33:24 +02:00
Takuya ASADA
8c403ea4e0 dist/debian: disable entire pybuild actions
Even after 25bc18b commited, we still see the build error similar to #3036 on
some environment, but not on dh_auto_install, it on dh_auto_test (see #3039).

So we need to disable entire pybuild actions, not just dh_auto_install.

Fixes #3039

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1512185097-23828-1-git-send-email-syuu@scylladb.com>
2017-12-02 19:36:43 +02:00
Takuya ASADA
25bc18b8ff dist/debian: skip running dh_auto_install on pybuild
We are getting package build error on dh_auto_install which is invoked by
pybuild.
But since we handle all installation on debian/scylla-server.install, we can
simply skip running dh_auto_install.

Fixes #3036

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1512065117-15708-1-git-send-email-syuu@scylladb.com>
2017-12-01 16:06:25 +02:00
Takuya ASADA
7380a6088b dist/debian: link libgcc dynamically
As we discussed on the thread (https://github.com/scylladb/scylla/issues/2941),
since we override symbols on libgcc, we need to link libgcc dynamically for
Ubuntu/Debian too (CentOS already do it).

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1511542866-21486-2-git-send-email-syuu@scylladb.com>
2017-11-25 20:09:51 +02:00
Takuya ASADA
df6546d151 dist/debian: switch to our PPA verions of gcc-72
Now we have gcc-7.2 on our PPA for Ubuntu 16.04/14.04, let's switch to it.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1511542866-21486-1-git-send-email-syuu@scylladb.com>
2017-11-25 20:09:51 +02:00
Takuya ASADA
c1b97d11ea dist/redhat: avoid hardcoding GPG key file path on scylla-epel-7-x86_64.cfg
Since we want to support cross building, we shouldn't hardcode GPG file path,
even these files provided on recent version of mock.

This fixes build error on some older build environment such as CentOS-7.2.

Fixes #3002

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1511277722-22917-1-git-send-email-syuu@scylladb.com>
2017-11-21 17:25:39 +02:00
Takuya ASADA
ab9d7cdc65 dist/debian: switch Debian 3rdparty packages to external build service
Switch Debian 3rdparty packages to our OBS repo
(https://build.opensuse.org/project/subprojects/home:scylladb).

We don't use 3rdparty packages on dist/debian/dep, so dropped them.
Also we switch Debian to gcc-7.2/boost-1.63 on same time.

Due to packaging issues following packages doesn't renamed our 3rdparty
package naming rule for now:
 - gcc-7: renamed as 'xxx-scylla72', instead of scylla-xxx-72.
 - boost1.63: doesn't renamed, also doesn't changed prefix to /opt/scylladb

Message-Id: <1510952411-20722-1-git-send-email-syuu@scylladb.com>
2017-11-20 13:07:04 +02:00
Takuya ASADA
07c191af41 dist/common/scripts/scylla_dev_mode_setup: include scylla_lib.sh
To use verify_args function we requires scylla_lib.sh, so include it.

Fixes #2945

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1510154173-18017-1-git-send-email-syuu@scylladb.com>
2017-11-15 11:31:14 +02:00
Takuya ASADA
f607a01cc5 dist/debian: link boost statically
Since we switched scylla-boost163 which isn't provided by distribution repo,
we need to link them statically.

Fixes #2946

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1510229553-29801-1-git-send-email-syuu@scylladb.com>
2017-11-09 14:51:00 +02:00