Commit Graph

110 Commits

Author SHA1 Message Date
Vlad Zolotarov
c74e1a9368 dist: don't distribute posix_net_conf.sh any more
We don't need it since we use perftune.py directly

Signed-off-by: Vlad Zolotarov <vladz@scylladb.com>
2018-10-31 15:27:00 -04:00
Takuya ASADA
5bf9a03d65 dist/debian: skip running dh_strip_nondeterminism
On some Fedora environment dh build tries to run
dh_strip_nondeterminism, and fails sice Fedora does not provide such
command.
(see:
http://jenkins.cloudius-systems.com/view/master/job/scylla-master/job/unified-deb/3/console)

To prevent the build error we need to skip it.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20181030062935.9930-1-syuu@scylladb.com>
2018-10-31 10:23:54 +02:00
Avi Kivity
925ef48fce Merge "Use relocatable package to generate .rpm/.deb" from Takuya
"
This patchset adds support generating .rpm/.deb from relocatable
package.
"

* 'reloc_rpmdeb_v5' of https://github.com/syuu1228/scylla:
  configure.py: run create-relocatable-package.py everytime
  configure.py: add SCYLLA-RELEASE-FILE/SCYLLA-VERSION-FILE targets
  configure.py: use {mode} instead of $mode on scylla-package.tar.gz build target
  dist/ami: build relocatable .rpm when --localrpm specified
  dist/debian: use relocatable package to produce .deb
  dist/redhat: use relocatable package to produce .rpm
  install-dependencies.sh: add libsystemd as dependencies
  install.sh: drop hardcoded distribution name, add --target option to specify distribution
  build: add script to build relocatable package
  build: compress relocatable package
  build: add files on relocatable package to support generating .rpm/.deb
2018-10-24 14:44:09 +03:00
Takuya ASADA
f3c3b9183c dist/debian: use relocatable package to produce .deb
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2018-10-24 11:29:47 +00:00
Glauber Costa
fc5635100d install seastar-addr2line and seastar-cpumap into scylla packages
It is very useful for investigations in scylla issues, and we have
been moving those scripts manually when needed. Make it officially
part of the scylla package.

Signed-off-by: Glauber Costa <glauber@scylladb.com>
Message-Id: <20181023184400.23187-1-glauber@scylladb.com>
2018-10-24 09:52:17 +03:00
Takuya ASADA
950dbdb466 dist/common/sysctl.d: add new conf file to set fs.aio-max-nr
We need raise fs.aio-max-nr to larger value since Seastar may allocates
more then 65535 AIO events (= kernel default value)

Fixes #3842

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20181023030449.15445-1-syuu@scylladb.com>
2018-10-23 11:01:07 +03:00
Takuya ASADA
d89114d1fc dist/debian: install GPG key for cross-building
We found on some Debian environment Ubuntu .deb build fails with
gpg error because lack of Ubuntu GPG key, so we need to install it before
start pbuilder.
Same as on Ubuntu, it needs to install Debian GPG key.

Fixes #3823

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20181008072246.13305-1-syuu@scylladb.com>
2018-10-08 10:43:25 +03:00
Takuya ASADA
2136479012 dist/debian: delete mounts.conf on scylla-server.postrm
Since we added mounts.conf on 687372bc48,
we need to delete the file on uninstall the package.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20180905204631.9265-1-syuu@scylladb.com>
2018-09-06 16:50:14 +03:00
Takuya ASADA
d78762d627 dist/debian: fix broken debian/changelog
It also need $MUSTACHE_DIST.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20180903094558.3862-1-syuu@scylladb.com>
2018-09-03 14:04:01 +03:00
Takuya ASADA
4a5157857a dist/debian: support package renaming on build script
To automatically rename packages on enterprise release, added package name
prefix as a variable on build_deb.sh.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20180828010445.11920-1-syuu@scylladb.com>
2018-08-28 09:25:07 +03:00
Takuya ASADA
f30b701872 dist/debian: fix install scylla-server.service
On previous commit we moved debian/scylla-server.service to
debian/scylla-server.scylla-server.service to explicitly specify
subpackage name, but it doesn't work for dh_installinit without '--name'
option.

Result of that current scylla-server .deb package missing
scylla-server.service, so we need to rename the service to original
file name.

Fixes #3675

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20180810221944.24837-1-syuu@scylladb.com>
2018-08-11 15:07:37 +03:00
Takuya ASADA
1bb463f7e5 dist/debian: install *.service on correct subpackage
We mistakenly installing scylla-housekeeping-*.service to scylla-conf
package, all *.service should explicitly specified subpackage name.

Fixes #3642

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20180801233042.307-1-syuu@scylladb.com>
2018-08-02 11:39:52 +03:00
Takuya ASADA
58f094e06d dist/debian: fix ImportError on pystache
Seems like pystache does not provides dependency, need to install it on
build_deb.sh.

Fixes #3627

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20180724164852.16094-1-syuu@scylladb.com>
2018-07-25 07:42:19 +03:00
Takuya ASADA
3cb7ddaf68 dist/debian/build_deb.sh: make build_deb.sh more simplified
Use is_debian()/is_ubuntu() to detect target distribution, also install
pystache by path since package name is different between Fedora and
CentOS.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20180703193224.4773-1-syuu@scylladb.com>
2018-07-04 11:12:26 +03:00
Takuya ASADA
4151120752 dist/debian: change owner of build/debs/ to current user
Currently build/debs/ is owned by root user since pbuilder requires to
run in root.
So chown them after finished building.

Fixes #3447

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20180613093213.28827-1-syuu@scylladb.com>
2018-06-20 16:48:17 +03:00
Takuya ASADA
9971576ecb dist: drop collectd support from package
Since scyllatop no longer needs collectd, now we are able to drop collectd.

resolves #3490

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1528961612-8528-1-git-send-email-syuu@scylladb.com>
2018-06-14 10:40:23 +03:00
Takuya ASADA
899f7641b6 dist/debian: fix pystache package name on Debian / Ubuntu
It's python-pystache, not python2-pystache.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2018-06-06 15:55:52 +09:00
Takuya ASADA
db9074707a dist/debian: switch to systemd-coredump on Debian 9
Debian 9 has newer systemd that supports systemd-coredump, so enable it.
2018-06-06 15:04:31 +09:00
Takuya ASADA
30386ed215 dist/debian: rename 99-scylla.conf to 99-scylla-coredump.conf
Since 99-scylla.conf is only used for setting coredump handler, rename
it to 99-scylla-coredump.conf.
2018-06-06 14:59:32 +09:00
Piotr Sarna
6130a00597 dist: add scylla/hints directory to scripts
/var/lib/scylla/hints directory was missing from dist-specific
scripts, which may cause package installations to fail.
Package building scripts and descriptions are updated/

Fixes #3495

Message-Id: <0f5596cb49500416820ece023b7f76a4e2427799.1528184949.git.sarna@scylladb.com>
2018-06-05 11:33:29 +03:00
Takuya ASADA
ad4ca1e166 dist: simplified build script templates
Currently, build_deb.sh looks very complicated because each of distribution
requires different parameter, and we are applying them by sed command one-by-one.

This patch will replace them by Mustache, it's simple and easy syntax
template language.
Both .rpm distributions and .deb distributions have pystache (a Python
implimentation of Mustache), we will use it.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20180604104026.22765-1-syuu@scylladb.com>
2018-06-04 14:38:52 +03:00
Takuya ASADA
e0d49aae37 dist/debian: fix missing --configfile parameter on pdebuild
We need to specify --configfile on pdebuild too, otherwise we will
always fail to build .deb on newly created build environment.
Only reason why we still able to build .deb is we already copied
.pbuilderrc to home directory on existing build environment.

Fixes #3456

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20180523204112.24669-1-syuu@scylladb.com>
2018-05-24 10:10:27 +03:00
Avi Kivity
3dd2f68712 dist: drop libunwind dependency
Since Seastar no longer (1f005fb434) requires libunwind, we can
drop it from our dependency list.  This helps the power build, for
which no libunwind is available.

Fixes #3453.
Message-Id: <20180523114750.10753-1-avi@scylladb.com>
2018-05-23 13:53:29 +02:00
Takuya ASADA
5407c34c73 dist/debian: depends to coreutils instead of realpath on Ubuntu 18.04
On Ubuntu 18.04 realpath package is dropped, it becomes part of coreutils.

Fixes #3445

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20180521031954.30815-1-syuu@scylladb.com>
2018-05-21 10:42:05 +03:00
Glauber Costa
f837d5b1f1 scripts: move scylla-blocktune to scripts location
scylla-blocktune currently lives in the top level but this is mostly
historical. When time comes for us to install it, the packaging systems
will copy it to /usr/lib/scylla with the others.

So for consistency let's make sure that it also lives in the scripts
directory.

Signed-off-by: Glauber Costa <glauber@scylladb.com>
2018-04-28 09:50:07 -04:00
Takuya ASADA
782ebcece4 dist/debian: add --jobs <njobs> option just like build_rpm.sh
On some build environment we may want to limit number of parallel jobs since
ninja-build runs ncpus jobs by default, it may too many since g++ eats very
huge memory.
So support --jobs <njobs> just like on rpm build script.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20180425205439.30053-1-syuu@scylladb.com>
2018-04-26 12:44:06 +03:00
Takuya ASADA
7b92c3fd3f dist: Drop AmbientCapabilities from scylla-server.service for Debian 8
Debian 8 causes "Invalid argument" when we used AmbientCapabilities on systemd
unit file, so drop the line when we build .deb package for Debian 8.
For other distributions, keep using the feature.

Fixes #3344

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20180423102041.2138-1-syuu@scylladb.com>
2018-04-23 13:27:14 +03:00
Takuya ASADA
01c36556bf dist/debian: use --configfile to specify pbuilderrc
Use --configfile to specify pbuilderrc, instead of copying it to home directory.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20180420024624.9661-1-syuu@scylladb.com>
2018-04-22 16:06:42 +03:00
Takuya ASADA
e99f43ef43 dist/debian: call lsb_release after command existance check
Fixes #3364

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1523907917-13188-1-git-send-email-syuu@scylladb.com>
2018-04-17 10:54:39 +03:00
Takuya ASADA
ace44784e8 dist/debian: use ~root as HOME to place .pbuilderrc
When 'always_set_home' is specified on /etc/sudoers pbuilder won't read
.pbuilderrc from current user home directory, and we don't have a way to change
the behavor from sudo command parameter.

So let's use ~root/.pbuilderrc and switch to HOME=/root when sudo executed,
this can work both environment which does specified always_set_home and doesn't
specified.

Fixes #3366

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1523926024-3937-1-git-send-email-syuu@scylladb.com>
2018-04-17 09:37:16 +03:00
Takuya ASADA
5a71d4f814 dist/debian: use apt-get instead of apt
To suppress following warning, use apt-get instead of apt:
"WARNING: apt does not have a stable CLI interface. Use with caution in scripts."

Fixes #3365

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1523909727-13343-1-git-send-email-syuu@scylladb.com>
2018-04-17 09:29:16 +03:00
Avi Kivity
52882d1bd9 dist: debian: try harder to set the target distribution
build_deb.sh relies on pbuilder picking up a ~/.pbuilderrc which we
copy from the script. According to the pbuilder manual, "~" will refer
to the root directory (since pbuilder is run via sudo). In practice
we've observed this working with "~" referring to the current user's
home directory, but also sometimes failing, while complaining
about /root/.pbuilderrc failing. When it fails, it fails to set
the correct distribution.

To be extra sure, also copy .pbuilderrc to root's home directory. This
way, whatever behavior pbuilder chooses to follow, it will have a
configuration file to read.
Message-Id: <20180410134508.9415-1-avi@scylladb.com>
2018-04-16 10:10:47 +02:00
Takuya ASADA
2045891cc2 dist/debian: use rebuilt libyaml-cpp on Debian9
On Debian9, distribution provided libyaml-cpp does not able to link against
scylla, use rebuilt one from our 3rdparty repo.

fixes #3221

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1521496288-12856-1-git-send-email-syuu@scylladb.com>
2018-03-20 12:30:47 +02:00
Takuya ASADA
378bf7cec0 dist/debian: switch Debian9 to boost-1.65
We switched Debian8/Ubuntu14/Ubuntu16 to boost-1.65 to fix #3090, but Debian9
stil uses distribution provided boost-1.62, it causes same build error.
So switch it to our boost-1.65, too.

See c636f552e0

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1521250590-16510-1-git-send-email-syuu@scylladb.com>
2018-03-18 10:23:43 +02:00
Takuya ASADA
945e6ec4f6 dist/debian: use 3rdparty ppa on Ubuntu 18.04
Currently Ubuntu 18.04 uses distribution provided g++ and boost, but it's easier
to maintain Scylla package to build with same version toolchain/libraries, so
switch to them.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1521075576-12064-1-git-send-email-syuu@scylladb.com>
2018-03-15 10:41:05 +02:00
Takuya ASADA
7f891e7a48 dist/debian: build only scylla, iotune
Since we don't package tests, we don't need to build them.
It reduces package building time.
2018-03-15 04:33:11 +09:00
Takuya ASADA
c636f552e0 dist/debian: switch to boost-1.65
We get following compile error on Debian/Ubuntu with boost-1.63:

/opt/scylladb/include/boost/intrusive/pointer_plus_bits.hpp:76:48: error: '*((void*)& __tmp +136)' is used uninitialized in this function [-Werror=uninitialized]
       n = pointer(uintptr_t(p) | (uintptr_t(n) & Mask));
                                         ~~~~~~~~~~~~~~^~~~~~~

This is known issue (https://github.com/boostorg/intrusive/issues/29), fixed
on boost-1.65.

Switch to boost-1.65 to fix the issue.

Fixes #3090
2018-03-14 22:13:24 +09:00
Takuya ASADA
c3b2e2580a dist/debian: switch to gcc-7.3
We have hit following bug on debug-mode binary:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=82560
Since it's fixed on gcc-7.3, we need to upgrade our gcc package.

See: https://groups.google.com/d/topic/scylladb-dev/RIdIpqMeTog/discussion
2018-03-08 00:06:32 +09:00
Takuya ASADA
3229a87fee dist/debian: Drop scylla-fstrim cron job from Debian 8/9
Since we installs scylla-fstrim systemd unit files on Debian 8/9, no need to
install cron job, so drop them.

Fixes #3249

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1519950212-16231-2-git-send-email-syuu@scylladb.com>
2018-03-04 16:13:06 +02:00
Takuya ASADA
759b4de7a5 dist/debian: drop systemd unit files on Ubuntu 14.04
Ubuntu 14.04 uses upstart as init program, don't need systemd unit files,
so drop them.

Fixes #3245

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1519950212-16231-1-git-send-email-syuu@scylladb.com>
2018-03-04 16:13:05 +02:00
Takuya ASADA
101e909483 dist/debian: install scylla-housekeeping upstart script correctly on Ubuntu 14.04
Since we splited scylla-housekeeping service to two different services for systemd, we don't share same service name between systemd and upstart anymore.
So handle it independently for each distribution, try to install
/etc/init/scylla-housekeeping.conf on Ubuntu 14.04.

Fixes #3239

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1519852659-10688-1-git-send-email-syuu@scylladb.com>
2018-03-01 10:36:11 +02:00
Takuya ASADA
bec2b015e3 dist/debian: link yaml-cpp statically
To avoid incompatibility between distribution provided libyaml-cpp, link it
statically.

Fixes #3164

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1517313320-10712-1-git-send-email-syuu@scylladb.com>
2018-01-30 14:22:02 +02:00
Takuya ASADA
f3c8574135 dist/debian: follow gcc-7.2 package naming changes on 3rdparty repo for Debian 9
Switch to renamed gcc-7.2 package on Debian 9, too.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1516191853-2562-1-git-send-email-syuu@scylladb.com>
2018-01-17 14:38:41 +02:00
Takuya ASADA
15e266eea4 dist/debian: fix package name typo on Debian 8
Correct package name is scylla-gcc72-g++-7, not scylla-g++-7.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1516189354-5880-1-git-send-email-syuu@scylladb.com>
2018-01-17 13:45:24 +02:00
Takuya ASADA
912a14eb9b dist/debian: follow renaming of gcc-7.2 packages on Ubuntu 14.04/16.04
Now we applied our scylla-$(pkg)$(ver) style package naming on gcc-7.2,
so switch to it.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1516103292-26942-1-git-send-email-syuu@scylladb.com>
2018-01-16 13:52:05 +02:00
Takuya ASADA
b68ee98310 dist/debian: make pbuilder works on Debian 9
On Debian 9, 'pbuilder create' fails because of lack of GPG key for
3rdparty repo, so we need --allow-untrusted on 'pbuilder create' and
'pbuilder update'.

Also, apt-key adv --fetch-keys does not works correctly on it, but we can use
"curl <URL> | apt-key add -" as workaround.

Fixes #3088

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1513797714-18067-1-git-send-email-syuu@scylladb.com>
2018-01-11 15:02:05 +02:00
Takuya ASADA
420b61b466 dist/debian: follow renaming of gcc-7.2 packages on Debian 8
Now we applied our scylla-$(pkg)$(ver) style package naming on gcc-7.2,
so switch to it.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1515522920-8266-1-git-send-email-syuu@scylladb.com>
2018-01-11 15:02:04 +02:00
Takuya ASADA
51013f561d dist/debian: rename boost1.63 to scylla-boost163 on Debian 8
We provided "boost1.63" package for Debian 8 since we couldn't build
"scylla-boost163" package witch is available on Ubuntu14/16, but I fixed the
problem and now we have it for Debian 8 too, so switch to it.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1514220163-25985-1-git-send-email-syuu@scylladb.com>
2017-12-25 18:51:36 +02:00
Takuya ASADA
c2e87f4677 dist/common/systemd: specify correct repo file path for housekeeping service on Ubuntu/Debian
Currently scylla-housekeeping-daily.service/-restart.service hardcoded
"--repo-files '/etc/yum.repos.d/scylla*.repo'" to specify CentOS .repo file,
but we use same .service for Ubuntu/Debian.
It doesn't work correctly, we need to specify .list file for Debian variants.

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1513385159-15736-1-git-send-email-syuu@scylladb.com>
2017-12-16 22:03:25 +02:00
Amos Kong
8fd5d27508 dist/debian: add scylla-tools-core to depends list
Signed-off-by: Amos Kong <amos@scylladb.com>
Message-Id: <db39cbda0e08e501633556ab238d816e357ad327.1512646123.git.amos@scylladb.com>
2017-12-07 13:40:10 +02:00