Commit Graph

36 Commits

Author SHA1 Message Date
Yaniv Kaul
fef565482c Update unified/build_unified.sh
fix sentence overall
2023-12-05 15:23:38 +02:00
Yaniv Kaul
ae2ab6000a Typos: fix typos in code
Fixes some more typos as found by codespell run on the code.
In this commit, there are more user-visible errors.

Refs: https://github.com/scylladb/scylladb/issues/16255
2023-12-05 15:18:11 +02:00
Kefu Chai
75f458f2a5 unified: respect --pkgs option
let's provide the default value, only if user does not specify --pkgs.
otherwise the --pkgs option is always ignored.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
2023-09-12 22:56:10 +08:00
Kefu Chai
84387e3856 unified: allow passing --pkgs with a semicolon-separated list
simpler than passing a space-separated list requiring escaping, which
is a source of headache.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
2023-09-12 22:56:10 +08:00
Kefu Chai
d9dcda9dd5 unified: prefer SCYLLA-PRODUCT-FILE in build_dir
unlike `configure.py`, the building system created by CMake do not
share the `SCYLLA-PRODUCT-FILE` across different builds. so we cannot
assume that build/SCYLLA-PRODUCT-FILE exists.

so, in this change, we check $BUILD_DIR/SCYLLA-PRODUCT-FILE first,
and fallback to $BUILD_DIR/../SCYLLA-PRODUCT-FILE. this should work
for both configure.py and CMake building system.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
2023-09-12 22:56:10 +08:00
Kefu Chai
fea3a11716 unified: derive UNIFIED_PKG from --build-dir
we should respect the --build-dir if --unified-pkg is not specified,
and deduce the path to unified pkg from BUILD_DIR.

so, in this change, we deduce the path to unified pkg from BUILD_DIR
unless --unified-pkg is specfied.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
2023-09-12 22:56:10 +08:00
Kefu Chai
4bb5af763b unified: add --build-dir option to build_unified.sh
this allows build_unified.sh to generate unified pkg in specified
directory, instead of assuming the naming convention of build/$mode.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
2023-09-12 22:56:10 +08:00
Kefu Chai
befc78274b install.sh: pass -version to java executable
currently, despite that we are moving from Java-8 to
Java-11, we still support both Java versions. and the
docker image used for testing Datatax driver has
not been updated to install java-11.

the "java" executable provided by openjdk-java-8 does
not support "--version" command line argument. java-11
accept both "-version" and "--version". so to cater
the needs of the the outdated docker image, we pass
"-version" to the selected java. so the test passes
if java-8 is found. a better fix is to update the
docker image to install java-11 though.

the output of "java -version" and "java --version" is
attached here as a reference:

```console
$ /usr/lib/jvm/java-1.8.0/bin/java --version
Unrecognized option: --version
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
```

```console
$ /usr/lib/jvm/java-1.8.0/bin/java -version
openjdk version "1.8.0_362"
OpenJDK Runtime Environment (build 1.8.0_362-b09)
OpenJDK 64-Bit Server VM (build 25.362-b09, mixed mode)
```

```console
/usr/lib/jvm/jre-11/bin/java --version
openjdk 11.0.19 2023-04-18
OpenJDK Runtime Environment (Red_Hat-11.0.19.0.7-2.fc38) (build 11.0.19+7)
OpenJDK 64-Bit Server VM (Red_Hat-11.0.19.0.7-2.fc38) (build 11.0.19+7, mixed mode, sharing)
```

```console
$ /usr/lib/jvm/jre-11/bin/java -version
openjdk version "11.0.19" 2023-04-18
OpenJDK Runtime Environment (Red_Hat-11.0.19.0.7-2.fc38) (build 11.0.19+7)
OpenJDK 64-Bit Server VM (Red_Hat-11.0.19.0.7-2.fc38) (build 11.0.19+7, mixed mode, sharing)
```

Fixes #14253
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>

Closes #14254
2023-06-15 15:42:09 +03:00
Kefu Chai
031f770557 install.sh: use scylla-jmx for detecting JRE
now that scylla-jmx has a dedicated script for detecting the existence
of OpenJDK, and this script is included in the unified package, let's
just leverage it instead of repeating it in `install.sh`.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>

Closes #13514
2023-05-19 11:22:57 +03:00
Israel Fruchter
ef229a5d23 Repackaging cqlsh
cqlsh is moving into it's own repository:
https://github.com/scylladb/scylla-cqlsh

* add cqlsh as submodule
* update scylla-java-tools to have cqlsh remove
* introduced new cqlsh artifcat (rpm/deb/tar)

Depends: https://github.com/scylladb/scylla-tools-java/pull/316
Ref: scylladb/scylladb#11569

Closes #11937

[avi: restore tools/java submodule location, adjust commit]
2023-03-12 20:22:33 +02:00
Takuya ASADA
da6c472db9 install.sh: Skip systemd existance check when --without-systemd
When --without-systemd specified, install.sh should skip systemd
existance check.

Fixes #11898

Closes #11934
2022-11-14 14:07:46 +02:00
Takuya ASADA
1a11a38add unified: move unified package contents to sub-directory
On most of the software distribution tar.gz, it has sub-directory to contain
everything, to prevent extract contents to current directory.
We should follow this style on our unified package too.

To do this we need to increment relocatable package version to '3.0'.

Fixes #8349

Closes #8867
2022-10-25 08:58:15 +03:00
Takuya ASADA
49d5e51d76 reloc: add support stripped binary installation for relocatable package
This add support stripped binary installation for relocatable package.
After this change, scylla and unified packages only contain stripped binary,
and introduce "scylla-debuginfo" package for debug symbol.
On scylla-debuginfo package, install.sh script will extract debug symbol
at /opt/scylladb/<dir>/.debug.

Note that we need to keep unstripped version of relocatable package for rpm/deb,
otherwise rpmbuild/debuild fails to create debug symbol package.
This version is renamed to scylla-unstripped-$version-$release.$arch.tar.gz.

See #8918

Signed-off-by: Takuya ASADA <syuu@scylladb.com>

Closes #9005
2022-10-13 15:11:32 +02:00
Takuya ASADA
cd5320fe60 install.sh: add --without-systemd option
Since we fail to write files to $USER/.config on Jenkins jobs, we need
an option to skip installing systemd units.
Let's add --without-systemd to do that.

Also, to detect the option availability, we need to increment
relocatable package version.

See scylladb/scylla-dtest#2819

Closes #11345
2022-09-12 13:04:00 +03:00
Yaron Kaikov
c42c5111eb SCYLLA-VERSION-GEN: use semver-compatible version
Setting Scylla to use semantic versioning. (Ref: https://semver.org/)

Closes: https://github.com/scylladb/scylla/issues/9543

Closes #10957
2022-07-25 18:06:28 +03:00
Takuya ASADA
752be6536a rename relocatable packages
Currently, we use following naming convention for relocatable package
filename:
  ${package_name}-${arch}-package-${version}.${release}.tar.gz
But this is very different with Linux standard packaging system such as
.rpm and .deb.
Let's align the convention to .rpm style, so new convention should be:
  ${package_name}-${version}-${release}.${arch}.tar.gz

Closes #9799

Closes #10891

* tools/java de8289690e...d0143b447c (1):
  > build_reloc.sh: rename relocatable packages

* tools/jmx fe351e8...06f2735 (1):
  > build_reloc.sh: rename relocatable packages

* tools/python3 e48dcc2...bf6e892 (1):
  > reloc/build_reloc.sh: rename relocatable packages
2022-07-19 15:46:49 +03:00
Avi Kivity
fcb8d040e8 treewide: use Software Package Data Exchange (SPDX) license identifiers
Instead of lengthy blurbs, switch to single-line, machine-readable
standardized (https://spdx.dev) license identifiers. The Linux kernel
switched long ago, so there is strong precedent.

Three cases are handled: AGPL-only, Apache-only, and dual licensed.
For the latter case, I chose (AGPL-3.0-or-later and Apache-2.0),
reasoning that our changes are extensive enough to apply our license.

The changes we applied mechanically with a script, except to
licenses/README.md.

Closes #9937
2022-01-18 12:15:18 +01:00
Takuya ASADA
462961ca51 unified: fix handling --supervisor option
We need to pass --supervisor option just for scylla-server module,
and also pass --packaging option to scylla-jmx module to avoid running
systemctl command, since the option may run in container, and container
may not have systemd.

Fixes #9141

Closes #9142
2021-08-18 13:17:08 +03:00
Takuya ASADA
fdc786b451 install.sh: add supervisor support
Bring supervisor support from dist/docker to install.sh, make it
installable from relocatable package.
This enables to use supervisor with nonroot / offline environment,
and also make relocatable package able to run in Docker environment.

Related #8849

Closes #8918
2021-07-27 12:51:29 +03:00
Takuya ASADA
edd54a9463 reloc: add arch to relocatable package filename
Add architecture name for relocatable packages, to support distributing
both x86_64 version and aarch64 version.

Also create symlink from new filename to old filename to keep
compatibility with older scripts.

Fixes #8675

Closes #8709

[update tools/python3 submodule:

* tools/python3 ad04e8e...afe2e7f (1):
  > reloc: add arch to relocatable package filename
]
2021-06-28 15:01:09 +03:00
Avi Kivity
a55b434a2b treewide: extent copyright statements to present day 2021-06-06 19:18:49 +03:00
Takuya ASADA
a3b25e3d29 unified/uninstall.sh: simplify uninstall.sh, delete all files correctly
Current uninstall.sh is trying to do similar logic with install.sh,
but it makes script larger meaninglessly, and also it failing to remove
few files under /opt/scylladb.

Let's just do rm -rf /opt/scylladb, and drop few other files located out
side of /opt/scylladb.

Closes #8662
2021-05-18 14:55:18 +02:00
Takuya ASADA
cbbd5b2b6f unified: abort install when non-bash shell detected
On Debian variants, sh -x ./install.sh will fail since our script in
written in bash, and /bin/sh in Debian variants is dash, not bash.

So detect non-bash shell and print error message, let users to run in
bash.

Fixes #8479

Closes #8484
2021-04-15 11:59:41 +02:00
Takuya ASADA
a34edf8169 install.sh: add systemd existance check
offline installer can run in non-systemd distributions, but it won't
work since we only have systemd units.
So check systemd existance and print error message.
2021-01-13 19:32:45 +09:00
Avi Kivity
bcced9f56b build: compress unified package faster
The unified package is quite large (1GB compressed), and it
is the last step in the build so its build time cannot be
parallized with other tasks. Compress it with pigz to take
advantage of multiple cores and speed up the build a little.

Closes #7670
2020-11-23 00:31:04 +02:00
Eliran Sinvani
8380ac93c5 build: Make artifacts product aware
This commit changes the build file generation and the package
creation scripts to be product aware. This will change the
relocatable package archives to be named after the product,
this commit deals with two main things:
1. Creating the actual Scylla server relocatable with a product
prefixed name - which is independent of any other change
2. Expect all other packages to create product prefixed archive -
which is dependant uppon the actual submodules creating
product prefixed archives.

If the support is not introduced in the submodules first this
will break the package build.

Tests: Scylla full build with the original product and a
different product name.

Closes #7581
2020-11-10 14:38:10 +02:00
Takuya ASADA
4410934829 install.sh: show warning nonroot mode when systemd does not support user mode
On older distribution such as CentOS7, it does not support systemd user mode.
On such distribution nonroot mode does not work, show warning message and
skip running systemctl --user.

Fixes #7071
2020-11-09 12:16:35 +02:00
Takuya ASADA
ba29074c42 install.sh: stop using symlinks for systemd units on nonroot mode
On some environment, systemctl enable <service> fails when we use symlink.
So just directly copy systemd units to ~/.config/systemd/user, instead of
creating symlink.

Fixes #7288

Closes #7290
2020-09-29 12:20:41 +03:00
Avi Kivity
311b6b827c build: unified tarball: reduce log spam
Don't print out the names of all files archived, there are too
many of them.

Closes #7191
2020-09-18 15:00:59 +03:00
Avi Kivity
05229d1d31 Merge "Add unified tarball to build "dist" target" from Pekka
"
This pull request fixes unified relocatable package dependency issues in
other build modes than release, and then adds unified tarball to the
"dist" build target.

Fixes #6949
"

* 'penberg/build/unified-to-dist/v1' of github.com:penberg/scylla:
  configure.py: Build unified tarball as part of "dist" target
  unified/build_unified: Use build/<mode>/dist/tar for dependency tarballs
  configure.py: Use build/<mode>/dist/tar for unified tarball dependencies
2020-09-14 11:29:28 +03:00
Takuya ASADA
233d0fc0e5 unified: don't proceed offline install when openjdk is not available
Currently, we run openjdk existance check after scylla main program installed.
We should do it before installing anything.
2020-09-13 12:39:05 +03:00
Pekka Enberg
138e723e56 unified/build_unified: Use build/<mode>/dist/tar for dependency tarballs
The build_unified.sh script has the same bug as configure.py had: it
looks for the python tarball in
build/<mode>/scylla-python3-package.tar.gz, but it's never generated
there. Fix up the problem by using build/<mode>/dist/tar location for
all dependency tarballs.
2020-09-11 12:37:44 +03:00
Amos Kong
5785947e28 unified/install.sh: set default python3/sysconfdir smartly
Users can set python3 and sysconfdir from cmdline of install.sh
according to the install mode (root or nonroot) and distro type.

It's helpful to correct the default python3/sysconfdir, otherwise
setup scripts or scylla-jmx doesn't work.

Fixes #7130

Signed-off-by: Amos Kong <amos@scylladb.com>
2020-08-31 15:54:51 +03:00
Takuya ASADA
f1255cb2d0 unified: add uninstall.sh
Provide an uninstaller for offline & nonroot installation.

Fixes #7076
2020-08-29 20:55:06 +03:00
Pekka Enberg
39400f58fb build_unified.sh: Compress generated tarball
Fixes #7039
Message-Id: <20200813124921.1648028-1-penberg@scylladb.com>
2020-08-16 14:41:01 +03:00
Takuya ASADA
e52ae78f79 reloc: support unified relocatable package
This introduce unified relocatable package, a single tarball to install
all Scylla packages.

Fixes #6626
See scylladb/scylla-pkg#1218
2020-07-15 20:29:31 +03:00