With the use of Docker image, some extra options needed to be exposed
to provide extended functionality when starting the image. The flags
added by this commit are:
- cluster-name: name of the Scylla cluster. cluster_name option in
scylla.yaml.
- rpc-address: IP address for client connections (CQL). rpc_address
option in scylla.yaml.
- endpoint-snitch: The snitch used to discover the cluster topology.
endpoint_snitch option in scylla.yaml.
- replace-address-first-boot: Replace a Scylla node by its IP.
replace_address_first_boot option in scylla.yaml.
Signed-off-by: Yannis Zarkadas <yanniszarkadas@gmail.com>
[ penberg@scylladb.com: fix up merge conflicts ]
Message-Id: <20181108234212.19969-2-yanniszarkadas@gmail.com>
dist/docker/redhat/docker-entrypoint.py:20:1: E722 do not use bare 'except'
dist/docker/redhat/commandlineparser.py:13:13: E128 continuation line
under-indented for visual indent
Signed-off-by: Alexys Jacob <ultrabug@gentoo.org>
Message-Id: <20181104120134.9598-1-ultrabug@gentoo.org>
We tune NIC and disks together now. Change the sysconfig parameter to
reflect this new semantics.
However if we detect an old parameter name in the scylla-server we would
still update it thereby keeping the support for old installations.
Signed-off-by: Vlad Zolotarov <vladz@scylladb.com>
Since our scripts were converted to Python, we can no longer
source them from a shell. Execute them directly instead. Also,
we now need to import configuration variables ourselves, since
scylla_prepare, being an independent process, won't do it for
us.
Fixes#3647
Message-Id: <20180802153017.11112-1-avi@scylladb.com>
By default Scylla docker runs without the security features.
This patch adds support for the user to supply different params values for the
authenticator and authorizer classes and allowing to setup a secure Scylla in
Docker.
For example if you want to run a secure Scylla with password and authorization:
docker run --name some-scylla -d scylladb/scylla --authenticator
PasswordAuthenticator --authorizer CassandraAuthorizer
Update the Docker documentation with the new command line options.
Signed-off-by: Noam Hasson <noam@scylladb.com>
Message-Id: <20180620122340.30394-1-noam@scylladb.com>
When provisioning a Scylla docker image with --developer-mode 0 (disabled)
scylla_raid_setup is not invoked. As a consequence the "data" directory is not
created and scylla_io_setup fails (steps to reproduce and error message provided
at the end).
This patch adds the same verifications present in scylla_io_setup to docker's
scyllasetup.py and creates the data directory in the case it is not present.
--
Steps to reproduce on AWS i3.2xlarge with Ubuntu 16.04:
sudo -s
apt update && apt upgrade -y && apt-get install docker.io -y
mdadm --create --verbose --force --run /dev/md0 --level=0 -c1024 --raid-devices=1 /dev/nvme0n1
mkfs.xfs /dev/md0 -f -K
mkdir /var/lib/scylla
mount -t xfs /dev/md0 /var/lib/scylla
docker run --name some-scylla \
--volume /var/lib/scylla:/var/lib/scylla \
-p 9042:9042 -p 7000:7000 -p 7001:7001 -p 7199:7199 \
-p 9160:9160 -p 9180:9180 -p 10000:10000 \
-d scylladb/scylla --overprovisioned 1 --developer-mode 0
docker logs some-scylla
running: (['/usr/lib/scylla/scylla_dev_mode_setup', '--developer-mode', '0'],)
running: (['/usr/lib/scylla/scylla_io_setup'],)
terminate called after throwing an instance of 'std::system_error'
what(): open: No such file or directory
ERROR:root:/var/lib/scylla/data did not pass validation tests, it may not be on XFS and/or has limited disk space.
This is a non-supported setup, and performance is expected to be very bad.
For better performance, placing your data on XFS-formatted directories is required.
To override this error, enable developer mode as follow:
sudo /usr/lib/scylla/scylla_dev_mode_setup --developer-mode 1
failed!
Traceback (most recent call last):
File "/docker-entrypoint.py", line 15, in <module>
setup.io()
File "/scyllasetup.py", line 34, in io
self._run(['/usr/lib/scylla/scylla_io_setup'])
File "/scyllasetup.py", line 23, in _run
subprocess.check_call(*args, **kwargs)
File "/usr/lib64/python3.4/subprocess.py", line 558, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/usr/lib/scylla/scylla_io_setup']' returned non-zero exit status 1
ls -latr /var/lib/scylla
total 4
drwxr-xr-x 44 root root 4096 Abr 24 13:02 ..
drwxr-xr-x 2 root root 6 Abr 24 13:10 .
Signed-off-by: Moreno Garcia <moreno@scylladb.com>
Message-Id: <20180424173729.22151-1-moreno@scylladb.com>
By default, overprovisioned is not enabled on docker unless it is
explicitly set. I have come to believe that this is a mistake.
If the user is running alone in the machine, and there are no other
processes pinned anywhere - including interrupts - not running
overprovisioned is the best choice.
But everywhere else, it is not: even if a user runs 2 docker containers
in the same machine and statically partitions CPUs with --smp (but
without cpuset) the docker containers will pin themselves to the same
sets of CPU, as they are totally unaware of each other.
It is also very common, specially in some virtualized environments, for
interrupts not to be properly distributed - being particularly keen on
being delivered on CPU0, a CPU which Scylla will pin by default.
Lastly, environments like Kubernetes simply don't support pinning at the
moment.
This patch enables the overprovisioned flag if it is explicitly set -
like we did before - but also by default unless --cpuset is set.
Signed-off-by: Glauber Costa <glauber@scylladb.com>
Message-Id: <20180331142131.842-1-glauber@scylladb.com>
This patch takes a modified version of the Ubuntu 14.04 housekeeping
service script and uses it in Docker to validate the current version.
To disable the version validation, pass the --disable-version-check flag
when running the container.
Message-Id: <20180220161231.1630-1-amnon@scylladb.com>
Users sometimes need to run their own yaml configuration files, and it
is currently annoying to deploy modified files on docker.
One possible solution is to bind mount the file into the docker
container using the -v switch, just like we already do for for the data
volume.
The problem with the aforementioned approach is that we have to change
the yaml file to insert the addresses, and that will change the file in
the host (or fail to happen, if we bind mount it read-only).
The solution I am proposing is to avoid touching the yaml file inside
the container altogether. Instead, we can deploy the address-related
arguments that we currently write to the yaml file as Scylla options.
Fixes#2113
Signed-off-by: Glauber Costa <glauber@scylladb.com>
Message-Id: <1490195141-19940-1-git-send-email-glauber@scylladb.com>
If early startup fails in docker-entrypoint.py, the container does not
start. It's therefore not very helpful to log to a file _within_ the
container...
Message-Id: <1490275943-23590-1-git-send-email-penberg@scylladb.com>
This patch exposes Scylla's Prometheus port by default. You can now use
the Scylla Monitoring project with the Docker image:
https://github.com/scylladb/scylla-grafana-monitoring
To configure the IP addresses, use the 'docker inspect' command to
determine Scylla's IP address (assuming your running container is called
'some-scylla'):
docker inspect --format='{{ .NetworkSettings.IPAddress }}' some-scylla
and then use that IP address in the prometheus/scylla_servers.yml
configuration file.
Fixes#1827
Message-Id: <1490008357-19627-1-git-send-email-penberg@scylladb.com>
wget is often used from scripts recording to logs; as it emits a log
line every second, the logs are huge and unreadable. Make it quieter.
Message-Id: <1477558534-32718-1-git-send-email-avi@scylladb.com>
In order to allow Scylla’s docker container to handle multiple network
interfaces, the start-scylla script was refactored:
- `$IP` is now called `$SCYLLA_LISTEN_ADDRESS`, so it is less likely to
be confused or interfere with other environment variables.
- `$SCYLLA_LISTEN_ADDRESS` now checks its value and also tries to
resolve a hostname, if no IP was set to it.
- `$SCYLLA_LISTEN_DEVICE` can now be set as environment variable and
contain any available NIC device name (e.g. `eth0`). The script
automatically retrieves the IP address from the device.
Usage:
1. With `$SCYLLA_LISTEN_ADDRESS` as IP:
`docker run -t -i --rm --name scylla -e SCYLLA_LISTEN_ADDRESS=192.168.1.100 scylladb/scylla`
2. With `$SCYLLA_LISTEN_ADDRESS` as hostname:
`docker run -t -i --rm --name scylla -e SCYLLA_LISTEN_ADDRESS=containername.network.lan scylladb/scylla`
3. With `$SCYLLA_LISTEN_DEVICE`:
`docker run -t -i --rm --name scylla -e SCYLLA_LISTEN_DEVICE=eth0 scylladb/scylla`
Message-Id: <20161003151230.67672-1-marius@twostairs.com>
Move scylla-server and scylla-jmx supervisord config files to separate
files and make the main supervisord.conf scan /etc/supervisord.conf.d/
directory. This makes it easier for people to extend the Docker image
and add their own services.
Message-Id: <1471588406-25444-1-git-send-email-penberg@scylladb.com>
allow user to use the `supervisorctl' program to start and stop
services. `exec` needed to be added to the scylla and scylla-jmx starter
scripts - otherwise supervisord loses track of the actual process we
want to manage.
Signed-off-by: Yoav Kleinberger <yoav@scylladb.com>
Message-Id: <1471442960-110914-1-git-send-email-yoav@scylladb.com>
Add '--smp', '--memory', and '--overprovisioned' options to the Docker
image. The options are written to /etc/scylla.d/docker.conf file, which
is picked up by the Scylla startup scripts.
You can now, for example, restrict your Docker container to 1 CPU and 1
GB of memory with:
$ docker run --name some-scylla penberg/scylla --smp 1 --memory 1G --overprovisioned 1
Needed by folks who want to run Scylla on Docker in production.
Cc: Sasha Levin <alexander.levin@verizon.com>
Message-Id: <1470680445-25731-1-git-send-email-penberg@scylladb.com>
We configure the hostname in the "CQLSH_HOST" environment variable but
that is only picked up if we first start the shell. Setup the hostname
in $HOME/.cqlshrc file instead so that we can start "cqlsh" directly:
docker exec -it scylla cqlsh
We don't have systemd running on the image so "journalctl" is useless.
Log to stdout instead which has the nice benefit of making "docker logs"
produce meaningful output on the host.
Switch to supervisord to manage the two processes we have: Scylla server
and Scylla JMX proxy. We need this to make the Docker image run under
Kubernetes, which now fails as follows as we try to start the systemd
init process:
Couldn't find an alternative telinit implementation to spawn.
I have not seen other people hitting the issue, except for GitLab Docker
image:
https://gitlab.com/gitlab-org/gitlab-ce/issues/18612
which "solved" the problem by not running init...
https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/838/diffs
Furthermore, the "supervisord" approach seems to be what people actually
use in Docker land:
http://blog.kunicki.org/blog/2016/02/12/multiple-entrypoints-in-docker/
The only downside is that we now sort of duplicate functionality that's
already in the systemd configuration files. However, we should work
towards Scylla figuring out its configuration rather than compose a long
list of command line arguments. Once we do that, the duplication in
Docker supervisord scripts disappears.
Previously, the Docker image could only be run interactively, which is
not conducive for running clusters. This patch makes the docker image
run in the background (using systemd). This makes the docker workflow
similar to working with virtual machines, i.e. the user launches a
container, and once it is running they can connect to it with
docker exec -it <container_name> bash
and immediately use `cqlsh` to control it.
In addition, the configuration of scylla is done using established
scripts, such as `scylla_dev_mode_setup`, `scylla_cpuset_setup` and
`scylla_io_setup`, whereas previously code from these scripts was
duplicated into the docker startup file.
To specify seeds for making a cluster, use the --seeds command line
argument, e.g.
docker run -d --privileged scylladb/scylla
docker run -d --privileged scylladb/scylla --seeds 172.17.0.2
other options include --developer-mode, --cpuset, --broadcast-address
The --developer-mode option mode is on by default - so that we don't fail users
who just want to play with this.
The Dockerfile entrypoint script was rewritten as a few Python modules.
The move to Python is meritted because:
* Using `sed` to manipulate YAML is fragile
* Lack of proper command line parsing resulted in introducing ad-hoc environment variables
* Shell scripts don't throw exceptions, and it's easy to forget to check exit codes for every single command
I've made an effort to make the entrypoint `go' script very simple and readable.
The goary details are hidden inside the other python modules.
Signed-off-by: Yoav Kleinberger <yoav@scylladb.com>
Message-Id: <1468938693-32168-1-git-send-email-yoav@scylladb.com>
Scylla will not start if the disk was not benchmarked
so start run io_tune with the right parameters.
Also add the cpu_set environment variables for passing
cpu set to iotune and scylla.
Signed-of-by: Benoît Canet <benoit@scylladb.com>
Message-Id: <1466412846-4760-2-git-send-email-benoit@scylladb.com>
This variable if set to true will activate
developer mode. It will be set by using the
-e option of docker run.
The xfs bind mount behavior and the cpuset behavior
will be set by using the relevant docker command
lines options and documented in the scylla/docker
howto.
Fixes: #1267
Signed-of-by: Benoît Canet <benoit@scylladb.com>
Message-Id: <1465213713-2537-1-git-send-email-benoit@scylladb.com>
Make the Docker image more user-friendly by starting up JMX proxy in the
background and install Scylla tools in the image. Also add a welcome
banner like we have with our AMI so that users have pointers to nodetool
and cqlsh, as well as our documentation.
Message-Id: <1460376059-3678-1-git-send-email-penberg@scylladb.com>
The default shell in Ubuntu is "dash" which causes the following error
when "scylla-start" script is executed:
/start-scylla: 8: /start-scylla: source: not found
Message-Id: <1459406561-20141-1-git-send-email-penberg@scylladb.com>
Switch to CentOS 7 as the Docker base image. It's more stable and
updated less frequently than Fedora. As a bonus, it's Thrift package
doesn't pull the world as a dependency which reduces image size from 700
MB to 380 MB.
Suggested by Avi.
Message-Id: <1451911969-26647-1-git-send-email-penberg@scylladb.com>
The Fedora base image has changed so we need to add "hostname" that's
used by the Docker-specific launch script to our image.
Fixes Scylla startup.
Signed-off-by: Pekka Enberg <penberg@scylladb.com>
Add a Dockerfile for building a ScyllaDB Docker image. The image is
based on Fedora 22 and ScyllaDB is installed from our RPM repository.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
