We can use iotunev2 (or any other I/O generator) to test for the limits
of the disks for the i2 and i3 instance classes. The values I got here
are the values I got from ~5 invocations of the (yet to be upstreamed)
iotune v2, with the IOPS numbers rounded for convenience of reading.
During the execution, I verified that the disks were saturated so we
can trust these numbers even if iotunev2 is merged in a different form.
The numbers are very consistent, unlike what we usually saw with the
first version of iotune.
Previously, we were just multiplying the concurrency number by the
number of disks. Now that we have better infrastructure, we will
manually test i3.large and i3.xlarge, since their disks are smaller
and slower.
For the other i3, and all instances in the i2 family storage scales up
by adding more disks. So we can keep multiplying the characteristics of
one known disk by the number of disks and assuming perfect scaling.
Example for i3, obtained with i3.2xlarge:
read_iops = 411k
read_bandwidth = 1.9GB/s
So for i3.16xlarge, we would have read_iops = 3.28M and 15GB/s - very
close to the numbers advertised by AWS.
Signed-off-by: Glauber Costa <glauber@scylladb.com>
m3 has 80GB SSDs in its largest form and I doubt anybody has ever
used it with Scylla.
I am also not aware of any c3 deployments. Since it is past generation,
it doesn't even show up in the default instance selector anymore.
I propose we drop AMI support for it. In practice, what that means is
that we won't auto-tune its I/O properties and people that want to use
it will have to run scylla_io_setup - like they do today with the EBS
instances.
Signed-off-by: Glauber Costa <glauber@scylladb.com>
We are not configuring the disks the way we want them with respect to
scheduler and nomerges. This is an oversigh that became clear now that
I started rewriting iotune-- since I will explicitly test for that. But
since this can affect the results, it should be here all along.
Signed-off-by: Glauber Costa <glauber@scylladb.com>
This patch makes the functions in scylla-blocktune available as a
library for other scripts - namely scylla_io_setup.
The filename, scylla-blocktune, is not the most convenient thing to call
from python so instead of just wrapping it in the usual test for
__main__ I am just splitting the file into two.
Another option would be to patch all callers to call
scylla_blocktune.py, but because we are usually not using extensions in
scripts that are meant to be called directly I decided for the split.
Signed-off-by: Glauber Costa <glauber@scylladb.com>
scylla-blocktune currently lives in the top level but this is mostly
historical. When time comes for us to install it, the packaging systems
will copy it to /usr/lib/scylla with the others.
So for consistency let's make sure that it also lives in the scripts
directory.
Signed-off-by: Glauber Costa <glauber@scylladb.com>
Since we unconditionally running blkdiscard on disks, we may get ioctl error
message on some disks which does not support TRIM.
This can be ignore but it's bad UX, so let's skip running blkdiscard when TRIM
is not supported on the disk.
Fixes#2774
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1517992904-13838-1-git-send-email-syuu@scylladb.com>
Currently we don't check data_file_directories existance before running iotune,
therefore it's shows unclear error message.
To make the message better, check the directory existance on scylla_io_setup.
Fixes#3137
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1517200647-6347-1-git-send-email-syuu@scylladb.com>
- Removed text from Report's "PURPOSE" section, which was referring to the "MANUAL CHECK LIST" (not needed anymore).
- Removed curl command (no longer using the api_address), instead using scylla --version
- Added -v flag in iptables command, for more verbosity
- Added support to for OEL (Oracle Enterprise Linux) - minor fix
- Some text changes - minor
- OEL support indentation fix + collecting all files under /etc/scylla
- Added line seperation under cp output message
Signed-off-by: Tomer Sandler <tomer@scylladb.com>
Message-Id: <20170828131429.4212-1-tomer@scylladb.com>
Choose the networking configuration mode according to the current contents of /etc/scylla.d/cpuset.conf.
If it doesn't exist - use the default mode.
If it exists - use the mode that has been used for generation of the CPU set.
Store the configuration into the /etc/scylla.d/perftune.yaml
Signed-off-by: Vlad Zolotarov <vladz@scylladb.com>
scylla_setup resets our configuration and perftune.yaml is a part of it.
perftune.yaml is generated based on the contents of cpuset.conf therefore we should reset
these together.
Signed-off-by: Vlad Zolotarov <vladz@scylladb.com>
The scylla_cpuset_setup uses a verify_args() function that is defined in the scylla_lib.sh.
Fixes#2716
Signed-off-by: Vlad Zolotarov <vladz@scylladb.com>
These two admin related packages will be packaged under the "app-admin"
category and not the "dev-db" one.
This fixes the detection path of the packages for scylla_setup.
Signed-off-by: Alexys Jacob <ultrabug@gentoo.org>
Message-Id: <20170817094756.21550-1-ultrabug@gentoo.org>
When user mistakenly forgot to pass parameter for a flag, our scripts misparses
next flag as the parameter.
ex) Correct usage is '--ntp-domain <domain> --setup-nic', but passed
'--ntp-domain --setup-nic'.
Result of that, next flag will ignore by scripts.
To prevent such behavior, reject any parameter that start with '--'.
Fixes#2609
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20170815114751.6223-1-syuu@scylladb.com>
We experienced 'Constructing RAID volume...' takes too much time on some AMIs,
this is because setup script stuck at 'yum -y install mdadm xfsprogs'.
We don't have to install these packages on AMI startup time, we should
preinstall them on AMI creating time.
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1502192796-21040-1-git-send-email-syuu@scylladb.com>
We have problem to run fstrim with nomerges=2, so we need to change
the parameter to 1 during fstrim execution.
To do this, this fix changes follow things:
- revert dropping scylla_fstrim on Ubuntu 16.04/CentOS
- disable distribution provided fstrim script
- enable scylla_fstrim on all distributions
- introduce --set-nomerges on scylla-blocktune
- scylla_fstrim call scylla-blocktune by following order:
- 'scylla-blocktune --set-nomerges 1'
- 'fstrim' for each devices
- 'scylla-blocktune --set-nomerges 2'
Fixes#2649
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1501531393-21109-1-git-send-email-syuu@scylladb.com>
A user reported scylla-server.service does not able to run on their cloud instance, because of hugeadm.
(hugeadm says the kernel does not support huge pages.)
We don't need it for posix mode, so move it in dpdk mode.
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1500367219-8728-1-git-send-email-syuu@scylladb.com>
Configuring cpufreq service on VMs/IaaS causes an error because it doesn't supported cpufreq.
To prevent causing error, skip whole configuration when the driver not loaded.
Fixes#2051
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1498809504-27029-1-git-send-email-syuu@scylladb.com>
Current script has a bug that overwrites symlink on
/etc/sysconfig/selinux by real file, the script not able to disable
SELinux because of it.
So keep symlink after modifying the file.
Fixes#2279
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1493663263-10573-1-git-send-email-syuu@scylladb.com>
This is a folded version of the following rework on the node health
check script:
- Added support for non-default cql + nodetool ports
- Script will not exit if either Scylla-server / Scylla-jmx / Both
services are not up and running. It will alert the user about it and
which output cannot be collected, but continue collecting everything
else.
- Removed lshw installation and non-needed use in commands
- Script supports RHEL/CentOS/Ubuntu14/Ubuntu16/Debian (tested on all
beside Debian, should behave the same as Ubuntu14/16)
- All Indentation issues fixed -> using only tab (no spaces)
consistently.
- >> vs. > was fixed as well in the needed places.
- Changes the ${VAR_NAME} instances to $VAR_NAME, and kept the {} only
where needed.
- Check Scylla service as Vlad recommended using 'ps -C'
- Fixed the CQL not listening error message.
- Added Sanity check if script is attempted to run on non-Fedora and
non-Debian OS -> alert the user and exit.
- Removed the MANUAL CHECK LIST section (moved to Google Forms)
- Added date in head of the report.
- Removed text from Report's "PURPOSE" section, which was referring to
the "MANUAL CHECK LIST" (not needed anymore).
[ penberg: Fold into a single commit and add proper license. ]
Acked-by: Vlad Zolotarov <vladz@scylladb.com>
Message-Id: <1493900076-29170-1-git-send-email-penberg@scylladb.com>
During the changes in the way the housekeeping check for newer version
and warn about it in the installation the UUID part was removed but kept
in the sarounding if.
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <20170426075724.7132-1-amnon@scylladb.com>
