Commit Graph

118 Commits

Author SHA1 Message Date
Glauber Costa
54bc0ce70d scylla_setup: make sure it works (again) in interactive mode
Commit 019a2e3a27 marked some arguments as required, which improved
the usability of scylla_setup.

The problem is that when we call scylla_setup in interactive mode,
no argument should be required. After the aforementioned commit
scylla_setup will either complain that the required arguments were
not passed if zero arguments are present, or skip interactive mode
if one of the mandatory ones is present.

This patch fixes that by checking whether or not we were invoked with
no command line arguments and lifting the requirements for mandatory
arguments in that case.

Signed-off-by: Glauber Costa <glauber@scylladb.com>
Message-Id: <20190122003621.11156-1-glauber@scylladb.com>
2019-01-22 16:54:55 +02:00
Glauber Costa
ca997b5f60 scylla_setup: warn users on the severity of answering no to IOTUne
The system won't work properly if IOTune is not run. While it is fair
to skip this step because it takes long-- indeed, it is common to provision
io.conf manually to be able to skip this step, first time users don't know
this and can have the impression that this is just a totally optional step.

Except the node won't boot up without it.

As a user nicely put recently in our mailing list:

"...in this case, it would be even simpler to forbid answering "no"
 to this not-so-optional step :)"

We should not forbid saying no to IOTune, but we should warn the user
about the consequences of doing so.

Fixes #4120

Signed-off-by: Glauber Costa <glauber@scylladb.com>
Message-Id: <20190121144506.17121-1-glauber@scylladb.com>
2019-01-21 16:55:50 +02:00
Amos Kong
09a3b11c2f scylla_setup: only ask for nic in interactive mode
Current scylla_setup still asks for nic even nic is already assigned in cmdline.

Fixes #3908

Signed-off-by: Amos Kong <amos@scylladb.com>
Message-Id: <6b867e17a5583c495c771a37d5fa1e8366b1d61b.1542337635.git.amos@scylladb.com>
2018-12-09 15:29:31 +02:00
Avi Kivity
d4f353d3c8 Merge "normalized python3 compatibility, shebang and encoding" from Alexys
"
This series of patches ensures that all the Python code base is python3 compliant
and consistent by applying the following logic:

- python3 classifier on setup.py to explicitly state our python compatibility matrix
- add UTF-8 encoding header
- correct every shebang to the same /usr/bin/env python3
- shebang is only added on scripts meant to be executed on their own (removed otherwise)
- migrate some leftover scripts from python2 to python3 with minimal QA

This work is important to prepare for a more drastic change on Python code styling
using the black formatter and the setting up of automated QA checks on Python code base.
"

* 'python3_everywhere' of https://github.com/numberly/scylla:
  scylla-housekeeping: fix python3 compat and shebang
  dist/ami/files/scylla_install_ami: python3 shebang
  dist/docker/redhat/docker-entrypoint.py: add encoding comment
  fix_system_distributed_tables.py: fix python3 compat and shebang
  gen_segmented_compress_params.py: add encoding comment
  idl-compiler.py: python3 shebang
  scylla-gdb.py: python3 shebang
  configure.py: python3 shebang
  tools/scyllatop/: add / normalize python3 shebang
  scripts/: add / normalize python3 shebang
  dist/common/scripts: add / normalize python3 shebang
  test.py: add encoding comment
  setup.py: add python3 classifiers
2018-12-06 12:16:57 +02:00
Alexys Jacob
02656fb00e dist/common/scripts: add / normalize python3 shebang 2018-11-28 23:55:26 +01:00
Dan Yasny
019a2e3a27 scylla_setup: Mark required args
Fixes #3945

Message-Id: <20181128220549.3083-1-dyasny@gmail.com>
2018-11-28 22:30:02 +00:00
Vlad Zolotarov
9a5373254a dist: scylla_setup and scylla_sysconfig_setup: change paremeters names to reflect new semantics
Change the name of the corresponding parameter (--setup-nic) to reflect
the fact that we tune not just NIC now but rather NIC and disks together.

Signed-off-by: Vlad Zolotarov <vladz@scylladb.com>
2018-10-31 15:27:00 -04:00
Vlad Zolotarov
0e47d8bb1d dist: use perftune.py to tune disks and NIC
Tune disks using perftune.py together with NIC.
This is needed because disk(s) and NIC tuning has to be
performed using the mode (for non-NVMe disks).

We tune disks based on the current content of /etc/scylla/scylla.yaml.

Don't use scylla-blocktune for optimizing disks' performance
any more.

Unite the decision to optimize the NIC and disks tuning.
Optimize or not optimize them both together.

Disable disk tuning for DPDK and "virtio" modes for now.

Signed-off-by: Vlad Zolotarov <vladz@scylladb.com>
2018-10-31 15:27:00 -04:00
Takuya ASADA
f3c3b9183c dist/debian: use relocatable package to produce .deb
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
2018-10-24 11:29:47 +00:00
Alexys Jacob
eebbae066a dist/common/scripts/scylla_setup: fix gentoo linux installed package detection
return code is expected to be 0 when installed package was found

Signed-off-by: Alexys Jacob <ultrabug@gentoo.org>
Message-Id: <20181002123433.4702-1-ultrabug@gentoo.org>
2018-10-07 16:46:02 +03:00
Takuya ASADA
2ef1b094d7 dist/common/scripts/scylla_setup: don't proceed RAID setup until user type 'done'
Need to wait user confirmation before running RAID setup.

See #3659
Fixes #3681

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20180810194507.1115-1-syuu@scylladb.com>
2018-08-11 18:48:05 +03:00
Takuya ASADA
b7cf3d7472 dist/common/scripts/scylla_setup: don't mention about interactive mode prompt when running on non-interactive mode
Skip showing message when it's non-interactive mode.

Fixes #3674

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20180810191945.32693-1-syuu@scylladb.com>
2018-08-11 18:48:03 +03:00
Takuya ASADA
ef9475dd3c dist/common/scripts/scylla_setup: check existance of housekeeping.cfg before asking to run version check
Skip asking to run version check when housekeeping.cfg is already
exists.
Fixes #3657

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20180807232313.15525-1-syuu@scylladb.com>
2018-08-11 18:48:02 +03:00
Takuya ASADA
ad7bc313f7 dist/common/scripts: pass format variables to colorprint()
When we use str.format() to pass variables on the message it will always
causes Exception like "KeyError: 'red'", since the message contains color
variables but it's not passed to str.format().
To avoid the error we need to pass all format variables to colorprint()
and run str.format() inside the function.

Fixes #3649

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20180803015216.14328-1-syuu@scylladb.com>
2018-08-08 18:37:50 +03:00
Takuya ASADA
652eb5ae0e dist/common/scripts/scylla_setup: fix typo on interactive setup
Scylls -> Scylla

Fixes #3656

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20180808002443.1374-1-syuu@scylladb.com>
2018-08-08 09:15:13 +03:00
Takuya ASADA
a300926495 dist/common/scripts/scylla_setup: use specified NIC ifname correctly
Interactive NIC selection prompt always returns 'eth0' as selected NIC name
mistakenly, need to fix.

Fixes #3651

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20180803020724.15155-1-syuu@scylladb.com>
2018-08-06 20:59:19 +03:00
Amos Kong
0fcdab8538 scylla_setup: nic setup dialog is only for interactive mode
Current code raises dialog even for non-interactive mode when we pass options
in executing scylla_setup. This blocked automatical artifact-test.

Fixes #3549

Signed-off-by: Amos Kong <amos@scylladb.com>
Message-Id: <58f90e1e2837f31d9333d7e9fb68ce05208323da.1531824972.git.amos@scylladb.com>
2018-07-17 16:31:18 +03:00
Takuya ASADA
ee61660b76 dist/common/scripts/scylla_ec2_check: support custom NIC ifname on EC2
Since some AMIs using consistent network device naming, primary NIC
ifname is not 'eth0'.
But we hardcoded NIC name as 'eth0' on scylla_ec2_check, we need to add
--nic option to specify custom NIC ifname.

Fixes #3584

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20180712142446.15909-1-syuu@scylladb.com>
2018-07-12 18:22:28 +03:00
Takuya ASADA
99d5ca03e7 dist/common/scripts: drop class concolor, use colorprint()
To print colored console output with simplar code, drop class concolor
and use colorprint() instead.
2018-07-08 01:13:36 +09:00
Takuya ASADA
d0f39ea31d dist/common/scripts/scylla_raid_setup: verify specified disks are unused
Currently only scylla_setup interactive mode verifies selected disks are
unused, on non-interactive mode we get mdadm/mkfs.xfs program error and
python backtrace when disks are busy.

So we should verify disks are unused also on scylla_raid_setup, print
out simpler error message.
2018-07-03 14:50:34 +09:00
Takuya ASADA
e0c16c4585 dist/common/scripts/scylla_setup: don't accept disk path if it's not block device
Need to ignore input when specified path is not block device.
2018-07-03 14:50:34 +09:00
Takuya ASADA
972ce88601 dist/common/scripts/scylla_setup: allow input multiple disk paths on RAID disk prompt
Allow "/dev/sda1,/dev/sdb1" style input on RAID disk prompt.
2018-06-29 01:37:19 +09:00
Takuya ASADA
f2132c61bd dist/common/scripts/scylla_setup: check disk is used in MDRAID
Check disk is used in MDRAID by /proc/mdstat.
2018-06-29 01:37:19 +09:00
Takuya ASADA
daccc10a06 dist/common/scripts/scylla_setup: move unmasking scylla-fstrim.timer on scylla_fstrim_setup
Currently, enabling scylla-fstrim.timer is part of 'enable-service', it
will be enabled even --no-fstrim-setup specified (or input 'No' on interactive setup prompt).

To apply --no-fstrim-setup we need to enabling scylla-fstrim.timer in
scylla_fstrim_setup instead of enable-service part of scylla_setup.

Fixes #3248
2018-06-29 01:37:19 +09:00
Takuya ASADA
fa6db21fea dist/common/scripts/scylla_setup: use print() instead of logging.error()
Align with other script scripts, use print().
2018-06-29 01:37:19 +09:00
Takuya ASADA
2401115e14 dist/common/scripts/scylla_setup: implement do_verify_package() for Gentoo Linux
Implement Gentoo Linux support on scylla_setup.
2018-06-29 01:37:19 +09:00
Takuya ASADA
5b4da4d4bd dist/common/scripts/scylla_setup: don't include the disk on unused list when it contains partitions
On current implementation, we are checking the partition is mounted, but
a disk contains the partition marked as unused.
To avoid the problem, we should skip a disk which contains partitions.

Fixes #3545
2018-06-29 01:37:19 +09:00
Takuya ASADA
83bc72b0ab dist/common/scripts/scylla_setup: skip running rest of the check when the disk detected as used
Don't need to run check when we already detected the disk as used.
2018-06-29 01:37:19 +09:00
Takuya ASADA
1650d37dae dist/common/scripts/scylla_setup: add a disk to selected list correctly
When a disk path typed on the RAID setup prompt, the script mistakenly
splits the input for each character,
like ['/', 'd', 'e', 'v', '/', 's', 'd', 'b'].

To fix the issue we need to use selected.append() instead of
selected +=.

See #3545
2018-06-29 01:37:19 +09:00
Takuya ASADA
4b5826ff5a dist/common/scripts/scylla_setup: fix wrong indent
list_block_devices() should return 'devices' on both re.match() is
matched and unmatched.
2018-06-29 01:37:19 +09:00
Takuya ASADA
d3a3d0f8de dist/common/scripts: convert scylla_setup to python3
Convert bash script to python3.
2018-06-22 12:30:37 +09:00
Takuya ASADA
9971576ecb dist: drop collectd support from package
Since scyllatop no longer needs collectd, now we are able to drop collectd.

resolves #3490

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1528961612-8528-1-git-send-email-syuu@scylladb.com>
2018-06-14 10:40:23 +03:00
Amos Kong
364c2551c8 scylla_setup: fix conditional statement of silent mode
Commit 300af65555 introdued a problem in
conditional statement, script will always abort in silent mode, it doesn't
care about the return value.

Fixes #3485

Signed-off-by: Amos Kong <amos@scylladb.com>
Message-Id: <1c12ab04651352964a176368f8ee28f19ae43c68.1528077114.git.amos@scylladb.com>
2018-06-04 10:14:06 +03:00
Laura Novich
e053da6f51 scylla_setup: adjust language
Edited the text for the scylla setup, improving readability for the prompts
with regards to grammar and usage.

Signed-off by: Laura Novich <laura@scylladb.com>
Message-Id: <CAGcEH3Xa6TFy=_rdz_=NP0b23vEDZmfRQzAdxV-f04C1p+AzTw@mail.gmail.com>
2018-05-29 09:56:41 +03:00
Takuya ASADA
300af65555 dist/common/scripts/scylla_setup: abort running script when one of setup failed in silent mode
Current script silently continues even one of setup fails, need to
abort.

Fixes #3433

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20180522180355.1648-1-syuu@scylladb.com>
2018-05-23 11:05:33 +03:00
Amnon Heiman
bc356a3c15 scylla_setup support private repo on debian during setup
Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <20170917145248.19677-1-amnon@scylladb.com>
2017-12-11 10:36:30 +02:00
Amos Kong
76ab8bf292 scylla_setup: parse --no-ec2-check option
The option was introduced by commit e645b0f ("dist/common/scripts: move
EC2 configuration verification to 'scylla_ec2_check'"), but it doesn't
parsed the option at all.

Fixes #2934

Signed-off-by: Amos Kong <amos@scylladb.com>
Acked-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <df21356e528c6e161f73f4408a201fedef8f52d9.1509744954.git.amos@scylladb.com>
2017-11-06 12:06:54 +02:00
Alexys Jacob
e5ff8efea3 dist: Fix Gentoo Linux scylla-jmx and scylla-tools packages detection
These two admin related packages will be packaged under the "app-admin"
category and not the "dev-db" one.

This fixes the detection path of the packages for scylla_setup.

Signed-off-by: Alexys Jacob <ultrabug@gentoo.org>
Message-Id: <20170817094756.21550-1-ultrabug@gentoo.org>
2017-08-17 13:20:43 +03:00
Takuya ASADA
0f9b095867 dist/common/scripts: prevent ignoreing flag that passed after another flag which requires parameter
When user mistakenly forgot to pass parameter for a flag, our scripts misparses
next flag as the parameter.
ex) Correct usage is '--ntp-domain <domain> --setup-nic', but passed
    '--ntp-domain --setup-nic'.
Result of that, next flag will ignore by scripts.
To prevent such behavior, reject any parameter that start with '--'.

Fixes #2609

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20170815114751.6223-1-syuu@scylladb.com>
2017-08-15 18:27:32 +03:00
Takuya ASADA
6ac254fbcb dist: change nomerges=1 on block devices during fstrim execution
We have problem to run fstrim with nomerges=2, so we need to change
the parameter to 1 during fstrim execution.
To do this, this fix changes follow things:
 - revert dropping scylla_fstrim on Ubuntu 16.04/CentOS
 - disable distribution provided fstrim script
 - enable scylla_fstrim on all distributions
 - introduce --set-nomerges on scylla-blocktune
 - scylla_fstrim call scylla-blocktune by following order:
   - 'scylla-blocktune --set-nomerges 1'
   - 'fstrim' for each devices
   - 'scylla-blocktune --set-nomerges 2'

Fixes #2649

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1501531393-21109-1-git-send-email-syuu@scylladb.com>
2017-08-03 13:00:34 +03:00
Takuya ASADA
e645b0fb13 dist/common/scripts: move EC2 configuration verification to 'scylla_ec2_check'
Currently we only have EC2 configuration verification on AMI, so move it to
/usr/lib/scylla and run it from scylla_setup, to make it usable for
non-AMI users.

Fixes #1997

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1498811107-29135-1-git-send-email-syuu@scylladb.com>
2017-07-03 09:59:28 +03:00
Amos Kong
f655639e5a scylla_setup: fix deadloop in inputting invalid option
example: # scylla_setup --invalid-opt

Fixes #2305

Signed-off-by: Amos Kong <amos@scylladb.com>
Message-Id: <9a4f631b126d8eaaae479fa99137db7a61a7c869.1493135357.git.amos@scylladb.com>
2017-05-28 13:02:10 +03:00
Takuya ASADA
bdec38d23c dist/common/scripts/scylla_setup: skip SELinux setup when it's already disabled
It doesn't make sence to ask "Do you want to disable SELinux?" when SELinux is
already disabled, so skip whole question.

Fixes #2411

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1495652423-20806-1-git-send-email-syuu@scylladb.com>
2017-05-28 13:00:10 +03:00
Alexys Jacob
c0756d97b8 scylla_setup: fix typos on cpu scaling messages
This fixes typos on CPU scaling related messages.

Signed-off-by: Alexys Jacob <ultrabug@gentoo.org>
Message-Id: <20170519143703.3574-1-ultrabug@gentoo.org>
2017-05-21 13:41:42 +03:00
Alexys Jacob
fa0944ac19 scylla_setup: add gentoo linux installation detection 2017-05-17 18:06:54 +02:00
Amnon Heiman
b59c95359d scylla_setup: Fix conditional when checking for newer version
During the changes in the way the housekeeping check for newer version
and warn about it in the installation the UUID part was removed but kept
in the sarounding if.

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <20170426075724.7132-1-amnon@scylladb.com>
2017-05-01 12:13:35 +03:00
Takuya ASADA
4262edd843 dist: use distribution standard fstrim script instead of our custom one
We recently introduced fstrim cronjob / systemd timer unit, but some of
distributions already has their own fstrim cronjob / systemd timer unit.
So let's use them when it's posssible.

Fixes #2233

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1491226727-10507-1-git-send-email-syuu@scylladb.com>
2017-04-04 16:44:45 +03:00
Takuya ASADA
72696eff22 dist/common/scripts/scylla_setup: add 'cancel' feature on RAID disk selector prompt
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1491217320-10211-2-git-send-email-syuu@scylladb.com>
2017-04-04 15:35:23 +03:00
Takuya ASADA
4eee3dd778 dist/common/scripts/scylla_setup: skip list DVD drive on block device list for RAID
Fixes #2230

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1491217320-10211-1-git-send-email-syuu@scylladb.com>
2017-04-04 15:35:22 +03:00
Takuya ASADA
e7697c37b2 dist/common/scripts/scylla_setup: warn twice before constructing RAID volume
Since RAID construction has possibility to destroy user data, warn twice before
executing scylla_raid_setup.

Fixes #1346

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1487891797-13109-1-git-send-email-syuu@scylladb.com>
2017-03-26 10:36:23 +03:00