Commit Graph

38 Commits

Author SHA1 Message Date
Amnon Heiman
52db99f25f scyllatop/livedata.py: Safe iteration over metrics
This patch change the code that iterates over the metrics to use a copy
of the metrics names to make it safe to remove the metrics from the
metrics object.

Fixes #7488

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
2020-10-27 22:44:02 +02:00
Benny Halevy
c22ad90c04 scyllatop: livedata, metric: expire absent metrics
Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
2019-09-15 19:48:09 +03:00
Benny Halevy
6e807a56e1 scyllatop: livedata: update all metrics based on new discovered list
Update current results dictionary using the Metric.discover method.

New results are added and missing results are marked as absent.
(Both full metrics or specific keys)

Previously, with prometheous, each metric.update called query_list
resulting in O(n^2) when all metric were updated, like in the scylla_top
dtest - causing test timeout when testing debug build.

Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
2019-09-15 19:45:34 +03:00
Benny Halevy
16de4600a0 scyllatop: metric: return discover results as dict
So that we can easily search by symbol for updating
multiple results in a single pass.

Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
2019-09-15 16:07:19 +03:00
Benny Halevy
02707621d4 scyllatop: metric: update_info in discover
So that all metric information can be retrieved in a single pass.

Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
2019-09-15 16:07:19 +03:00
Benny Halevy
3861460d3b scyllatop: metric: refactor update method
Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
2019-09-15 16:07:19 +03:00
Benny Halevy
99ab60fc27 scyllatop: metric: add_to_results
In preparation to changing results to a dict
use a method to add a new metric to the results.

Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
2019-09-15 16:07:19 +03:00
Benny Halevy
b489556807 scyllatop: metric: refactor discover and discover_with_help
Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
2019-09-15 16:07:19 +03:00
Benny Halevy
8f7c721907 scyllatop: livedata: get rid of _setupUserSpecifiedMetrics
Add self._metricPatterns member and merge _setupUserSpecifiedMetrics
with _initializeMetrics.

Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
2019-09-15 16:07:19 +03:00
Benny Halevy
c17aee0dd3 scyllatop: add debug logging
Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
2019-09-15 16:07:19 +03:00
Avi Kivity
8e9989685d scyllatop: complete conversion to python3
d2dbbba139 converted scyllatop's interperter to Python 3, but neglected to do
the actual conversion. This patch does so, by running 2to3 over allfiles and adding
an additional bytes->string decode step in prometheus.py. Superfluous 2to3 changes
to print() calls were removed.
Message-Id: <20190117124121.7409-1-avi@scylladb.com>
2019-01-17 12:50:25 +00:00
Alexys Jacob
d2dbbba139 tools/scyllatop/: add / normalize python3 shebang 2018-11-28 23:57:03 +01:00
Amnon Heiman
6bcde841bd scyllatop: Nicer error message when fail opening a log file or connecting
scyllatop uses a log file, if opening the file fails, the user should
get a clear response not an exception trace.

The same is true for connecting to scylla

After this patch the following:
$ scyllatop.py -L /usr/lib/scyllatop.log
scyllatop failed opening log file: '/usr/lib/scyllatop.log' With an error: [Errno 13] Permission denied: '/usr/lib/scyllatop.log'

Fixes #3860

Signed-off-by: Amnon Heiman <amnon@scylladb.com>
Message-Id: <20181021065525.22749-1-amnon@scylladb.com>
2018-10-24 09:50:45 +03:00
Alexys Jacob
c80d7b97cc scyllatop: more coding style fixes
tools/scyllatop/metric.py:2:1: F401 're' imported but unused
tools/scyllatop/metric.py:53:20: E221 multiple spaces before operator
tools/scyllatop/metric.py:69:20: E221 multiple spaces before operator

Signed-off-by: Alexys Jacob <ultrabug@gentoo.org>
Message-Id: <20180917153308.7240-1-ultrabug@gentoo.org>
2018-09-17 18:39:53 +03:00
Alexys Jacob
46d101c1f2 scyllatop: coding style fixes
tools/scyllatop/prometheus.py:3:1: F401 'sys' imported but unused
tools/scyllatop/prometheus.py:7:1: E302 expected 2 blank lines, found 1
tools/scyllatop/prometheus.py:12:5: E301 expected 1 blank line, found 0
tools/scyllatop/prometheus.py:17:1: W293 blank line contains whitespace
tools/scyllatop/prometheus.py:22:82: E225 missing whitespace around operator

Signed-off-by: Alexys Jacob <ultrabug@gentoo.org>
Message-Id: <20180914110847.1862-1-ultrabug@gentoo.org>
2018-09-17 15:45:43 +03:00
Amnon Heiman
bc7503feee Scyllatop to use prometheus by default
Scylla now expose the prometheus API by default. This patch chagnes
scyllatop to use the Prometheus API, the collect API is still available.

The main changes in the patch:
* Move collectd specific logic inside collectd.
* Add support for help information.
* Add command line to configure prometheus end point and to enable
collectd.

* Add a prometheus class that collect information from prometheus.

Fixes: #1541
Message-Id: <20180531124156.26336-1-amnon@scylladb.com>
2018-05-31 18:00:22 +03:00
Alexys Jacob
bd3517efd8 scyllatop: PEP8 python coding style compliance
this patch fixes the following remarks:
./defaults.py:2:9: E126 continuation line over-indented for hanging indent
./fake.py:15:1: E305 expected 2 blank lines after class or function definition, found 1
./livedata.py:49:17: F402 import 'metric' from line 5 shadowed by loop variable
./scyllatop.py:44:1: E305 expected 2 blank lines after class or function definition, found 1

Signed-off-by: Alexys Jacob <ultrabug@gentoo.org>
Message-Id: <20180119162939.17866-1-ultrabug@gentoo.org>
2018-01-21 17:15:29 +02:00
Yoav Kleinberger
624165da79 scyllatop: dump all output to stdout instead of running a fancy console interface
Sometimes the user would like to dump all the metrics into a file or
pipe it to another program, as requested in issue #1506.
This patch makes scyllatop check if stdout is connected to a TTY,
and if not - it does not fire up the fancy urwid UI but instead, just
writes all it's collected metrics to stdout.

Optionally, the user tell the program to quit after a specific
number of iterations via the -n or --iterations flag

Signed-off-by: Yoav Kleinberger <yoav@scylladb.com>
Message-Id: <1471777516-9903-1-git-send-email-yoav@scylladb.com>
2016-08-31 08:31:36 +03:00
Glauber Costa
f7706d51d1 scyllatop: fix typo
Keyborad -> Keyboard

Signed-off-by: Glauber Costa <glauber@scylladb.com>
Message-Id: <349f20fd69be2f2e05ae0b7800e34a336cd2472b.1468248179.git.glauber@scylladb.com>
2016-07-11 18:27:49 +03:00
Yoav Kleinberger
0ad940bfc3 tools/scyllatop: fix crash due to mouse events
due to an urwid-library technicality, some mouse events like scroll or
click would crash scyllatop. This patch fixes this problem.

closes issue #1396.

Signed-off-by: Yoav Kleinberger <yoav@scylladb.com>
Message-Id: <1467294117-19218-1-git-send-email-yoav@scylladb.com>
2016-07-05 19:08:55 +03:00
Yoav Kleinberger
49cba035ea tools/scyllatop: leave terminal in a functioning state when user quits with CTRL-C
closes issue #1417.

Signed-off-by: Yoav Kleinberger <yoav@scylladb.com>
Message-Id: <1467556769-11851-1-git-send-email-yoav@scylladb.com>
2016-07-03 17:43:46 +03:00
Yoav Kleinberger
43071bf488 tools/scyllatop: handle absentee metrics
Sometimes a metric previously reported from collectd is not available
anymore. Previously, this caused scyllatop to log and exception to the
user - which in effect destroyes the user experience and inhibits
monitoring other metrics. This patch makes ScyllaTop handle this
problem. It will display such metrics and 'not available', and exclude
them from some and average computations.

Closes issue #1287.

Signed-off-by: Yoav Kleinberger <yoav@scylladb.com>
Message-Id: <1465301178-27544-1-git-send-email-yoav@scylladb.com>
2016-06-08 16:35:55 +03:00
Yoav Kleinberger
26c0d86401 tools/scyllatop: improved user interface: scrollable views
NOTE: scyllatop now requires the urwid library

previously, if there were more metrics that lines in the terminal
window, the user could not see some of the metrics.  Now the user can
scroll.

As an added bonus, the program will not crash when the window size
changes.

Signed-off-by: Yoav Kleinberger <yoav@scylladb.com>
Message-Id: <1464098832-5755-1-git-send-email-yoav@scylladb.com>
2016-05-26 13:36:28 +03:00
Yoav Kleinberger
de7952a8db tools/scyllatop: log input from collectd for easier debugging
When running with DEBUG verbosity, scyllatop will now log every single
value it receives from collectd. When you suspect that scyllatop is
somehow distorting values, this is a good way to check it.

Signed-off-by: Yoav Kleinberger <yoav@scylladb.com>
Message-Id: <1463320730-6631-1-git-send-email-yoav@scylladb.com>
2016-05-15 19:17:10 +03:00
Yoav Kleinberger
1543253bfd scyllatop: differentiate metrics coming from different hosts
Fix issue #1173.
Previously scyllatop aggregated metrics coming from a cluster with many
hosts so that individual contributions could not be recognized. This is
now changed so that aggregation is also by hostname.

Signed-off-by: Yoav Kleinberger <yoav@scylladb.com>
Message-Id: <8a4d8b82216d8c1aa855026ff31bcfd8bfac7e47.1461150261.git.yoav@scylladb.com>
2016-04-20 20:20:09 +02:00
Yoav Kleinberger
91269d0c15 tools/scyllatop: add sums to aggregate view
the aggregate view now supports both sums and means.

Signed-off-by: Yoav Kleinberger <yoav@scylladb.com>
Message-Id: <1328af8efb113a786d7402b0704220108bfb28db.1458749600.git.yoav@scylladb.com>
2016-03-23 18:49:57 +02:00
Yoav Kleinberger
d2cfb86dc8 tools/scyllatop: defend against unexpected strings from collectd
Signed-off-by: Yoav Kleinberger <yoav@scylladb.com>
Message-Id: <cd7ecf6b3b82bd2027179cbec4e689a946469e9a.1458740337.git.yoav@scylladb.com>
2016-03-23 16:05:59 +02:00
Yoav Kleinberger
97bb7a35d9 tools/scyllatop: some sensible default metrics
Previosly if the user did not specify any metrics, scyllatop use
whatever it could find. Now we have some preset defaults which are
probably more interesting.

Signed-off-by: Yoav Kleinberger <yoav@scylladb.com>
Message-Id: <1458658804-377-1-git-send-email-yoav@scylladb.com>
2016-03-22 17:04:13 +02:00
Yoav Kleinberger
1cd01cd2ab tools/scyllatop: defend against curses "out of screen bounds" error
Fixes issue #945 (hopefully)
This issue was probably the result of trying to write outside the
confines of the window. The views.Base class now defends against this.

Signed-off-by: Yoav Kleinberger <yoav@scylladb.com>
Message-Id: <9735806b211567f3239e187d87437c484f532291.1457265435.git.yoav@scylladb.com>
2016-03-07 18:02:26 +01:00
Yoav Kleinberger
651aa06c32 tools/scyllatop: fix mistake in help message
Signed-off-by: Yoav Kleinberger <yoav@scylladb.com>
Message-Id: <ae5f11d7df954dc7561db94cda2d73bd8233f1e5.1456510513.git.yoav@scylladb.com>
2016-02-28 12:42:12 +02:00
Yoav Kleinberger
872079d999 tools/scyllatop: correct mistake in help text
Signed-off-by: Yoav Kleinberger <yoav@scylladb.com>
Message-Id: <01844d90f2d942a051d128b03ae12578ac0bb69c.1456324697.git.yoav@scylladb.com>
2016-02-25 12:49:48 +02:00
Takuya ASADA
28dd202613 scyllatop: add --logfile argument to specify path to log file
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1456333116-7389-2-git-send-email-syuu@scylladb.com>
2016-02-25 10:33:41 +02:00
Takuya ASADA
af3a8ead21 scyllatop: output error message both on log file and stdout
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1456333116-7389-1-git-send-email-syuu@scylladb.com>
2016-02-25 10:33:40 +02:00
Yoav Kleinberger
c3ce9e53cb tools/scyllatop: support glob patterns to specifiy metrics
Signed-off-by: Yoav Kleinberger <yoav@scylladb.com>
Message-Id: <42f84cdeeb75c3719230028a13a1dd8499673d4c.1456319441.git.yoav@scylladb.com>
2016-02-24 15:35:45 +02:00
Yoav Kleinberger
f822359d96 bugfix: fixed broken --print-config option
Signed-off-by: Yoav Kleinberger <yoav@scylladb.com>
Message-Id: <57b452106cdcd9ceb09da4c63781650cefe48040.1456234464.git.yoav@scylladb.com>
2016-02-23 15:35:44 +02:00
Pekka Enberg
4a4074ad21 tools/scyllatop: Sort metrics by name
This makes the output much easier to read, especially if you have tons
of metrics specified.

Message-Id: <1456230377-3149-1-git-send-email-penberg@scylladb.com>
2016-02-23 14:35:57 +02:00
Pekka Enberg
1f6cac8839 tools/scyllatop: Use 'erase' to clear the screen
The 'clear' function explicitly clears the screen and repaints it which
causes really annoying flicker. Use 'erase' to make scyllatop more
pleasant on the eyes.

Message-Id: <1456229348-2194-1-git-send-email-penberg@scylladb.com>
2016-02-23 14:12:48 +02:00
Yoav Kleinberger
74fbc62129 ScyllaTop: top-like tool to see live scylla metrics
requires a local collectd configured with the unix-sock plugin,
use the --help option for more. Run it with:

        $ scyllatop.py --help

Signed-off-by: Yoav Kleinberger <yoav@scylladb.com>
Message-Id: <bd3f8c7e120996fc464f41f60130c82e3fb55ac6.1456164703.git.yoav@scylladb.com>
2016-02-23 12:32:47 +02:00