Commit Graph

17 Commits

Author SHA1 Message Date
Avi Kivity
c85671ce51 scripts: refresh-submodules: don't omit last (first) commit
`git log --format` doesn't add a newline after the last line. This
causes `read` to ignore that line, losing the last line (corresponding
to the first commit).

Use `git log --tformat` instead, which terminates the last line.

Closes scylladb/scylladb#27317
2025-11-27 18:46:27 +02:00
Pavel Emelyanov
339f08b24a scripts: Enhance refresh_submodules.sh with nested summary
Currently when refreshing submodule, the script puts a plain list of
non-merge commits into commit message. The resulting summary contains
everything, but is hard to understand. E.g. if updating seastar today
the summary would start with

    * seastar 26badcb1...86c4893b (55):
      > util: make SEASTAR_ASSERT() failure generate SIGABRT
      > core: fix high CPU use at idle on high core count machines
      > http::reply: Add 308 (permanent redirect) and make pretty-print handle unknown values
      > reactor: Relax friendship with file_data_source_impl
      > fstream: Use direct io_stats reference
      > thread_pool: Relax coupling with reactor
      > reactor: Mark some IO classes management methods private
      > http: Deprecate json_exception
      > fair_queue: Move io_throttler to io_queue.hh
      > fair_queue: Move metrics from to io_queue::stream
      > fair_queue: Remove io_throttler from tests
      > fair_queue_test: Remove io-throttler from fair-queue
      > fair_queue: Remove capacity getters
      > fair_queue: Move grab_result into io_queue::stream too
      > fair_queue: Move throtting code to io_queue.cc
      > fair_queue: Move throttling code to io_queue::stream class
      > fair_queue: Open-code dispatch_requests() into users
      > fair_queue: Split dispatch_requests() into top() and pop_front()
      > fair_queue: Swap class push back and dispatch
      > fair_queue: Configure forgiving factor externally
      ...

That's not very informative, because the update includes several large
"merges" that have their summary which is missing here. This update
changes the way summary is generated to include merges and their
summaries and all merged commits are listed as sub-lines, like this

    * seastar 26badcb1...86c4893b (26):
      > util: make SEASTAR_ASSERT() failure generate SIGABRT
      > core: fix high CPU use at idle on high core count machines
      > Merge 'Move output IO throttler to IO queue level' from Pavel Emelyanov
        fair_queue: Move io_throttler to io_queue.hh
        fair_queue: Move metrics from to io_queue::stream
        fair_queue: Remove io_throttler from tests
        fair_queue_test: Remove io-throttler from fair-queue
        fair_queue: Remove capacity getters
        fair_queue: Move grab_result into io_queue::stream too
        fair_queue: Move throtting code to io_queue.cc
        fair_queue: Move throttling code to io_queue::stream class
        fair_queue: Open-code dispatch_requests() into users
        fair_queue: Split dispatch_requests() into top() and pop_front()
        fair_queue: Swap class push back and dispatch
        fair_queue: Configure forgiving factor externally
        fair_queue: Move replenisher kick to dispatch caller
        io_queue: Introduce io_queue::stream
        fair_queue: Merge two grab_capacity overloads
        fair_queue: Detatch outcoming capacity grabbing from main dispatch loop
        fair_queue: Move available tokens update into if branch
        io_queue: Rename make_fair_group_config into configure_throttler
        io_queue: Rename get_fair_group into get_throttler
        fair_queue: Rename fair_group -> io_throttler
      > http::reply: Add 308 (permanent redirect) and make pretty-print handle unknown values
      > Merge 'Relax reactor coupling with file_data_source_impl' from Pavel Emelyanov
        reactor: Relax friendship with file_data_source_impl
        fstream: Use direct io_stats reference
      > thread_pool: Relax coupling with reactor
      > reactor: Mark some IO classes management methods private
      ...

Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>

Closes scylladb/scylladb#24834
2025-07-21 14:48:30 +03:00
Avi Kivity
5e1cf90a51 build: replace tools/java submodule with packaged cassandra-stress
We no longer use tools/java (scylladb/scylla-tools-java.git) for
nodetool or cqlsh; only cassandra-stress. Since that is available
in package form install that and excise the tools/java submodule
from the source tree.

pgo/ is adjusted to use the packaged cassandra-stress (and the cqlsh
submodule).

A few jmx references are dropped as well.

Frozen toolchain regenerated.

Optimized clang from

  https://devpkg.scylladb.com/clang/clang-19.1.7-Fedora-41-aarch64.tar.gz
  https://devpkg.scylladb.com/clang/clang-19.1.7-Fedora-41-x86_64.tar.gz

Closes scylladb/scylladb#23698
2025-04-15 10:11:28 +03:00
Avi Kivity
cf72c31617 treewide: improve bash error reporting
bash error handling and reporting is atrocious. Without -e it will
just ignore errors. With -e it will stop on errors, but not report
where the error happened (apart from exiting itself with an error code).

Improve that with the `trap ERR` command. Note that this won't be invoked
on intentional error exit with `exit 1`.

We apply this on every bash script that contains -e or that it appears
trivial to set it in. Non-trivial scripts without -e are left unmodified,
since they might intentionally invoke failing scripts.

Closes scylladb/scylladb#22747
2025-02-10 18:28:52 +03:00
Avi Kivity
f3eade2f62 treewide: relicense to ScyllaDB-Source-Available-1.0
Drop the AGPL license in favor of a source-available license.
See the blog post [1] for details.

[1] https://www.scylladb.com/2024/12/18/why-were-moving-to-a-source-available-license/
2024-12-18 17:45:13 +02:00
Kefu Chai
8d79811c6a scripts/refresh-submodules.sh: use the correct sha1 in title
0d4ffe1d69 introduced a regression where
it used the sha1 of the local "master" branch instead of the remote's
"master" branch in the title of the commit message.

in this change, let's use the origin/${branch}'s sha1 in the title.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>

Closes #13974
2023-05-22 12:10:03 +03:00
Kefu Chai
0d4ffe1d69 scripts/refresh-submodules.sh: include all commits in summary
before this change, we suse `git submodule summary ${submodule}`
for collecting the titles of commits in between current HEAD and
origin/master. normally, this works just fine. but it fails to
collect all commits if the origin/master happens to reference
a merge commit. for instance, if we have following history like:

1. merge foo
2. bar
3. foo
4. baz  <--- submodule is pointing here.

`git submodule summary` would just print out the titles of commits
of 1 and 3.

so, in this change, instead of relying on `git submodule summary`,
we just collect the commits using `git log`. but we preserve the
output format used by `git submodule summary` to be consistent with
the previous commits bumping up the submodules. please note, in
this change instead of matching the output of `git submodule summary`,
we use `git merge-base --is-ancestor HEAD origin/master` to check
if we are going to create a fastforward change, this is less fragile.

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>

Closes #13366
2023-04-06 11:27:14 +03:00
Kefu Chai
c307c60d04 scripts: correct a typo in comment
s/refreh/refresh/

Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>

Closes #13357
2023-03-29 13:44:47 +03:00
Pavel Emelyanov
cb4fe65a78 scripts: Allow specifying submodule branch to refresh from
There's a script to automate fetching submodule changes. However, this
script alays fetches remote master branch, which's not always the case.
For example, for branch-5.0/next-5.0 pair the correct scylla-seastar
branch would be the branch-5.0 one, not master.

With this change updating a submodule from a custom branch would be like

   refresh-submodules.sh <submodule>:<branch>

Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Message-Id: <20220322093623.15748-1-xemul@scylladb.com>
2022-03-22 15:18:25 +02:00
Pavel Emelyanov
22d131d40a Revert "scripts: Detect remote branch to fetch submodules from"
This reverts commit 87df37792c.

Scylla branches are not mapped to seastar branches 1-1, so getting
the upstream scylla branch doesn't point to the correct seastar one.
2022-03-22 15:18:25 +02:00
Pavel Emelyanov
87df37792c scripts: Detect remote branch to fetch submodules from
There's a script to automate fetching submodule changes. However, this
script alays fetches remote master branch, which's not always the case.
The correct branch can be detected by checking the current remote
tracking scylla branch which should coincide with the submodule one.

Signed-off-by: Pavel Emelyanov <xemul@scylladb.com>
Message-Id: <20220317085018.11529-1-xemul@scylladb.com>
2022-03-17 12:21:29 +02:00
Avi Kivity
fcb8d040e8 treewide: use Software Package Data Exchange (SPDX) license identifiers
Instead of lengthy blurbs, switch to single-line, machine-readable
standardized (https://spdx.dev) license identifiers. The Linux kernel
switched long ago, so there is strong precedent.

Three cases are handled: AGPL-only, Apache-only, and dual licensed.
For the latter case, I chose (AGPL-3.0-or-later and Apache-2.0),
reasoning that our changes are extensive enough to apply our license.

The changes we applied mechanically with a script, except to
licenses/README.md.

Closes #9937
2022-01-18 12:15:18 +01:00
Avi Kivity
a55b434a2b treewide: extent copyright statements to present day 2021-06-06 19:18:49 +03:00
Nadav Har'El
d5121d1476 scripts/refresh-submodules.sh: allow choosing which submodule to refresh
Currently, scripts/refresh-submodules.sh always refreshes all
submodules, i.e., takes the latest version of all of all of them and
commits it. But sometimes, a committer only wants to refresh a specific
submodule, and doesn't want to deal with the implications of updating
a different one.

As a recent example, for issue #8230, I wanted to update the tools/java
submodule, which included a fix for sstableloader, without updating the
Seastar submodule - which contained completely irrelevant changes.

So in this patch we add the ability to override the default list of
submodules that refresh-submodules.sh uses, with one or more command
line parameters. For example:

    scripts/refresh-submodules.sh tools/java

will update only tools/java.

Signed-off-by: Nadav Har'El <nyh@scylladb.com>
Message-Id: <20210411151421.309483-1-nyh@scylladb.com>
2021-04-11 18:35:04 +03:00
Pekka Enberg
8f17ca2d1a scripts/refresh-submodules.sh: Add python3 submodule
Message-Id: <20200928075422.377888-1-penberg@scylladb.com>
2020-09-29 16:06:32 +03:00
Pekka Enberg
9d183aed2d scripts: Fix submodule names in refresh-submodules.sh
The submodules were moved under tools/jmx and tools/java.
Message-Id: <20200720112447.754850-1-penberg@scylladb.com>
2020-07-20 14:28:39 +03:00
Pekka Enberg
3d128f5b51 scripts: Rename sync-submodules.sh to refresh-submodules.sh
Rename the script as per Nadav's suggestion and update documentation
within the script.
Message-Id: <20200618123446.32496-1-penberg@scylladb.com>
2020-06-18 15:39:23 +03:00