The SCYLLA-VERSION-GEN file skips updating the SCYLLA-*-FILE files if
the commit hash from SCYLLA-RELEASE-FILE is the same. The original
reason for this was to prevent the date in the version string from
changing if multiple modes are built across midnight
(scylladb/scylla-pkg#826). However - intentionally or not - it serves
another purpose: it prevents an infinite loop in the build process.
If the build.ninja file needs to be rebuilt, the configure.py script
unconditionally calls ./SCYLLA-VERSION-GEN. On the other hand, if one
of the SCYLLA-*-FILE files is updated then this triggers rebuild
of build.ninja. Apparently, this is sufficient for ninja to enter an
infinite loop.
However, the check assumes that the RELEASE is in the format
<build identifier>.<date>.<commit hash>
and assumes that none of the components have a dot inside - otherwise it
breaks and just works incorrectly. Specifically, when building a private
version, it is recommended to set the build identifier to
`count.yourname`.
Previously, before 85219e9, this problem wasn't noticed most likely
because reconfigure process was broken and stopped overwriting
the build.ninja file after the first iteration.
Fix the problem by fixing the logic that extracts the commit hash -
instead of looking at the third dot-separated field counting from the
left side, look at the last field.
Fixes: scylladb/scylladb#21027Closesscylladb/scylladb#21049
Doing so is a pitfall that will make one waste a lot of time rebuilding
the packages, just because at the end it turns out that the version has
illegal characters in it. The author of this patch has certainly fallen
into this pitfall a lot of times.
Closesscylladb/scylladb#18429
before this change the argument passed to --date-stamp option is
ignored, as we don't reference the date-stamp specified with this option
at all. instead, we always overwrite it with the the output of
`date --utc +%Y%m%d`, if we are going to reference this value.
so, in this change instead of unconditionally overwriting it, we
keep its value intact if it is already set.
the change which introduced this regression was 839d8f40e6Fixes#15894
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
Closesscylladb/scylladb#15895
actually, we never use the its output in our workflow. and the
output is distracting when building the package. so, in this
change, let's print it only on demand. this feature is preserved
just in case some of us would want to use this script for getting
the version number string.
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
Closes#15327
remove unnecessary bashism, so that this script can be interpreted
by a POSIX shell.
/bin/sh is specified in the shebang line. on debian derivatives,
/bin/sh is dash, which is POSIX compliant. but this script is
written in the bash dialect.
before this change, we could run into following build failure
when building the tree on Debian:
[7/904] ./SCYLLA-VERSION-GEN
./SCYLLA-VERSION-GEN: 37: [[: not found
after this change, the build is able to proceed.
Signed-off-by: Kefu Chai <kefu.chai@scylladb.com>
Closes#12530
The extant logic for deriving the value of SCYLLA_RELEASE from the
source tree has those assumptions:
* The tree being built includes Git metadata.
* The value of `date` is trustworthy and interesting.
* There are no uncommitted changes (those relevant to building,
anyway).
The above assumptions are either irrelevant or problematic in pure
build environments (such as the sandbox set up by `nix-build`):
* Pure builds use cleaned-up sources with all timestamps reset to Unix
time 0. Those cleaned-up sources are saved (in the Nix store, for
example) and content-hashed, so leaving the (possibly huge) Git
metadata increases the time to copy the sources and wastes disk
space (in fact, Nix in flake mode strips `.git` unconditionally).
* Pure builds run in a sandbox where time is, likewise, reset to Unix
time 0, so the output of `date` is neither informative nor useful.
Now, the only build step that uses Git metadata in the first place is
the SCYLLA_RELEASE value derivation logic. So, essentially, answering
the question "is the Git metadata needed to build Scylla" is a matter
of definition, and is up to us. If we elect to ignore Git metadata
and current time, we can derive SCYLLA_RELEASE value from the content
hash of the cleaned-up tree, regardless of the way that tree was
arrived at.
This change makes it possible to skip the derivation of SCYLLA_RELEASE
value from Git metadata and current time by way of setting
SCYLLA_RELEASE in the environment.
Signed-off-by: Michael Livshin <michael.livshin@scylladb.com>
When starting `Build` job we have a situation when `x86` and `arm`
starting in different dates causing the all process to fail
As suggested by @avikivity , adding a date-stamp parameter and will pass
it through downstream jobs to get one release for each job
Ref: scylladb/scylla-pkg#3008
Closes#11234
This helps keep packages built on different machines have the
same datestamp, if started on the same time.
* tools/java 05ec511bbb...fd10821045 (1):
> build: use utc for build datestamp
* tools/jmx 48d37f3...d6225c5 (1):
> build: use utc for build datestamp
* tools/python3 c51db54...8a77e76 (1):
> build: use utc for build datestamp
[avi: commit own patches as this one requires excessive coordination
across submodules, for something quite innocuous]
Ref #9563 (doesn't really fix it, but helps a little)
This change allows to invoke the script in out-of-source
builds: `git log` now uses `-C` option with the directory
containing the script.
Also, the destination path can now be overriden by providing
`-o|--output-dir PATH` option. By default it's set to the
`build` directory relative to the script location.
Usage message is now shown, when '-h|--help' option is
specified.
Signed-off-by: Pavel Solodovnikov <pa.solodovnikov@scylladb.com>
Message-Id: <20210831120257.46920-1-pa.solodovnikov@scylladb.com>
Now that scylla-ccm and scylla-dtest conform to PEP-440
version comparison (See https://www.python.org/dev/peps/pep-0440/)
we can safely change scylla version on master to be the development
branch for the next release.
The version order logic is:
4.3.dev is followed by
4.3.rc[i] followed by
4.3.[n]
Note that also according to
https://blog.jasonantman.com/2014/07/how-yum-and-rpm-compare-versions/
4.3.dev < 4.3.rc[i] < 4.3.[n]
as "dev" < "rc" by alphabetical order
and both "dev" and "rc*" < any number, based on the general
rule that alphabetical strings compare as less than numbers.
Test: unit
Dtest: gating
Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
Message-Id: <20201015151153.726637-1-bhalevy@scylladb.com>
On our build system we tries to build relocatable package multiple times on
same revision of the repository, it executes ./SCYLLA-VERSION-GEN for each time.
When the build job invoked at midnight and it did not finished until 12:00AM,
first build and last build has different SCYLLA-RELEASE-FILE, since it contains
current date.
To prevent it, skip updating SCYLLA-*-FILE when git hash unchanged.
Fixesscylladb/scylla-pkg#826
When we add product name customization, we mistakenly defined the
parameter on each package build script.
Number of script is increasing since we recently added relocatable
python3 package, we should merge it in single place.
Also we should save the parameter on relocatable package, just like
version-release parameters.
So move the definition to SCYLLA-VERSION-GEN, save it to
build/SCYLLA-PRODUCT-FILE then archive it to relocatable package.
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <20190417163335.10191-1-syuu@scylladb.com>
Add a counter field to RELEASE, just before the date, and fix it at zero.
This allows custom package builds to override it in a way that sorts before
the official packages.
Example:
Official release: 1.6.0-0.20160120.<githash>
Custom release 1: 1.6.0-1.avi.20160121.<githash>
Custom release 2: 1.6.0-2.avi.20160122.<githash>
The counter (0/1/2) ensures that the build number dominates over the date
when sorting.
Message-Id: <20170122102814.19649-1-avi@scylladb.com>
yum command think "development-xxxx.xxxx" is older than "0.x", so nightly package mistakenly update with release version.
To prevent this problem, we should add greater number prior to "development".
Also same on Ubuntu package.
Signed-off-by: Takuya ASADA <syuu@scylladb.com>
Message-Id: <1452592877-29721-1-git-send-email-syuu@scylladb.com>
This adds version number generation in the build system. Version numbers
follow the format:
<version>-<release>
where release consists of:
<date>-<git-hash>
The version and release numbers are generated by the SCYLLA-VERSION-GEN
script and they are stored in SCYLLA-VERSION-FILE and
SCYLLA-RELEASE-FILE files so that other parts of the build system can
easily pick them up.
For builds that happen from release tarballs, for example,
SCYLLA-VERSION-GEN looks for a "version" file in the tree and just uses
that.
Basically, we're doing pretty much the same as Git is doing in its build
system.
Signed-off-by: Pekka Enberg <penberg@cloudius-systems.com>
