Fail the build if we don't check with sparse in both the kernel and
userspace utils. Add a filtering wrapper to the kernel build so that we
have a place to filter out uninteresting errors from kernel sources that
we're building against.
Signed-off-by: Zach Brown <zab@versity.com>
Instead of hashing headers, define an interop version. Do not mount
superblocks that have a different version, either higher or lower.
Since this is pretty much the same as the format hash except it's a
constant, minimal code changes are needed.
Initial dev version is 0, with the intent that version will be bumped to
1 immediately prior to tagging initial release version.
Update README. Fix comments.
Add interop version to notes and modinfo.
Signed-off-by: Andy Grover <agrover@versity.com>
For some reason, the make dist rule in kmod/ put the spec file in a
scoutfs-$ver/ directory, instead of scoutfs-kmod-$ver/ like the rest of
the files and instead of scoutfs-utils-$ver/ that the spec file for
utils is put in the utils dist tarball.
This adds -kmod to the path for the spec file so that it matches the
rest of the kmod dist tarball.
Signed-off-by: Zach Brown <zab@versity.com>
This adds in the makefile targets and spec file template we need to
build RPMs. Most of the heavy lifting is taken care of by our docker
container and the rpmbuild.sh script distributed in it.
The git versioning comes from 'git describe --long', which gives us the
tag, the number of commits and the abbreviated commit name. This allows
us to use the number of commits as the RPM release version, letting yum
understand how to process 'yum update' ordering.
yum update shows us the proper processing, along with how our versioning
lines up in the RPMs:
---> Package kmod-scoutfs.x86_64 0:0-0.3.gb83d29d.el7 will be updated
---> Package kmod-scoutfs.x86_64 0:0-0.4.g2e5324e.el7 will be an update
The rpm file name is: kmod-scoutfs-0-0.4.g2e5324e.el7.x86_64.rpm
When we build release RPMS, we'll toggle _release, giving us a rpm name
and version like kmod-scoutfs-0-1.4.g2e5324e.el7.x86_64.rpm. The toggle
of 0/1 is enough to tell yum that all of the non-release RPMs with the
same version are older than the released RPMs. This allows for the
release to yum update cleanly over development versions.
The git hash helps map RPM names to the git version and the contents of
the .note-git_describe, for this RPM it was: heads/nic/rpms-0-g2e5324.
The RPM doesn't contain the branch name, but we can add that and other
info later if needed.
We are not naming the module for a kernel version, that does not seem to
be standard practice upstream. Instead, we'll make use of our
Artifactory repos and upload the RPMs to the correct places (7.3 vs 7.4
directories, etc).
We had an excessive number of layers between scoutfs and the dlm code in
the kernel. We had dlmglue, the scoutfs locks, and task refs. Each
layer had structs that track the lifetime of the layer below it. We
were about to add another layer to hold on to locks just a bit longer so
that we can avoid down conversion and transaction commit storms under
contention.
This collapses all those layers into simple state machine in lock.c that
manages the mode of dlm locks on behalf of the file system.
The users of the lock interface are mainly unchanged. We did change
from a heavier trylock to a lighter nonblock lock attempt and have to
change the single rare readpage use. Lock fields change so a few
external users of those fields change.
This not only removes a lot of code it also contains functional
improvements. For example, it can now convert directly to CW locks with
a single lock request instead of having to use two by first converting
to NL.
It introduces the concept of an unlock grace period. Locks won't be
dropped on behalf of other nodes soon after being unlocked so that tasks
have a chance to batch up work before the other node gets a chance.
This can result in two orders of magnitude improvements in the time it
takes to, say, change a set of xattrs on the same file population from
two nodes concurrently.
There are significant changes to trace points, counters, and debug files
that follow the implementation changes.
Signed-off-by: Zach Brown <zab@versity.com>
I accidentally left this off with the initial dlmglue commit. I enabled
it here so that I could see our CW locks happening in real time. We
don't print lock name yet but that will be remedied in a future patch.
Turning this on gives us a debugfs file,
/sys/kernel/debug/scoutfs/<fsid>/locking_state
which exports the full lock state to userspace. The information
exported on each lock is extensive. The export includes each locks name
level, blocking level, request state, flags, etc. We also get a count of
lock attempts and failures for each level (cw, pr, ex). In addition we
also get the total time and max time waited on a given lock request.
Signed-off-by: Mark Fasheh <mfasheh@versity.com>
Calculate the hash of format.h and ioctl.h and make sure the hash stored
in the super during mkfs matches our calculated hash on mount.
Signed-off-by: Zach Brown <zab@versity.com>
We insist on a warning free build but it's up to human diligence to
discover and address warnings. We've also caught errors when compilers
in automated testing saw problems that the compilers in developer
environments didn't. That is, a human only could have noticed by
investigating the output from successful test runs.
Let's put some weight behind our promise of a warning free build and
turn gcc warnings into errors.
Signed-off-by: Zach Brown <zab@versity.com>
It's handy to quickly find the git commit that built a given module. We
add a MOD_INFO() tag for it so we can see it in modinfo on the built
module. We add a ELF note that the kernel tracks in
/sys/modules/$m/notes/ when the module is loaded.
Signed-off-by: Zach Brown <zab@versity.com>
We don't need the dlm to track key ranges if we implement ranges by
mapping keys to resources which represent ranges of the key space.
Signed-off-by: Zach Brown <zab@versity.com>
To actually use it, we first have to copy symbols over from the dlm build
into the scoutfs source directory. Make that happen automatically for us in
the Makefile.
The only users of locking at the moment are mount, unmount and xattr
read/write. Adding more locking calls should be a straight-forward endeavor.
The LVB based server ip communication didn't work out, and LVBS as they are
written don't make sense in a range locking world. So instead, we record the
server ip address in the superblock. This is protected by the listen lock,
which also arbitrates which node will be the manifest server.
We take and drop the dlm lock on each lock/unlock call. Lock caching will
come in a future patch.
Signed-off-by: Mark Fasheh <mfasheh@versity.com>
Also wire it into the build system. We have to figure out how to get scoutfs
pulling in the right headers but that can wait until we have something more
usable.
Signed-off-by: Mark Fasheh <mfasheh@versity.com>
We were duplicating the make args a few times so make a little ARGS
variable.
Default to the /lib/modules/$(uname -r) installed kernel source if
SK_KSRC isn't set.
And only try a sparse build that can fail if we can execute the sparse
command.
Signed-off-by: Zach Brown <zab@versity.com>
Reviewed-by: Mark Fasheh <mfasheh@versity.com>
Adding in an 'all' target allows us to use canned build scripts for any
of the scoutfs related repositories.
Signed-off-by: Nic Henke <nic.henke@versity.com>
Signed-off-by: Zach Brown <zab@zabbo.net>
When running make in a limited shell or in docker, there is no PWD from
shell. By using CURDIR we avoid worrying about the environment and let
make take care of this for us.
Signed-off-by: Nic Henke <nic.henke@versity.com>
Signed-off-by: Zach Brown <zab@zabbo.net>
Now that we know that it's easy to fix sparse build failures against
RHEL kernel headers we can require sparse builds when developing.
Signed-off-by: Zach Brown <zab@versity.com>
This is the initial commit of the repo that will track development
against distro kernels.
This is an import of a prototype branch in the upstream kernel that only
had a few initial commits. It needed to move to the old readdir
interface and use find_or_create_page() instead of pagecache_get_page()
to build in older distro kernels.
