alphacentri/scoutfs-notify
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

v1.29-notify-2: drop config surface, add scoutfs-notifyd

Browse Source 
Reworks the series to Option B from the design discussion: no mount
options, no sysfs toggles, single well-known socket per scoutfs
volume at /run/scoutfs/<fsid>/notify.sock.

Changes vs v1.29-notify-1:

- Kmod: dropped notify_events / notify_ring_kb mount options and
  their sysfs attrs.  Ring is always allocated at mount with a
  compile-time 64 KiB size.  notify_enabled is now flipped by the
  ioctl handler on reader attach / detach, so emit hooks are
  zero-cost unless a watcher is connected.
- New third patch: scoutfs-notifyd userspace relay.  Single-threaded
  epoll daemon that drains the ioctl and broadcasts each record as
  a SOCK_SEQPACKET message to connected clients.  Shipped with a
  systemd template unit and man page.
- Packaging: utils/Makefile and scoutfs-utils.spec.in install the
  daemon at /usr/sbin/scoutfs-notifyd and the unit at
  /usr/lib/systemd/system/scoutfs-notifyd@.service.

Patches 1 and 2 have been re-scoped; re-apply expected to succeed
on scoutfs v1.29.
This commit is contained in:
William Gill
2026-04-22 13:59:44 -05:00
parent 50b9e27f75
commit 94a4fd53c6
4 changed files with 988 additions and 331 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+39 -43
View File
@@ -1,26 +1,28 @@
# scoutfs-notify
Observer-only file access notifications for [ScoutFS](https://github.com/versity/scoutfs).
Maintained as a rebasable `git format-patch` series so it can be layered onto
each upstream release with minimal maintenance.
Observer-only file access notifications for [ScoutFS](https://github.com/versity/scoutfs),
distributed as a rebasable `git format-patch` series plus a small userspace
relay daemon. Layers onto each upstream release with minimal maintenance.
## What it adds
## What the series adds
* New mount option `notify_events=0|1` (default `0`) that turns on a per-mount
ring buffer of file **open** and **read** events.
* New mount option `notify_ring_kb=N` (4..4096, default 64) sizing that ring.
* New ioctl `SCOUTFS_IOC_READ_NOTIFY` (nr 25) that lets a single privileged
userspace reader drain the ring. Reader is expected to relay events over a
Unix socket to a watcher daemon.
* Three percpu counters: `notify_emitted`, `notify_dropped_ring_full`,
`notify_reader_attached`.
Three patches against the scoutfs source tree:
The notification path is strictly observer-only: if the ring fills, records
are dropped but the monotonic `seq` field still advances so readers see a
gap. `notify_events=0` reduces the hook to a single predicted-false branch.
Nothing in the data-waiter state machine is touched by this series; a future
patch series may add data-waiter observation events (reserved type values 3+).
1. **Kmod core** — a per-mount 64 KiB ring of 64-byte notification records and
a single-reader drain ioctl (`SCOUTFS_IOC_READ_NOTIFY`, nr 25). Emit is
non-blocking, drop-on-full; the monotonic `seq` field exposes drops to
consumers. Three percpu counters (`notify_emitted`,
`notify_dropped_ring_full`, `notify_reader_attached`). No mount option,
no sysfs toggle.
2. **Kmod hooks** — `->open` wrapper and READ emit in
`scoutfs_file_aio_read` / `scoutfs_file_read_iter`. Every hook is a
single predicted-false branch when no reader is attached. Nothing in
the data-waiter state machine is touched.
3. **scoutfs-notifyd** — userspace daemon that binds
`/run/scoutfs/<fsid>/notify.sock` (AF_UNIX SOCK_SEQPACKET, mode 0600,
root-only), drains the ring, and broadcasts each record to connected
clients. Shipped with a systemd template unit
`scoutfs-notifyd@<mountpoint>.service`.
## Base
@@ -28,45 +30,39 @@ Currently rebased against:
scoutfs v1.29
See [base.txt](./base.txt) for the exact tag the patches target.
See [base.txt](./base.txt).
## Applying
On a git checkout of scoutfs at the base tag:
```sh
./apply.sh /path/to/scoutfs
```
The script runs `git am --3way` against each `patches/*.patch`. Requires the
target to be a git working tree.
For a non-git tarball, apply with:
```sh
cd /path/to/scoutfs
for p in /path/to/scoutfs-notify/patches/*.patch; do
patch -p1 < "$p"
done
```
Runs `git am --3way` on each patch. For a tarball instead of a git tree,
use `patch -p1` in a loop (see script).
## Rebasing onto a new upstream release
```sh
# In your scoutfs working clone
git fetch --tags
git checkout -B notify v1.30 # new upstream tag
git am --3way patches/*.patch # from this repo
# ... resolve any conflicts, git am --continue ...
git checkout -B notify v1.30
git am --3way patches/*.patch
# resolve any conflicts, git am --continue
git format-patch v1.30..notify -o patches/
# Update base.txt and commit the refreshed patches
# update base.txt and commit
```
## Tagging
## Tag pinning
Tag releases of the patch set with both the upstream version and a patch
revision so consumers can pin precisely:
Each release of this patch set is tagged as `v<upstream>-notify-<rev>`:
v1.29-notify-1
v1.29-notify-2
v1.30-notify-1
v1.29-notify-1 (retired — included mount options)
v1.29-notify-2 (current — daemon-driven, no knobs)
## Quick smoke test after installation
```sh
systemctl enable --now scoutfs-notifyd@mnt-scoutfs.service
socat - UNIX-CONNECT:/run/scoutfs/$(stat -c %d /mnt/scoutfs)/notify.sock \
| xxd | head # watch events stream as you touch files
```
patches/0001-notify-core-file-access-notification-infrastructure.patch
+99 -269
View File
@@ -1,46 +1,56 @@
From b57f3ec7e3c3d971c8f4b6825b7d8837f6668efb Mon Sep 17 00:00:00 2001
From 7564fa7095161816c28c774af97636a8b9ad6f17 Mon Sep 17 00:00:00 2001
From: William Gill <claude@williamgill.net>
Date: Wed, 22 Apr 2026 10:17:24 -0500
Subject: [PATCH 1/2] notify: core file-access notification infrastructure
Date: Wed, 22 Apr 2026 13:46:39 -0500
Subject: [PATCH 1/3] notify: core file-access notification infrastructure
Adds an optional, observer-only notification stream to the scoutfs
kernel module. File open/read events are recorded in a per-mount
ring buffer and drained by a single privileged userspace reader
through a new ioctl.
Adds a per-mount observer-only notification ring and an ioctl to
drain it. The feature has no user-facing configuration: the ring
is always allocated at mount and emits are enabled automatically
while a userspace reader is attached.
Behavior:
What this patch adds:
- Disabled by default. A new mount option, notify_events=0|1,
turns the stream on; notify_ring_kb=N sizes the ring (4..4096 KiB,
default 64 KiB).
- New ioctl SCOUTFS_IOC_READ_NOTIFY (nr 25) fills the caller's
array of scoutfs_ioctl_notify_event records; supports a
timeout_ms wait policy. Requires CAP_SYS_ADMIN.
- Ring is lossy: on overflow records are dropped but the monotonic
seq field still advances, so a reader detects gaps. Three
percpu counters (notify_emitted, notify_dropped_ring_full,
- New files kmod/src/notify.{h,c}. A fixed-size 64 KiB ring of
64-byte records, a single-reader drain ioctl, and setup/destroy
hooks for scoutfs_sb_info.
- New ioctl SCOUTFS_IOC_READ_NOTIFY (nr 25). Fills the caller's
array of scoutfs_ioctl_notify_event; supports a timeout_ms wait
policy. CAP_SYS_ADMIN required. atomic_cmpxchg on
reader_attached returns -EBUSY to concurrent readers.
- Per-super state: scoutfs_sb_info gains "struct notify_info *"
and "bool notify_enabled". notify_enabled is flipped by the
ioctl handler on reader attach/detach, so emit hooks short-
circuit at a single predicted-false branch when no reader is
listening.
- Three percpu counters (notify_emitted, notify_dropped_ring_full,
notify_reader_attached) surface operational state via sysfs.
- Single-reader: concurrent readers receive -EBUSY.
- Setup/destroy wired into super.c alongside the other per-super
subsystems.
This patch only adds the infrastructure (new files, mount options,
ioctl, lifecycle, counters). No scoutfs code path calls
scoutfs_notify_emit() yet, so behavior is bit-identical to stock
with notify_events=0 or notify_events=1.
Design properties:
Event type values 1..2 are defined (OPEN, READ); 3+ are reserved
for future data-waiter observation events.
- Emit is non-blocking: one leaf spinlock, no allocations, no
sleeping locks, drop-on-full with seq still advancing so readers
see gaps.
- Ring size (64 KiB = 1024 events) is a compile-time constant.
- No mount option, no sysfs toggle. The watcher daemon's
attach/detach is the only gate.
- Event type values 1..2 are defined for file OPEN and READ.
Values 3+ are reserved for future data-waiter observation.
This patch adds only the infrastructure. No scoutfs code path
calls scoutfs_notify_emit() yet; the hook sites arrive in a
follow-up patch.
---
kmod/src/Makefile | 1 +
kmod/src/counters.h | 3 +
kmod/src/ioctl.c | 3 +
kmod/src/ioctl.h | 78 +++++++++
kmod/src/notify.c | 389 ++++++++++++++++++++++++++++++++++++++++++++
kmod/src/notify.c | 373 ++++++++++++++++++++++++++++++++++++++++++++
kmod/src/notify.h | 39 +++++
kmod/src/options.c | 91 +++++++++++
kmod/src/options.h | 2 +
kmod/src/super.c | 3 +
kmod/src/super.h | 5 +
10 files changed, 614 insertions(+)
8 files changed, 505 insertions(+)
create mode 100644 kmod/src/notify.c
create mode 100644 kmod/src/notify.h
@@ -180,10 +190,10 @@ index c0d2285..fc58f8c 100644
#endif
diff --git a/kmod/src/notify.c b/kmod/src/notify.c
new file mode 100644
index 0000000..94fab17
index 0000000..7a547a2
--- /dev/null
+++ b/kmod/src/notify.c
@@ -0,0 +1,389 @@
@@ -0,0 +1,373 @@
+/*
+ * Copyright (C) 2026 Versity Software, Inc. All rights reserved.
+ *
@@ -207,8 +217,15 @@ index 0000000..94fab17
+ * reader, and returns. The sequence counter advances on drops so the
+ * reader sees a gap rather than a seamless stream.
+ *
+ * Userspace drains the ring via SCOUTFS_IOC_READ_NOTIFY. Only one
+ * reader may be attached at a time.
+ * There is no user-visible configuration. A per-super ring is always
+ * allocated at mount, but sbi->notify_enabled — the fast flag the
+ * emit hooks check — is set only while a userspace reader is attached
+ * via SCOUTFS_IOC_READ_NOTIFY. Without an attached reader, hook sites
+ * short-circuit at a single predicted-false branch and no events are
+ * produced.
+ *
+ * Only one reader may be attached at a time; the ioctl handler uses
+ * atomic_cmpxchg so a second concurrent caller receives -EBUSY.
+ */
+
+#include <linux/kernel.h>
@@ -228,7 +245,6 @@ index 0000000..94fab17
+#include <linux/bitops.h>
+
+#include "super.h"
+#include "options.h"
+#include "counters.h"
+#include "ioctl.h"
+#include "notify.h"
@@ -253,8 +269,13 @@ index 0000000..94fab17
+ u64 next_seq;
+};
+
+#define NOTIFY_MIN_KB 4
+#define NOTIFY_MAX_KB 4096
+/*
+ * Fixed ring size. Each record is 64 bytes; 64 KiB holds 1024 events.
+ * Deliberately not user-configurable so the feature has no knobs.
+ */
+#define NOTIFY_RING_BYTES (64 * 1024)
+#define NOTIFY_RING_CAPACITY (NOTIFY_RING_BYTES / \
+ sizeof(struct scoutfs_ioctl_notify_event))
+
+static u32 ring_len_locked(struct notify_info *ni)
+{
@@ -271,64 +292,15 @@ index 0000000..94fab17
+ return ring_len_locked(ni) == ni->capacity;
+}
+
+/*
+ * Allocate the ring buffer. Uses kvmalloc to accept large user-chosen
+ * sizes without failing under fragmentation.
+ */
+static int alloc_ring(struct notify_info *ni, u32 capacity)
+{
+ size_t bytes;
+
+ if (!is_power_of_2(capacity))
+ return -EINVAL;
+
+ if (check_mul_overflow((size_t)capacity,
+ sizeof(struct scoutfs_ioctl_notify_event),
+ &bytes))
+ return -EINVAL;
+
+ ni->ring = kvzalloc(bytes, GFP_KERNEL);
+ if (!ni->ring)
+ return -ENOMEM;
+
+ ni->capacity = capacity;
+ ni->mask = capacity - 1;
+ ni->head = 0;
+ ni->tail = 0;
+ return 0;
+}
+
+/*
+ * Compute ring capacity in records from the options-configured size in
+ * KiB. Rounds down to a power of two so the mask-based index math is
+ * valid. Returns a record count, not a byte count.
+ */
+static u32 capacity_from_kb(u32 kb)
+{
+ u64 bytes;
+ u32 records;
+
+ if (kb < NOTIFY_MIN_KB)
+ kb = NOTIFY_MIN_KB;
+ if (kb > NOTIFY_MAX_KB)
+ kb = NOTIFY_MAX_KB;
+
+ bytes = (u64)kb << 10;
+ records = (u32)(bytes / sizeof(struct scoutfs_ioctl_notify_event));
+ if (records < 2)
+ records = 2;
+
+ /* round down to power of two */
+ return 1U << (fls(records) - 1);
+}
+
+int scoutfs_notify_setup(struct super_block *sb)
+{
+ struct scoutfs_sb_info *sbi = SCOUTFS_SB(sb);
+ struct scoutfs_mount_options opts;
+ struct notify_info *ni;
+ u32 capacity;
+ int ret;
+ size_t bytes;
+
+ /* compile-time sanity: capacity must be a power of two */
+ BUILD_BUG_ON((NOTIFY_RING_CAPACITY & (NOTIFY_RING_CAPACITY - 1)) != 0);
+ BUILD_BUG_ON(NOTIFY_RING_CAPACITY < 2);
+
+ ni = kzalloc(sizeof(*ni), GFP_KERNEL);
+ if (!ni)
@@ -339,24 +311,26 @@ index 0000000..94fab17
+ atomic_set(&ni->reader_attached, 0);
+ ni->next_seq = 1;
+
+ scoutfs_options_read(sb, &opts);
+ capacity = capacity_from_kb(opts.notify_ring_kb);
+
+ ret = alloc_ring(ni, capacity);
+ if (ret < 0) {
+ bytes = (size_t)NOTIFY_RING_CAPACITY *
+ sizeof(struct scoutfs_ioctl_notify_event);
+ ni->ring = kvzalloc(bytes, GFP_KERNEL);
+ if (!ni->ring) {
+ kfree(ni);
+ return ret;
+ return -ENOMEM;
+ }
+
+ ni->capacity = NOTIFY_RING_CAPACITY;
+ ni->mask = NOTIFY_RING_CAPACITY - 1;
+ ni->head = 0;
+ ni->tail = 0;
+
+ sbi->notify_info = ni;
+
+ /*
+ * Publish the enable flag last so emit callers observe a fully
+ * initialized ring.
+ * notify_enabled stays false until a reader attaches. Emit hooks
+ * will short-circuit until then, so the feature is zero-cost in
+ * the common no-reader case.
+ */
+ smp_wmb();
+ WRITE_ONCE(sbi->notify_enabled, opts.notify_events ? true : false);
+
+ WRITE_ONCE(sbi->notify_enabled, false);
+ return 0;
+}
+
@@ -384,8 +358,8 @@ index 0000000..94fab17
+/*
+ * Best-effort observer-only event emit.
+ *
+ * - Callers must gate this on sbi->notify_enabled; we re-check under
+ * the spinlock so shutdown is race-free.
+ * - Callers must gate this on sbi->notify_enabled; we re-check the
+ * pointer under the spinlock so shutdown is race-free.
+ * - On ring-full we drop the record but still consume a sequence
+ * number so the reader sees exactly the number of drops.
+ * - Never returns an error and never blocks scoutfs.
@@ -452,7 +426,11 @@ index 0000000..94fab17
+ empty = ring_empty_locked(ni);
+ spin_unlock_irqrestore(&ni->lock, irqflags);
+
+ return !empty || !READ_ONCE(sbi->notify_enabled);
+ /*
+ * Wake on events, on detach-requested (e.g. unmount flips the
+ * enable flag), or on our own reader having gone away.
+ */
+ return !empty || !atomic_read(&ni->reader_attached);
+}
+
+/*
@@ -537,6 +515,14 @@ index 0000000..94fab17
+ attached = true;
+ scoutfs_inc_counter(sb, notify_reader_attached);
+
+ /*
+ * Reader is now attached. Flip the emit fast-path flag on so
+ * hook sites start producing events. We publish the bool after
+ * the cmpxchg so no racing reader can observe enabled=true with
+ * reader_attached=false.
+ */
+ WRITE_ONCE(sbi->notify_enabled, true);
+
+ spin_lock_irqsave(&ni->lock, irqflags);
+ empty = ring_empty_locked(ni);
+ spin_unlock_irqrestore(&ni->lock, irqflags);
@@ -569,8 +555,16 @@ index 0000000..94fab17
+ ret = drain_ring(ni, uevents, args.events_nr);
+
+out:
+ if (attached)
+ if (attached) {
+ /*
+ * Turn the fast-path flag off BEFORE clearing
+ * reader_attached. Any concurrent emit either sees
+ * enabled=true (and produces to an attached reader) or
+ * enabled=false (and does nothing).
+ */
+ WRITE_ONCE(sbi->notify_enabled, false);
+ atomic_set(&ni->reader_attached, 0);
+ }
+ return ret;
+}
diff --git a/kmod/src/notify.h b/kmod/src/notify.h
@@ -618,170 +612,6 @@ index 0000000..fe6a2d9
+long scoutfs_ioc_read_notify(struct file *file, unsigned long arg);
+
+#endif /* _SCOUTFS_NOTIFY_H_ */
diff --git a/kmod/src/options.c b/kmod/src/options.c
index b7565d7..90e7713 100644
--- a/kmod/src/options.c
+++ b/kmod/src/options.c
@@ -38,6 +38,8 @@ enum {
Opt_log_merge_wait_timeout_ms,
Opt_metadev_path,
Opt_noacl,
+ Opt_notify_events,
+ Opt_notify_ring_kb,
Opt_orphan_scan_delay_ms,
Opt_quorum_heartbeat_timeout_ms,
Opt_quorum_slot_nr,
@@ -54,6 +56,8 @@ static const match_table_t tokens = {
{Opt_log_merge_wait_timeout_ms, "log_merge_wait_timeout_ms=%s"},
{Opt_metadev_path, "metadev_path=%s"},
{Opt_noacl, "noacl"},
+ {Opt_notify_events, "notify_events=%s"},
+ {Opt_notify_ring_kb, "notify_ring_kb=%s"},
{Opt_orphan_scan_delay_ms, "orphan_scan_delay_ms=%s"},
{Opt_quorum_heartbeat_timeout_ms, "quorum_heartbeat_timeout_ms=%s"},
{Opt_quorum_slot_nr, "quorum_slot_nr=%s"},
@@ -138,6 +142,10 @@ static void free_options(struct scoutfs_mount_options *opts)
#define DEFAULT_TCP_KEEPALIVE_TIMEOUT_MS (60 * MSEC_PER_SEC)
+#define MIN_NOTIFY_RING_KB 4U
+#define DEFAULT_NOTIFY_RING_KB 64U
+#define MAX_NOTIFY_RING_KB 4096U
+
static void init_default_options(struct scoutfs_mount_options *opts)
{
memset(opts, 0, sizeof(*opts));
@@ -147,6 +155,8 @@ static void init_default_options(struct scoutfs_mount_options *opts)
opts->ino_alloc_per_lock = SCOUTFS_LOCK_INODE_GROUP_NR;
opts->lock_idle_count = DEFAULT_LOCK_IDLE_COUNT;
opts->log_merge_wait_timeout_ms = DEFAULT_LOG_MERGE_WAIT_TIMEOUT_MS;
+ opts->notify_events = false;
+ opts->notify_ring_kb = DEFAULT_NOTIFY_RING_KB;
opts->orphan_scan_delay_ms = -1;
opts->quorum_heartbeat_timeout_ms = SCOUTFS_QUORUM_DEF_HB_TIMEO_MS;
opts->quorum_slot_nr = -1;
@@ -309,6 +319,29 @@ static int parse_options(struct super_block *sb, char *options, struct scoutfs_m
sb->s_flags &= ~SB_POSIXACL;
break;
+ case Opt_notify_events:
+ ret = match_int(args, &nr);
+ if (ret < 0 || nr < 0 || nr > 1) {
+ scoutfs_err(sb, "invalid notify_events option, bool must only be 0 or 1");
+ if (ret == 0)
+ ret = -EINVAL;
+ return ret;
+ }
+ opts->notify_events = nr;
+ break;
+
+ case Opt_notify_ring_kb:
+ ret = match_int(args, &nr);
+ if (ret < 0 || nr < MIN_NOTIFY_RING_KB || nr > MAX_NOTIFY_RING_KB) {
+ scoutfs_err(sb, "invalid notify_ring_kb option, must be between %u and %u",
+ MIN_NOTIFY_RING_KB, MAX_NOTIFY_RING_KB);
+ if (ret == 0)
+ ret = -EINVAL;
+ return ret;
+ }
+ opts->notify_ring_kb = nr;
+ break;
+
case Opt_orphan_scan_delay_ms:
if (opts->orphan_scan_delay_ms != -1) {
scoutfs_err(sb, "multiple orphan_scan_delay_ms options provided, only provide one.");
@@ -442,6 +475,8 @@ int scoutfs_options_show(struct seq_file *seq, struct dentry *root)
seq_printf(seq, ",metadev_path=%s", opts.metadev_path);
if (!is_acl)
seq_puts(seq, ",noacl");
+ seq_printf(seq, ",notify_events=%u", opts.notify_events);
+ seq_printf(seq, ",notify_ring_kb=%u", opts.notify_ring_kb);
seq_printf(seq, ",orphan_scan_delay_ms=%u", opts.orphan_scan_delay_ms);
if (opts.quorum_slot_nr >= 0)
seq_printf(seq, ",quorum_slot_nr=%d", opts.quorum_slot_nr);
@@ -651,6 +686,60 @@ static ssize_t metadev_path_show(struct kobject *kobj, struct kobj_attribute *at
}
SCOUTFS_ATTR_RO(metadev_path);
+static ssize_t notify_events_show(struct kobject *kobj, struct kobj_attribute *attr,
+ char *buf)
+{
+ struct super_block *sb = SCOUTFS_SYSFS_ATTRS_SB(kobj);
+ struct scoutfs_mount_options opts;
+
+ scoutfs_options_read(sb, &opts);
+
+ return snprintf(buf, PAGE_SIZE, "%u", opts.notify_events);
+}
+static ssize_t notify_events_store(struct kobject *kobj, struct kobj_attribute *attr,
+ const char *buf, size_t count)
+{
+ struct super_block *sb = SCOUTFS_SYSFS_ATTRS_SB(kobj);
+ struct scoutfs_sb_info *sbi = SCOUTFS_SB(sb);
+ DECLARE_OPTIONS_INFO(sb, optinf);
+ char nullterm[20];
+ long val;
+ int len;
+ int ret;
+
+ len = min(count, sizeof(nullterm) - 1);
+ memcpy(nullterm, buf, len);
+ nullterm[len] = '\0';
+
+ ret = kstrtol(nullterm, 0, &val);
+ if (ret < 0 || val < 0 || val > 1) {
+ scoutfs_err(sb, "invalid notify_events option, bool must be 0 or 1");
+ return -EINVAL;
+ }
+
+ write_seqlock(&optinf->seqlock);
+ optinf->opts.notify_events = val;
+ write_sequnlock(&optinf->seqlock);
+
+ /* mirror into sbi so emit fast path can check with one load */
+ WRITE_ONCE(sbi->notify_enabled, val ? true : false);
+
+ return count;
+}
+SCOUTFS_ATTR_RW(notify_events);
+
+static ssize_t notify_ring_kb_show(struct kobject *kobj, struct kobj_attribute *attr,
+ char *buf)
+{
+ struct super_block *sb = SCOUTFS_SYSFS_ATTRS_SB(kobj);
+ struct scoutfs_mount_options opts;
+
+ scoutfs_options_read(sb, &opts);
+
+ return snprintf(buf, PAGE_SIZE, "%u", opts.notify_ring_kb);
+}
+SCOUTFS_ATTR_RO(notify_ring_kb);
+
static ssize_t orphan_scan_delay_ms_show(struct kobject *kobj, struct kobj_attribute *attr,
char *buf)
{
@@ -747,6 +836,8 @@ static struct attribute *options_attrs[] = {
SCOUTFS_ATTR_PTR(lock_idle_count),
SCOUTFS_ATTR_PTR(log_merge_wait_timeout_ms),
SCOUTFS_ATTR_PTR(metadev_path),
+ SCOUTFS_ATTR_PTR(notify_events),
+ SCOUTFS_ATTR_PTR(notify_ring_kb),
SCOUTFS_ATTR_PTR(orphan_scan_delay_ms),
SCOUTFS_ATTR_PTR(quorum_heartbeat_timeout_ms),
SCOUTFS_ATTR_PTR(quorum_slot_nr),
diff --git a/kmod/src/options.h b/kmod/src/options.h
index b37bbd7..12b976e 100644
--- a/kmod/src/options.h
+++ b/kmod/src/options.h
@@ -12,6 +12,8 @@ struct scoutfs_mount_options {
int lock_idle_count;
unsigned int log_merge_wait_timeout_ms;
char *metadev_path;
+ bool notify_events;
+ unsigned int notify_ring_kb;
unsigned int orphan_scan_delay_ms;
int quorum_slot_nr;
u64 quorum_heartbeat_timeout_ms;
diff --git a/kmod/src/super.c b/kmod/src/super.c
index 3c83716..3c028b0 100644
--- a/kmod/src/super.c
patches/0002-notify-file-open-read-hook-sites.patch
+22 -19
View File
@@ -1,36 +1,39 @@
From dcf118e874fa4af97144c8e0c960ac42c7e17245 Mon Sep 17 00:00:00 2001
From 1b6569c33f7337c6af16933441241c84874a6f5d Mon Sep 17 00:00:00 2001
From: William Gill <claude@williamgill.net>
Date: Wed, 22 Apr 2026 10:18:11 -0500
Subject: [PATCH 2/2] notify: file open/read hook sites
Date: Wed, 22 Apr 2026 13:47:13 -0500
Subject: [PATCH 2/3] notify: file open/read hook sites
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Wires the notification emit path into the two scoutfs file
operations that carry user-visible activity we want to observe.
OPEN (data.c):
- A new scoutfs_file_open() wrapper is installed as ->open in
scoutfs_file_fops. The wrapper calls generic_file_open() to
preserve the existing VFS default semantics for regular files,
then, only on success and only when notifications are enabled,
emits a SCOUTFS_NOTIFY_TYPE_OPEN record. The file mode is
inspected for FMODE_WRITE to set SCOUTFS_NOTIFY_F_WRITE_OPEN.
A new scoutfs_file_open() wrapper is installed as ->open in
scoutfs_file_fops. The wrapper calls generic_file_open() to
preserve the existing VFS default semantics for regular files,
then — only on success, and only when notifications are enabled
— emits a SCOUTFS_NOTIFY_TYPE_OPEN record. The file mode is
inspected for FMODE_WRITE to set SCOUTFS_NOTIFY_F_WRITE_OPEN.
READ (file.c):
- Both the aio_read (KC_LINUX_HAVE_FOP_AIO_READ) and read_iter
paths get an emit placed past the existing data-waiter retry
check, guarded on (ret > 0) so only successful reads are
reported and retries never double-count. start_pos is captured
at entry of read_iter before generic_file_read_iter advances
iocb->ki_pos.
Both the aio_read (KC_LINUX_HAVE_FOP_AIO_READ) and read_iter
paths get an emit placed past the existing data-waiter retry
check, guarded on (ret > 0) so only successful reads are
reported and retries never double-count. start_pos is captured
at entry of read_iter before generic_file_read_iter advances
iocb->ki_pos.
Every hook is behind unlikely(READ_ONCE(sbi->notify_enabled)), so
with the default notify_events=0 the path is a single
when no userspace reader is attached the hook is a single
predicted-false branch. No scoutfs state is mutated, no error is
propagated, and no existing control flow is altered.
Nothing in the data-waiter state machine (scoutfs_data_wait_check,
scoutfs_data_wait, scoutfs_data_wait_changed, the waiter rbtree, or
SCOUTFS_IOC_DATA_WAITING / DATA_WAIT_ERR) is touched by this patch.
That work is deferred to a future series.
scoutfs_data_wait, scoutfs_data_wait_changed, the waiter rbtree,
or SCOUTFS_IOC_DATA_WAITING / DATA_WAIT_ERR) is touched. That
work is deferred to a future series.
---
kmod/src/data.c | 28 ++++++++++++++++++++++++++++
kmod/src/file.c | 11 +++++++++++
patches/0003-notify-scoutfs-notifyd-userspace-relay-daemon.patch
+828
View File
@@ -0,0 +1,828 @@
From 4a2b2f062c4d733444f1fa40257725625d9bebe3 Mon Sep 17 00:00:00 2001
From: William Gill <claude@williamgill.net>
Date: Wed, 22 Apr 2026 13:47:45 -0500
Subject: [PATCH 3/3] notify: scoutfs-notifyd userspace relay daemon
Adds the userspace side of the file access notification feature.
The kmod provides a ring and a drain ioctl; this daemon binds the
well-known socket, drains the ring, and broadcasts each record to
connected clients.
Socket:
/run/scoutfs/<fsid>/notify.sock
AF_UNIX / SOCK_SEQPACKET
mode 0600, owner root:root
The fsid is discovered at startup via SCOUTFS_IOC_STATFS_MORE on
the given mountpoint, so the daemon never consults its
environment or a config file for the path. The shipped systemd
unit creates /run/scoutfs/ via RuntimeDirectory=scoutfs.
Protocol:
Each broadcast is one SOCK_SEQPACKET message carrying exactly one
64-byte struct scoutfs_ioctl_notify_event record. Clients issue
recv(2) in a loop; each returned message is one event.
Behavior:
- Single-threaded with epoll. The drain ioctl uses a 100 ms
timeout so the loop paces itself; new clients, client
disconnects, and shutdown signals are handled between ioctl
calls non-blocking.
- Slow-client protection: sends use MSG_DONTWAIT | MSG_NOSIGNAL.
Any send error drops the client immediately.
- MAX_CLIENTS is 128; extras are rejected with a log warning.
- Reader exclusivity is owned by the kmod ioctl (returns -EBUSY
to a second attached reader); starting a second instance of
the daemon against the same mount logs the error and exits.
- Graceful shutdown on SIGTERM / SIGINT / SIGQUIT via signalfd.
Packaging:
- utils/Makefile: new target notifyd/scoutfs-notifyd built from
utils/notifyd/scoutfs-notifyd.c with -I../kmod/src for the
shared ioctl.h. No extra libs beyond libc.
- utils/scoutfs-utils.spec.in: installs the binary at
/usr/sbin/scoutfs-notifyd and the template unit at
/usr/lib/systemd/system/scoutfs-notifyd@.service.
- utils/man/scoutfs-notifyd.8 documents the tool, socket
protocol, and drop policy.
Activation:
systemctl enable --now scoutfs-notifyd@mnt-scoutfs.service
where the instance name is the escaped mountpoint path. Watchers
then connect to /run/scoutfs/<fsid>/notify.sock and recv() events.
---
utils/Makefile | 21 +-
utils/man/scoutfs-notifyd.8 | 120 ++++++
utils/notifyd/scoutfs-notifyd.c | 501 +++++++++++++++++++++++++
utils/notifyd/scoutfs-notifyd@.service | 40 ++
utils/scoutfs-utils.spec.in | 4 +
5 files changed, 683 insertions(+), 3 deletions(-)
create mode 100644 utils/man/scoutfs-notifyd.8
create mode 100644 utils/notifyd/scoutfs-notifyd.c
create mode 100644 utils/notifyd/scoutfs-notifyd@.service
diff --git a/utils/Makefile b/utils/Makefile
index e0f7614..3535928 100644
--- a/utils/Makefile
+++ b/utils/Makefile
@@ -18,7 +18,10 @@ BIN := src/scoutfs
OBJ := $(patsubst %.c,%.o,$(wildcard src/*.c))
DEPS := $(wildcard */*.d)
-all: $(BIN)
+NOTIFYD := notifyd/scoutfs-notifyd
+NOTIFYD_OBJ := $(patsubst %.c,%.o,$(wildcard notifyd/*.c))
+
+all: $(BIN) $(NOTIFYD)
ifneq ($(DEPS),)
-include $(DEPS)
@@ -29,13 +32,25 @@ QU = @echo
VE = @
else
QU = @:
-VE =
+VE =
endif
$(BIN): $(OBJ)
$(QU) [BIN $@]
$(VE)gcc -o $@ $^ -luuid -lm -lcrypto -lblkid
+# scoutfs-notifyd is a single-source daemon; it needs -I../kmod/src for
+# ioctl.h but no external libraries beyond libc.
+$(NOTIFYD): $(NOTIFYD_OBJ)
+ $(QU) [BIN $@]
+ $(VE)gcc -o $@ $^
+
+notifyd/%.o notifyd/%.d: notifyd/%.c Makefile sparse.sh
+ $(QU) [CC $<]
+ $(VE)gcc $(CFLAGS) -I../kmod/src -MD -MP -MF notifyd/$*.d -c $< -o notifyd/$*.o
+ $(QU) [SP $<]
+ $(VE)./sparse.sh -Wbitwise -D__CHECKER__ $(CFLAGS) -I../kmod/src $<
+
%.o %.d: %.c Makefile sparse.sh
$(QU) [CC $<]
$(VE)gcc $(CFLAGS) -MD -MP -MF $*.d -c $< -o $*.o
@@ -65,4 +80,4 @@ dist: $(RPM_DIR) scoutfs-utils.spec
tar rf $(TARFILE) --transform="s@.*\(src/.*\)@scoutfs-utils-$(RPM_VERSION)/\1@" $(FMTIOC_KMOD)
clean:
- @rm -f $(BIN) $(OBJ) $(DEPS) .sparse.*
+ @rm -f $(BIN) $(OBJ) $(NOTIFYD) $(NOTIFYD_OBJ) $(DEPS) .sparse.*
diff --git a/utils/man/scoutfs-notifyd.8 b/utils/man/scoutfs-notifyd.8
new file mode 100644
index 0000000..a0dcd46
--- /dev/null
+++ b/utils/man/scoutfs-notifyd.8
@@ -0,0 +1,120 @@
+.TH scoutfs-notifyd 8
+.SH NAME
+scoutfs-notifyd \- scoutfs file access notification relay daemon
+
+.SH SYNOPSIS
+.B scoutfs-notifyd
+.I mountpoint
+
+.SH DESCRIPTION
+The
+.B scoutfs-notifyd
+daemon drains the file access notification ring of a scoutfs mount
+and broadcasts each event to every process connected to
+.I /run/scoutfs/<fsid>/notify.sock.
+
+Events are observer-only records of file
+.B open
+and
+.B read
+activity, produced by the scoutfs kernel module for the mount
+identified by
+.IR mountpoint .
+The daemon is the single privileged reader of the ring and is expected
+to be started by the systemd template unit
+.I scoutfs-notifyd@.service
+with the instance name set to the escaped mountpoint path.
+
+.SH OPTIONS
+
+.TP
+.I mountpoint
+An absolute path to a directory on the scoutfs volume to monitor.
+Typically this is the mountpoint itself. The daemon opens the path,
+reads the volume's fsid via the
+.B SCOUTFS_IOC_STATFS_MORE
+ioctl, and builds the listener socket path from that fsid.
+
+.SH SOCKET
+
+The listener is an
+.B AF_UNIX
+.B SOCK_SEQPACKET
+socket at
+.I /run/scoutfs/<fsid>/notify.sock
+with mode
+.B 0600
+and owner
+.BR root:root .
+
+Each broadcast is one atomic message carrying a single packed 64-byte
+record matching
+.BR "struct scoutfs_ioctl_notify_event" .
+Clients read exactly one record per
+.BR recv (2)
+call. The fields are:
+
+.nf
+.RS 4
+__u64 seq; /* monotonic; gap = dropped events */
+__u64 ino; /* scoutfs inode number */
+__u64 offset; /* byte offset of the read, 0 for OPEN */
+__u64 length; /* byte length of the read, 0 for OPEN */
+__u64 time_ns; /* CLOCK_REALTIME at the event */
+__u32 pid; /* task group id */
+__u32 uid; /* effective uid (init user namespace) */
+__u8 type; /* 1 = OPEN, 2 = READ */
+__u8 flags; /* bit 0 = opened writable */
+__u8 _pad[6];
+.RE
+.fi
+
+Numeric values 3 and higher in
+.B type
+are reserved for future scoutfs events (data-waiter observation).
+
+.SH DROP POLICY
+
+The in-kernel ring is lossy. When it fills, records are discarded but
+the monotonic
+.B seq
+field still advances, so consumers that track seq will detect gaps
+rather than silent loss.
+
+The daemon forwards to clients with
+.B MSG_DONTWAIT
+and
+.BR MSG_NOSIGNAL .
+A client that cannot keep up is disconnected immediately; it is
+expected to reconnect.
+
+.SH PERMISSIONS
+
+The daemon requires
+.B CAP_SYS_ADMIN
+to call
+.BR SCOUTFS_IOC_READ_NOTIFY .
+The shipped systemd unit grants that capability and strips everything
+else. Only one reader may be attached to a scoutfs mount's ring at a
+time; starting a second daemon against the same mount returns
+.BR EBUSY .
+
+.SH EXIT STATUS
+
+The daemon exits 0 on clean shutdown (SIGTERM, SIGINT, SIGQUIT) and
+non-zero on error.
+
+.SH FILES
+.TP
+.I /run/scoutfs/<fsid>/notify.sock
+Listener socket, per scoutfs fsid.
+
+.TP
+.I /usr/lib/systemd/system/scoutfs-notifyd@.service
+Template unit. Enable and start a specific mount with, e.g.,
+.BR "systemctl enable --now scoutfs-notifyd@mnt-scoutfs.service" .
+
+.SH SEE ALSO
+.BR scoutfs (5),
+.BR scoutfs (8),
+.BR systemd.unit (5)
diff --git a/utils/notifyd/scoutfs-notifyd.c b/utils/notifyd/scoutfs-notifyd.c
new file mode 100644
index 0000000..d81c74e
--- /dev/null
+++ b/utils/notifyd/scoutfs-notifyd.c
@@ -0,0 +1,501 @@
+/*
+ * Copyright (C) 2026 Versity Software, Inc. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ */
+
+/*
+ * scoutfs-notifyd
+ *
+ * Drains the scoutfs file access notification ring (via
+ * SCOUTFS_IOC_READ_NOTIFY) and fans each record out to every client
+ * connected to /run/scoutfs/<fsid>/notify.sock.
+ *
+ * Usage: scoutfs-notifyd <mountpoint>
+ *
+ * Design notes:
+ *
+ * - One process, one thread. A short blocking ioctl (100 ms) drives
+ * the loop; new clients and shutdown signals are handled between
+ * ioctl calls via epoll with timeout 0. No cross-thread state.
+ *
+ * - Socket type is SOCK_SEQPACKET so each broadcast is one atomic
+ * 64-byte message on the wire. Clients read exactly one
+ * struct scoutfs_ioctl_notify_event per recv().
+ *
+ * - Permissions: socket owner root, mode 0600. The listener directory
+ * /run/scoutfs/<fsid>/ is created with mode 0755.
+ *
+ * - Slow-client protection: sends use MSG_DONTWAIT | MSG_NOSIGNAL.
+ * Any send error (EAGAIN, EPIPE, ECONNRESET, ...) drops the client.
+ * The kmod-side ring absorbs the brief lag.
+ *
+ * - Exit cleanly on SIGTERM/SIGINT/SIGQUIT; systemd restarts on
+ * failure per the shipped unit file.
+ */
+
+#include <errno.h>
+#include <fcntl.h>
+#include <signal.h>
+#include <stdarg.h>
+#include <stdbool.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/epoll.h>
+#include <sys/ioctl.h>
+#include <sys/signalfd.h>
+#include <sys/socket.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <sys/un.h>
+#include <syslog.h>
+#include <unistd.h>
+#include <linux/types.h>
+
+#include "ioctl.h"
+
+#define PROGNAME "scoutfs-notifyd"
+
+#define RUN_DIR_FMT "/run/scoutfs/%llu"
+#define SOCK_PATH_FMT "/run/scoutfs/%llu/notify.sock"
+
+#define IOCTL_BATCH_EVENTS 64
+#define IOCTL_TIMEOUT_MS 100
+#define MAX_CLIENTS 128
+#define SOCK_BACKLOG 16
+
+static volatile sig_atomic_t g_shutdown;
+
+struct ctx {
+ int mnt_fd;
+ int listen_fd;
+ int epoll_fd;
+ int signal_fd;
+ char sock_path[128];
+ char run_dir[128];
+ int clients[MAX_CLIENTS];
+ int nclients;
+};
+
+static void logmsg(int prio, const char *fmt, ...)
+ __attribute__((format(printf, 2, 3)));
+
+static void logmsg(int prio, const char *fmt, ...)
+{
+ va_list ap;
+
+ va_start(ap, fmt);
+ vsyslog(prio, fmt, ap);
+ va_end(ap);
+
+ /* also to stderr when not yet daemonized (systemd captures it) */
+ va_start(ap, fmt);
+ vfprintf(stderr, fmt, ap);
+ fputc('\n', stderr);
+ va_end(ap);
+}
+
+/*
+ * Open the mountpoint and resolve its fsid via SCOUTFS_IOC_STATFS_MORE.
+ * The fsid is unique per scoutfs mount and forms the per-mount socket
+ * path component.
+ */
+static int get_fsid(const char *mountpoint, uint64_t *fsid_out, int *mnt_fd_out)
+{
+ struct scoutfs_ioctl_statfs_more sm = { 0 };
+ int fd;
+
+ fd = open(mountpoint, O_RDONLY | O_DIRECTORY);
+ if (fd < 0) {
+ logmsg(LOG_ERR, "open(%s): %s", mountpoint, strerror(errno));
+ return -1;
+ }
+
+ if (ioctl(fd, SCOUTFS_IOC_STATFS_MORE, &sm) < 0) {
+ logmsg(LOG_ERR, "SCOUTFS_IOC_STATFS_MORE on %s: %s (is this a scoutfs mount?)",
+ mountpoint, strerror(errno));
+ close(fd);
+ return -1;
+ }
+
+ *fsid_out = sm.fsid;
+ *mnt_fd_out = fd;
+ return 0;
+}
+
+/*
+ * mkdir -p for a single-level run dir. We only have to create the
+ * leaf; systemd (or the shipped tmpfiles.d) creates /run/scoutfs.
+ */
+static int ensure_run_dir(const char *dir)
+{
+ struct stat st;
+
+ if (mkdir("/run/scoutfs", 0755) < 0 && errno != EEXIST) {
+ logmsg(LOG_ERR, "mkdir /run/scoutfs: %s", strerror(errno));
+ return -1;
+ }
+ if (mkdir(dir, 0755) < 0 && errno != EEXIST) {
+ logmsg(LOG_ERR, "mkdir %s: %s", dir, strerror(errno));
+ return -1;
+ }
+ if (stat(dir, &st) < 0 || !S_ISDIR(st.st_mode)) {
+ logmsg(LOG_ERR, "%s: not a directory", dir);
+ return -1;
+ }
+ return 0;
+}
+
+static int create_listener(const char *path)
+{
+ struct sockaddr_un addr = { .sun_family = AF_UNIX };
+ int fd;
+
+ if (strlen(path) >= sizeof(addr.sun_path)) {
+ logmsg(LOG_ERR, "socket path too long: %s", path);
+ return -1;
+ }
+ strncpy(addr.sun_path, path, sizeof(addr.sun_path) - 1);
+
+ fd = socket(AF_UNIX, SOCK_SEQPACKET | SOCK_CLOEXEC | SOCK_NONBLOCK, 0);
+ if (fd < 0) {
+ logmsg(LOG_ERR, "socket(): %s", strerror(errno));
+ return -1;
+ }
+
+ /* Remove any stale socket left by a previous instance. */
+ (void)unlink(path);
+
+ if (bind(fd, (struct sockaddr *)&addr, sizeof(addr)) < 0) {
+ logmsg(LOG_ERR, "bind(%s): %s", path, strerror(errno));
+ close(fd);
+ return -1;
+ }
+
+ /* Root-only access. chmod after bind so there's no brief window. */
+ if (chmod(path, 0600) < 0) {
+ logmsg(LOG_ERR, "chmod(%s, 0600): %s", path, strerror(errno));
+ close(fd);
+ unlink(path);
+ return -1;
+ }
+
+ if (listen(fd, SOCK_BACKLOG) < 0) {
+ logmsg(LOG_ERR, "listen(): %s", strerror(errno));
+ close(fd);
+ unlink(path);
+ return -1;
+ }
+
+ return fd;
+}
+
+static int setup_signalfd(void)
+{
+ sigset_t mask;
+ int fd;
+
+ sigemptyset(&mask);
+ sigaddset(&mask, SIGTERM);
+ sigaddset(&mask, SIGINT);
+ sigaddset(&mask, SIGQUIT);
+
+ if (sigprocmask(SIG_BLOCK, &mask, NULL) < 0) {
+ logmsg(LOG_ERR, "sigprocmask: %s", strerror(errno));
+ return -1;
+ }
+
+ fd = signalfd(-1, &mask, SFD_CLOEXEC | SFD_NONBLOCK);
+ if (fd < 0)
+ logmsg(LOG_ERR, "signalfd: %s", strerror(errno));
+ return fd;
+}
+
+static int epoll_add(int epfd, int fd, uint32_t events, uint64_t tag)
+{
+ struct epoll_event ev = { .events = events, .data.u64 = tag };
+
+ return epoll_ctl(epfd, EPOLL_CTL_ADD, fd, &ev);
+}
+
+/*
+ * Tag encoding so epoll events route cleanly without a per-fd lookup.
+ * tag[63..56] = kind
+ * tag[ 7.. 0] = client index (when kind == KIND_CLIENT)
+ */
+#define KIND_LISTEN 1
+#define KIND_SIGNAL 2
+#define KIND_CLIENT 3
+
+#define TAG_KIND(t) ((int)((t) >> 56))
+#define TAG_INDEX(t) ((int)((t) & 0xff))
+#define TAG_MAKE(k, i) (((uint64_t)(k) << 56) | (uint32_t)(i))
+
+static void client_close(struct ctx *c, int idx)
+{
+ if (idx < 0 || idx >= c->nclients)
+ return;
+ (void)epoll_ctl(c->epoll_fd, EPOLL_CTL_DEL, c->clients[idx], NULL);
+ close(c->clients[idx]);
+ c->clients[idx] = c->clients[c->nclients - 1];
+ c->nclients--;
+ /* re-tag the swapped-in client to its new index */
+ if (idx < c->nclients) {
+ struct epoll_event ev = {
+ .events = EPOLLIN | EPOLLHUP | EPOLLERR,
+ .data.u64 = TAG_MAKE(KIND_CLIENT, idx),
+ };
+ (void)epoll_ctl(c->epoll_fd, EPOLL_CTL_MOD,
+ c->clients[idx], &ev);
+ }
+}
+
+static void client_accept(struct ctx *c)
+{
+ int fd;
+
+ for (;;) {
+ fd = accept4(c->listen_fd, NULL, NULL,
+ SOCK_CLOEXEC | SOCK_NONBLOCK);
+ if (fd < 0) {
+ if (errno == EAGAIN || errno == EWOULDBLOCK)
+ return;
+ logmsg(LOG_WARNING, "accept: %s", strerror(errno));
+ return;
+ }
+
+ if (c->nclients >= MAX_CLIENTS) {
+ logmsg(LOG_WARNING, "client limit (%d) reached, rejecting",
+ MAX_CLIENTS);
+ close(fd);
+ continue;
+ }
+
+ if (epoll_add(c->epoll_fd, fd,
+ EPOLLIN | EPOLLHUP | EPOLLERR,
+ TAG_MAKE(KIND_CLIENT, c->nclients)) < 0) {
+ logmsg(LOG_WARNING, "epoll_add client: %s",
+ strerror(errno));
+ close(fd);
+ continue;
+ }
+
+ c->clients[c->nclients++] = fd;
+ }
+}
+
+/*
+ * Send one event to all connected clients. Errors drop the client.
+ * Iterate backwards so client_close's swap-remove doesn't skip entries.
+ */
+static void broadcast_event(struct ctx *c,
+ const struct scoutfs_ioctl_notify_event *ev)
+{
+ int i;
+
+ for (i = c->nclients - 1; i >= 0; i--) {
+ ssize_t sent = send(c->clients[i], ev, sizeof(*ev),
+ MSG_DONTWAIT | MSG_NOSIGNAL);
+ if (sent == (ssize_t)sizeof(*ev))
+ continue;
+ /*
+ * SOCK_SEQPACKET is atomic per-message; partial send
+ * shouldn't happen, but treat anything less as a drop.
+ */
+ client_close(c, i);
+ }
+}
+
+/*
+ * Drain kernel events with a short timeout. A positive return means
+ * we should keep polling without delay; zero means go to epoll_wait
+ * with a longer sleep.
+ */
+static int drain_kernel(struct ctx *c)
+{
+ struct scoutfs_ioctl_notify_event events[IOCTL_BATCH_EVENTS];
+ struct scoutfs_ioctl_read_notify req = {
+ .events_ptr = (uint64_t)(uintptr_t)events,
+ .events_nr = IOCTL_BATCH_EVENTS,
+ .timeout_ms = IOCTL_TIMEOUT_MS,
+ .flags = 0,
+ };
+ int ret;
+ int i;
+
+ ret = ioctl(c->mnt_fd, SCOUTFS_IOC_READ_NOTIFY, &req);
+ if (ret < 0) {
+ if (errno == ETIMEDOUT || errno == EINTR)
+ return 0;
+ if (errno == EBUSY) {
+ logmsg(LOG_ERR,
+ "another reader is attached; backing off");
+ sleep(1);
+ return 0;
+ }
+ logmsg(LOG_ERR, "SCOUTFS_IOC_READ_NOTIFY: %s",
+ strerror(errno));
+ g_shutdown = 1;
+ return -1;
+ }
+
+ for (i = 0; i < ret; i++)
+ broadcast_event(c, &events[i]);
+
+ return ret;
+}
+
+static int handle_epoll(struct ctx *c, int timeout_ms)
+{
+ struct epoll_event evs[16];
+ int n;
+ int i;
+
+ n = epoll_wait(c->epoll_fd, evs,
+ (int)(sizeof(evs) / sizeof(evs[0])), timeout_ms);
+ if (n < 0) {
+ if (errno == EINTR)
+ return 0;
+ logmsg(LOG_ERR, "epoll_wait: %s", strerror(errno));
+ return -1;
+ }
+
+ for (i = 0; i < n; i++) {
+ uint64_t tag = evs[i].data.u64;
+ int kind = TAG_KIND(tag);
+
+ switch (kind) {
+ case KIND_LISTEN:
+ client_accept(c);
+ break;
+
+ case KIND_SIGNAL:
+ g_shutdown = 1;
+ return 0;
+
+ case KIND_CLIENT:
+ /* any readability / hangup → drop client */
+ client_close(c, TAG_INDEX(tag));
+ break;
+
+ default:
+ logmsg(LOG_WARNING, "unexpected epoll tag kind %d",
+ kind);
+ break;
+ }
+ }
+
+ return 0;
+}
+
+static void cleanup(struct ctx *c)
+{
+ int i;
+
+ for (i = 0; i < c->nclients; i++)
+ close(c->clients[i]);
+ c->nclients = 0;
+
+ if (c->listen_fd >= 0)
+ close(c->listen_fd);
+ if (c->sock_path[0])
+ unlink(c->sock_path);
+ if (c->signal_fd >= 0)
+ close(c->signal_fd);
+ if (c->epoll_fd >= 0)
+ close(c->epoll_fd);
+ if (c->mnt_fd >= 0)
+ close(c->mnt_fd);
+}
+
+static void usage(FILE *f)
+{
+ fprintf(f,
+ "usage: %s <mountpoint>\n"
+ "\n"
+ " Drains the scoutfs file access notification ring for the\n"
+ " given mount and broadcasts events to clients connected to\n"
+ " /run/scoutfs/<fsid>/notify.sock.\n"
+ "\n"
+ " Requires CAP_SYS_ADMIN. Socket is root-only (mode 0600).\n",
+ PROGNAME);
+}
+
+int main(int argc, char **argv)
+{
+ struct ctx c = {
+ .mnt_fd = -1,
+ .listen_fd = -1,
+ .epoll_fd = -1,
+ .signal_fd = -1,
+ };
+ uint64_t fsid;
+ int ret = EXIT_FAILURE;
+
+ openlog(PROGNAME, LOG_PID | LOG_PERROR, LOG_DAEMON);
+
+ if (argc != 2 || argv[1][0] == '-') {
+ usage(stderr);
+ goto out;
+ }
+
+ if (get_fsid(argv[1], &fsid, &c.mnt_fd) < 0)
+ goto out;
+
+ snprintf(c.run_dir, sizeof(c.run_dir),
+ RUN_DIR_FMT, (unsigned long long)fsid);
+ snprintf(c.sock_path, sizeof(c.sock_path),
+ SOCK_PATH_FMT, (unsigned long long)fsid);
+
+ if (ensure_run_dir(c.run_dir) < 0)
+ goto out;
+
+ c.listen_fd = create_listener(c.sock_path);
+ if (c.listen_fd < 0)
+ goto out;
+
+ c.signal_fd = setup_signalfd();
+ if (c.signal_fd < 0)
+ goto out;
+
+ c.epoll_fd = epoll_create1(EPOLL_CLOEXEC);
+ if (c.epoll_fd < 0) {
+ logmsg(LOG_ERR, "epoll_create1: %s", strerror(errno));
+ goto out;
+ }
+
+ if (epoll_add(c.epoll_fd, c.listen_fd, EPOLLIN,
+ TAG_MAKE(KIND_LISTEN, 0)) < 0 ||
+ epoll_add(c.epoll_fd, c.signal_fd, EPOLLIN,
+ TAG_MAKE(KIND_SIGNAL, 0)) < 0) {
+ logmsg(LOG_ERR, "epoll_ctl: %s", strerror(errno));
+ goto out;
+ }
+
+ logmsg(LOG_INFO, "listening on %s", c.sock_path);
+
+ while (!g_shutdown) {
+ /*
+ * The ioctl blocks up to IOCTL_TIMEOUT_MS when the ring
+ * is empty, so this loop naturally paces itself and the
+ * daemon is idle between events. Client / signal fds
+ * are drained non-blocking between ioctl calls.
+ */
+ if (drain_kernel(&c) < 0)
+ break;
+ if (handle_epoll(&c, 0) < 0)
+ break;
+ }
+
+ logmsg(LOG_INFO, "shutting down");
+ ret = EXIT_SUCCESS;
+
+out:
+ cleanup(&c);
+ closelog();
+ return ret;
+}
diff --git a/utils/notifyd/scoutfs-notifyd@.service b/utils/notifyd/scoutfs-notifyd@.service
new file mode 100644
index 0000000..2e87f17
--- /dev/null
+++ b/utils/notifyd/scoutfs-notifyd@.service
@@ -0,0 +1,40 @@
+[Unit]
+Description=ScoutFS file access notification daemon for %I
+Documentation=man:scoutfs-notifyd(8)
+# The mount must exist before we can open it and read its fsid.
+RequiresMountsFor=/%I
+
+[Service]
+Type=simple
+User=root
+Group=root
+UMask=0077
+# scoutfs-notifyd creates /run/scoutfs/<fsid>/ and binds the socket
+# inside it; the top-level /run/scoutfs is created here so the daemon
+# never has to.
+RuntimeDirectory=scoutfs
+RuntimeDirectoryMode=0755
+ExecStart=/usr/sbin/scoutfs-notifyd /%I
+Restart=on-failure
+RestartSec=5s
+StartLimitBurst=5
+
+# The daemon needs CAP_SYS_ADMIN to drive the read-notify ioctl.
+# Everything else can be stripped.
+CapabilityBoundingSet=CAP_SYS_ADMIN
+AmbientCapabilities=CAP_SYS_ADMIN
+NoNewPrivileges=yes
+PrivateTmp=yes
+ProtectSystem=strict
+ProtectHome=yes
+ProtectKernelTunables=yes
+ProtectKernelModules=yes
+ProtectControlGroups=yes
+RestrictAddressFamilies=AF_UNIX
+LockPersonality=yes
+MemoryDenyWriteExecute=yes
+RestrictRealtime=yes
+RestrictSUIDSGID=yes
+
+[Install]
+WantedBy=multi-user.target
diff --git a/utils/scoutfs-utils.spec.in b/utils/scoutfs-utils.spec.in
index fb24b81..0aa91f5 100644
--- a/utils/scoutfs-utils.spec.in
+++ b/utils/scoutfs-utils.spec.in
@@ -52,19 +52,23 @@ cp man/*.5.gz $RPM_BUILD_ROOT%{_mandir}/man5/.
cp man/*.7.gz $RPM_BUILD_ROOT%{_mandir}/man7/.
cp man/*.8.gz $RPM_BUILD_ROOT%{_mandir}/man8/.
install -m 755 -D src/scoutfs $RPM_BUILD_ROOT%{_sbindir}/scoutfs
+install -m 755 -D notifyd/scoutfs-notifyd $RPM_BUILD_ROOT%{_sbindir}/scoutfs-notifyd
install -m 644 -D src/ioctl.h $RPM_BUILD_ROOT%{_includedir}/scoutfs/ioctl.h
install -m 644 -D src/format.h $RPM_BUILD_ROOT%{_includedir}/scoutfs/format.h
install -m 755 -D fenced/scoutfs-fenced $RPM_BUILD_ROOT%{_libexecdir}/scoutfs-fenced/scoutfs-fenced
install -m 644 -D fenced/scoutfs-fenced.service $RPM_BUILD_ROOT%{_unitdir}/scoutfs-fenced.service
install -m 644 -D fenced/scoutfs-fenced.conf.example $RPM_BUILD_ROOT%{_sysconfdir}/scoutfs/scoutfs-fenced.conf.example
+install -m 644 -D notifyd/scoutfs-notifyd@.service $RPM_BUILD_ROOT%{_unitdir}/scoutfs-notifyd@.service
%files
%defattr(644,root,root,755)
%{_mandir}/man*/scoutfs*.gz
/%{_unitdir}/scoutfs-fenced.service
+/%{_unitdir}/scoutfs-notifyd@.service
%{_sysconfdir}/scoutfs
%defattr(755,root,root,755)
%{_sbindir}/scoutfs
+%{_sbindir}/scoutfs-notifyd
%{_libexecdir}/scoutfs-fenced
%files -n scoutfs-devel
--
2.49.0.windows.1