ValueCache re-emitted a known value as an edition on a higher seq alone,
while OpValueCache requires the content to differ as well. Since
Value::operator== ignores seq, identical content re-announced with a
bumped seq was re-emitted as an add: a source-counting OpValueCache
then counted the same node twice and kept the value after the node
expired or left the search. Apply the same rule at both levels.
Value::operator== ignores seq, so a node re-announcing identical content
with a bumped seq is the same value refreshed, not an edition. ValueCache
must not re-emit it: a source-counting OpValueCache above would count the
node twice and keep the value after the node is gone.
onValuesAdded ignores a different payload that does not carry a higher
seq (e.g. the same signed content re-signed with a randomized
signature), but onValuesExpired only skipped strictly older versions.
The uncounted source could therefore decrement the refcount and expire
a value still held by others. Apply the same rule on both sides.
A source reporting a different payload with the same seq is ignored on
add and not counted; its later expiration must not release the value
still held by the counted sources.
~DhtProxyServer destroyed requests_ whose ~http::Request notifies DONE;
the handler registered by sendPushNotification then erased from the map
being destroyed, corrupting the heap (double free under ASan) whenever a
push request was still pending, e.g. an unreachable push server. Detach
the state-change callbacks and release the requests before the members
are destroyed.
Storage::store reported edited_values=1 whenever the stored Sp<Value>
differed from the incoming one, even with equal content. A local put()
of an already-stored value was therefore announced to local listeners
as a new value a second time, unbalancing listen caches that count how
many sources hold a value. Only report an edition when the content
actually changed; the expiration is still refreshed.
SearchCache::get called the GetCallback unconditionally; Dht::query
passes none, which threw bad_function_call as soon as a listen cache
existed for the key. Return false without a callback so the network
path runs. A callback returning false means the consumer has enough
values, not a failure: report done(true) as the network path does.
DhtProxyClient push refresh expired every previously cached value once
the refresh get completed, including values that were just re-delivered
or edited, producing phantom expire callbacks. Track the ids received by
the get and only expire cached values missing from it, and skip the
reconciliation when the get failed so a transient error does not expire
values still present on the DHT.
ValueCache emitted any content change of a known value as an edition
while OpValueCache only accepts a higher seq. A node reporting an older
version made ValueCache replace its copy and re-emit versions that were
then dropped or, with source counting, double counted downstream. Use
the same rule at both levels.
OpValueCache counted every re-delivery of an identical value as a new
source, but the proxy client is a single logical source that re-sends
values on push refresh or listen restart: the refcount grew unbounded
and expirations were never propagated. Add a countSources flag to
OpValueCache, OpCache and SearchCache; the proxy client disables it.
When a value is edited (same id, higher seq), only the reporting source
is known to hold the new version: replace it in place, reset the
refcount to 1 and emit a single add. Ignore older versions from lagging
sources on add, and ignore their expiration so they cannot release a
newer version still held by others.
ValueCache emits an edited value as an add instead of silently
refreshing its old copy, and applies a grace period of one type
expiration while the listen is synced so that a single lost refresh
notification does not drop a value the remote node still holds.
SearchCache::get only answers from the cache once the listen is synced,
so a get never returns a partial view that the network would complete.
Add the single-source OpCacheTester cases enabled by countSources.
OpCacheTester and ValueCacheTester exercise the listen caches in
isolation: add/edit/expire flows, per-source refcounting, stale
versions, sync status, SearchCache::get and node drop.
DhtRunnerTester and DhtProxyTester gain end-to-end listen tests for
value edition, duplicate puts, multi-source and partial expiration,
get after a synced listen, and the push refresh/expire paths.
The proxy fixture joins the proxy DhtRunner before destroying the
DhtProxyServer so no DHT callback can reach a destroyed server, and
push notifications are never awaited while holding the mutex taken by
listen callbacks: the result is produced on the DHT thread.
Register the new executables in CMake and Meson.
asio sockets are not thread-safe. http::Connection::close() used to close
the socket synchronously on the calling thread. Request::cancel() and
Request::~Request() (via terminate()) are called from arbitrary threads
(e.g. DhtProxyClient::stop(), cancelAllListeners(), cancelListen()),
so the close raced with in-flight operations on the io_context thread.
In particular, asio::async_connect's range_connect_op closes and
re-opens the socket when trying the next endpoint (common on mobile
where the IPv6 endpoint fails first). A concurrent close() from another
thread corrupts the epoll reactor's per-descriptor state, leading to
crashes in epoll_reactor::deregister_descriptor / register_descriptor
called from range_connect_op::process.
Dispatch the close to the io_context thread when not already on it.
Also keep the connection alive during the TLS handshake, like every
other asynchronous operation, so that the destructor can never run
while an operation is pending.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Track subscribe/permanent-put expiration with std::chrono::system_clock
(wall clock) instead of steady_clock in both the proxy client and server,
so persisted expirations survive restarts and map directly to epoch time.
- DhtProxyServer::clock is now system_clock; expireTimer/expireNotifyTimer
use asio::system_timer.
- DhtProxyClient::refreshSubscriberTimer uses asio::system_timer and
system_clock::now() for subscriber refresh.
to_system_time computed its saturation bounds by subtracting a
steady_clock duration from system_clock::time_point::max(). When the
two clocks have different periods (microseconds for system_clock and
nanoseconds for steady_clock on libc++), the common type promotes the
bound to nanoseconds and overflows, so every future time point was
converted to system_clock::time_point::max().
Cast the delta to the target duration before comparing against the
bounds, and saturate from_system_time before the reverse cast for
symmetry.
An error array was read at index 0 without checking its size, so a packet
carrying "e":[] dereferenced a null element pointer and crashed the process
on unauthenticated UDP input.
When a message had an error array but no "a", "r" or "u" map, the same object
was then walked through the map view of the msgpack union. A two-element array
appeared to hold two key/value pairs, reading past its allocation. A request
field that was not a map was never type-checked either.
Validate the error array size, restrict request parsing to "a", "r" and "u",
and require those to be maps. Add parser regressions for the empty error
array, the error-only message and the non-map request; the first one
segfaults without this change.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Key partial messages by source and transaction ID, buffer bounded fragments received before their header, and avoid charging reordered requests twice against rate limits. Cover source collisions, early fragments, empty data and large values.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Reject field-compressed responses with no selected fields before computing the value count, avoiding a division by zero on malformed network input. Add a parser regression test.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Dht::onDisconnected() only starts a bootstrap when bootstrapJob is
unset, reading a set handle as "a bootstrap is already scheduled".
That reading is wrong: Scheduler::run() erases the job from its timers
and calls it, but never clears the shared handle the caller holds, and
Scheduler::cancel() leaves the handle alone too when the job is no
longer in the timers. The handle therefore stays set for as long as the
Dht lives, once a bootstrap job has run.
That is harmless while bootstrap() reschedules itself, which it does as
long as the node is disconnected. It is not harmless on the early
return: a job that fires while a ping is still in flight, or while a
dubious contact keeps the status at Connecting, returns without
rescheduling. From then on the chain is dead and onDisconnected() will
not restart it, so a node that goes on to lose every contact stays
disconnected for good. Only an explicit addBootstrap() or a
connectivityChanged() can revive it, and a node whose network simply
went away gets neither.
Clearing the handle on the early return restores the meaning
onDisconnected() relies on.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
gnutls_pubkey_export reports the size of the exported PEM including its
terminating null byte, and PublicKey::toString hands that size straight
back to the caller. The returned string therefore ends with a null byte
that is not part of the PEM text.
Certificate::toString does not have the problem because
gnutls_x509_crt_export excludes the terminator, so the two exports
disagree on what a PEM string is.
The stray byte travels: Value::toJson stores owner->toString() in the
"owner" field, so a value published through the proxy carries a null
byte inside a JSON string, which is valid JSON but a surprise for any
consumer that treats the field as text.
Resize to the reported size and drop the terminator when it is there.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
RevocationList::sign() always overwrote the CRL Number extension: random
on the first signature, incremented afterwards. This leaves no way for a
certificate authority replicated over several devices to keep numbers
unique, since every device branching from the same list mints the same
number for a different content.
Add an optional `number` parameter, used verbatim when non-zero, so
callers can partition the number space (RFC 5280 5.2.3 requires
monotonicity for a given CRL scope and issuer, not globally).
Also fix the default increment path: the extension holds a DER integer
with leading zeros stripped, so reading it straight into an uint64_t only
worked for numbers wide enough to occupy all 8 bytes. Signing a list
numbered 5 produced 0x0500000000000001 instead of 6. Read and write the
number as a big-endian byte string instead, which makes the `endian()`
helper unnecessary.
Add tests covering the default numbering, explicit numbers (including
values with leading zero bytes and the largest positive 8-byte integer),
pack/unpack round-trips, and a device-partitioned numbering scheme.
In SearchCache::expire, if there is an op whose expiration is equal to
`now`, then it's not processed and the function returns `now` as the
next expiration timepoint. This can lead to the opExpirationJob in
Search::cancelListen getting rescheduled at the same time in an infinite
loop.
Since 15eb221c the node's bound port is persisted alongside the DHT
state and reused on the next run for a stable identity. If that port
is still held by another process - typically a fast application
restart racing the previous instance's shutdown - UdpSocket
constructor throws (both the IPv4 and IPv6 binds fail) and
DhtRunner::run() propagates the exception: the node never starts.
In Jami this left every account stuck in the connecting state after a
quick restart, with 'DhtException occurred: Can't bind socket' logged
per account, and no retry.
The persisted port is only a best-effort hint: when binding it fails,
retry with OS-assigned ports instead of throwing. The persisted file
is then refreshed with the newly bound ports as before.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Cover starting a node whose persisted port (persist_path + "_port.txt")
is already bound by another node, as happens when an application is
restarted while the previous instance still holds the socket. The node
is expected to start on a fallback port instead of failing.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The pushType field (593e59e0) lets the DHT proxy server tag push
notifications with the connection type ("pt" in the FCM payload), so
mobile clients can tell incoming calls apart from background noise.
It was only serialized in msgpack. Clients using DhtProxyClient
send their puts as JSON (Value::toJson()), where the field was
silently dropped, and the proxy server parsed the incoming JSON with
Value(const Json::Value&), which never read it back. As a result any
value put through a proxy (e.g. a call request from a mobile device)
reached the recipient's device with an empty "pt", was classified as
noise, and did not wake up the callee: calls from proxied devices
never rang, while the same call from a full DHT node (desktop) did.
Serialize pushType as "pt" in Value::toJson() when set, and parse it
back in the JSON constructor, mirroring the msgpack behavior.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Cover Value::toJson() / Value(const Json::Value&) preservation of the
pushType field, and its absence from the JSON when empty. This is the
serialization path used by DhtProxyClient::put(), so mobile clients
behind a DHT proxy depend on it to propagate the push notification
type.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adrien Beraud
Cyrille BéraudandCopilot
dependabot[bot]
Cyrille
François-Simon Fauteux-Chapleau