Compare commits

...
175 Commits
Author SHA1 Message Date
Adrien Beraud fdf5030694 proxy client: always resubscribe on connectivity change 2026-09-03 14:24:34 -04:00
Adrien Beraud 6868d34738 proxy: increase OP_MARGIN to 8 hours 2026-09-03 14:24:34 -04:00
Adrien Beraud 7337cc459c proxy: use system_clock for subscriber expiration timeouts
Track subscribe/permanent-put expiration with std::chrono::system_clock
(wall clock) instead of steady_clock in both the proxy client and server,
so persisted expirations survive restarts and map directly to epoch time.

- DhtProxyServer::clock is now system_clock; expireTimer/expireNotifyTimer
  use asio::system_timer.
- DhtProxyClient::refreshSubscriberTimer uses asio::system_timer and
  system_clock::now() for subscriber refresh.
2026-09-03 14:05:49 -04:00
Adrien Beraud 8baf53d078 utils: fix clock conversion overflow with mismatched durations
to_system_time computed its saturation bounds by subtracting a
steady_clock duration from system_clock::time_point::max(). When the
two clocks have different periods (microseconds for system_clock and
nanoseconds for steady_clock on libc++), the common type promotes the
bound to nanoseconds and overflows, so every future time point was
converted to system_clock::time_point::max().

Cast the delta to the target duration before comparing against the
bounds, and saturate from_system_time before the reverse cast for
symmetry.
2026-09-01 22:56:16 -04:00
Adrien Beraud 82ab21dc3f crypto: support explicit EC curve generation 2026-09-01 21:34:36 -04:00
Adrien Beraud d676aef027 http: support legacy X509 lookup signature 2026-08-06 14:05:48 -04:00
Adrien Beraud a2d9fdf334 Recompute runner wakeup after import schedules value expirations. 2026-08-06 13:59:48 -04:00
Adrien Beraud 0cec4135c0 update to 4.3.0 2026-08-05 16:35:01 -04:00
Adrien Beraud 6f76686a3e ci: update to Debian 13 2026-08-05 16:19:46 -04:00
Adrien Beraud c211b3c376 dht: persist values with system time 2026-08-05 16:14:15 -04:00
Adrien Beraud 0179b6ecc7 test: cover system time state persistence 2026-08-05 16:14:15 -04:00
Adrien Beraud 44e1ad839e net: harden fragmented reply completion 2026-08-05 14:29:03 -04:00
Adrien Beraud dddd97daff tests: cover partial reply hardening 2026-08-05 14:29:03 -04:00
Adrien Beraud 632bec459e net: avoid partial key allocation for regular messages 2026-08-05 14:29:03 -04:00
Cyrille BéraudandCopilot 910ca10a5f net: reject malformed error and request fields
An error array was read at index 0 without checking its size, so a packet
carrying "e":[] dereferenced a null element pointer and crashed the process
on unauthenticated UDP input.

When a message had an error array but no "a", "r" or "u" map, the same object
was then walked through the map view of the msgpack union. A two-element array
appeared to hold two key/value pairs, reading past its allocation. A request
field that was not a map was never type-checked either.

Validate the error array size, restrict request parsing to "a", "r" and "u",
and require those to be maps. Add parser regressions for the empty error
array, the error-only message and the non-map request; the first one
segfaults without this change.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-08-05 14:29:03 -04:00
Cyrille BéraudandCopilot 0826822a4d net: harden fragmented message reassembly
Key partial messages by source and transaction ID, buffer bounded fragments received before their header, and avoid charging reordered requests twice against rate limits. Cover source collisions, early fragments, empty data and large values.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-08-05 14:29:03 -04:00
Cyrille BéraudandCopilot 8aa6b009c3 net: reject empty field selections
Reject field-compressed responses with no selected fields before computing the value count, avoiding a division by zero on malformed network input. Add a parser regression test.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-08-05 14:29:03 -04:00
Cyrille BéraudandCopilot 2caa21d65f dht: let a node that lost the network try to rejoin it
Dht::onDisconnected() only starts a bootstrap when bootstrapJob is
unset, reading a set handle as "a bootstrap is already scheduled".
That reading is wrong: Scheduler::run() erases the job from its timers
and calls it, but never clears the shared handle the caller holds, and
Scheduler::cancel() leaves the handle alone too when the job is no
longer in the timers. The handle therefore stays set for as long as the
Dht lives, once a bootstrap job has run.

That is harmless while bootstrap() reschedules itself, which it does as
long as the node is disconnected. It is not harmless on the early
return: a job that fires while a ping is still in flight, or while a
dubious contact keeps the status at Connecting, returns without
rescheduling. From then on the chain is dead and onDisconnected() will
not restart it, so a node that goes on to lose every contact stays
disconnected for good. Only an explicit addBootstrap() or a
connectivityChanged() can revive it, and a node whose network simply
went away gets neither.

Clearing the handle on the early return restores the meaning
onDisconnected() relies on.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-08-03 11:34:25 -04:00
Adrien Beraud cf64fa454c crypto: encode certificate serials in network order 2026-08-02 17:24:55 -04:00
Adrien Beraud ef9ef12195 tests: cover positive certificate serials 2026-08-02 17:24:55 -04:00
Cyrille BéraudandCopilot 74a1918a4d crypto: strip the terminating null byte from an exported public key
gnutls_pubkey_export reports the size of the exported PEM including its
terminating null byte, and PublicKey::toString hands that size straight
back to the caller. The returned string therefore ends with a null byte
that is not part of the PEM text.

Certificate::toString does not have the problem because
gnutls_x509_crt_export excludes the terminator, so the two exports
disagree on what a PEM string is.

The stray byte travels: Value::toJson stores owner->toString() in the
"owner" field, so a value published through the proxy carries a null
byte inside a JSON string, which is valid JSON but a surprise for any
consumer that treats the field as text.

Resize to the reported size and drop the terminator when it is there.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-08-02 17:20:04 -04:00
dependabot[bot] 56a60fb557 build(deps): bump pypa/cibuildwheel from 4.1.0 to 4.1.1
Bumps [pypa/cibuildwheel](https://github.com/pypa/cibuildwheel) from 4.1.0 to 4.1.1.
- [Release notes](https://github.com/pypa/cibuildwheel/releases)
- [Changelog](https://github.com/pypa/cibuildwheel/blob/main/docs/changelog.md)
- [Commits](https://github.com/pypa/cibuildwheel/compare/v4.1.0...v4.1.1)

---
updated-dependencies:
- dependency-name: pypa/cibuildwheel
  dependency-version: 4.1.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-07-30 10:17:04 -04:00
Adrien Beraud f98df1d615 crypto: allow signing a revocation list with an explicit CRL number
RevocationList::sign() always overwrote the CRL Number extension: random
on the first signature, incremented afterwards. This leaves no way for a
certificate authority replicated over several devices to keep numbers
unique, since every device branching from the same list mints the same
number for a different content.

Add an optional `number` parameter, used verbatim when non-zero, so
callers can partition the number space (RFC 5280 5.2.3 requires
monotonicity for a given CRL scope and issuer, not globally).

Also fix the default increment path: the extension holds a DER integer
with leading zeros stripped, so reading it straight into an uint64_t only
worked for numbers wide enough to occupy all 8 bytes. Signing a list
numbered 5 produced 0x0500000000000001 instead of 6. Read and write the
number as a big-endian byte string instead, which makes the `endian()`
helper unnecessary.

Add tests covering the default numbering, explicit numbers (including
values with leading zero bytes and the largest positive 8-byte integer),
pack/unpack round-trips, and a device-partitioned numbering scheme.
2026-07-29 13:26:23 -04:00
Adrien Beraud 8f779f2efb http: fix build on OpenSSL 4.0 2026-07-20 11:25:02 -04:00
Adrien Beraud 699073acd8 update version to 4.2.0 2026-07-17 14:09:54 -04:00
Cyrille 02d49b9f6a http: run completed resolver callbacks on io executor 2026-07-15 14:07:14 -04:00
Adrien Beraud 85b0bc52c8 test: reproduce completed resolver callback race 2026-07-15 14:07:14 -04:00
Adrien Beraud af02c3a117 ci: install brew packages for macOS 15 2026-07-15 12:16:28 -04:00
François-Simon Fauteux-Chapleau 74b99453ce op_cache: fix boundary condition
In SearchCache::expire, if there is an op whose expiration is equal to
`now`, then it's not processed and the function returns `now` as the
next expiration timepoint. This can lead to the opExpirationJob in
Search::cancelListen getting rescheduled at the same time in an infinite
loop.
2026-07-13 14:38:27 -04:00
Adrien Beraud cba3d0e18f update version to 4.1.1 2026-07-06 15:50:00 -04:00
Adrien Beraud f9332985f5 thread_pool: only include ciso646 on C++<20 2026-07-06 15:45:01 -04:00
Cyrille BéraudandCopilot 513d0eb803 dhtrunner: fall back to random port when saved port is taken
Since 15eb221c the node's bound port is persisted alongside the DHT
state and reused on the next run for a stable identity. If that port
is still held by another process - typically a fast application
restart racing the previous instance's shutdown - UdpSocket
constructor throws (both the IPv4 and IPv6 binds fail) and
DhtRunner::run() propagates the exception: the node never starts.

In Jami this left every account stuck in the connecting state after a
quick restart, with 'DhtException occurred: Can't bind socket' logged
per account, and no retry.

The persisted port is only a best-effort hint: when binding it fails,
retry with OS-assigned ports instead of throwing. The persisted file
is then refreshed with the newly bound ports as before.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-07-06 15:44:47 -04:00
Cyrille BéraudandCopilot 139852a531 tests: dhtrunner: cover saved-port fallback
Cover starting a node whose persisted port (persist_path + "_port.txt")
is already bound by another node, as happens when an application is
restarted while the previous instance still holds the socket. The node
is expected to start on a fallback port instead of failing.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-07-06 15:44:47 -04:00
Cyrille BéraudandCopilot 3a73ad3aca value: serialize pushType in JSON
The pushType field (593e59e0) lets the DHT proxy server tag push
notifications with the connection type ("pt" in the FCM payload), so
mobile clients can tell incoming calls apart from background noise.

It was only serialized in msgpack. Clients using DhtProxyClient
send their puts as JSON (Value::toJson()), where the field was
silently dropped, and the proxy server parsed the incoming JSON with
Value(const Json::Value&), which never read it back. As a result any
value put through a proxy (e.g. a call request from a mobile device)
reached the recipient's device with an empty "pt", was classified as
noise, and did not wake up the callee: calls from proxied devices
never rang, while the same call from a full DHT node (desktop) did.

Serialize pushType as "pt" in Value::toJson() when set, and parse it
back in the JSON constructor, mirroring the msgpack behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-07-04 11:57:49 -04:00
Cyrille BéraudandCopilot 9031cf06c4 tests: value: add pushType JSON round-trip test
Cover Value::toJson() / Value(const Json::Value&) preservation of the
pushType field, and its absence from the JSON when empty. This is the
serialization path used by DhtProxyClient::put(), so mobile clients
behind a DHT proxy depend on it to propagate the push notification
type.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-07-04 11:57:49 -04:00
dependabot[bot] 7b1369931e build(deps): bump actions/cache from 5 to 6
Bumps [actions/cache](https://github.com/actions/cache) from 5 to 6.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/v5...v6)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-29 19:15:42 -04:00
dependabot[bot] 64364602d7 build(deps): bump pypa/cibuildwheel from 3.4.1 to 4.1.0
Bumps [pypa/cibuildwheel](https://github.com/pypa/cibuildwheel) from 3.4.1 to 4.1.0.
- [Release notes](https://github.com/pypa/cibuildwheel/releases)
- [Changelog](https://github.com/pypa/cibuildwheel/blob/main/docs/changelog.md)
- [Commits](https://github.com/pypa/cibuildwheel/compare/v3.4.1...v4.1.0)

---
updated-dependencies:
- dependency-name: pypa/cibuildwheel
  dependency-version: 4.1.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-22 10:56:34 -04:00
dependabot[bot] 2dbec3df97 build(deps): bump actions/checkout from 6 to 7
Bumps [actions/checkout](https://github.com/actions/checkout) from 6 to 7.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v6...v7)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-22 10:53:41 -04:00
Adrien Beraud 507e2de3c4 update version to 4.1.0 2026-06-19 00:27:25 -04:00
dependabot[bot] c37abed715 build(deps): bump https://github.com/microsoft/vcpkg
Bumps [https://github.com/microsoft/vcpkg](https://github.com/microsoft/vcpkg) from HEAD to 2026.06.01. This release includes the previously tagged commit.
- [Release notes](https://github.com/microsoft/vcpkg/releases)
- [Commits](https://github.com/microsoft/vcpkg/compare/66c0373dc7fca549e5803087b9487edfe3aca0a1...f3e10653cc27d62a37a3763cd84b38bca07c6075)

---
updated-dependencies:
- dependency-name: https://github.com/microsoft/vcpkg
  dependency-version: 2026.06.01
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-19 00:12:50 -04:00
Adrien Beraud 0eb1c33ba5 ci: update vcpkg 2026-06-19 00:10:28 -04:00
Adrien Beraud 45918186a2 ci: update vcpkg 2026-06-19 00:08:20 -04:00
Adrien Beraud 29cb909228 build/meson: base64 requires simdutf 2026-06-18 23:52:24 -04:00
Adrien Beraud 9f6a03bf3d build/meson: require appropriate dependencies for proxy server 2026-06-18 20:34:45 -04:00
Adrien Beraud 732da0e478 default types: add ValueId to TrustRequest 2026-06-11 16:52:03 -04:00
Adrien Béraud 9a60329f66 proxy server: use token as request url 2026-06-03 20:00:34 -04:00
Adrien Béraud 0bb393ec36 http: allow to build Request by std::string url 2026-06-03 19:30:53 -04:00
vx32 947610003b http: allow to build Url from std::string, cleanup 2026-06-03 19:14:32 -04:00
Adrien Béraud dcbe158554 proxy server: allow url-encoded base64 for UnifiedPush 2026-06-03 19:12:40 -04:00
vx32 38e9a7fd61 base64: add support for url-encoded base64 2026-06-03 17:01:19 -04:00
François-Simon Fauteux-Chapleau 7cf3e8b738 thread_pool: notify when task queue becomes empty
This prevents ThreadPool::stop from hanging if it is called with `wait`
set to true (which is notably the case in ThreadPool::join) and the
pool's task queue is non-empty.
2026-05-21 10:17:08 -04:00
François-Simon Fauteux-Chapleau e924ef5620 tests: add regression test for hang in thread pool
The worker threads in a ThreadPool don't signal the pool's condition
variable when completing the last task in the queue. As a consequence,
ThreadPool::join() hangs if called while the task queue is non-empty.
2026-05-21 10:17:08 -04:00
Adrien Béraud d96115b445 docs: update manpage for dhtnode 2026-05-18 10:50:23 -04:00
Adrien Béraud bb96d2d80e dhtchat: adapt for new API 2026-05-18 10:36:48 -04:00
Adrien Béraud 8b96652b32 dhtrunner: add signatures for putEncrypted(InfoHash, PkId to, Value&&) 2026-05-18 10:36:48 -04:00
Adrien Béraud d1235660d5 crypto: add support to build with Nettle 4.0 2026-05-13 14:36:13 -04:00
Adrien Béraud 23c6b5547b python: update for API changes in 4.0 2026-05-13 14:18:08 -04:00
Adrien Béraud 61c4a3880f update version to 4.0.0 2026-05-11 09:28:57 -04:00
Adrien Béraud 44f2aaa876 remove deprecated APIs 2026-05-11 09:27:56 -04:00
Adrien Béraud a363d573e3 dht: fix persisted permanent values never expiring
When values are exported and re-imported, permanent values retain their
time_point::max() expiration but lose their permanent flag. This caused
them to be stored as non-permanent with an effectively infinite
expiration, so they would never expire.

Cap the expiration of non-permanent values to created + type.expiration,
ensuring that imported permanent values get a finite expiration matching
their value type.
2026-05-09 00:39:49 -04:00
Adrien Béraud 3ff9338980 tests: add test for permanent value import expiration
Add testImportPermanentValueExpires to verify that values originally
stored as permanent (with time_point::max() expiration) properly expire
after being exported and re-imported, since the permanent flag is lost
during the import process.
2026-05-09 00:39:49 -04:00
Adrien Béraud ea3c0f0d93 update version to 3.7.9 2026-05-02 20:40:25 -04:00
Adrien Béraud f0cf074576 BUILD.md: add llhttp dependency and remove CMAKE_INSTALL_PREFIX 2026-05-02 19:08:02 -04:00
Adrien Béraud 760586ed4a python: build for python 3.14 2026-05-02 19:06:25 -04:00
dependabot[bot] b951f4f40a build(deps): bump softprops/action-gh-release from 2 to 3
Bumps [softprops/action-gh-release](https://github.com/softprops/action-gh-release) from 2 to 3.
- [Release notes](https://github.com/softprops/action-gh-release/releases)
- [Changelog](https://github.com/softprops/action-gh-release/blob/master/CHANGELOG.md)
- [Commits](https://github.com/softprops/action-gh-release/compare/v2...v3)

---
updated-dependencies:
- dependency-name: softprops/action-gh-release
  dependency-version: '3'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-04-18 13:55:23 +10:00
dependabot[bot] 63cc650a72 build(deps): bump pypa/cibuildwheel from 3.4.0 to 3.4.1
Bumps [pypa/cibuildwheel](https://github.com/pypa/cibuildwheel) from 3.4.0 to 3.4.1.
- [Release notes](https://github.com/pypa/cibuildwheel/releases)
- [Changelog](https://github.com/pypa/cibuildwheel/blob/main/docs/changelog.md)
- [Commits](https://github.com/pypa/cibuildwheel/compare/v3.4.0...v3.4.1)

---
updated-dependencies:
- dependency-name: pypa/cibuildwheel
  dependency-version: 3.4.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-04-18 13:54:34 +10:00
Adrien Béraud c020d2e1ef update version to 3.7.8 2026-04-02 16:37:17 -04:00
copilot-swe-agent[bot]andaberaud aae1bb2ae5 infohash: fix unaligned memory access on strict-alignment architectures (sparc64)
Agent-Logs-Url: https://github.com/savoirfairelinux/opendht/sessions/d4106989-db2c-4d34-b1ea-4ca2ecab524a

Co-authored-by: aberaud <101827+aberaud@users.noreply.github.com>
2026-04-02 16:08:33 -04:00
Pavan Koushik Nellore 593e59e093 proxy: add pushType field and propagate through proxy notifications 2026-04-02 15:03:11 -04:00
dependabot[bot] 86d5065aca build(deps): bump microsoft/setup-msbuild from 2 to 3
Bumps [microsoft/setup-msbuild](https://github.com/microsoft/setup-msbuild) from 2 to 3.
- [Release notes](https://github.com/microsoft/setup-msbuild/releases)
- [Commits](https://github.com/microsoft/setup-msbuild/compare/v2...v3)

---
updated-dependencies:
- dependency-name: microsoft/setup-msbuild
  dependency-version: '3'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-03-25 16:25:14 -04:00
dependabot[bot] 308d8ff82b build(deps): bump actions/upload-artifact from 6 to 7
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 6 to 7.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v6...v7)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-03-19 18:32:26 -04:00
dependabot[bot] 34b9ddd023 build(deps): bump docker/build-push-action from 6 to 7
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 6 to 7.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](https://github.com/docker/build-push-action/compare/v6...v7)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-03-19 18:32:04 -04:00
dependabot[bot] 88cb14ad95 build(deps): bump actions/download-artifact from 7 to 8
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 7 to 8.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/v7...v8)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-version: '8'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-03-19 18:31:48 -04:00
Adrien Béraud e453a47242 update version to 3.7.7 2026-03-16 11:05:38 -04:00
Adrien Béraud 199090b959 network engine: log exceptions 2026-03-15 19:42:09 -04:00
Adrien Béraud d250e6a349 dhtrunner: cleanup 2026-03-15 10:43:46 -04:00
Adrien Béraud 99da0be2b7 misc: remove lock mutex template argument 2026-03-15 00:00:33 -04:00
Adrien Béraud 22d219c695 dht: close socket on auth error 2026-03-14 18:19:06 -04:00
Adrien Béraud ccd4865df5 dht: allow search table update with listen token refresh 2026-03-14 16:17:32 -04:00
Adrien Beraud 5afc452f32 build/cmake: set SIMDUTF_CXX_STANDARD to match CMAKE_CXX_STANDARD 2026-03-14 11:37:53 -04:00
Adrien Beraud a06596d365 tests: unlock mutex before join 2026-03-14 11:37:50 -04:00
Adrien Béraud aa260c7b40 tests: add testShutdownCompletesWithPendingPut 2026-03-14 11:37:43 -04:00
Adrien Béraud 7183847e44 tests: use random ports 2026-03-13 22:59:41 -04:00
Adrien Béraud 884cb09220 tests: bound dhtrunner async waits 2026-03-13 22:00:09 -04:00
Adrien Béraud 04edaafa57 dht: simplify bootstrap logic 2026-03-13 20:59:24 -04:00
Adrien Béraud 6be58884aa tests: set timout for test_crypto to 60s 2026-03-13 20:58:53 -04:00
Adrien Béraud d50d3d4f32 tests: set timout for test_dhtrunner to 60s 2026-03-13 20:52:56 -04:00
Adrien Béraud 8acafe844d dht: release pending bootstrap after first attempt 2026-03-13 20:39:40 -04:00
Adrien Béraud 688b713c09 tests: cover bootstrap put failure after disconnect 2026-03-13 20:39:37 -04:00
Adrien Béraud 017df59dde update version to 3.7.6 2026-03-13 18:30:43 -04:00
Adrien Béraud 964800a506 tests: disable test_networkengine on MSVC 2026-03-13 16:30:36 -04:00
Adrien Béraud 4f423d066f tests: attempt to fix test_networkengine on MSVC 2026-03-13 16:09:49 -04:00
Adrien Béraud ae49b96c2d tests: attempt to fix test_networkengine on MSVC 2026-03-13 16:02:33 -04:00
Adrien Béraud 3701b21425 tests: attempt to fix test_networkengine on MSVC 2026-03-13 15:49:52 -04:00
Adrien Béraud 55af602026 tests: fix build with GCC 14 2026-03-13 15:32:45 -04:00
Adrien Béraud eda121297b proxy client: protect searches when updating push token 2026-03-13 15:10:12 -04:00
Adrien Béraud 603e777c9e dht: refresh token on listen confirmation 2026-03-13 15:10:12 -04:00
Adrien Béraud bef4e61fc6 tests: add test for update of listen token 2026-03-13 15:10:12 -04:00
Adrien Béraud 42971d9587 tests: add test for proxy client token change 2026-03-13 15:10:12 -04:00
Adrien Béraud 4a43326de1 tests: add more proxy tests 2026-03-13 13:06:36 -04:00
Adrien Béraud 8fcb19627d dht: mark listen as non-synced on auth failure 2026-03-13 13:06:08 -04:00
Adrien Béraud b6ea2ed8e2 proxy client: resubscribe on new endpoint 2026-03-13 13:05:34 -04:00
Adrien Béraud ac520bd587 tests: make sure bootstrap then put always succeed 2026-03-13 12:03:09 -04:00
Adrien Béraud 4bd796cddd dht: state is connecting if bootstrap is pending 2026-03-13 11:53:32 -04:00
Adrien Béraud ad372bf232 dht: restart searches on connect 2026-03-13 11:52:57 -04:00
Adrien Béraud 299b4510f5 AGENTS.md: add commit instructions 2026-03-13 11:07:01 -04:00
Adrien Béraud ae44ac98fb dht: open new socket if previous was closed 2026-03-13 11:06:50 -04:00
Adrien Béraud c37ff1b167 tests: add test for reopens socket after node expiration 2026-03-13 11:06:30 -04:00
Adrien Béraud f94f072ad6 dht: send token in listen confirmation 2026-03-13 00:53:05 -04:00
Adrien Béraud c51beca374 dht: properly persist expiration time 2026-03-13 00:38:31 -04:00
Adrien Béraud 4b9a6489c3 dht: persist storage usage 2026-03-13 00:28:38 -04:00
Adrien Béraud 34722cab7a dht: clarify storage log 2026-03-13 00:04:49 -04:00
Adrien Béraud 9e1dc3f793 dht: log stats about imported values 2026-03-13 00:04:49 -04:00
Adrien Béraud 0eacd7fe6c tools: move to dht::tools namespace 2026-03-13 00:04:49 -04:00
Adrien Béraud 37f4469961 update version to 3.7.3 2026-03-09 16:08:26 -04:00
Adrien Béraud e9c4a31949 Support listing tests and running a single test by name 2026-03-09 13:57:56 -04:00
dependabot[bot] 1b5d9569ec build(deps): bump pypa/cibuildwheel from 3.3.1 to 3.4.0
Bumps [pypa/cibuildwheel](https://github.com/pypa/cibuildwheel) from 3.3.1 to 3.4.0.
- [Release notes](https://github.com/pypa/cibuildwheel/releases)
- [Changelog](https://github.com/pypa/cibuildwheel/blob/main/docs/changelog.md)
- [Commits](https://github.com/pypa/cibuildwheel/compare/v3.3.1...v3.4.0)

---
updated-dependencies:
- dependency-name: pypa/cibuildwheel
  dependency-version: 3.4.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-03-09 10:35:40 -04:00
dependabot[bot] e70f76f7f9 build(deps): bump docker/setup-buildx-action from 3 to 4
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 3 to 4.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](https://github.com/docker/setup-buildx-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-03-09 10:34:43 -04:00
dependabot[bot] 0f25da371e build(deps): bump docker/setup-qemu-action from 3 to 4
Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action) from 3 to 4.
- [Release notes](https://github.com/docker/setup-qemu-action/releases)
- [Commits](https://github.com/docker/setup-qemu-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: docker/setup-qemu-action
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-03-09 10:33:43 -04:00
dependabot[bot] f33f66c038 build(deps): bump docker/login-action from 3 to 4
Bumps [docker/login-action](https://github.com/docker/login-action) from 3 to 4.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](https://github.com/docker/login-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-03-09 10:33:06 -04:00
dependabot[bot] 1d08ff1489 build(deps): bump docker/metadata-action from 5 to 6
Bumps [docker/metadata-action](https://github.com/docker/metadata-action) from 5 to 6.
- [Release notes](https://github.com/docker/metadata-action/releases)
- [Commits](https://github.com/docker/metadata-action/compare/v5...v6)

---
updated-dependencies:
- dependency-name: docker/metadata-action
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-03-09 10:32:35 -04:00
Adrien Béraud b892be092a infohash: make core APIs constexpr
Make Hash default constructor, hex constructor, data(), const
operator[] and fromString constexpr. Change fromString to be
non-throwing: it now uses a validity flag and fills the hash with zeros
on parse error instead of throwing. Add a constexpr toHexArray helper
that returns a std::array<char, N*2> for compile-time hex conversion.
Update tests to include <string_view>, use std::string_view, add
constexpr InfoHash usage and a static_assert for the hex array.
2026-03-08 23:34:50 -04:00
Adrien Béraud 9a902fa80c dht: fix incorrect id in log 2026-03-08 12:27:00 -04:00
Adrien Béraud 984d720fac meson: export opendht_EXPORTS 2026-03-08 12:26:23 -04:00
Adrien Béraud 931d79b701 dht: handle out-of-order partial messages 2026-03-08 12:26:23 -04:00
Adrien Béraud 58a5791063 ci: fix ccpp.yml syntax 2026-03-07 14:52:14 -05:00
Adrien Béraud 33b2bf12fa dht: fix recycling of existing search 2026-03-06 23:19:51 -05:00
Adrien Béraud 08e9f0b763 network_engine: always erase partial message after processing 2026-03-06 23:19:51 -05:00
Adrien Béraud 12cb93f9cf AGENTS.md: update 2026-03-06 23:19:51 -05:00
Adrien Béraud 71d14bb0fe proxy stats: use localhost by default 2026-03-02 16:32:38 -05:00
Adrien Béraud 635a73141a build/cmake: use three dots as policy_max separator 2026-03-01 21:39:53 -05:00
Adrien Béraud de4327c056 proxy stats: add storage stats 2026-02-27 16:52:07 -05:00
Adrien Béraud 348d6be920 proxy stats: add storage stats 2026-02-27 16:00:16 -05:00
Adrien Béraud dbdf2ba94d update version to 3.7.2 2026-02-27 15:27:44 -05:00
Adrien Béraud ef7505baaa tests/dhtrunner: adapt to use longId 2026-02-27 15:12:31 -05:00
Adrien Béraud 9ad6b3a608 dhtrunner, securedht: mark findCertificate(InfoHash hash) as deprecated 2026-02-27 14:26:39 -05:00
Adrien Béraud a32d76d312 securedht: cleanup 2026-02-27 00:18:20 -05:00
Adrien Béraud 8ebecfa25a securedht: stop storing certificate at sha1(publicKey) 2026-02-27 00:12:40 -05:00
Adrien Béraud ae3c2ae0d3 build/meson: use c++20 by default 2026-02-26 22:53:06 -05:00
Adrien Béraud 808ef0c51e rename tests to test_x format 2026-02-26 22:52:48 -05:00
Adrien Béraud 6cf6892767 add AGENTS.md 2026-02-26 22:51:41 -05:00
Adrien Béraud 9ebe6e8bb6 tools: pretty print byte count 2026-02-24 00:15:47 -05:00
Adrien Béraud a5455c19a6 update version to 3.7.1 2026-02-22 23:34:54 -05:00
Adrien Béraud 166566dbf4 remove automake files 2026-02-22 23:34:16 -05:00
Adrien Béraud 9b1534e6fe update README.md 2026-02-22 21:42:24 -05:00
Adrien Béraud 0263104373 proxy client: implement getLocalStoreSize 2026-02-22 18:55:02 -05:00
Adrien Béraud 0fc7e451e6 dhtrunner: move getNodeInfo implementation to securedht 2026-02-22 17:16:28 -05:00
Adrien Béraud 85cbee0f1a dht: add local storage stats in NodeInfo 2026-02-22 16:27:35 -05:00
Adrien Béraud 724b5e592e dht: use independent storage quota for local values 2026-02-22 16:27:35 -05:00
Adrien Béraud 28001840fa dht: avoid infinite loop when largest quota empty
Ensure the selected store_quota entry is non-empty before entering the
expiry loop. If an empty entry is encountered, log a warning and break
out to prevent repeatedly attempting to expire from an empty quota
(which could hang the expiry routine).
2026-02-21 18:34:07 -05:00
Adrien Béraud 047c10cf85 dht: handle empty values when updating storage quota 2026-02-21 18:34:00 -05:00
Adrien Béraud 152f9a2f54 dht: take shared_ptr by const-ref in sortedBucketInsert 2026-02-19 22:02:32 -05:00
Adrien Béraud 62243fb4f6 query: optimize msgpack_pack 2026-02-19 21:50:07 -05:00
Adrien Béraud 01eab83a7b value: move filter 2026-02-19 21:44:43 -05:00
Adrien Béraud 2c71662af7 dht_proxy_client: prevent disconnect on local ip change 2026-02-09 15:05:43 -05:00
Adrien Béraud 64032eb6d2 utils: use fmt::format for getaddrinfo error
Use fmt::format to build the std::invalid_argument message in
SockAddr::resolve
instead of manual std::string concatenation when getaddrinfo fails.
2026-02-09 13:26:02 -05:00
Adrien Béraud d91f3bdbbd dhtrunner: implement getStorageLimit 2026-02-09 13:26:00 -05:00
Adrien Béraud 1abaddba3e ci: bump VCPKG_VERSION to 2026.01.16
Bump VCPKG_VERSION in .github/workflows/windows-build.yml from
2025.07.25 to 2026.01.16 so the Windows CI workflow uses the newer
vcpkg release.
2026-02-05 21:53:28 -05:00
Adrien Béraud 65b537f6e9 ci: test builds with C++17/20/23
Add a cxx_standard matrix (17, 20, 23) to the Ubuntu and macOS CI jobs
and pass
-DCMAKE_CXX_STANDARD=${{ matrix.cxx_standard }} and
-DCMAKE_CXX_STANDARD_REQUIRED=ON to CMake so the builds are exercised
across
C++17, C++20 and C++23.
2026-02-05 21:53:14 -05:00
Adrien Béraud f44646ff82 ci(windows): test builds with C++17/20/23 standards
Add a cxx_standard matrix axis to the Windows build jobs and pass
-DCMAKE_CXX_STANDARD and -DCMAKE_CXX_STANDARD_REQUIRED=ON to CMake so
builds are produced for C++17, C++20 and C++23. Include the chosen
standard in the uploaded artifact names for both shared and static
builds.
2026-02-05 21:34:44 -05:00
Adrien Béraud 73dc55155e utils: make unpackMsg take const Blob& and inline
Change the template unpackMsg to accept a const Blob& to avoid an
unnecessary copy of the Blob argument. Move the non-template unpackMsg
(returning msgpack::unpacked) into the header as an inline function and
remove its out-of-line definition from src/utils.cpp. Update
declarations accordingly.
2026-02-05 21:31:42 -05:00
Adrien Béraud aa000ee586 http: cleanup 2026-02-05 21:31:42 -05:00
Adrien Béraud ec4c3e343c dht: format request targets with fmt 2026-02-05 21:31:42 -05:00
copilot-swe-agent[bot]andaberaud 4b24f1da38 Replace deprecated systemd directives with current alternatives
Replace ReadOnlyDirectories with ReadOnlyPaths and ReadWriteDirectories with ReadWritePaths as per modern systemd.exec(5) documentation

Co-authored-by: aberaud <101827+aberaud@users.noreply.github.com>
2026-02-03 16:35:58 -05:00
dependabot[bot] c47048121b build(deps): bump https://github.com/microsoft/vcpkg
Bumps [https://github.com/microsoft/vcpkg](https://github.com/microsoft/vcpkg) from HEAD to 2026.01.16. This release includes the previously tagged commit.
- [Release notes](https://github.com/microsoft/vcpkg/releases)
- [Commits](https://github.com/microsoft/vcpkg/compare/dd3097e305afa53f7b4312371f62058d2e665320...66c0373dc7fca549e5803087b9487edfe3aca0a1)

---
updated-dependencies:
- dependency-name: https://github.com/microsoft/vcpkg
  dependency-version: 2026.01.16
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-02-02 09:33:51 -05:00
Adrien Béraud 638ffdc669 logger: add missing header 2026-01-29 22:55:47 -05:00
Adrien Béraud 40ffafa756 logger: use file and line number 2026-01-29 22:38:57 -05:00
Adrien Béraud e6836f076f logger: protect children with mutex 2026-01-29 22:38:31 -05:00
Adrien Béraud 03c1158d7e build: use /Zc:__cplusplus on MSVC 2026-01-27 16:09:07 -05:00
Adrien Béraud 0018e577d9 build: add simdutf to opendht.pc when downloaded 2026-01-27 15:07:15 -05:00
Adrien Béraud eb7d94ee79 Revert "build: add private link to simdutf static to pkgconfig"
This reverts commit ea6ca025fc.
2026-01-27 15:06:23 -05:00
Adrien Béraud ea6ca025fc build: add private link to simdutf static to pkgconfig 2026-01-27 14:45:07 -05:00
Adrien Béraud 1105a68f93 build: update simdutf to 8.0.0 2026-01-27 14:31:41 -05:00
Adrien Béraud 76ae1cd6f2 logger: make setFilter match substrings
Allow to filter based on partial tag match
2026-01-27 14:29:43 -05:00
Adrien Béraud c59c0ef64d log: add tag and filter mechanism 2026-01-26 17:31:37 -05:00
108 changed files with 6068 additions and 2681 deletions

No files matched your search

+20 -10
View File
@@ -7,16 +7,16 @@ env:
UBUNTU_FULL_DEPS: "libncurses5-dev libreadline-dev nettle-dev libasio-dev libgnutls28-dev libuv1-dev python3-pip python3-dev python3-setuptools libcppunit-dev libjsoncpp-dev libfmt-dev libmsgpack-dev libargon2-dev"
UBUNTU_MESON_DEPS: "ninja-build libncurses5-dev libreadline-dev nettle-dev libasio-dev libgnutls28-dev libuv1-dev python3-dev python3-setuptools python3-pip libcppunit-dev libjsoncpp-dev libfmt-dev libmsgpack-dev libargon2-dev"
UBUNTU_MINIMAL_DEPS: "libncurses5-dev libreadline-dev nettle-dev libfmt-dev libgnutls28-dev libcppunit-dev libmsgpack-dev libargon2-dev"
DEBIAN12_DEPS: "git libtool pkg-config wget cmake libncurses5-dev libreadline-dev nettle-dev libasio-dev libgnutls28-dev libuv1-dev python3-pip python3-dev python3-setuptools libcppunit-dev libjsoncpp-dev libfmt-dev libmsgpack-dev libargon2-dev"
DEBIAN13_DEPS: "git libtool pkg-config wget cmake libncurses5-dev libreadline-dev nettle-dev libasio-dev libgnutls28-dev libuv1-dev python3-pip python3-dev python3-setuptools libcppunit-dev libjsoncpp-dev libfmt-dev libmsgpack-dev libargon2-dev"
on: [push, pull_request]
jobs:
build-ubuntu-meson:
build-ubuntu-meson:
name: Ubuntu/GCC Meson build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
- name: Install Dependencies
run: |
@@ -36,8 +36,11 @@ jobs:
build-ubuntu-minimal:
name: Ubuntu/GCC minimal build
runs-on: ubuntu-latest
strategy:
matrix:
cxx_standard: [17, 20, 23]
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
- name: Install Minimal Dependencies
run: |
@@ -50,6 +53,8 @@ jobs:
mkdir build && cd build
cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local \
-DCMAKE_BUILD_TYPE=Debug \
-DCMAKE_CXX_STANDARD=${{ matrix.cxx_standard }} \
-DCMAKE_CXX_STANDARD_REQUIRED=ON \
-DOPENDHT_C=Off -DBUILD_TESTING=On \
-DOPENDHT_PEER_DISCOVERY=Off -DOPENDHT_PYTHON=Off \
-DOPENDHT_TOOLS=On -DOPENDHT_PROXY_SERVER=Off -DOPENDHT_PROXY_CLIENT=Off
@@ -60,12 +65,12 @@ jobs:
- name: Run Unit tests
run: ctest --output-on-failure --test-dir build
build-debian12:
name: Debian 12/GCC build
build-debian13:
name: Debian 13/GCC build
runs-on: ubuntu-latest
container:
image: debian:12
image: debian:13
volumes:
- .:/workspace
options: --workdir /workspace
@@ -73,12 +78,12 @@ jobs:
DEBIAN_FRONTEND: noninteractive
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
- name: Install Dependencies
run: |
apt-get update
apt-get install -y $DEBIAN12_DEPS
apt-get install -y $DEBIAN13_DEPS
- name: Build llhttp dependency
run: |
@@ -122,8 +127,11 @@ jobs:
build-macos:
name: macOS/Clang build
runs-on: macos-15
strategy:
matrix:
cxx_standard: [17, 20, 23]
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
- name: Install Homebrew Dependencies
run: brew install msgpack-cxx asio gnutls nettle readline fmt jsoncpp argon2 openssl cppunit
@@ -158,6 +166,8 @@ jobs:
cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local \
-DOPENDHT_DOWNLOAD_DEPS=ON \
-DCMAKE_BUILD_TYPE=Debug \
-DCMAKE_CXX_STANDARD=${{ matrix.cxx_standard }} \
-DCMAKE_CXX_STANDARD_REQUIRED=ON \
-DOPENDHT_C=On -DBUILD_TESTING=On \
-DOPENDHT_PEER_DISCOVERY=On -DOPENDHT_PYTHON=Off \
-DOPENDHT_TOOLS=On -DOPENDHT_PROXY_SERVER=On \
+1 -1
View File
@@ -6,7 +6,7 @@ jobs:
name: Clang Static Analysis
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
- name: deps
run: |
sudo apt update && \
+1 -1
View File
@@ -27,7 +27,7 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v7
- name: Initialize CodeQL
uses: github/codeql-action/init@v4
+9 -9
View File
@@ -16,18 +16,18 @@ jobs:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, ubuntu-24.04-arm, macos-latest]
os: [ubuntu-latest, ubuntu-24.04-arm, macos-15]
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
# Run locally with: pipx run cibuildwheel --platform linux python
- name: Build wheels
uses: pypa/cibuildwheel@v3.3.1
uses: pypa/cibuildwheel@v4.1.1
with:
package-dir: python
- uses: actions/upload-artifact@v6
- uses: actions/upload-artifact@v7
with:
name: opendht-wheels-${{ matrix.os }}
path: ./wheelhouse/*.whl
@@ -36,7 +36,7 @@ jobs:
name: Build wheels on Windows
runs-on: windows-latest
env:
VCPKG_VERSION: "2025.07.25"
VCPKG_VERSION: "2026.06.01"
VCPKG_INSTALLATION_ROOT: "C:\\vcpkg"
VCPKG_DISABLE_METRICS: "1"
VCPKG_DEFAULT_TRIPLET: x64-windows-release
@@ -44,11 +44,11 @@ jobs:
VCPKG_DEFAULT_BINARY_CACHE: ${{ github.workspace }}\vcpkg_binary_cache
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
- name: Cache vcpkg installed packages
id: cache-vcpkg
uses: actions/cache@v5
uses: actions/cache@v6
with:
path: |
${{ env.VCPKG_INSTALLED_DIR }}
@@ -86,7 +86,7 @@ jobs:
pipx run cibuildwheel python
shell: pwsh
- uses: actions/upload-artifact@v6
- uses: actions/upload-artifact@v7
with:
name: opendht-wheels-win_amd64
path: ./build/wheelhouse/*.whl
@@ -95,7 +95,7 @@ jobs:
runs-on: ubuntu-latest
needs: [build_wheels, build_wheels_windows]
steps:
- uses: actions/download-artifact@v7
- uses: actions/download-artifact@v8
with:
pattern: opendht-wheels-*
path: dist
+56 -56
View File
@@ -25,16 +25,16 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v7
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
uses: docker/setup-qemu-action@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
uses: docker/setup-buildx-action@v4
- name: Log in to the Container registry
uses: docker/login-action@v3
uses: docker/login-action@v4
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
@@ -42,12 +42,12 @@ jobs:
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v5
uses: docker/metadata-action@v6
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME_DEPS }}
- name: Build and push Docker image
uses: docker/build-push-action@v6
uses: docker/build-push-action@v7
with:
context: .
file: docker/DockerfileDeps
@@ -66,16 +66,16 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v7
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
uses: docker/setup-qemu-action@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
uses: docker/setup-buildx-action@v4
- name: Log in to the Container registry
uses: docker/login-action@v3
uses: docker/login-action@v4
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
@@ -83,13 +83,13 @@ jobs:
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v5
uses: docker/metadata-action@v6
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
- name: Build and push Docker image
uses: docker/build-push-action@v6
uses: docker/build-push-action@v7
with:
context: .
file: docker/Dockerfile
@@ -107,16 +107,16 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v7
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
uses: docker/setup-qemu-action@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
uses: docker/setup-buildx-action@v4
- name: Log in to the Container registry
uses: docker/login-action@v3
uses: docker/login-action@v4
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
@@ -124,12 +124,12 @@ jobs:
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v5
uses: docker/metadata-action@v6
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME_DEPS_LLVM }}
- name: Build and push Docker image
uses: docker/build-push-action@v6
uses: docker/build-push-action@v7
with:
context: .
file: docker/DockerfileDepsLlvm
@@ -148,16 +148,16 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v7
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
uses: docker/setup-qemu-action@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
uses: docker/setup-buildx-action@v4
- name: Log in to the Container registry
uses: docker/login-action@v3
uses: docker/login-action@v4
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
@@ -165,13 +165,13 @@ jobs:
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v5
uses: docker/metadata-action@v6
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME_LLVM }}
- name: Build and push Docker image
uses: docker/build-push-action@v6
uses: docker/build-push-action@v7
with:
context: .
file: docker/DockerfileLlvm
@@ -190,16 +190,16 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v7
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
uses: docker/setup-qemu-action@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
uses: docker/setup-buildx-action@v4
- name: Log in to the Container registry
uses: docker/login-action@v3
uses: docker/login-action@v4
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
@@ -207,13 +207,13 @@ jobs:
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v5
uses: docker/metadata-action@v6
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME_DHTNODE }}
- name: Build and push Docker image
uses: docker/build-push-action@v6
uses: docker/build-push-action@v7
with:
context: .
file: docker/DockerfileDhtnode
@@ -231,16 +231,16 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v7
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
uses: docker/setup-qemu-action@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
uses: docker/setup-buildx-action@v4
- name: Log in to the Container registry
uses: docker/login-action@v3
uses: docker/login-action@v4
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
@@ -248,13 +248,13 @@ jobs:
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v5
uses: docker/metadata-action@v6
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME_ALPINE_DEPS }}
- name: Build and push Docker image
uses: docker/build-push-action@v6
uses: docker/build-push-action@v7
with:
context: .
file: docker/DockerfileDepsAlpine
@@ -273,16 +273,16 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v7
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
uses: docker/setup-qemu-action@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
uses: docker/setup-buildx-action@v4
- name: Log in to the Container registry
uses: docker/login-action@v3
uses: docker/login-action@v4
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
@@ -290,13 +290,13 @@ jobs:
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v5
uses: docker/metadata-action@v6
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME_ALPINE }}
- name: Build and push Docker image
uses: docker/build-push-action@v6
uses: docker/build-push-action@v7
with:
context: .
file: docker/DockerfileAlpine
@@ -309,7 +309,7 @@ jobs:
name: Build Windows MSI Installer
runs-on: windows-latest
env:
VCPKG_VERSION: "2025.07.25"
VCPKG_VERSION: "2026.06.01"
VCPKG_INSTALLATION_ROOT: "C:\\vcpkg"
VCPKG_DISABLE_METRICS: "1"
VCPKG_DEFAULT_TRIPLET: x64-windows
@@ -319,12 +319,12 @@ jobs:
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v7
with:
submodules: recursive
- name: Set up MSVC developer environment
uses: microsoft/setup-msbuild@v2
uses: microsoft/setup-msbuild@v3
- name: Install WIX using .net tools
run: |
@@ -334,7 +334,7 @@ jobs:
- name: Cache vcpkg installed packages
id: cache-vcpkg
uses: actions/cache@v5
uses: actions/cache@v6
with:
path: |
${{ env.VCPKG_INSTALLED_DIR }}
@@ -403,7 +403,7 @@ jobs:
}
- name: Upload MSI Artifact
uses: actions/upload-artifact@v6
uses: actions/upload-artifact@v7
with:
name: opendht-windows-msi
path: build/${{ steps.cpack_msi.outputs.msi_path }} # Path to the MSI in the build directory
@@ -413,7 +413,7 @@ jobs:
name: Build Windows dhtnode
runs-on: windows-latest
env:
VCPKG_VERSION: "2025.07.25"
VCPKG_VERSION: "2026.06.01"
VCPKG_INSTALLATION_ROOT: "C:\\vcpkg"
VCPKG_DISABLE_METRICS: "1"
VCPKG_DEFAULT_TRIPLET: x64-windows-static-release
@@ -423,16 +423,16 @@ jobs:
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v7
with:
submodules: recursive
- name: Set up MSVC developer environment
uses: microsoft/setup-msbuild@v2
uses: microsoft/setup-msbuild@v3
- name: Cache vcpkg static installed packages
id: cache-vcpkg-static
uses: actions/cache@v5
uses: actions/cache@v6
with:
path: |
${{ env.VCPKG_INSTALLED_DIR }}
@@ -483,7 +483,7 @@ jobs:
cmake --build build_static --config Release --parallel $([System.Environment]::ProcessorCount)
- name: Upload Static dhtnode Artifact
uses: actions/upload-artifact@v6
uses: actions/upload-artifact@v7
with:
name: opendht-windows-dhtnode
path: |
@@ -499,23 +499,23 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v7
- name: Download Windows MSI artifact
uses: actions/download-artifact@v7
uses: actions/download-artifact@v8
with:
name: opendht-windows-msi
path: downloaded_artifacts/windows_msi
- name: Download Static dhtnode artifact
uses: actions/download-artifact@v7
uses: actions/download-artifact@v8
with:
name: opendht-windows-dhtnode
path: downloaded_artifacts/static_dhtnode
- name: Create Release
id: create_release
uses: softprops/action-gh-release@v2
uses: softprops/action-gh-release@v3
with:
draft: true
files: |
+25 -19
View File
@@ -9,7 +9,7 @@ on:
branches: [ main, master ]
env:
VCPKG_VERSION: "2025.07.25"
VCPKG_VERSION: "2026.06.01"
VCPKG_INSTALLATION_ROOT: "C:\\vcpkg"
VCPKG_DISABLE_METRICS: "1"
VCPKG_INSTALLED_DIR: ${{ github.workspace }}\vcpkg_installed
@@ -27,10 +27,10 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v7
- name: Set up MSVC
uses: microsoft/setup-msbuild@v2
uses: microsoft/setup-msbuild@v3
- name: Generate Cache Key
id: generate_cache_key
@@ -41,7 +41,7 @@ jobs:
- name: Cache vcpkg installed packages
id: cache-vcpkg
uses: actions/cache@v5
uses: actions/cache@v6
with:
path: |
${{ env.VCPKG_INSTALLED_DIR }}
@@ -75,10 +75,10 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v7
- name: Set up MSVC
uses: microsoft/setup-msbuild@v2
uses: microsoft/setup-msbuild@v3
- name: Generate Cache Key for Static Libs
id: generate_static_cache_key
@@ -89,7 +89,7 @@ jobs:
- name: Cache vcpkg static installed packages
id: cache-vcpkg-static
uses: actions/cache@v5
uses: actions/cache@v6
with:
path: |
${{ env.VCPKG_INSTALLED_DIR }}
@@ -121,17 +121,18 @@ jobs:
matrix:
build_type: [Debug, Release]
arch: [x64]
cxx_standard: [17, 20, 23]
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v7
- name: Set up MSVC
uses: microsoft/setup-msbuild@v2
uses: microsoft/setup-msbuild@v3
- name: Restore vcpkg installed packages
id: cache-vcpkg-restore
uses: actions/cache/restore@v5
uses: actions/cache/restore@v6
with:
path: |
${{ env.VCPKG_INSTALLED_DIR }}
@@ -152,6 +153,8 @@ jobs:
-DVCPKG_INSTALLED_DIR=${{ env.VCPKG_INSTALLED_DIR }} `
-DVCPKG_MANIFEST_FEATURES=test `
-DCMAKE_BUILD_TYPE=${{ matrix.build_type }} `
-DCMAKE_CXX_STANDARD=${{ matrix.cxx_standard }} `
-DCMAKE_CXX_STANDARD_REQUIRED=ON `
-DBUILD_TESTING=ON `
-DOPENDHT_PROXY_CLIENT=ON `
-DOPENDHT_PROXY_SERVER=ON `
@@ -178,9 +181,9 @@ jobs:
- name: Upload build artifacts
if: matrix.build_type == 'Release'
uses: actions/upload-artifact@v6
uses: actions/upload-artifact@v7
with:
name: opendht-windows-${{ matrix.arch }}-${{ matrix.build_type }}
name: opendht-windows-${{ matrix.arch }}-cxx${{ matrix.cxx_standard }}-${{ matrix.build_type }}
path: |
build/**/*.exe
build/**/*.dll
@@ -195,17 +198,18 @@ jobs:
matrix:
build_type: [Release]
arch: [x64]
cxx_standard: [17, 20, 23]
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v7
- name: Set up MSVC
uses: microsoft/setup-msbuild@v2
uses: microsoft/setup-msbuild@v3
- name: Restore vcpkg static installed packages
id: cache-vcpkg-static-restore
uses: actions/cache/restore@v5
uses: actions/cache/restore@v6
with:
path: |
${{ env.VCPKG_INSTALLED_DIR }}
@@ -226,6 +230,8 @@ jobs:
-DVCPKG_INSTALLED_DIR=${{ env.VCPKG_INSTALLED_DIR }} `
-DVCPKG_MANIFEST_FEATURES=test `
-DCMAKE_BUILD_TYPE=${{ matrix.build_type }} `
-DCMAKE_CXX_STANDARD=${{ matrix.cxx_standard }} `
-DCMAKE_CXX_STANDARD_REQUIRED=ON `
-DBUILD_SHARED_LIBS=OFF `
-DBUILD_TESTING=ON `
-DOPENDHT_PROXY_CLIENT=ON `
@@ -250,9 +256,9 @@ jobs:
ctest --build-config ${{ matrix.build_type }} --output-on-failure --parallel 4
- name: Upload static build artifacts
uses: actions/upload-artifact@v6
uses: actions/upload-artifact@v7
with:
name: opendht-windows-static-${{ matrix.arch }}-${{ matrix.build_type }}
name: opendht-windows-static-${{ matrix.arch }}-cxx${{ matrix.cxx_standard }}-${{ matrix.build_type }}
path: |
build/**/*.exe
build/**/*.lib
@@ -269,7 +275,7 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v7
- name: Setup MinGW
uses: msys2/setup-msys2@v2
@@ -369,7 +375,7 @@ jobs:
- name: Upload MinGW artifacts
if: matrix.build_type == 'Release'
uses: actions/upload-artifact@v6
uses: actions/upload-artifact@v7
with:
name: opendht-windows-mingw-${{ matrix.build_type }}
path: |
+74
View File
@@ -0,0 +1,74 @@
# OpenDHT – Agent Instructions
## Build
### Primary: CMake (C++17)
```sh
mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Debug -DBUILD_TESTING=ON -GNinja
ninja
```
### Common CMake options
| Option | Default | Purpose |
|--------|---------|---------|
| `OPENDHT_TOOLS` | ON | CLI tools (dhtnode, dhtchat, …) |
| `OPENDHT_C` | OFF | C language bindings |
| `OPENDHT_PYTHON` | OFF | Python 3 Cython bindings |
| `OPENDHT_PROXY_SERVER` | OFF | REST API proxy server (requires RESTinio + llhttp) |
| `OPENDHT_PROXY_CLIENT` | OFF | REST API proxy client |
| `OPENDHT_PEER_DISCOVERY` | OFF | Multicast local peer discovery |
| `OPENDHT_PUSH_NOTIFICATIONS` | OFF | Push notification support |
| `OPENDHT_SANITIZE` | OFF | AddressSanitizer + stack protector |
### Alternative: Meson (C++20)
```sh
meson setup build . && cd build && ninja
```
### Run tests
```sh
# Full suite
ctest --output-on-failure --test-dir build
# Single test executable
./build/test_testName
```
## dhtnode
The `dhtnode` CLI tool is a simple DHT node implementation using `DhtRunner`.
---
## Commit
After implementing a feature or fixing a bug, commit your changes in clean, atomic commits with descriptive messages with format:
`topic: short description`
If any test are added or modified, commit them separately before the fix (if any).
---
## Architecture
OpenDHT is a Kademlia-based DHT library. The implementation is layered:
```
DhtRunner (public API — thread-safe, manages lifecycle & I/O threads)
└── SecureDht (GnuTLS crypto: signing, encryption, identity)
└── Dht (core DHT logic: routing, storage, find/get/put operations)
└── NetworkEngine (raw UDP sockets, message parsing, IPv4/IPv6)
```
- **`DhtRunner`** (`include/opendht/dhtrunner.h`, `src/dhtrunner.cpp`) — entry point for all users of the library. Wraps everything in a thread-pool and exposes async callbacks as well as blocking calls.
- **`Dht`** (`include/opendht/dht.h`, `src/dht.cpp`) — Kademlia protocol: routing table, k-buckets, value replication, expiry.
- **`SecureDht`** (`include/opendht/securedht.h`, `src/securedht.cpp`) — transparent encryption/signing layer on top of `Dht`. Identity is a GnuTLS certificate.
- **`NetworkEngine`** (`include/opendht/network_engine.h`, `src/network_engine.cpp`) — UDP socket layer, serializes/deserializes msgpack messages.
- **`RoutingTable`** (`include/opendht/routing_table.h`, `src/routing_table.cpp`) — Kademlia routing table with bucket splitting.
- **`Value`** (`include/opendht/value.h`) — the storable unit: binary blob, optional signature, optional encryption; typed via `ValueType`.
- **`InfoHash`** (`include/opendht/infohash.h`) — 20-byte SHA-1 identifier; alias `Hash<HASH_LEN>`.
Optional components (`DhtProxyServer`, `DhtProxyClient`, `PeerDiscovery`) are compiled in only when the corresponding CMake options are enabled.
---
+23 -12
View File
@@ -21,11 +21,11 @@ Optionally, install RESTinio to enable the OpenDHT proxy client and server:
mkdir /usr/include/nonstd
wget https://raw.githubusercontent.com/martinmoene/expected-lite/master/include/nonstd/expected.hpp -O /usr/include/nonstd/expected.hpp
wget https://github.com/Stiffstream/restinio/releases/download/v.0.7.3/restinio-0.7.3.tar.bz2
tar -xjf restinio-0.7.3.tar.bz2
cd restinio-0.7.3/dev
wget https://github.com/Stiffstream/restinio/releases/download/v.0.7.9/restinio-0.7.9.tar.bz2
tar -xjf restinio-0.7.9.tar.bz2
cd restinio-0.7.9/dev
cmake -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=Off -DRESTINIO_SAMPLE=Off -DRESTINIO_BENCHMARK=Off -DRESTINIO_WITH_SOBJECTIZER=Off -DRESTINIO_DEP_STANDALONE_ASIO=system -DRESTINIO_DEP_LLHTTP=system -DRESTINIO_DEP_FMT=system -DRESTINIO_DEP_EXPECTED_LITE=system .
make -j2 && make install
make && make install
```
To run openDHT on a different computer from the build computer, the following runtime dependencies are required:
@@ -48,11 +48,11 @@ Optionally, install RESTinio to enable the OpenDHT proxy client and server:
mkdir /usr/include/nonstd
wget https://raw.githubusercontent.com/martinmoene/expected-lite/master/include/nonstd/expected.hpp -O /usr/include/nonstd/expected.hpp
wget https://github.com/Stiffstream/restinio/releases/download/v.0.7.3/restinio-0.7.3.tar.bz2
tar -xjf restinio-0.7.3.tar.bz2
cd restinio-0.7.3/dev
wget https://github.com/Stiffstream/restinio/releases/download/v.0.7.9/restinio-0.7.9.tar.bz2
tar -xjf restinio-0.7.9.tar.bz2
cd restinio-0.7.9/dev
cmake -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=Off -DRESTINIO_SAMPLE=Off -DRESTINIO_BENCHMARK=Off -DRESTINIO_WITH_SOBJECTIZER=Off -DRESTINIO_DEP_STANDALONE_ASIO=system -DRESTINIO_DEP_LLHTTP=system -DRESTINIO_DEP_FMT=system -DRESTINIO_DEP_EXPECTED_LITE=system .
make -j2 && make install
make && make install
```
### Fedora
@@ -67,7 +67,7 @@ sudo dnf install python3-Cython python3-devel redhat-rpm-config
### macOS
```sh
brew install gnutls msgpack-cxx argon2 asio readline jsoncpp fmt
brew install gnutls msgpack-cxx argon2 asio readline jsoncpp fmt llhttp
```
## Build
@@ -81,7 +81,7 @@ git clone https://github.com/savoirfairelinux/opendht.git
# Build and install
cd opendht
mkdir build && cd build
cmake -DCMAKE_INSTALL_PREFIX=/usr ..
cmake ..
make -j$(nproc)
sudo make install
```
@@ -100,10 +100,21 @@ cmake -DOPENDHT_PYTHON=ON \
-DOPENDHT_PROXY_SERVER=ON \
-DOPENDHT_PROXY_SERVER_IDENTITY=ON \
-DOPENDHT_PROXY_CLIENT=ON \
-DOPENDHT_PUSH_NOTIFICATIONS=ON \
-DCMAKE_INSTALL_PREFIX=/usr ..
-DOPENDHT_PUSH_NOTIFICATIONS=ON ..
```
#### Common CMake options
| Option | Default | Purpose |
|--------|---------|---------|
| `OPENDHT_TOOLS` | ON | CLI tools (dhtnode, dhtchat, …) |
| `OPENDHT_C` | OFF | C language bindings |
| `OPENDHT_PYTHON` | OFF | Python 3 Cython bindings |
| `OPENDHT_PROXY_SERVER` | OFF | REST API proxy server (requires RESTinio + llhttp) |
| `OPENDHT_PROXY_CLIENT` | OFF | REST API proxy client |
| `OPENDHT_PEER_DISCOVERY` | OFF | Multicast local peer discovery |
| `OPENDHT_PUSH_NOTIFICATIONS` | OFF | Push notification support |
| `OPENDHT_SANITIZE` | OFF | AddressSanitizer + stack protector |
## Windows/MSVC
Building on Windows with MSVC is supported with VCPKG and CMake.
+36 -23
View File
@@ -1,4 +1,4 @@
cmake_minimum_required (VERSION 3.21..3.31)
cmake_minimum_required (VERSION 3.21...4.4)
if(POLICY CMP0091)
cmake_policy(SET CMP0091 NEW)
endif()
@@ -9,8 +9,8 @@ if(POLICY CMP0167)
cmake_policy(SET CMP0167 NEW)
endif()
set (opendht_VERSION_MAJOR 3)
set (opendht_VERSION_MINOR 7.0)
set (opendht_VERSION_MAJOR 4)
set (opendht_VERSION_MINOR 3.0)
set (opendht_VERSION ${opendht_VERSION_MAJOR}.${opendht_VERSION_MINOR})
set (PACKAGE_VERSION ${opendht_VERSION})
set (VERSION "${opendht_VERSION}")
@@ -130,15 +130,16 @@ if (NOT MSVC)
message(STATUS "simdutf not found with pkgconfig")
if (OPENDHT_DOWNLOAD_DEPS)
message(STATUS "Fallback mode specified, downloading simdutf")
set(SIMDUTF_CXX_STANDARD ${CMAKE_CXX_STANDARD} CACHE INTERNAL "C++ standard for simdutf downloader")
set(SIMDUTF_TESTS OFF CACHE INTERNAL "Disable tests for simdutf downloader")
set(SIMDUTF_TOOLS OFF CACHE INTERNAL "Disable tools for simdutf downloader")
set(SIMDUTF_CXX_STANDARD 17 CACHE INTERNAL "Set C++ standard for simdutf downloader")
FetchContent_Declare(simdutf
GIT_REPOSITORY "https://github.com/simdutf/simdutf.git"
GIT_TAG "v7.5.0"
GIT_TAG "v8.0.0"
)
FetchContent_MakeAvailable(simdutf)
set(SIMDUTF_LIBRARIES simdutf::simdutf)
set(simdutf_lib ", simdutf")
else()
message(SEND_ERROR "simdutf not found and the fallback downloader is not set! Install simdutf or use OPENDHT_DOWNLOAD_DEPS option")
endif()
@@ -416,7 +417,7 @@ else ()
-DSTATIC_GETOPT)
set(DISABLE_MSC_WARNINGS "/wd4101 /wd4244 /wd4267 /wd4273 /wd4804 /wd4834 /wd4996")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${DISABLE_MSC_WARNINGS}")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /MP /bigobj /utf-8 /EHsc")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /MP /bigobj /utf-8 /EHsc /Zc:__cplusplus")
endif ()
set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -DMSGPACK_NO_BOOST -DMSGPACK_DISABLE_LEGACY_NIL -DMSGPACK_DISABLE_LEGACY_CONVERT")
@@ -853,30 +854,42 @@ if (BUILD_TESTING)
endif()
# unit testing
list (APPEND test_FILES
tests/infohashtester.h
tests/infohashtester.cpp
tests/valuetester.h
tests/valuetester.cpp
tests/cryptotester.h
tests/cryptotester.cpp
tests/dhtrunnertester.h
tests/dhtrunnertester.cpp
tests/threadpooltester.h
tests/threadpooltester.cpp
tests/test_infohash.h
tests/test_infohash.cpp
tests/test_value.h
tests/test_value.cpp
tests/test_crypto.h
tests/test_crypto.cpp
tests/test_dhtrunner.h
tests/test_dhtrunner.cpp
tests/test_storage.h
tests/test_storage.cpp
tests/test_threadpool.h
tests/test_threadpool.cpp
tests/test_parsedmessage.h
tests/test_parsedmessage.cpp
tests/test_networkengine.h
tests/test_networkengine.cpp
)
if (OPENDHT_PROXY_CLIENT)
list (APPEND test_FILES
tests/test_dhtproxy_client.h
tests/test_dhtproxy_client.cpp
)
endif()
if (OPENDHT_TESTS_NETWORK)
if (OPENDHT_PROXY_SERVER AND OPENDHT_PROXY_CLIENT)
list (APPEND test_FILES
tests/httptester.h
tests/httptester.cpp
tests/dhtproxytester.h
tests/dhtproxytester.cpp
tests/test_http.h
tests/test_http.cpp
tests/test_dhtproxy.h
tests/test_dhtproxy.cpp
)
endif()
if (OPENDHT_PEER_DISCOVERY)
list (APPEND test_FILES
tests/peerdiscoverytester.h
tests/peerdiscoverytester.cpp
tests/test_peerdiscovery.h
tests/test_peerdiscovery.cpp
)
endif()
endif()
@@ -884,7 +897,7 @@ if (BUILD_TESTING)
# We keep using the common tests_runner.cpp (CppUnit registry based) and compile it with each test file.
foreach(test_src ${test_FILES})
if (test_src MATCHES ".*\\.cpp$")
get_filename_component(test_name ${test_src} NAME_WE) # e.g. infohashtester
get_filename_component(test_name ${test_src} NAME_WE) # e.g. test_infohash
set(target_name ${test_name})
add_executable(${target_name}
tests/tests_runner.cpp
+9 -7
View File
@@ -140,20 +140,22 @@ asyncio.run(dht_async_demo("unique_key"))
```
## Dependencies
Core dependencies:
- msgpack-c 1.2+, used for data serialization.
- GnuTLS 3.3+, used for cryptographic operations.
- Nettle 2.4+, a GnuTLS dependency for crypto.
- {fmt} 9.0+, for log formatting.
- (optional) restinio used for the REST API.
- (optional) llhttp used for the REST API.
- (optional) jsoncpp 1.7.4-3+, used for the REST API.
Optional dependencies for the REST API (DHT proxy client/server):
- (optional) restinio
- (optional) llhttp
- (optional) jsoncpp 1.7.4-3+
- (optional) simdutf
Build environment:
- Build tested with GCC 9+ (GNU/Linux, Windows with MinGW), Clang/LLVM (GNU/Linux, Android, macOS, iOS).
- Build tested with Microsoft Visual Studio 2019, 2022
## Contact
IRC: join us on Libera.chat at [`#opendht`](https://web.libera.chat/#opendht).
## License
Copyright (c) 2014-2026 Savoir-faire Linux Inc.
-15
View File
@@ -1,15 +0,0 @@
lib_LTLIBRARIES = libopendht-c.la
noinst_HEADERS = opendht_c.h
AM_CPPFLAGS = -DOPENDHT_C_BUILD -isystem @top_srcdir@/include @JsonCpp_CFLAGS@ @MsgPack_CFLAGS@
if OPENDHT_SHARED
AM_CPPFLAGS += -Dopendht_c_EXPORTS
endif
libopendht_c_la_LIBADD = $(LIBOBJS) ../src/.libs/libopendht.la
libopendht_c_la_SOURCES = \
opendht.cpp
nobase_include_HEADERS = \
opendht_c.h
+2 -2
View File
@@ -600,7 +600,7 @@ dht_runner_put_signed(
void
dht_runner_put_encrypted(dht_runner* r,
const dht_infohash* h,
const dht_infohash* to,
const dht_pkid* to,
const dht_value* v,
dht_done_cb done_cb,
void* cb_user_data,
@@ -608,7 +608,7 @@ dht_runner_put_encrypted(dht_runner* r,
{
auto runner = reinterpret_cast<dht::DhtRunner*>(r);
auto hash = reinterpret_cast<const dht::InfoHash*>(h);
auto toHash = reinterpret_cast<const dht::InfoHash*>(to);
auto toHash = reinterpret_cast<const dht::PkId*>(to);
auto value = reinterpret_cast<const ValueSp*>(v);
runner->putEncrypted(
*hash,
+1 -1
View File
@@ -195,7 +195,7 @@ OPENDHT_C_PUBLIC void dht_runner_put_signed(dht_runner* runner,
bool permanent);
OPENDHT_C_PUBLIC void dht_runner_put_encrypted(dht_runner* runner,
const dht_infohash* hash,
const dht_infohash* to,
const dht_pkid* to,
const dht_value* value,
dht_done_cb done_cb,
void* cb_user_data,
-16
View File
@@ -1,16 +0,0 @@
dist_man1_MANS = dhtnode.1
noinst_HEADERS = \
Doxyfile
if HAVE_DOXYGEN
doxyfile.stamp: Doxyfile
$(DOXYGEN) Doxyfile
echo stamp > doxyfile.stamp
CLEANFILES = doxyfile.stamp
all-local: doxyfile.stamp
clean-local:
rm -rf $(top_srcdir)/doc/man $(top_srcdir)/doc/html
endif
+145 -54
View File
@@ -1,102 +1,193 @@
.TH DHTNODE 1 2019-06-08
.TH DHTNODE 1 2026-04-18
.SH NAME
.B dhtnode
- a simple OpenDHT command line node runner.
.SH SYNOPSIS
.B dhtnode
[\fB\-h\fR]
[\fB\-V\fR]
[\fB\-v\fR [\fB\-l\fR \fIlogfile\fR] [\fB\-L\fR]]
[\fB\-i\fR [\fB\-\-save\-identity\fR \fIfile\fR]]
[\fB\-i\fR [\fB\-I\fR \fIfile\fR]]
[\fB\-c\fR \fIfile\fR]
[\fB\-k\fR \fIfile\fR]
[\fB\-m\fR \fIpassword\fR]
[\fB\-d\fR] [\fB\-s\fR]
[\fB\-D\fR] [\fB\-U\fR] [\fB\-P\fR]
[\fB\-n\fR \fInetwork_id\fR]
[\fB\-p\fR \fIlocal_port\fR]
[\fB\-b\fR \fIbootstrap_host\fR[:\fIport\fR]]
[\fB\-\-certificate\fR \fIfile\fR]
[\fB\-\-privkey\fR \fIfile\fR]
[\fB\-\-privkey\-password\fR \fIpassword\fR]
[\fB\-\-proxyserver\fR \fIport\fR]
[\fB\-\-proxyclient\fR \fIserver\fR]
[\fB\-f\fR \fIpath\fR]
[\fB\-S\fR \fIport\fR]
[\fB\-e\fR \fIport\fR]
[\fB\-a\fR \fIaddress\fR]
[\fB\-w\fR \fIfile\fR]
[\fB\-K\fR \fIfile\fR]
[\fB\-M\fR \fIpassword\fR]
[\fB\-C\fR \fIserver\fR]
[\fB\-y\fR \fIendpoint\fR]
[\fB\-z\fR \fIkey\fR]
[\fB\-u\fR \fIbundleid\fR]
.SH DESCRIPTION
Runs an OpenDHT node, with a CLI (default) or as a daemon (with \fB'-d'\fP or \fB'-s'\fP).
Runs an OpenDHT node, with an interactive CLI (default) or as a daemon (with \fB'-d'\fP or \fB'-s'\fP).
Commands available in the interactive shell are:
.EX
h, help Print this help message.
q, quit Quit the program.
x, quit Quit the program.
log Start/stop printing DHT logs.
Node information:
ll Print basic information and stats about the current node.
ls Print basic information about current searches.
ld Print basic information about currenty stored values on this node.
lr Print the full current routing table of this node
ls [key] Print basic information about current search(es).
ld [key] Print basic information about currenty stored values on this
node (or for a specific key).
lr Print the full current routing table of this node.
Operations on the DHT:
b ip:port Ping potential node at given IP address/port.
g [key] Get values at [key].
l [key] Listen for value changes at [key].
p [key] [str] Put string value at [key].
s [key] [str] Put string value at [key], signed with our generated
private key.
e [key] [dest] [str] Put string value at [key], encrypted for [dest] with
its public key (if found).
b <ip:port> Ping potential node at given IP address/port.
g <key> Get values at <key>.
l <key> Listen for value changes at <key>.
cl <key> <token> Cancel listen for <token> and <key>.
p <key> <str> Put string value at <key>.
pp <key> <str> Put string value at <key> (persistent version).
cpp <key> <id> Cancel persistent put operation for <key> and
value <id>.
s <key> <str> Put string value at <key>, signed with our
generated private key.
e <key> <dest> <str> Put string value at <key>, encrypted for <dest>
with its public key (if found).
cc Trigger connectivity changed signal.
Operations with the proxy (if compiled with proxy server support):
pst [port] <pushServer> Start the proxy interface on port.
psx [port] Start the proxy SSL interface on port.
psp [port] Stop the proxy interface on port.
Operations with the proxy (if compiled with proxy client support):
stt [server_address] <device_key> Start the proxy client.
stp Stop the proxy client.
rs [token] Resubscribe to opendht.
rp [token] Inject a push notification.
.EE
.SH OPTIONS
.TP
\fB\-h\fP
Prints some help.
\fB\-h\fP, \fB\-\-help\fP
Prints help and exits.
.TP
\fB\-v\fP
Enable the verbose mode (log to stdout/stderr by default)
\fB\-V\fP, \fB\-\-version\fP
Prints version information and exits.
.TP
\fB\-l\fP \fIlog_file\fP
Write log to file instead of stdout/stderr
\fB\-v\fP, \fB\-\-verbose\fP
Enable verbose mode (log to stdout/stderr by default).
.TP
\fB\-L\fP
Write log to syslog instead of stdout/stderr
\fB\-l\fP \fIlogfile\fP, \fB\-\-logfile\fP \fIlogfile\fP
Write log to file instead of stdout/stderr.
.TP
\fB\-i\fP
Generate cryptographic identity for the node.
\fB\-L\fP, \fB\-\-syslog\fP
Write log to syslog instead of stdout/stderr (enables verbose mode).
.TP
\fB\-\-save\-identity\fP \fIfile\fP
\fB\-i\fP, \fB\-\-identity\fP
Generate a cryptographic identity (RSA key pair and self-signed certificate)
for the node. Required to perform cryptographic operations (encrypt/sign values)
if no private key file is provided.
.TP
\fB\-I\fP \fIfile\fP, \fB\-\-save\-identity\fP \fIfile\fP
Save generated identity (certificate and private key) to given file prefix.
Files are saved as \fIfile\fP and \fIfile\fP_ca.
.TP
\fB\-\-certificate\fP \fIfile\fP
Load identity certificate from given file.
\fB\-c\fP \fIfile\fP, \fB\-\-certificate\fP \fIfile\fP
Load identity certificate from given file. If specified, the certificate is
published on the DHT.
.TP
\fB\-\-privkey\fP \fIfile\fP
Load identity private key from given file.
\fB\-k\fP \fIfile\fP, \fB\-\-privkey\fP \fIfile\fP
Load identity private key from given file. Required to perform cryptographic
operations if no identity is generated.
.TP
\fB\-\-privkey\-password\fP \fIpassword\fP
\fB\-m\fP \fIpassword\fP, \fB\-\-privkey\-password\fP \fIpassword\fP
Password to use for private key encryption or decryption (optional).
.TP
\fB\-d\fP
Run the program in daemon mode (will fork in the background).
\fB\-d\fP, \fB\-\-daemonize\fP
Run the program in daemon mode (will fork in the background). When verbose mode
is enabled, logs to syslog by default.
.TP
\fB\-s\fP
\fB\-s\fP, \fB\-\-service\fP
Run the program in service mode (non\-forking daemon).
.TP
\fB\-D\fP
\fB\-D\fP, \fB\-\-peer\-discovery\fP
Enables multicast automatic local peer discovery.
.TP
\fB\-f\fP \fIfile\fP
Specify a file path to persist/load the node state.
\fB\-U\fP, \fB\-\-no\-rate\-limit\fP
Disable rate limiting (allows unlimited requests and storage).
.TP
\fB\-n\fP \fInetwork_id\fP
Specify the network id. This let you connect to distinct networks and prevents
the merge of two different networks (available since OpenDHT v0.6.1).
\fB\-P\fP, \fB\-\-public\-stable\fP
Enable public stable mode.
.TP
\fB\-p\fP \fIlocal_port\fP
Use UDP port \fIlocal_port\fP for the program to bind to.
\fB\-f\fP \fIpath\fP, \fB\-\-persist\fP \fIpath\fP
Specify a file path to persist/load the node state (routing table and values).
.TP
\fB\-b\fP \fIbootstrap_host\fP[:\fIport\fP]
The program needs to be given a node to connect to the network. You use this
option to provide the ip address of that node.
\fB\-n\fP \fInetwork_id\fP, \fB\-\-net\fP \fInetwork_id\fP
Specify the network id. This lets you connect to distinct networks and prevents
the merge of two different networks. Defaults to 0 (the main public network).
.TP
\fB\-\-proxyserver\fP \fIlocal_port\fP
Run a proxy server bound to this DHT node on HTTP port \fIlocal_port\fP
\fB\-p\fP \fIlocal_port\fP, \fB\-\-port\fP \fIlocal_port\fP
Use UDP port \fIlocal_port\fP for the program to bind to. If not set, will use
any available port. The default OpenDHT port for public nodes is 4222.
.TP
\fB\-\-proxyclient\fP \fIserver\fP
Run this DHT node in proxy client mode, and connect to \fIserver\fP
\fB\-b\fP \fIbootstrap_host\fP[:\fIport\fP], \fB\-\-bootstrap\fP \fIbootstrap_host\fP[:\fIport\fP]
Specify a bootstrap node address to connect to the network.
.TP
\fB\-S\fP \fIlocal_port\fP, \fB\-\-proxyserver\fP \fIlocal_port\fP
Run a REST API proxy server bound to this DHT node on HTTP port \fIlocal_port\fP.
.TP
\fB\-e\fP \fIlocal_port\fP, \fB\-\-proxyserverssl\fP \fIlocal_port\fP
Run a REST API proxy server with SSL bound to this DHT node on HTTPS port \fIlocal_port\fP.
.TP
\fB\-a\fP \fIaddress\fP, \fB\-\-proxy\-addr\fP \fIaddress\fP
Proxy server bind address.
.TP
\fB\-w\fP \fIfile\fP, \fB\-\-proxy\-certificate\fP \fIfile\fP
Load proxy SSL certificate from given file.
.TP
\fB\-K\fP \fIfile\fP, \fB\-\-proxy\-privkey\fP \fIfile\fP
Load proxy SSL private key from given file.
.TP
\fB\-M\fP \fIpassword\fP, \fB\-\-proxy\-privkey\-password\fP \fIpassword\fP
Password for the proxy private key encryption or decryption (optional).
.TP
\fB\-C\fP \fIserver\fP, \fB\-\-proxyclient\fP \fIserver\fP
Run this DHT node in proxy client mode, and connect to \fIserver\fP.
.TP
\fB\-y\fP \fIendpoint\fP, \fB\-\-pushserver\fP \fIendpoint\fP
Push notification server endpoint.
.TP
\fB\-z\fP \fIkey\fP, \fB\-\-devicekey\fP \fIkey\fP
Device key for push notifications.
.TP
\fB\-u\fP \fIbundleid\fP, \fB\-\-bundleid\fP \fIbundleid\fP
Bundle ID for push notifications.
.SH EXAMPLES
Run a node using bootstrap.jami.net as a bootstrap:
.PP
.EX
dhtnode -b bootstrap.jami.net:4222
.EE
.PP
Run a node as a non-forking daemon on port 4222:
.PP
.EX
dhtnode -b bootstrap.jami.net -p 4222 -s
.EE
.PP
Run a proxy server on port 8000:
.PP
.EX
dhtnode -b bootstrap.jami.net:4222 --proxyserver 8000
.EE
.PP
Use a proxy client to connect to a proxy server:
.PP
.EX
dhtnode --proxyclient dhtproxy.jami.net:80
.EE
.SH AUTHORS
.TP
Program written by
+23 -9
View File
@@ -23,15 +23,15 @@ struct Node;
*/
enum class NodeStatus {
Disconnected, // 0 nodes
Connecting, // 1+ nodes
Connecting, // bootstrap pending or 1+ nodes
Connected // 1+ good nodes
};
inline constexpr const char*
inline constexpr std::string_view
statusToStr(NodeStatus status)
{
return status == NodeStatus::Connected ? "connected"
: (status == NodeStatus::Connecting ? "connecting" : "disconnected");
return status == NodeStatus::Connected ? "connected"sv
: (status == NodeStatus::Connecting ? "connecting"sv : "disconnected"sv);
}
struct OPENDHT_PUBLIC NodeStats
@@ -65,6 +65,8 @@ struct OPENDHT_PUBLIC NodeInfo
size_t ongoing_ops {0};
size_t storage_values {0};
size_t storage_size {0};
size_t local_storage_values {0};
size_t local_storage_size {0};
in_port_t bound4 {0};
in_port_t bound6 {0};
@@ -77,7 +79,17 @@ struct OPENDHT_PUBLIC NodeInfo
explicit NodeInfo(const Json::Value& v);
#endif
MSGPACK_DEFINE_MAP(id, node_id, ipv4, ipv6)
MSGPACK_DEFINE_MAP(id,
node_id,
ipv4,
ipv6,
ongoing_ops,
storage_values,
storage_size,
local_storage_values,
local_storage_size,
bound4,
bound6)
};
/**
@@ -113,9 +125,12 @@ struct OPENDHT_PUBLIC Config
/* If non-0, overrides the default maximum number of searches. -1 means no limit. */
ssize_t max_searches {0};
/* If non-0, overrides the default maximum store size. -1 means no limit. */
/* If non-0, overrides the default maximum store size for remote values. -1 means no limit. */
ssize_t max_store_size {0};
/* If non-0, overrides the default maximum store size for locally stored values. -1 means no limit. */
ssize_t max_local_store_size {0};
/* If non-0, overrides the default maximum store key count. -1 means no limit. */
ssize_t max_store_keys {0};
@@ -159,7 +174,8 @@ enum class OPENDHT_PUBLIC PushNotificationResult : uint8_t {
Error
};
static constexpr size_t DEFAULT_STORAGE_LIMIT {1024 * 1024 * 64};
static constexpr size_t STORAGE_LIMIT_DEFAULT {1024 * 1024 * 64};
static constexpr size_t STORAGE_LIMIT_UNLIMITED {(size_t) -1};
using ValuesExport = std::pair<InfoHash, Blob>;
@@ -171,8 +187,6 @@ using ShutdownCallback = std::function<void()>;
using IdentityAnnouncedCb = std::function<void(bool)>;
using PublicAddressChangedCb = std::function<void(std::vector<SockAddr>)>;
using CertificateStoreQueryLegacy
= std::function<std::vector<std::shared_ptr<crypto::Certificate>>(const InfoHash& pk_id)>;
using CertificateStoreQuery = std::function<std::vector<std::shared_ptr<crypto::Certificate>>(const PkId& pk_id)>;
using DoneCallback = std::function<void(bool success, const std::vector<std::shared_ptr<Node>>& nodes)>;
+17 -5
View File
@@ -204,7 +204,13 @@ struct OPENDHT_PUBLIC PrivateKey
* @param algo : the public key algorithm to use, GNUTLS_PK_RSA or GNUTLS_PK_RSA_OAEP
*/
static PrivateKey generate(unsigned key_length = 4096, gnutls_pk_algorithm_t algo = GNUTLS_PK_RSA);
static PrivateKey generateEC();
/**
* Generate a new EC key pair
* @param curve : the elliptic curve to use. If GNUTLS_ECC_CURVE_INVALID is provided, the default curve will be used
* (currently GNUTLS_ECC_CURVE_SECP384R1).
*/
static PrivateKey generateEC(gnutls_ecc_curve_t curve = GNUTLS_ECC_CURVE_INVALID);
gnutls_privkey_t key {};
gnutls_x509_privkey_t x509_key {};
@@ -214,7 +220,7 @@ private:
PrivateKey& operator=(const PrivateKey&) = delete;
Blob decryptBloc(const uint8_t* src, size_t src_size) const;
mutable std::mutex publicKeyMutex_ {};
mutable std::mutex publicKeyMutex_;
mutable std::shared_ptr<PublicKey> publicKey_ {};
};
@@ -267,9 +273,15 @@ public:
/**
* Sign this revocation list using provided key and certificate.
* Validity_period sets the duration until next update (default to no next update).
* If `number` is non-zero, it is stored verbatim (big-endian) as the CRL Number
* extension. Otherwise the number is initialized randomly on the first signature
* and incremented on subsequent ones.
*/
void sign(const PrivateKey&, const Certificate&, duration validity_period = {});
void sign(const Identity& id) { sign(*id.first, *id.second); }
void sign(const PrivateKey&, const Certificate&, duration validity_period = {}, uint64_t number = 0);
void sign(const Identity& id, duration validity_period = {}, uint64_t number = 0)
{
sign(*id.first, *id.second, validity_period, number);
}
bool isSignedBy(const Certificate& issuer) const;
@@ -685,7 +697,7 @@ private:
std::set<std::shared_ptr<RevocationList>, crlNumberCmp> revocation_lists;
mutable std::mutex publicKeyMutex_ {};
mutable std::mutex publicKeyMutex_;
mutable std::shared_ptr<PublicKey> publicKey_ {};
};
+6
View File
@@ -159,7 +159,13 @@ public:
{}
static Value::Filter getFilter() { return EncryptedValue::getFilter(); }
virtual void unpackValue(const dht::Value& v) override
{
BaseClass::unpackValue(v);
id = v.id;
}
dht::Value::Id id {0}; /// Value ID of the trust request, used for deserialization only.
std::string service;
std::string conversationId;
Blob payload;
+20 -5
View File
@@ -296,17 +296,28 @@ public:
}
/**
* Set the in-memory storage limit in bytes
* Set the in-memory storage limit in bytes for remote values
*/
void setStorageLimit(size_t limit = DEFAULT_STORAGE_LIMIT) override { max_store_size = limit; }
void setStorageLimit(size_t limit = 0) override { max_store_size = limit == 0 ? STORAGE_LIMIT_DEFAULT : limit; }
size_t getStorageLimit() const override { return max_store_size; }
/**
* Set the in-memory storage limit in bytes for locally stored values
*/
void setLocalStorageLimit(size_t limit = 0) override
{
max_local_store_size = limit == 0 ? STORAGE_LIMIT_UNLIMITED : limit;
}
size_t getLocalStorageLimit() const override { return max_local_store_size; }
/**
* Returns the total memory usage of stored values and the number
* of stored values.
*/
std::pair<size_t, size_t> getStoreSize() const override { return {total_store_size, total_values}; }
std::pair<size_t, size_t> getLocalStoreSize() const override;
std::vector<SockAddr> getPublicAddress(sa_family_t family = 0) override;
PushNotificationResult pushNotificationReceived(const std::map<std::string, std::string>&) override
@@ -400,13 +411,16 @@ private:
std::vector<std::pair<std::string, std::string>> bootstrap_nodes {};
std::chrono::steady_clock::duration bootstrap_period {BOOTSTRAP_PERIOD};
Sp<Scheduler::Job> bootstrapJob {};
bool bootstrap_pending {false};
std::map<InfoHash, Storage> store;
std::map<SockAddr, StorageBucket, SockAddr::ipCmp> store_quota;
std::unique_ptr<StorageBucket> local_store_quota;
size_t total_values {0};
size_t total_store_size {0};
size_t max_store_keys {MAX_HASHES};
size_t max_store_size {DEFAULT_STORAGE_LIMIT};
size_t max_store_size {STORAGE_LIMIT_DEFAULT};
size_t max_local_store_size {STORAGE_LIMIT_UNLIMITED};
size_t max_searches {MAX_SEARCHES};
size_t search_id {0};
@@ -451,7 +465,8 @@ private:
const Sp<Value>& value,
time_point created,
const SockAddr& sa = {},
bool permanent = false);
bool permanent = false,
time_point expiration = time_point::min());
bool storageRefresh(const InfoHash& id, Value::Id vid);
void expireStore();
void expireStorage(InfoHash h);
@@ -613,7 +628,7 @@ private:
/* when we receive a listen request */
net::RequestAnswer onListen(
Sp<Node> node, const InfoHash& hash, const Blob& token, size_t socket_id, const Query& query, int version = 0);
void onListenDone(const Sp<Node>& status, net::RequestAnswer& a, Sp<Search>& sr);
void onListenDone(const Sp<Node>& node, net::RequestAnswer& a, Sp<Search>& sr);
/* when we receive an announce request */
net::RequestAnswer onAnnounce(Sp<Node> node,
const InfoHash& hash,
+10 -13
View File
@@ -18,9 +18,6 @@ class OPENDHT_PUBLIC DhtInterface
{
public:
DhtInterface() = default;
DhtInterface(const Logger& l)
: logger_(std::make_shared<Logger>(l))
{}
DhtInterface(const std::shared_ptr<Logger>& l)
: logger_(l)
{}
@@ -221,26 +218,26 @@ public:
virtual std::vector<unsigned> getNodeMessageStats(bool in = false) = 0;
/**
* Set the in-memory storage limit in bytes
* Set the in-memory storage limit in bytes for remote values
*/
virtual void setStorageLimit(size_t limit = DEFAULT_STORAGE_LIMIT) = 0;
virtual void setStorageLimit(size_t limit = 0) = 0;
virtual size_t getStorageLimit() const = 0;
/**
* Set the in-memory storage limit in bytes for locally stored values
*/
virtual void setLocalStorageLimit(size_t limit = 0) = 0;
virtual size_t getLocalStorageLimit() const = 0;
/**
* Returns the total memory usage of stored values and the number
* of stored values.
*/
virtual std::pair<size_t, size_t> getStoreSize() const = 0;
virtual std::vector<SockAddr> getPublicAddress(sa_family_t family = 0) = 0;
virtual std::pair<size_t, size_t> getLocalStoreSize() const = 0;
virtual void setLogger(const Logger& l)
{
if (logger_)
*logger_ = l;
else
logger_ = std::make_shared<Logger>(l);
}
virtual std::vector<SockAddr> getPublicAddress(sa_family_t family = 0) = 0;
virtual void setLogger(const std::shared_ptr<Logger>& l) { logger_ = l; }
+3
View File
@@ -276,6 +276,7 @@ public:
void insertNode(const InfoHash&, const SockAddr&) override {}
void insertNode(const NodeExport&) override {}
std::pair<size_t, size_t> getStoreSize() const override { return {}; }
std::pair<size_t, size_t> getLocalStoreSize() const override { return {}; }
std::vector<NodeExport> exportNodes() const override { return {}; }
std::vector<ValuesExport> exportValues() const override { return {}; }
void importValues(const std::vector<ValuesExport>&) override {}
@@ -288,6 +289,8 @@ public:
std::vector<unsigned> getNodeMessageStats(bool) override { return {}; }
void setStorageLimit(size_t) override {}
virtual size_t getStorageLimit() const override { return 0; }
void setLocalStorageLimit(size_t) override {}
virtual size_t getLocalStorageLimit() const override { return STORAGE_LIMIT_UNLIMITED; }
void connectivityChanged(sa_family_t) override { getProxyInfos(); }
void connectivityChanged() override
{
+9 -9
View File
@@ -76,7 +76,7 @@ public:
asio::io_context& io_context() const;
using clock = std::chrono::steady_clock;
using clock = std::chrono::system_clock;
using time_point = clock::time_point;
struct PushStats
@@ -405,8 +405,8 @@ private:
std::string pushToken;
std::string clientId;
std::shared_ptr<PushSessionContext> sessionCtx;
std::unique_ptr<asio::steady_timer> expireTimer;
std::unique_ptr<asio::steady_timer> expireNotifyTimer;
std::unique_ptr<asio::system_timer> expireTimer;
std::unique_ptr<asio::system_timer> expireNotifyTimer;
Sp<Value> value;
PushType type;
std::string topic;
@@ -419,13 +419,13 @@ private:
p.pack("value");
p.pack(value);
p.pack("exp");
p.pack(to_time_t(expiration));
p.pack(std::chrono::system_clock::to_time_t(expiration));
if (not clientId.empty()) {
p.pack("cid");
p.pack(clientId);
}
if (sessionCtx) {
std::lock_guard<std::mutex> l(sessionCtx->lock);
std::lock_guard l(sessionCtx->lock);
p.pack("sid");
p.pack(sessionCtx->sessionId);
}
@@ -464,8 +464,8 @@ private:
std::string clientId;
std::shared_ptr<PushSessionContext> sessionCtx;
std::future<size_t> internalToken;
std::unique_ptr<asio::steady_timer> expireTimer;
std::unique_ptr<asio::steady_timer> expireNotifyTimer;
std::unique_ptr<asio::system_timer> expireTimer;
std::unique_ptr<asio::system_timer> expireNotifyTimer;
PushType type;
std::string topic;
@@ -476,9 +476,9 @@ private:
p.pack("cid");
p.pack(clientId);
p.pack("exp");
p.pack(to_time_t(expiration));
p.pack(std::chrono::system_clock::to_time_t(expiration));
if (sessionCtx) {
std::lock_guard<std::mutex> l(sessionCtx->lock);
std::lock_guard l(sessionCtx->lock);
p.pack("sid");
p.pack(sessionCtx->sessionId);
}
+18 -29
View File
@@ -61,8 +61,7 @@ public:
std::unique_ptr<net::DatagramSocket> sock;
std::shared_ptr<PeerDiscovery> peerDiscovery {};
StatusCallback statusChangedCallback {};
CertificateStoreQueryLegacy certificateStore {};
CertificateStoreQuery certificateStorePkId {};
CertificateStoreQuery certificateStore {};
IdentityAnnouncedCb identityAnnouncedCb {};
PublicAddressChangedCb publicAddressChangedCb {};
std::unique_ptr<std::mt19937_64> rng {};
@@ -286,21 +285,6 @@ public:
}
void putSigned(const std::string& key, Value&& value, DoneCallbackSimple cb = {}, bool permanent = false);
void putEncrypted(
InfoHash hash, InfoHash to, std::shared_ptr<Value> value, DoneCallback cb = {}, bool permanent = false);
void putEncrypted(
InfoHash hash, InfoHash to, std::shared_ptr<Value> value, DoneCallbackSimple cb, bool permanent = false)
{
putEncrypted(hash, to, value, bindDoneCb(cb), permanent);
}
void putEncrypted(InfoHash hash, InfoHash to, Value&& value, DoneCallback cb = {}, bool permanent = false);
void putEncrypted(InfoHash hash, InfoHash to, Value&& value, DoneCallbackSimple cb, bool permanent = false)
{
putEncrypted(hash, to, std::forward<Value>(value), bindDoneCb(cb), permanent);
}
void putEncrypted(const std::string& key, InfoHash to, Value&& value, DoneCallback cb = {}, bool permanent = false);
void putEncrypted(InfoHash hash,
const std::shared_ptr<crypto::PublicKey>& to,
std::shared_ptr<Value> value,
@@ -315,13 +299,11 @@ public:
putEncrypted(hash, to, value, bindDoneCb(cb), permanent);
}
[[deprecated("Use the shared_ptr version instead")]]
void putEncrypted(InfoHash hash,
const std::shared_ptr<crypto::PublicKey>& to,
Value&& value,
DoneCallback cb = {},
bool permanent = false);
[[deprecated("Use the shared_ptr version instead")]]
void putEncrypted(InfoHash hash,
const std::shared_ptr<crypto::PublicKey>& to,
Value&& value,
@@ -338,6 +320,14 @@ public:
{
putEncrypted(hash, to, value, bindDoneCb(cb), permanent);
}
void putEncrypted(InfoHash hash, const PkId& to, Value&& value, DoneCallback cb = {}, bool permanent = false)
{
putEncrypted(hash, to, std::make_shared<Value>(std::move(value)), std::move(cb), permanent);
}
void putEncrypted(InfoHash hash, const PkId& to, Value&& value, DoneCallbackSimple cb, bool permanent = false)
{
putEncrypted(hash, to, std::make_shared<Value>(std::move(value)), bindDoneCb(cb), permanent);
}
/**
* Insert known nodes to the routing table, without necessarly ping them.
@@ -406,15 +396,17 @@ public:
std::pair<size_t, size_t> getStoreSize() const;
void getStorageLimit() const;
void setStorageLimit(size_t limit = DEFAULT_STORAGE_LIMIT);
size_t getStorageLimit() const;
void setStorageLimit(size_t limit = 0);
size_t getLocalStorageLimit() const;
void setLocalStorageLimit(size_t limit = 0);
std::vector<NodeExport> exportNodes() const;
std::vector<ValuesExport> exportValues() const;
void setLogger(const Sp<Logger>& logger = {});
void setLogger(const Logger& logger) { setLogger(std::make_shared<Logger>(logger)); }
/**
* Only print logs related to the given InfoHash (if given), or disable filter (if zeroes).
@@ -448,12 +440,9 @@ public:
void getPublicAddress(std::function<void(std::vector<SockAddr>&&)>, sa_family_t af = AF_UNSPEC);
// securedht methods
void findCertificate(InfoHash hash, std::function<void(const std::shared_ptr<crypto::Certificate>&)>);
void findCertificate(PkId hash, std::function<void(const std::shared_ptr<crypto::Certificate>&)>);
void registerCertificate(const std::shared_ptr<crypto::Certificate>& cert);
void setLocalCertificateStore(CertificateStoreQueryLegacy&& query_method);
void setLocalCertificateStore(CertificateStoreQuery&& query_method);
/**
@@ -496,7 +485,7 @@ public:
*/
time_point loop()
{
std::lock_guard<std::mutex> lck(dht_mtx);
std::lock_guard lck(dht_mtx);
return loop_();
}
@@ -576,16 +565,16 @@ private:
*/
void resetDht();
mutable std::mutex dht_mtx {};
mutable std::mutex dht_mtx;
std::thread dht_thread {};
std::condition_variable cv {};
std::mutex sock_mtx {};
std::mutex sock_mtx;
net::PacketList rcv {};
decltype(rcv) rcv_free {};
std::queue<std::function<void(SecureDht&)>> pending_ops_prio {};
std::queue<std::function<void(SecureDht&)>> pending_ops {};
std::mutex storage_mtx {};
std::mutex storage_mtx;
std::atomic<State> running {State::Idle};
std::atomic_size_t ongoing_ops {0};
+8 -14
View File
@@ -51,11 +51,11 @@ using ConnectHandlerCb = std::function<void(const asio::error_code& ec, const as
using ssl_socket_t = restinio::impl::tls_socket_t;
using socket_t = asio::ip::tcp::socket;
class OPENDHT_PUBLIC Url
struct OPENDHT_PUBLIC Url
{
public:
Url() = default;
Url(std::string_view url);
Url(std::string url);
std::string url;
std::string protocol {"http"};
std::string_view user;
@@ -149,7 +149,7 @@ public:
using ResolverCb
= std::function<void(const asio::error_code& ec, const std::vector<asio::ip::tcp::endpoint>& endpoints)>;
Resolver(asio::io_context& ctx, const std::string& url, std::shared_ptr<log::Logger> logger = {});
Resolver(asio::io_context& ctx, std::string url, std::shared_ptr<log::Logger> logger = {});
Resolver(asio::io_context& ctx,
std::string_view host,
std::string_view service,
@@ -161,10 +161,6 @@ public:
std::vector<asio::ip::tcp::endpoint> endpoints,
const bool ssl = false,
std::shared_ptr<log::Logger> logger = {});
Resolver(asio::io_context& ctx,
const std::string& url,
std::vector<asio::ip::tcp::endpoint> endpoints,
std::shared_ptr<log::Logger> logger = {});
~Resolver();
@@ -216,19 +212,19 @@ public:
// resolves implicitly
Request(asio::io_context& ctx,
const std::string& url,
std::string url,
const Json::Value& json,
OnJsonCb jsoncb,
std::shared_ptr<log::Logger> logger = {});
Request(asio::io_context& ctx, const std::string& url, OnJsonCb jsoncb, std::shared_ptr<log::Logger> logger = {});
Request(asio::io_context& ctx, std::string url, OnJsonCb jsoncb, std::shared_ptr<log::Logger> logger = {});
Request(asio::io_context& ctx, const std::string& url, std::shared_ptr<log::Logger> logger = {});
Request(asio::io_context& ctx, std::string url, std::shared_ptr<log::Logger> logger = {});
Request(asio::io_context& ctx,
std::string_view host,
std::string_view service,
const bool ssl = false,
std::shared_ptr<log::Logger> logger = {});
Request(asio::io_context& ctx, const std::string& url, OnDoneCb onDone, std::shared_ptr<log::Logger> logger = {});
Request(asio::io_context& ctx, std::string url, OnDoneCb onDone, std::shared_ptr<log::Logger> logger = {});
// user defined resolver
Request(asio::io_context& ctx, std::shared_ptr<Resolver> resolver, sa_family_t family = AF_UNSPEC);
@@ -344,8 +340,6 @@ private:
dht::crypto::Identity client_identity_;
std::shared_ptr<dht::crypto::Certificate> server_ca_;
std::string service_;
std::string host_;
unsigned int id_;
static std::atomic_uint ids_;
+98 -51
View File
@@ -42,8 +42,7 @@ OPENDHT_PUBLIC void hash(const uint8_t* data, size_t data_length, uint8_t* hash,
}
/**
* Represents an Hash,
* a byte array of N bytes.
* Represents an Hash, a byte array of N bytes.
* Hashes identify nodes and values in the Dht.
*/
template<size_t N>
@@ -54,31 +53,25 @@ public:
typedef typename T::iterator iterator;
typedef typename T::const_iterator const_iterator;
Hash() noexcept { data_.fill(0); }
Hash(const uint8_t* h, size_t data_len)
{
if (data_len < N)
data_.fill(0);
else
std::copy_n(h, N, data_.begin());
}
constexpr Hash() noexcept
: data_ {}
{}
constexpr Hash(const uint8_t* h, size_t data_len)
: data_(fromBytes(h, data_len))
{}
/**
* Constructor from an hexadecimal string (without "0x").
* hex must be at least 2.HASH_LEN characters long.
* If too long, only the first 2.HASH_LEN characters are read.
* hex must be at least 2.N characters long.
* If too long, only the first 2.N characters are read.
*/
explicit Hash(std::string_view hex)
{
if (hex.size() < 2 * N)
data_.fill(0);
else
fromString(hex.data());
}
explicit constexpr Hash(std::string_view hex)
: data_(fromString(hex))
{}
Hash(const msgpack::object& o) { msgpack_unpack(o); }
static constexpr size_t size() noexcept { return N; }
const uint8_t* data() const { return data_.data(); }
constexpr const uint8_t* data() const { return data_.data(); }
uint8_t* data() { return data_.data(); }
iterator begin() { return data_.begin(); }
const_iterator cbegin() const { return data_.cbegin(); }
@@ -87,10 +80,21 @@ public:
static constexpr inline Hash zero() noexcept { return Hash {}; }
bool operator==(const Hash& h) const { return data_ == h.data_; }
bool operator!=(const Hash& h) const { return !(*this == h); }
constexpr bool operator==(const Hash& h) const
{
#if __cplusplus >= 202002L
return data_ == h.data_;
#else
for (size_t i = 0; i < N; ++i) {
if (data_[i] != h.data_[i])
return false;
}
return true;
#endif
}
constexpr bool operator!=(const Hash& h) const { return !(*this == h); }
bool operator<(const Hash& o) const
constexpr bool operator<(const Hash& o) const
{
for (unsigned i = 0; i < N; i++) {
if (data_[i] != o.data_[i])
@@ -99,7 +103,7 @@ public:
return false;
}
Hash operator^(const Hash& o) const
constexpr Hash operator^(const Hash& o) const
{
Hash result;
for (auto i = 0u; i < N; i++) {
@@ -108,8 +112,9 @@ public:
return result;
}
explicit operator bool() const
explicit constexpr operator bool() const
{
#if __cplusplus >= 202002L
auto a = reinterpret_cast<const uint32_t*>(data_.data());
auto b = reinterpret_cast<const uint32_t*>(data_.data() + N);
for (; a != b; a++) {
@@ -117,24 +122,32 @@ public:
return true;
}
return false;
#else
for (size_t i = 0; i < N; ++i) {
if (data_[i] != 0)
return true;
}
return false;
#endif
}
uint8_t& operator[](size_t index) { return data_[index]; }
const uint8_t& operator[](size_t index) const { return data_[index]; }
constexpr const uint8_t& operator[](size_t index) const { return data_[index]; }
/**
* Find the lowest 1 bit in an id.
* Result will allways be lower than 8*N
*/
inline int lowbit() const
constexpr int lowbit() const
{
int i, j;
for (i = N - 1; i >= 0; i--)
int i = N - 1;
for (; i >= 0; i--)
if (data_[i] != 0)
break;
if (i < 0)
return -1;
for (j = 7; j >= 0; j--)
int j = 7;
for (; j >= 0; j--)
if ((data_[i] & (0x80 >> j)) != 0)
break;
return 8 * i + j;
@@ -146,11 +159,10 @@ public:
}
/** Find how many bits two ids have in common. */
static inline unsigned commonBits(const Hash& id1, const Hash& id2)
static constexpr inline unsigned commonBits(const Hash& id1, const Hash& id2)
{
unsigned i, j;
uint8_t x;
for (i = 0; i < N; i++) {
unsigned i = 0;
for (; i < N; i++) {
if (id1.data_[i] != id2.data_[i])
break;
}
@@ -158,9 +170,9 @@ public:
if (i == N)
return 8 * N;
x = id1.data_[i] ^ id2.data_[i];
uint8_t x = id1.data_[i] ^ id2.data_[i];
j = 0;
unsigned j = 0;
while ((x & 0x80) == 0) {
x <<= 1;
j++;
@@ -170,7 +182,7 @@ public:
}
/** Determine whether id1 or id2 is closer to this */
int xorCmp(const Hash& id1, const Hash& id2) const
constexpr int xorCmp(const Hash& id1, const Hash& id2) const
{
for (unsigned i = 0; i < N; i++) {
if (id1.data_[i] == id2.data_[i])
@@ -182,7 +194,7 @@ public:
return 0;
}
bool getBit(unsigned nbit) const
constexpr bool getBit(unsigned nbit) const
{
auto& num = *(data_.cbegin() + (nbit / 8));
unsigned bit = 7 - (nbit % 8);
@@ -257,8 +269,9 @@ public:
}
private:
T data_;
void fromString(const char*);
alignas(alignof(uint32_t)) T data_;
static constexpr T fromBytes(const uint8_t*, size_t);
static constexpr T fromString(std::string_view);
};
#define HASH_LEN 20u
@@ -281,30 +294,51 @@ operator>>(std::istream& s, Hash<N>& h)
std::array<char, h.size() * 2> dat;
s.exceptions(std::istream::eofbit | std::istream::failbit);
s.read(&(*dat.begin()), dat.size());
fromString(dat.data());
h = Hash<N>(std::string_view(dat.data(), dat.size()));
return s;
}
template<size_t N>
void
Hash<N>::fromString(const char* in)
constexpr typename Hash<N>::T
Hash<N>::fromBytes(const uint8_t* in, size_t size)
{
auto hex2bin = [](char c) -> uint8_t {
T data {};
if (size < N)
return data;
for (size_t i = 0; i < N; ++i)
data[i] = in[i];
return data;
}
template<size_t N>
constexpr typename Hash<N>::T
Hash<N>::fromString(std::string_view in)
{
T data {};
if (in.size() < 2 * N)
return data;
constexpr auto hex2bin = [](char c, bool& valid) -> uint8_t {
if (c >= 'a' and c <= 'f')
return 10 + c - 'a';
else if (c >= 'A' and c <= 'F')
return 10 + c - 'A';
else if (c >= '0' and c <= '9')
return c - '0';
else
throw std::domain_error("not an hex character");
else {
valid = false;
return 0;
}
};
try {
for (size_t i = 0; i < N; i++)
data_[i] = (hex2bin(in[2 * i]) << 4) | hex2bin(in[2 * i + 1]);
} catch (const std::domain_error&) {
data_.fill(0);
bool valid = true;
for (size_t i = 0; i < N && valid; i++) {
uint8_t high = hex2bin(in[2 * i], valid);
uint8_t low = hex2bin(in[2 * i + 1], valid);
data[i] = (high << 4) | low;
}
if (!valid)
return {};
return data;
}
template<size_t N>
@@ -369,6 +403,19 @@ toHex(const std::vector<uint8_t>& data)
return toHex(data.data(), data.size());
}
template<size_t N>
constexpr std::array<char, N * 2>
toHexArray(const Hash<N>& h)
{
std::array<char, N * 2> ret {};
for (size_t i = 0; i < N; i++) {
const auto& m = hex_map[h[i]];
ret[i * 2] = m[0];
ret[i * 2 + 1] = m[1];
}
return ret;
}
template<size_t N>
const char*
Hash<N>::to_c_str() const
+151 -65
View File
@@ -8,111 +8,197 @@
#include <fmt/printf.h>
#include <functional>
#include <mutex>
#include <string_view>
#include <type_traits>
#include <cstdarg>
#if __cplusplus >= 202002L
#include <source_location>
#endif
namespace dht {
namespace log {
#if __cplusplus >= 202002L
/**
* Extracts the filename from a full file path at compile time.
*/
consteval std::string_view
getfilename(std::string_view path)
{
size_t pos = path.size();
for (size_t i = path.size(); i-- > 0;) {
if (path[i] == '/' || path[i] == '\\') {
pos = i + 1;
break;
}
}
return path.substr(pos);
}
consteval std::string_view
getfunctionname(std::string_view path)
{
size_t pos = 0;
size_t end = path.size();
for (size_t i = 0; i < path.size(); i++) {
if (path[i] == ' ' || path[i] == '\t' || path[i] == ':') {
pos = i + 1;
} else if (path[i] == '(') {
end = i;
break;
}
}
return path.substr(pos, end - pos);
}
#endif
struct source_loc
{
std::string_view file;
std::uint_least32_t line;
std::string_view function;
#if __cplusplus >= 202002L
consteval source_loc(const std::source_location& loc)
: file(getfilename(loc.file_name()))
, line(loc.line())
, function(getfunctionname(loc.function_name()))
{}
#else
constexpr source_loc()
: file()
, line(0)
, function()
{}
#endif
};
enum class LogLevel { debug, warning, error };
using LogMethod = std::function<void(LogLevel, std::string&&)>;
using LogMethod = std::function<void(source_loc, LogLevel, std::string_view, std::string&&)>;
template<typename... Args>
struct LogFormat
{
fmt::format_string<Args...> fmt;
source_loc loc;
#if __cplusplus >= 202002L
template<typename S>
consteval LogFormat(const S& s, const std::source_location& l = std::source_location::current())
: fmt(s)
, loc(l)
{}
#else
template<typename S>
constexpr LogFormat(const S& s)
: fmt(s)
, loc()
{}
#endif
};
#if __cplusplus < 202002L
template<class _Tp>
struct type_identity
{
typedef _Tp type;
};
template<class _Tp>
using type_identity_t = typename type_identity<_Tp>::type;
#else
using std::type_identity_t;
#endif
struct OPENDHT_PUBLIC Logger
{
Logger() = delete;
Logger(LogMethod&& l)
: logger(std::move(l))
Logger(LogMethod&& l, std::string tag = "")
: logger_(std::move(l))
, tag_(std::move(tag))
, prefix_(tag_.empty() ? "" : fmt::format("[{}] ", tag_))
{
if (!logger)
throw std::invalid_argument {"logger and loggerf must be set"};
if (!logger_)
throw std::invalid_argument {"logger must be set"};
}
void setFilter(const InfoHash& f)
Logger(const Logger& parent, std::string tag)
: logger_(parent.logger_)
, tag_(std::move(tag))
, prefix_(fmt::format("{}[{}] ", parent.prefix_, tag_))
{}
std::shared_ptr<Logger> createChild(std::string tag)
{
filter_ = f;
filterEnable_ = static_cast<bool>(filter_);
std::lock_guard lock {children_mutex_};
auto child = std::make_shared<Logger>(*this, std::move(tag));
children_.push_back(child);
return child;
}
inline void log0(LogLevel level, fmt::string_view format, fmt::printf_args args) const
void setFilter(std::string_view filter)
{
if (not filterEnable_)
logger(level, fmt::vsprintf(format, args));
std::lock_guard lock {children_mutex_};
enable_ = filter.empty() or tag_.find(filter) != std::string_view::npos;
for (auto it = children_.begin(); it != children_.end();) {
if (auto c = it->lock()) {
c->setFilter(enable_ ? std::string_view {} : filter);
++it;
} else {
it = children_.erase(it);
}
}
}
inline void log1(LogLevel level, const InfoHash& f, fmt::string_view format, fmt::printf_args args) const
void setFilter(const InfoHash& f) { setFilter(f.to_view()); }
inline void log0(source_loc loc, LogLevel level, fmt::string_view format, fmt::printf_args args) const
{
if (not filterEnable_ or f == filter_)
logger(level, fmt::vsprintf(format, args));
}
inline void log2(
LogLevel level, const InfoHash& f1, const InfoHash& f2, fmt::string_view format, fmt::printf_args args) const
{
if (not filterEnable_ or f1 == filter_ or f2 == filter_)
logger(level, fmt::vsprintf(format, args));
if (enable_)
logger_(loc, level, prefix_, fmt::vsprintf(format, args));
}
template<typename... Args>
inline void debug(fmt::format_string<Args...> format, Args&&... args) const
inline void debug(LogFormat<type_identity_t<Args>...> format, Args&&... args) const
{
logger(LogLevel::debug, fmt::format(format, std::forward<Args>(args)...));
if (enable_)
logger_(format.loc, LogLevel::debug, prefix_, fmt::format(format.fmt, std::forward<Args>(args)...));
}
template<typename... Args>
inline void warn(fmt::format_string<Args...> format, Args&&... args) const
inline void warn(LogFormat<type_identity_t<Args>...> format, Args&&... args) const
{
logger(LogLevel::warning, fmt::format(format, std::forward<Args>(args)...));
if (enable_)
logger_(format.loc, LogLevel::warning, prefix_, fmt::format(format.fmt, std::forward<Args>(args)...));
}
template<typename... Args>
inline void error(fmt::format_string<Args...> format, Args&&... args) const
inline void error(LogFormat<type_identity_t<Args>...> format, Args&&... args) const
{
logger(LogLevel::error, fmt::format(format, std::forward<Args>(args)...));
if (enable_)
logger_(format.loc, LogLevel::error, prefix_, fmt::format(format.fmt, std::forward<Args>(args)...));
}
template<typename... T>
inline void d(fmt::format_string<T...> format, T&&... args) const
inline void d(LogFormat<type_identity_t<T>...> format, T&&... args) const
{
log0(LogLevel::debug, format, fmt::make_printf_args(args...));
log0(format.loc, LogLevel::debug, format.fmt, fmt::make_printf_args(args...));
}
template<typename... T>
inline void d(const InfoHash& f, fmt::format_string<T...> format, T&&... args) const
inline void w(LogFormat<type_identity_t<T>...> format, T&&... args) const
{
log1(LogLevel::debug, f, format, fmt::make_printf_args(args...));
log0(format.loc, LogLevel::warning, format.fmt, fmt::make_printf_args(args...));
}
template<typename... T>
inline void d(const InfoHash& f1, const InfoHash& f2, fmt::format_string<T...> format, T&&... args) const
inline void e(LogFormat<type_identity_t<T>...> format, T&&... args) const
{
log2(LogLevel::debug, f1, f2, format, fmt::make_printf_args(args...));
}
template<typename... T>
inline void w(fmt::format_string<T...> format, T&&... args) const
{
log0(LogLevel::warning, format, fmt::make_printf_args(args...));
}
template<typename... T>
inline void w(const InfoHash& f, fmt::format_string<T...> format, T&&... args) const
{
log1(LogLevel::warning, f, format, fmt::make_printf_args(args...));
}
template<typename... T>
inline void w(const InfoHash& f1, const InfoHash& f2, fmt::format_string<T...> format, T&&... args) const
{
log2(LogLevel::warning, f1, f2, format, fmt::make_printf_args(args...));
}
template<typename... T>
inline void e(fmt::format_string<T...> format, T&&... args) const
{
log0(LogLevel::error, format, fmt::make_printf_args(args...));
}
template<typename... T>
inline void e(const InfoHash& f, fmt::format_string<T...> format, T&&... args) const
{
log1(LogLevel::error, f, format, fmt::make_printf_args(args...));
}
template<typename... T>
inline void e(const InfoHash& f1, const InfoHash& f2, fmt::format_string<T...> format, T&&... args) const
{
log2(LogLevel::error, f1, f2, format, fmt::make_printf_args(args...));
log0(format.loc, LogLevel::error, format.fmt, fmt::make_printf_args(args...));
}
private:
LogMethod logger = {};
bool filterEnable_ {false};
InfoHash filter_ {};
const LogMethod logger_ = {};
const std::string tag_ {};
const std::string prefix_ {};
std::mutex children_mutex_;
std::vector<std::weak_ptr<Logger>> children_ {};
bool enable_ {true};
};
} // namespace log
+8 -3
View File
@@ -489,8 +489,12 @@ private:
static constexpr size_t MTU {1280};
static constexpr size_t MAX_PACKET_VALUE_SIZE {600};
static constexpr size_t MAX_MESSAGE_VALUE_SIZE {56 * 1024};
static constexpr size_t RX_MAX_EARLY_BYTES {MAX_MESSAGE_VALUE_SIZE + MAX_VALUE_SIZE + 32};
static constexpr size_t RX_MAX_EARLY_PARTS {RX_MAX_EARLY_BYTES / MTU + 2};
void process(std::unique_ptr<ParsedMessage>&&, const SockAddr& from);
bool isExpectedReply(const ParsedMessage&, const SockAddr& from);
bool tryCompleteMessage(PartialMessage&, const SockAddr& from);
bool rateLimit(const SockAddr& addr);
@@ -527,7 +531,8 @@ private:
{
return packValueHeader(buf, values.begin(), values.end());
}
void maintainRxBuffer(Tid tid);
using PartialMessageKey = std::pair<SockAddr, Tid>;
void maintainRxBuffer(const PartialMessageKey& key);
/*************
* Answers *
@@ -547,7 +552,7 @@ private:
std::pair<Blob, Blob> bufferNodes(
sa_family_t af, const InfoHash& id, want_t want, std::vector<Sp<Node>>& nodes, std::vector<Sp<Node>>& nodes6);
/* answer to a listen request */
void sendListenConfirmation(const SockAddr& addr, Tid tid);
void sendListenConfirmation(const SockAddr& addr, Tid tid, const Blob& token = {});
/* answer to put request */
void sendValueAnnounced(const SockAddr& addr, Tid, Value::Id);
/* answer in case of error */
@@ -573,7 +578,7 @@ private:
// requests handling
std::map<Tid, Sp<Request>> requests {};
std::map<Tid, PartialMessage> partial_messages;
std::map<PartialMessageKey, PartialMessage> partial_messages;
MessageStats in_stats {}, out_stats {};
std::set<SockAddr> blacklist {};
+6 -6
View File
@@ -58,7 +58,7 @@ public:
inline void setOnReceive(OnReceive&& cb)
{
std::lock_guard<std::mutex> lk(lock);
std::lock_guard lk(lock);
rx_callback = std::move(cb);
}
@@ -67,12 +67,12 @@ public:
SockAddr getBound(sa_family_t family = AF_UNSPEC) const
{
std::lock_guard<std::mutex> lk(lock);
std::lock_guard lk(lock);
return getBoundRef(family);
}
in_port_t getPort(sa_family_t family = AF_UNSPEC) const
{
std::lock_guard<std::mutex> lk(lock);
std::lock_guard lk(lock);
return getBoundRef(family).getPort();
}
@@ -102,7 +102,7 @@ protected:
inline void onReceived(PacketList&& packets)
{
std::lock_guard<std::mutex> lk(lock);
std::lock_guard lk(lock);
if (rx_callback) {
auto r = rx_callback(std::move(packets));
if (not r.empty() and toRecycle_.size() < RX_QUEUE_MAX_SIZE)
@@ -134,12 +134,12 @@ public:
bool hasIPv4() const override
{
std::lock_guard<std::mutex> lk(lock);
std::lock_guard lk(lock);
return s4 != -1;
}
bool hasIPv6() const override
{
std::lock_guard<std::mutex> lk(lock);
std::lock_guard lk(lock);
return s6 != -1;
}
+1 -1
View File
@@ -8,7 +8,7 @@ namespace dht {
namespace proxy {
constexpr const std::chrono::hours OP_TIMEOUT {24}; // one day
constexpr const std::chrono::hours OP_MARGIN {2}; // two hours
constexpr const std::chrono::hours OP_MARGIN {8}; // eight hours
using ListenToken = uint64_t;
+23 -30
View File
@@ -113,14 +113,6 @@ public:
* and put it in the DHT.
* The operation will be immediate if the recipient' public key is known (otherwise it will be retrived first).
*/
void putEncrypted(
const InfoHash& hash, const InfoHash& to, Sp<Value> val, DoneCallback callback, bool permanent = false);
[[deprecated("Use the shared_ptr version instead")]]
void putEncrypted(const InfoHash& hash, const InfoHash& to, Value&& v, DoneCallback callback, bool permanent = false)
{
putEncrypted(hash, to, std::make_shared<Value>(std::move(v)), callback, permanent);
}
void putEncrypted(const InfoHash& hash,
const crypto::PublicKey& to,
Sp<Value> val,
@@ -145,17 +137,11 @@ public:
Value decrypt(const Value& v);
void findCertificate(const InfoHash& node, const std::function<void(const Sp<crypto::Certificate>)>& cb);
void findPublicKey(const InfoHash& node, const std::function<void(const Sp<crypto::PublicKey>)>& cb);
void findCertificate(const PkId& id, const std::function<void(const Sp<crypto::Certificate>)>& cb);
void findPublicKey(const PkId& id, const std::function<void(const Sp<crypto::PublicKey>)>& cb);
void registerCertificate(const Sp<crypto::Certificate>& cert);
Sp<crypto::Certificate> getCertificate(const InfoHash& node) const;
Sp<crypto::PublicKey> getPublicKey(const InfoHash& node) const;
Sp<crypto::Certificate> getCertificate(const PkId& node) const;
Sp<crypto::PublicKey> getPublicKey(const PkId& node) const;
@@ -164,10 +150,6 @@ public:
* The search key used is the public key ID, so there may be multiple certificates retured, signed with
* the same private key.
*/
void setLocalCertificateStore(CertificateStoreQueryLegacy&& query_method)
{
localQueryMethodLegacy_ = std::move(query_method);
}
void setLocalCertificateStore(CertificateStoreQuery&& query_method) { localQueryMethod_ = std::move(query_method); }
void setOnPublicAddressChanged(PublicAddressChangedCb cb) override { dht_->setOnPublicAddressChanged(cb); }
@@ -179,15 +161,35 @@ public:
inline const InfoHash& getNodeId() const override { return dht_->getNodeId(); }
std::pair<size_t, size_t> getStoreSize() const override { return dht_->getStoreSize(); }
std::pair<size_t, size_t> getLocalStoreSize() const override { return dht_->getLocalStoreSize(); }
std::string getStorageLog() const override { return dht_->getStorageLog(); }
std::string getStorageLog(const InfoHash& h) const override { return dht_->getStorageLog(h); }
void setStorageLimit(size_t limit = DEFAULT_STORAGE_LIMIT) override { dht_->setStorageLimit(limit); }
void setStorageLimit(size_t limit = 0) override { dht_->setStorageLimit(limit); }
size_t getStorageLimit() const override { return dht_->getStorageLimit(); }
void setLocalStorageLimit(size_t limit = 0) override { dht_->setLocalStorageLimit(limit); }
size_t getLocalStorageLimit() const override { return dht_->getLocalStorageLimit(); }
std::vector<NodeExport> exportNodes() const override { return dht_->exportNodes(); }
std::vector<ValuesExport> exportValues() const override { return dht_->exportValues(); }
void importValues(const std::vector<ValuesExport>& v) override { dht_->importValues(v); }
NodeStats getNodesStats(sa_family_t af) const override { return dht_->getNodesStats(af); }
NodeInfo getNodeInfo()
{
NodeInfo info;
info.id = getId();
info.node_id = getNodeId();
info.ipv4 = getNodesStats(AF_INET);
info.ipv6 = getNodesStats(AF_INET6);
std::tie(info.storage_size, info.storage_values) = getStoreSize();
std::tie(info.local_storage_size, info.local_storage_values) = getLocalStoreSize();
if (auto sock = getSocket()) {
info.bound4 = sock->getBoundRef(AF_INET).getPort();
info.bound6 = sock->getBoundRef(AF_INET6).getPort();
}
return info;
}
std::vector<unsigned> getNodeMessageStats(bool in = false) override { return dht_->getNodeMessageStats(in); }
std::string getRoutingTablesLog(sa_family_t af) const override { return dht_->getRoutingTablesLog(af); }
std::string getSearchesLog(sa_family_t af) const override { return dht_->getSearchesLog(af); }
@@ -292,12 +294,6 @@ public:
return dht_->pushNotificationReceived(notification);
}
void setLogger(const Logger& logger) override
{
DhtInterface::setLogger(logger);
dht_->setLogger(logger);
}
void setLogger(const std::shared_ptr<Logger>& logger) override
{
DhtInterface::setLogger(logger);
@@ -331,13 +327,10 @@ private:
// method to query the local certificate store
CertificateStoreQuery localQueryMethod_ {};
CertificateStoreQueryLegacy localQueryMethodLegacy_ {};
// our certificate cache
std::map<InfoHash, Sp<crypto::Certificate>> nodesCertificates_ {};
std::map<InfoHash, Sp<crypto::PublicKey>> nodesPubKeys_ {};
std::map<PkId, Sp<crypto::Certificate>> nodesCertificatesLong_ {};
std::map<PkId, Sp<crypto::PublicKey>> nodesPubKeysLong_ {};
std::map<PkId, Sp<crypto::Certificate>> nodesCertificates_ {};
std::map<PkId, Sp<crypto::PublicKey>> nodesPubKeys_ {};
std::atomic_bool forward_all_ {false};
bool enableCache_ {false};
+3 -2
View File
@@ -98,6 +98,9 @@ public:
{
return len == o.len && std::memcmp((const uint8_t*) get(), (const uint8_t*) o.get(), len) == 0;
}
bool operator==(const SockAddr& o) const { return equals(o); }
bool operator!=(const SockAddr& o) const { return !equals(o); }
SockAddr& operator=(const SockAddr& o)
{
set(o.get(), o.getLength());
@@ -327,8 +330,6 @@ private:
}
};
OPENDHT_PUBLIC bool operator==(const SockAddr& a, const SockAddr& b);
} // namespace dht
#if FMT_VERSION >= 90000
+12 -8
View File
@@ -10,7 +10,9 @@
#include <future>
#include <functional>
#include <ciso646> // fix windows compiler bug
#if __cplusplus < 202002L
#include <ciso646>
#endif
namespace dht {
@@ -53,12 +55,14 @@ public:
void detach();
private:
std::mutex lock_ {};
std::mutex lock_;
std::condition_variable cv_ {};
std::condition_variable stopCv_ {};
std::queue<std::function<void()>> tasks_ {};
std::vector<std::unique_ptr<std::thread>> threads_;
unsigned readyThreads_ {0};
bool running_ {true};
bool stopping_ {false};
unsigned minThreads_;
const unsigned maxThreads_;
@@ -81,7 +85,7 @@ public:
private:
std::reference_wrapper<ThreadPool> threadPool_;
const unsigned maxConcurrent_ {1};
std::mutex lock_ {};
std::mutex lock_;
unsigned current_ {0};
std::queue<std::function<void()>> tasks_ {};
@@ -104,7 +108,7 @@ public:
void run(std::function<void()>&& task)
{
std::lock_guard<std::mutex> lock(state_->mtx);
std::lock_guard lock(state_->mtx);
if (state_->shutdown_)
return;
state_->pendingTasks++;
@@ -114,7 +118,7 @@ public:
private:
struct SharedState
{
std::mutex mtx {};
std::mutex mtx;
std::condition_variable cv {};
unsigned pendingTasks {0};
unsigned ongoingTasks {0};
@@ -125,7 +129,7 @@ private:
void destroy(bool wait = true)
{
std::unique_lock<std::mutex> lock(mtx);
std::unique_lock lock(mtx);
if (destroyed)
return;
if (wait) {
@@ -141,7 +145,7 @@ private:
void run(const std::function<void()>& task)
{
{
std::lock_guard<std::mutex> lock(mtx);
std::lock_guard lock(mtx);
pendingTasks--;
ongoingTasks++;
}
@@ -149,7 +153,7 @@ private:
return;
task();
{
std::lock_guard<std::mutex> lock(mtx);
std::lock_guard lock(mtx);
ongoingTasks--;
cv.notify_all();
}
+19 -5
View File
@@ -85,8 +85,16 @@ using system_clock = std::chrono::system_clock;
using time_point = clock::time_point;
using duration = clock::duration;
time_point from_time_t(std::time_t t);
std::time_t to_time_t(time_point t);
OPENDHT_PUBLIC time_point from_system_time(system_clock::time_point t,
time_point steady_now,
system_clock::time_point system_now);
OPENDHT_PUBLIC time_point from_system_time(system_clock::time_point t);
OPENDHT_PUBLIC system_clock::time_point to_system_time(time_point t,
time_point steady_now,
system_clock::time_point system_now);
OPENDHT_PUBLIC system_clock::time_point to_system_time(time_point t);
OPENDHT_PUBLIC time_point from_time_t(std::time_t t);
OPENDHT_PUBLIC std::time_t to_time_t(time_point t);
template<class DT>
static std::string
@@ -126,6 +134,8 @@ print_time_relative(TimePoint now, TimePoint d)
return (d > now) ? concat("in "sv, print_duration(d - now)) : concat(print_duration(now - d), " ago"sv);
}
OPENDHT_PUBLIC std::string printByteCount(size_t bytes);
template<typename Duration = duration>
class uniform_duration_distribution : public std::uniform_int_distribution<typename Duration::rep>
{
@@ -172,15 +182,19 @@ packMsg(const Type& t)
template<typename Type>
Type
unpackMsg(Blob b)
unpackMsg(const Blob& b)
{
msgpack::unpacked msg_res = msgpack::unpack((const char*) b.data(), b.size());
return msg_res.get().as<Type>();
}
msgpack::unpacked unpackMsg(Blob b);
inline msgpack::unpacked
unpackMsg(const Blob& b)
{
return msgpack::unpack((const char*) b.data(), b.size());
}
msgpack::object* findMapValue(const msgpack::object& map, const char* key, size_t length);
OPENDHT_PUBLIC msgpack::object* findMapValue(const msgpack::object& map, const char* key, size_t length);
inline msgpack::object*
findMapValue(const msgpack::object& map, const char* key)
+17 -4
View File
@@ -31,6 +31,7 @@ using namespace std::literals;
static constexpr auto VALUE_KEY_ID("id");
static const std::string VALUE_KEY_DAT("dat");
static const std::string VALUE_KEY_PRIO("p");
static const std::string VALUE_KEY_PUSHTYPE("pt");
static const std::string VALUE_KEY_SIGNATURE("sig");
static const std::string VALUE_KEY_SEQ("seq");
@@ -451,6 +452,7 @@ struct OPENDHT_PUBLIC Value
, signature(std::move(o.signature))
, cypher(std::move(o.cypher))
, priority(o.priority)
, pushType(std::move(o.pushType))
{}
template<typename Type>
@@ -574,7 +576,7 @@ struct OPENDHT_PUBLIC Value
template<typename Packer>
void msgpack_pack(Packer& pk) const
{
pk.pack_map(2 + (priority ? 1 : 0));
pk.pack_map(2 + (priority ? 1 : 0) + (!pushType.empty() ? 1 : 0));
pk.pack(VALUE_KEY_ID);
pk.pack(id);
pk.pack(VALUE_KEY_DAT);
@@ -583,6 +585,10 @@ struct OPENDHT_PUBLIC Value
pk.pack(VALUE_KEY_PRIO);
pk.pack(priority);
}
if (!pushType.empty()) {
pk.pack(VALUE_KEY_PUSHTYPE);
pk.pack(pushType);
}
}
template<typename Packer>
@@ -672,6 +678,13 @@ struct OPENDHT_PUBLIC Value
*/
unsigned priority {0};
/**
* Push notification type — set by the sender so that the proxy server
* can include it in the FCM/APNs payload without decrypting the value.
* Example: "audioCall", "videoCall"
*/
std::string pushType {};
inline bool isSignatureChecked() const { return signatureChecked; }
inline bool isDecrypted() const { return decrypted; }
bool checkSignature();
@@ -1026,9 +1039,9 @@ struct OPENDHT_PUBLIC Query
void msgpack_pack(Packer& pk) const
{
pk.pack_map(2);
pk.pack(std::string("s"));
pk.pack("s"sv);
pk.pack(select); /* packing field selectors */
pk.pack(std::string("w"));
pk.pack("w"sv);
pk.pack(where); /* packing filters */
}
@@ -1081,7 +1094,7 @@ template<typename T, typename std::enable_if<std::is_base_of<Value::Serializable
Value::Filter
getFilterSet(Value::Filter f)
{
return Value::Filter::chain({Value::TypeFilter(T::TYPE), T::getFilter(), f});
return Value::Filter::chain({Value::TypeFilter(T::TYPE), T::getFilter(), std::move(f)});
}
template<typename T, typename std::enable_if<!std::is_base_of<Value::SerializableBase, T>::value, T>::type* = nullptr>
+85 -62
View File
@@ -5,9 +5,9 @@ project(
'cython',
license: 'MIT',
license_files: 'LICENSE',
version: '3.7.0',
version: '4.3.0',
meson_version: '>=1.1.0',
default_options: ['cpp_std=c++17', 'warning_level=3'],
default_options: ['cpp_std=c++20', 'warning_level=3'],
)
gnutls = dependency('gnutls')
@@ -15,10 +15,23 @@ nettle = dependency('nettle')
msgpack = dependency('msgpack-cxx', required: false)
argon2 = dependency('libargon2')
fmt = dependency('fmt')
openssl = dependency('openssl', required: get_option('proxy_client'))
jsoncpp = dependency('jsoncpp', required: get_option('proxy_client'))
llhttp = dependency('llhttp', 'libllhttp', required: get_option('proxy_client'))
simdutf = dependency('simdutf', required: get_option('proxy_client'))
proxy_enabled = get_option('proxy_client').enabled() or get_option('proxy_server').enabled()
openssl = dependency('openssl', required: proxy_enabled)
jsoncpp = dependency('jsoncpp', required: proxy_enabled)
llhttp = dependency('llhttp', 'libllhttp', required: proxy_enabled)
simdutf = dependency('simdutf', required: proxy_enabled)
# restinio is a header-only library that may not ship a pkg-config file
restinio_dep = dependency('restinio', required: false)
if not restinio_dep.found()
cpp = meson.get_compiler('cpp')
if cpp.has_header('restinio/all.hpp', required: proxy_enabled)
restinio_dep = declare_dependency()
endif
endif
io_uring = host_machine.system() == 'linux' ? dependency('liburing', required: false) : disabler()
deps = [
@@ -33,6 +46,10 @@ deps = [
llhttp,
]
if restinio_dep.found()
deps += restinio_dep
endif
if host_machine.system() == 'linux' and io_uring.found()
deps += io_uring
endif
@@ -79,7 +96,7 @@ if get_option('indexation').enabled()
opendht_src += 'src/indexation/pht.cpp'
add_project_arguments('-DOPENDHT_INDEXATION', language: 'cpp')
endif
if jsoncpp.found()
if jsoncpp.found() and simdutf.found()
opendht_src += ['src/base64.cpp']
add_project_arguments('-DOPENDHT_JSONCPP', language: 'cpp')
conf_data.set('jsoncpp_lib', ', jsoncpp')
@@ -118,6 +135,17 @@ opendht = library(
install: true,
)
opendht_dep_args = []
if get_option('default_library') != 'static'
opendht_dep_args += '-Dopendht_EXPORTS'
endif
opendht_dep = declare_dependency(
link_with: opendht,
include_directories: opendht_interface_inc,
compile_args: opendht_dep_args,
)
# Install library headers.
install_subdir('include', install_dir: get_option('includedir'), strip_directory: true)
@@ -128,6 +156,13 @@ conf_data.set('libdir', join_paths(get_option('prefix'), get_option('libdir')))
conf_data.set('includedir', join_paths(get_option('prefix'), get_option('includedir')))
conf_data.set('argon2_lib', ', libargon2')
conf_data.set('simdutf_lib', ', simdutf')
conf_data.set('opendht_public_cflags', '')
conf_data.set('opendht_c_public_cflags', '')
if get_option('default_library') != 'static'
conf_data.set('opendht_public_cflags', ' -Dopendht_EXPORTS')
conf_data.set('opendht_c_public_cflags', ' -Dopendht_c_EXPORTS')
endif
configure_file(
input: 'opendht.pc.in',
@@ -143,9 +178,7 @@ if get_option('c').enabled()
opendht_c = library(
'opendht-c',
['c/opendht.cpp', 'c/opendht_c.h'],
include_directories: opendht_interface_inc,
link_with: opendht,
dependencies: [msgpack, jsoncpp, fmt],
dependencies: [opendht_dep, msgpack, jsoncpp, fmt],
cpp_args: ['-DOPENDHT_C_BUILD', '-Dopendht_c_EXPORTS'],
install: true,
)
@@ -181,10 +214,8 @@ if get_option('python').require(
py.extension_module(
'opendht',
'python/opendht.pyx',
include_directories: opendht_interface_inc,
override_options: ['cython_language=cpp'],
link_with: [opendht],
dependencies: [msgpack, jsoncpp, fmt, dep_py],
dependencies: [opendht_dep, msgpack, jsoncpp, fmt, dep_py],
install: true,
)
endif
@@ -193,25 +224,19 @@ if get_option('tools').enabled()
dhtnode = executable(
'dhtnode',
'tools/dhtnode.cpp',
include_directories: opendht_interface_inc,
link_with: opendht,
dependencies: [readline, jsoncpp, msgpack, fmt, openssl],
dependencies: [opendht_dep, readline, jsoncpp, msgpack, fmt, openssl],
install: true,
)
dhtchat = executable(
'dhtchat',
'tools/dhtchat.cpp',
include_directories: opendht_interface_inc,
link_with: opendht,
dependencies: [readline, jsoncpp, msgpack, fmt, openssl],
dependencies: [opendht_dep, readline, jsoncpp, msgpack, fmt, openssl],
install: true,
)
dhtscanner = executable(
'dhtscanner',
'tools/dhtscanner.cpp',
include_directories: opendht_interface_inc,
link_with: opendht,
dependencies: [readline, jsoncpp, msgpack, fmt, openssl],
dependencies: [opendht_dep, readline, jsoncpp, msgpack, fmt, openssl],
install: true,
)
if llhttp.found()
@@ -224,9 +249,7 @@ if get_option('tools').enabled()
durl = executable(
'durl',
'tools/durl.cpp',
include_directories: opendht_interface_inc,
link_with: opendht,
dependencies: durl_deps,
dependencies: [opendht_dep] + durl_deps,
)
endif
endif
@@ -237,62 +260,68 @@ if get_option('tests').enabled()
test_infohash = executable(
'test_infohash',
'tests/infohashtester.cpp',
'tests/test_infohash.cpp',
'tests/tests_runner.cpp',
include_directories: opendht_interface_inc,
link_with: opendht,
dependencies: [cppunit, jsoncpp, fmt, openssl, msgpack],
dependencies: [opendht_dep, cppunit, jsoncpp, fmt, openssl, msgpack],
)
test('Infohash', test_infohash)
test_value = executable(
'test_value',
'tests/valuetester.cpp',
'tests/test_value.cpp',
'tests/tests_runner.cpp',
include_directories: opendht_interface_inc,
link_with: opendht,
dependencies: [cppunit, jsoncpp, fmt, openssl, msgpack],
dependencies: [opendht_dep, cppunit, jsoncpp, fmt, openssl, msgpack],
)
test('Value', test_value)
test_parsedmessage = executable(
'test_parsedmessage',
'tests/test_parsedmessage.cpp',
'tests/tests_runner.cpp',
include_directories: opendht_inc,
dependencies: [opendht_dep, cppunit, jsoncpp, fmt, openssl, msgpack],
)
test('ParsedMessage', test_parsedmessage)
test_networkengine = executable(
'test_networkengine',
'tests/test_networkengine.cpp',
'tests/tests_runner.cpp',
include_directories: opendht_inc,
dependencies: [opendht_dep, cppunit, jsoncpp, fmt, openssl, msgpack],
)
test('NetworkEngine', test_networkengine)
test_crypto = executable(
'test_crypto',
'tests/cryptotester.cpp',
'tests/test_crypto.cpp',
'tests/tests_runner.cpp',
include_directories: opendht_interface_inc,
link_with: opendht,
dependencies: [cppunit, jsoncpp, fmt, openssl, msgpack, gnutls],
dependencies: [opendht_dep, cppunit, jsoncpp, fmt, openssl, msgpack, gnutls],
)
test('Crypto', test_crypto)
test('Crypto', test_crypto, timeout: 60)
test_dhtrunner = executable(
'test_dhtrunner',
'tests/dhtrunnertester.cpp',
'tests/test_dhtrunner.cpp',
'tests/tests_runner.cpp',
include_directories: opendht_interface_inc,
link_with: opendht,
dependencies: [cppunit, jsoncpp, fmt, openssl, msgpack],
dependencies: [opendht_dep, cppunit, jsoncpp, fmt, openssl, msgpack],
)
test('DhtRunner', test_dhtrunner)
test('DhtRunner', test_dhtrunner, timeout: 60)
test_threadpool = executable(
'test_threadpool',
'tests/threadpooltester.cpp',
'tests/test_threadpool.cpp',
'tests/tests_runner.cpp',
include_directories: opendht_interface_inc,
link_with: opendht,
dependencies: [cppunit, jsoncpp, fmt, openssl, msgpack],
dependencies: [opendht_dep, cppunit, jsoncpp, fmt, openssl, msgpack],
)
test('ThreadPool', test_threadpool)
if get_option('proxy_client').enabled() or get_option('proxy_server').enabled()
test_http = executable(
'test_http',
'tests/httptester.cpp',
'tests/test_http.cpp',
'tests/tests_runner.cpp',
include_directories: opendht_interface_inc,
link_with: opendht,
dependencies: [cppunit, jsoncpp, fmt, openssl, msgpack],
dependencies: [opendht_dep, cppunit, jsoncpp, fmt, openssl, msgpack],
)
test('Http', test_http)
endif
@@ -300,11 +329,9 @@ if get_option('tests').enabled()
if get_option('peer_discovery').enabled()
test_peerdiscovery = executable(
'test_peerdiscovery',
'tests/peerdiscoverytester.cpp',
'tests/test_peerdiscovery.cpp',
'tests/tests_runner.cpp',
include_directories: opendht_interface_inc,
link_with: opendht,
dependencies: [cppunit, jsoncpp, fmt, openssl, msgpack],
dependencies: [opendht_dep, cppunit, jsoncpp, fmt, openssl, msgpack],
)
test('PeerDiscovery', test_peerdiscovery)
endif
@@ -312,11 +339,9 @@ if get_option('tests').enabled()
if get_option('proxy_server').enabled()
test_proxy = executable(
'test_dhtproxy',
'tests/dhtproxytester.cpp',
'tests/test_dhtproxy.cpp',
'tests/tests_runner.cpp',
include_directories: opendht_interface_inc,
link_with: opendht,
dependencies: [cppunit, jsoncpp, fmt, openssl, msgpack],
dependencies: [opendht_dep, cppunit, jsoncpp, fmt, openssl, msgpack],
)
test('DhtProxy', test_proxy)
@@ -325,9 +350,7 @@ if get_option('tests').enabled()
'test_dhtproxystress',
'tests/dhtproxy_stress_tester.cpp',
'tests/tests_runner.cpp',
include_directories: opendht_interface_inc,
link_with: opendht,
dependencies: [cppunit, jsoncpp, fmt, openssl, msgpack],
dependencies: [opendht_dep, cppunit, jsoncpp, fmt, openssl, msgpack],
)
test('DhtProxy Stress', test_proxystress)
endif
+1 -1
View File
@@ -7,4 +7,4 @@ Description: Distributed Hash Table library (C Wrapper)
Version: @VERSION@
Libs: -L${libdir} -lopendht-c
Requires.private: opendht gnutls >= 3.1 @argon2_lib@
Cflags: -I${includedir}
Cflags: -I${includedir}@opendht_c_public_cflags@
+1 -1
View File
@@ -9,4 +9,4 @@ Libs: -L${libdir} -lopendht -lfmt
Libs.private: @http_lib@ -pthread
Requires: gnutls >= 3.3@jsoncpp_lib@@openssl_lib@
Requires.private: nettle >= 2.4@argon2_lib@@llhttp_lib@@iouring_lib@@simdutf_lib@
Cflags: -I${includedir}
Cflags: -I${includedir}@opendht_public_cflags@
-30
View File
@@ -1,30 +0,0 @@
if USE_CYTHON
noinst_HEADERS = \
opendht.pyx \
opendht_cpp.pxd \
pyproject.toml
PYTHON_INSTALL_RECORD = $(builddir)/install_record.txt
pybuild.stamp:
LDFLAGS="-L$(top_srcdir)/src/.libs" $(PYTHON) -m build .
echo stamp > pybuild.stamp
CLEANFILES = pybuild.stamp
all-local: pybuild.stamp
clean-local:
rm -rf $(builddir)/build $(builddir)/*.so $(PYTHON_INSTALL_RECORD)
install-exec-local:
$(PYTHON) -m pip install --no-deps --no-build-isolation . \
--root=$(DESTDIR)/ --prefix=$(prefix)
rm -rf $(builddir)/build
if HAVE_PIP
uninstall-local:
/usr/bin/yes | $(PIP) uninstall $(PACKAGE)
endif
endif
+1 -6
View File
@@ -820,13 +820,8 @@ cdef class DhtRunner(_WithID):
key._infohash, (<PkId>to)._pkid, val._value,
cpp.bindDoneCb(done_callback, <void*>cb_obj), permanent
)
elif isinstance(to, InfoHash):
self.thisptr.get().putEncrypted(
key._infohash, (<InfoHash>to)._infohash, val._value,
cpp.bindDoneCb(done_callback, <void*>cb_obj), permanent
)
else:
raise TypeError("'to' must be PublicKey, PkId, or InfoHash")
raise TypeError("'to' must be PublicKey or PkId")
except:
ref.Py_DECREF(cb_obj)
raise
-1
View File
@@ -282,7 +282,6 @@ cdef extern from "opendht/dhtrunner.h" namespace "dht":
void get(InfoHash key, GetCallback get_cb, DoneCallback done_cb, nullptr_t f, Where w)
void put(InfoHash key, shared_ptr[Value] val, DoneCallback done_cb, time_point created, bool permanent)
void putSigned(InfoHash key, shared_ptr[Value] val, DoneCallback done_cb, bool permanent)
void putEncrypted(InfoHash key, InfoHash to, shared_ptr[Value] val, DoneCallback done_cb, bool permanent)
void putEncrypted(InfoHash key, PkId to, shared_ptr[Value] val, DoneCallback done_cb, bool permanent)
void putEncrypted(InfoHash key, shared_ptr[PublicKey] to, shared_ptr[Value] val, DoneCallback done_cb, bool permanent)
void cancelPut(InfoHash key, shared_ptr[Value] val)
+5 -5
View File
@@ -4,11 +4,11 @@ build-backend = "setuptools.build_meta"
[tool.cibuildwheel]
build = [
'cp3{9,10,11,12,13}-manylinux_x86_64',
'cp3{10,11,12,13}-manylinux_aarch64',
'cp3{11,12,13}-macosx_x86_64',
'cp3{11,12,13}-macosx_arm64',
'cp3{11,12,13}-win_amd64',
'cp3{9,10,11,12,13,14}-manylinux_x86_64',
'cp3{10,11,12,13,14}-manylinux_aarch64',
'cp3{11,12,13,14}-macosx_x86_64',
'cp3{11,12,13,14}-macosx_arm64',
'cp3{11,12,13,14}-win_amd64',
]
[tool.cibuildwheel.linux]
+1 -1
View File
@@ -226,7 +226,7 @@ impl DhtRunner {
pub fn put_encrypted<'a>(
&mut self,
h: &InfoHash,
to: &InfoHash,
to: &PkId,
v: Box<Value>,
done_cb: &'a mut (dyn FnMut(bool)),
permanent: bool,
+1 -1
View File
@@ -190,7 +190,7 @@ extern {
cb_user_data: *mut c_void,
permanent: bool);
pub fn dht_runner_put_encrypted(dht: *mut DhtRunner, h: *const InfoHash,
to: *const InfoHash, v: *const Value,
to: *const PkId, v: *const Value,
done_cb: extern fn(bool, *mut c_void),
cb_user_data: *mut c_void,
permanent: bool);
-82
View File
@@ -1,82 +0,0 @@
lib_LTLIBRARIES = libopendht.la
libopendht_la_CPPFLAGS = @CPPFLAGS@ -I$(top_srcdir)/include/opendht @Argon2_CFLAGS@ @JsonCpp_CFLAGS@ @MsgPack_CFLAGS@ @OpenSSL_CFLAGS@ @Fmt_CFLAGS@ @Llhttp_CFLAGS@ @Liburing_CFLAGS@ @simdutf_CFLAGS@
libopendht_la_LIBADD = @Argon2_LIBS@ @JsonCpp_LIBS@ @GnuTLS_LIBS@ @Nettle_LIBS@ @OpenSSL_LIBS@ @Fmt_LIBS@ @Llhttp_LIBS@ @Liburing_LIBS@ @simdutf_LIBS@
libopendht_la_LDFLAGS = @LDFLAGS@ -version-number @OPENDHT_MAJOR_VERSION@:@OPENDHT_MINOR_VERSION@:@OPENDHT_PATCH_VERSION@
libopendht_la_SOURCES = \
dht.cpp \
storage.h \
listener.h \
request.h \
search.h \
value_cache.h \
op_cache.h \
op_cache.cpp \
net.h \
parsed_message.h \
node_cache.cpp \
callbacks.cpp \
routing_table.cpp \
network_engine.cpp \
utils.cpp \
node.cpp \
value.cpp \
crypto.cpp \
securedht.cpp \
dhtrunner.cpp \
default_types.cpp \
log.cpp \
network_utils.cpp \
thread_pool.cpp
nobase_include_HEADERS = \
../include/opendht.h \
../include/opendht/def.h \
../include/opendht/dht.h \
../include/opendht/callbacks.h \
../include/opendht/node_cache.h \
../include/opendht/node_export.h \
../include/opendht/routing_table.h \
../include/opendht/network_engine.h \
../include/opendht/scheduler.h \
../include/opendht/rate_limiter.h \
../include/opendht/utils.h \
../include/opendht/sockaddr.h \
../include/opendht/infohash.h \
../include/opendht/node.h \
../include/opendht/value.h \
../include/opendht/crypto.h \
../include/opendht/securedht.h \
../include/opendht/dhtrunner.h \
../include/opendht/default_types.h \
../include/opendht/log.h \
../include/opendht/logger.h \
../include/opendht/network_utils.h \
../include/opendht/rng.h \
../include/opendht/thread_pool.h
if ENABLE_PROXY_SERVER
libopendht_la_SOURCES += dht_proxy_server.cpp
nobase_include_HEADERS += ../include/opendht/dht_proxy_server.h
endif
if ENABLE_PROXY_CLIENT
libopendht_la_SOURCES += dht_proxy_client.cpp
nobase_include_HEADERS += ../include/opendht/dht_proxy_client.h ../include/opendht/dht_interface.h
endif
libopendht_la_SOURCES += base64.h base64.cpp
if PROXY_CLIENT_OR_SERVER
libopendht_la_SOURCES += http.cpp compat/os_cert.cpp
nobase_include_HEADERS += ../include/opendht/proxy.h ../include/opendht/http.h compat/os_cert.h
endif
if ENABLE_PEER_DISCOVERY
libopendht_la_SOURCES += peer_discovery.cpp
nobase_include_HEADERS += ../include/opendht/peer_discovery.h
endif
if ENABLE_INDEXATION
libopendht_la_SOURCES += indexation/pht.cpp
nobase_include_HEADERS += ../include/opendht/indexation/pht.h
endif
+15
View File
@@ -36,4 +36,19 @@ base64_decode(std::string_view str)
return buffer;
}
std::vector<unsigned char>
base64_default_or_url_decode(std::string_view str)
{
std::vector<unsigned char> buffer(simdutf::maximal_binary_length_from_base64(str.data(), str.size()));
simdutf::result r = simdutf::base64_to_binary(str.data(),
str.size(),
(char*) buffer.data(),
simdutf::base64_options::base64_default_or_url);
if (r.error) {
throw std::invalid_argument(concat("Invalid base64 input: "sv, simdutf::error_to_string(r.error)));
} else {
buffer.resize(r.count);
}
return buffer;
}
} // namespace dht
+8 -1
View File
@@ -21,5 +21,12 @@ std::string base64_encode(const std::vector<unsigned char>& str);
* @return a base64-decoded buffer
*/
std::vector<unsigned char> base64_decode(std::string_view str);
/**
* Decode a buffer in base64 or url style base64 (including '-' '_' ).
*
* @param str the input buffer
* @return a base64-decoded buffer
*/
std::vector<unsigned char> base64_default_or_url_decode(std::string_view str);
} // namespace dht
} // namespace dht
+12
View File
@@ -139,6 +139,12 @@ NodeInfo::toJson() const
val["ipv4"] = ipv4.toJson();
val["ipv6"] = ipv6.toJson();
val["ops"] = Json::Value::LargestUInt(ongoing_ops);
val["storage_values"] = Json::Value::LargestUInt(storage_values);
val["storage_size"] = Json::Value::LargestUInt(storage_size);
val["local_storage_values"] = Json::Value::LargestUInt(local_storage_values);
val["local_storage_size"] = Json::Value::LargestUInt(local_storage_size);
val["port_ipv4"] = Json::Value::LargestUInt(bound4);
val["port_ipv6"] = Json::Value::LargestUInt(bound6);
return val;
}
@@ -150,6 +156,12 @@ NodeInfo::NodeInfo(const Json::Value& v)
ipv4 = NodeStats(v["ipv4"]);
ipv6 = NodeStats(v["ipv6"]);
ongoing_ops = v["ops"].asLargestUInt();
storage_values = v["storage_values"].asLargestUInt();
storage_size = v["storage_size"].asLargestUInt();
local_storage_values = v["local_storage_values"].asLargestUInt();
local_storage_size = v["local_storage_size"].asLargestUInt();
bound4 = v["port_ipv4"].asLargestUInt();
bound6 = v["port_ipv6"].asLargestUInt();
}
#endif
+89 -54
View File
@@ -11,6 +11,7 @@ extern "C" {
#include <nettle/gcm.h>
#include <nettle/aes.h>
#include <nettle/hmac.h>
#include <nettle/version.h>
#include <gnutls/crypto.h>
#include <argon2.h>
@@ -98,6 +99,46 @@ private:
#define GCM_DIGEST_SIZE GCM_BLOCK_SIZE
#endif
static void
gcmAes128Digest(gcm_aes128_ctx* ctx, uint8_t* digest)
{
#if NETTLE_VERSION_MAJOR >= 4
gcm_aes128_digest(ctx, digest);
#else
gcm_aes128_digest(ctx, GCM_DIGEST_SIZE, digest);
#endif
}
static void
gcmAes192Digest(gcm_aes192_ctx* ctx, uint8_t* digest)
{
#if NETTLE_VERSION_MAJOR >= 4
gcm_aes192_digest(ctx, digest);
#else
gcm_aes192_digest(ctx, GCM_DIGEST_SIZE, digest);
#endif
}
static void
gcmAes256Digest(gcm_aes256_ctx* ctx, uint8_t* digest)
{
#if NETTLE_VERSION_MAJOR >= 4
gcm_aes256_digest(ctx, digest);
#else
gcm_aes256_digest(ctx, GCM_DIGEST_SIZE, digest);
#endif
}
static void
hmacSha256Digest(hmac_sha256_ctx* ctx, uint8_t* digest)
{
#if NETTLE_VERSION_MAJOR >= 4
hmac_sha256_digest(ctx, digest);
#else
hmac_sha256_digest(ctx, SHA256_DIGEST_SIZE, digest);
#endif
}
Blob
aesEncrypt(const uint8_t* data, size_t data_length, const Blob& key)
{
@@ -115,19 +156,19 @@ aesEncrypt(const uint8_t* data, size_t data_length, const Blob& key)
gcm_aes128_set_key(&aes, key.data());
gcm_aes128_set_iv(&aes, GCM_IV_SIZE, ret.data());
gcm_aes128_encrypt(&aes, data_length, ret.data() + GCM_IV_SIZE, data);
gcm_aes128_digest(&aes, GCM_DIGEST_SIZE, ret.data() + GCM_IV_SIZE + data_length);
gcmAes128Digest(&aes, ret.data() + GCM_IV_SIZE + data_length);
} else if (key.size() == AES_LENGTHS[1]) {
struct gcm_aes192_ctx aes;
gcm_aes192_set_key(&aes, key.data());
gcm_aes192_set_iv(&aes, GCM_IV_SIZE, ret.data());
gcm_aes192_encrypt(&aes, data_length, ret.data() + GCM_IV_SIZE, data);
gcm_aes192_digest(&aes, GCM_DIGEST_SIZE, ret.data() + GCM_IV_SIZE + data_length);
gcmAes192Digest(&aes, ret.data() + GCM_IV_SIZE + data_length);
} else if (key.size() == AES_LENGTHS[2]) {
struct gcm_aes256_ctx aes;
gcm_aes256_set_key(&aes, key.data());
gcm_aes256_set_iv(&aes, GCM_IV_SIZE, ret.data());
gcm_aes256_encrypt(&aes, data_length, ret.data() + GCM_IV_SIZE, data);
gcm_aes256_digest(&aes, GCM_DIGEST_SIZE, ret.data() + GCM_IV_SIZE + data_length);
gcmAes256Digest(&aes, ret.data() + GCM_IV_SIZE + data_length);
}
return ret;
@@ -160,19 +201,19 @@ aesDecrypt(const uint8_t* data, size_t data_length, const Blob& key)
gcm_aes128_set_key(&aes, key.data());
gcm_aes128_set_iv(&aes, GCM_IV_SIZE, data);
gcm_aes128_decrypt(&aes, data_sz, ret.data(), data + GCM_IV_SIZE);
gcm_aes128_digest(&aes, GCM_DIGEST_SIZE, digest.data());
gcmAes128Digest(&aes, digest.data());
} else if (key.size() == AES_LENGTHS[1]) {
struct gcm_aes192_ctx aes;
gcm_aes192_set_key(&aes, key.data());
gcm_aes192_set_iv(&aes, GCM_IV_SIZE, data);
gcm_aes192_decrypt(&aes, data_sz, ret.data(), data + GCM_IV_SIZE);
gcm_aes192_digest(&aes, GCM_DIGEST_SIZE, digest.data());
gcmAes192Digest(&aes, digest.data());
} else if (key.size() == AES_LENGTHS[2]) {
struct gcm_aes256_ctx aes;
gcm_aes256_set_key(&aes, key.data());
gcm_aes256_set_iv(&aes, GCM_IV_SIZE, data);
gcm_aes256_decrypt(&aes, data_sz, ret.data(), data + GCM_IV_SIZE);
gcm_aes256_digest(&aes, GCM_DIGEST_SIZE, digest.data());
gcmAes256Digest(&aes, digest.data());
}
if (not std::equal(digest.begin(), digest.end(), data + data_length - GCM_DIGEST_SIZE)) {
@@ -446,7 +487,7 @@ PrivateKey::getPublicKey() const
const std::shared_ptr<PublicKey>&
PrivateKey::getSharedPublicKey() const
{
std::lock_guard<std::mutex> lock(publicKeyMutex_);
std::lock_guard lock(publicKeyMutex_);
if (not publicKey_) {
auto pk = std::make_shared<PublicKey>();
if (auto err = gnutls_pubkey_import_privkey(pk->pk, key, GNUTLS_KEY_KEY_CERT_SIGN | GNUTLS_KEY_CRL_SIGN, 0))
@@ -530,6 +571,12 @@ PublicKey::toString() const
}
if (err != GNUTLS_E_SUCCESS)
throw CryptoException(std::string("Could not print public key: ") + gnutls_strerror(err));
// GnuTLS reports the PEM size including the terminating null byte, which
// does not belong to the text and would end up embedded in whatever the
// string is copied into, such as the "owner" field of a serialized value.
ret.resize(sz);
if (not ret.empty() and ret.back() == '\0')
ret.pop_back();
return ret;
}
@@ -959,7 +1006,7 @@ Certificate::getPublicKey() const
const std::shared_ptr<PublicKey>&
Certificate::getSharedPublicKey() const
{
std::lock_guard<std::mutex> lock(publicKeyMutex_);
std::lock_guard lock(publicKeyMutex_);
if (not publicKey_) {
auto pk = std::make_shared<PublicKey>();
if (auto err = gnutls_pubkey_import_x509(pk->pk, cert, 0))
@@ -1247,14 +1294,16 @@ PrivateKey::generate(unsigned key_length, gnutls_pk_algorithm_t algo)
}
PrivateKey
PrivateKey::generateEC()
PrivateKey::generateEC(gnutls_ecc_curve_t curve)
{
gnutls_x509_privkey_t key;
if (gnutls_x509_privkey_init(&key) != GNUTLS_E_SUCCESS)
throw CryptoException("Unable to initialize private key.");
int err = gnutls_x509_privkey_generate(key,
GNUTLS_PK_EC,
gnutls_sec_param_to_pk_bits(GNUTLS_PK_EC, GNUTLS_SEC_PARAM_ULTRA),
curve == GNUTLS_ECC_CURVE_INVALID
? gnutls_sec_param_to_pk_bits(GNUTLS_PK_EC, GNUTLS_SEC_PARAM_ULTRA)
: GNUTLS_CURVE_TO_BITS(curve),
0);
if (err != GNUTLS_E_SUCCESS) {
gnutls_x509_privkey_deinit(key);
@@ -1355,8 +1404,11 @@ setRandomSerial(gnutls_x509_crt_t cert)
{
std::random_device rdev;
std::uniform_int_distribution<int64_t> dist {1};
int64_t cert_serial = dist(rdev);
gnutls_x509_crt_set_serial(cert, &cert_serial, sizeof(cert_serial));
const auto cert_serial = dist(rdev);
std::array<uint8_t, sizeof(cert_serial)> packed_serial;
for (size_t i = 0; i < packed_serial.size(); i++)
packed_serial[i] = cert_serial >> (8 * (packed_serial.size() - 1 - i));
gnutls_x509_crt_set_serial(cert, packed_serial.data(), packed_serial.size());
}
Certificate
@@ -1832,34 +1884,8 @@ RevocationList::getUpdateTime() const
return std::chrono::system_clock::from_time_t(t);
}
enum class Endian : uint32_t { LITTLE = 0, BIG = 1 };
template<typename T>
T
endian(T w, Endian endian = Endian::BIG)
{
// this gets optimized out into if (endian == host_endian) return w;
union {
uint64_t quad;
uint32_t islittle;
} t;
t.quad = 1;
if (t.islittle ^ (uint32_t) endian)
return w;
T r = 0;
// decent compilers will unroll this (gcc)
// or even convert straight into single bswap (clang)
for (size_t i = 0; i < sizeof(r); i++) {
r <<= 8;
r |= w & 0xff;
w >>= 8;
}
return r;
}
void
RevocationList::sign(const PrivateKey& key, const Certificate& ca, duration validity)
RevocationList::sign(const PrivateKey& key, const Certificate& ca, duration validity, uint64_t crlNumber)
{
if (auto err = gnutls_x509_crl_set_version(crl, 2))
throw CryptoException(std::string("Unable to set CRL version: ") + gnutls_strerror(err));
@@ -1869,19 +1895,28 @@ RevocationList::sign(const PrivateKey& key, const Certificate& ca, duration vali
throw CryptoException(std::string("Unable to set CRL update time: ") + gnutls_strerror(err));
if (auto err = gnutls_x509_crl_set_next_update(crl, std::chrono::system_clock::to_time_t(next_update)))
throw CryptoException(std::string("Unable to set CRL next update time: ") + gnutls_strerror(err));
uint64_t number {0};
size_t number_sz {sizeof(number)};
unsigned critical {0};
gnutls_x509_crl_get_number(crl, &number, &number_sz, &critical);
uint64_t number {crlNumber};
if (number == 0) {
// initialize to a random number
number_sz = sizeof(number);
std::random_device rdev;
std::generate_n((uint8_t*) &number, sizeof(number), std::bind(rand_byte, std::ref(rdev)));
} else
number = endian(endian(number) + 1);
if (auto err = gnutls_x509_crl_set_number(crl, &number, sizeof(number)))
throw CryptoException(std::string("Unable to set CRL update time: ") + gnutls_strerror(err));
// The CRL Number extension holds a DER integer of variable length, so it must be
// read as a big-endian byte string rather than straight into an uint64_t.
auto current = getNumber();
for (uint8_t byte : current)
number = (number << 8) | byte;
if (number == 0) {
// Initialize to a random number.
std::random_device rdev;
do {
std::generate_n((uint8_t*) &number, sizeof(number), std::bind(rand_byte, std::ref(rdev)));
} while (number == 0);
} else
number++;
}
// Store the number in network byte order.
uint8_t packedNumber[sizeof(number)];
for (size_t i = 0; i < sizeof(packedNumber); i++)
packedNumber[i] = (uint8_t) ((number >> (8 * (sizeof(packedNumber) - 1 - i))) & 0xff);
if (auto err = gnutls_x509_crl_set_number(crl, packedNumber, sizeof(packedNumber)))
throw CryptoException(std::string("Unable to set CRL number: ") + gnutls_strerror(err));
if (auto err = gnutls_x509_crl_sign2(crl, ca.cert, key.x509_key, GNUTLS_DIG_SHA512, 0))
throw CryptoException(std::string("Unable to sign certificate revocation list: ") + gnutls_strerror(err));
// to be able to actually use the CRL we need to serialize/deserialize it
@@ -2045,7 +2080,7 @@ hkdf_extract(const Blob& salt, const Blob& ikm)
hmac_sha256_set_key(&ctx, salt.size(), salt.data());
hmac_sha256_update(&ctx, ikm.size(), ikm.data());
Blob prk(SHA256_DIGEST_SIZE);
hmac_sha256_digest(&ctx, SHA256_DIGEST_SIZE, prk.data());
hmacSha256Digest(&ctx, prk.data());
return prk;
}
@@ -2063,7 +2098,7 @@ hkdf_expand(const Blob& prk, const uint8_t* info, size_t info_len, size_t len)
hmac_sha256_update(&ctx, info_len, info);
hmac_sha256_update(&ctx, 1, &i);
T.resize(SHA256_DIGEST_SIZE);
hmac_sha256_digest(&ctx, SHA256_DIGEST_SIZE, T.data());
hmacSha256Digest(&ctx, T.data());
okm.insert(okm.end(), T.begin(), T.end());
i++;
}
@@ -2223,7 +2258,7 @@ webPushEncrypt(const Blob& p256dh,
Blob ciphertext(plaintext.size() + GCM_DIGEST_SIZE);
gcm_aes128_encrypt(&aes, plaintext.size(), ciphertext.data(), plaintext.data());
gcm_aes128_digest(&aes, GCM_DIGEST_SIZE, ciphertext.data() + plaintext.size());
gcmAes128Digest(&aes, ciphertext.data() + plaintext.size());
// 6. Construct Output
Blob output;
+452 -401
View File
File diff suppressed because it is too large. Load diff
+137 -141
View File
@@ -34,20 +34,20 @@ struct DhtProxyClient::Listener
Sp<OperationState> opstate;
std::shared_ptr<http::Request> request;
#ifdef OPENDHT_PUSH_NOTIFICATIONS
std::unique_ptr<asio::steady_timer> refreshSubscriberTimer;
std::unique_ptr<asio::system_timer> refreshSubscriberTimer;
#endif
};
struct PermanentPut
{
PermanentPut(const Sp<Value>& v, std::unique_ptr<asio::steady_timer>&& j, const Sp<std::atomic_bool>& o)
PermanentPut(const Sp<Value>& v, std::unique_ptr<asio::system_timer>&& j, const Sp<std::atomic_bool>& o)
: value(v)
, refreshPutTimer(std::move(j))
, ok(o)
{}
Sp<Value> value;
std::unique_ptr<asio::steady_timer> refreshPutTimer;
std::unique_ptr<asio::system_timer> refreshPutTimer;
Sp<std::atomic_bool> ok;
};
@@ -127,25 +127,25 @@ DhtProxyClient::DhtProxyClient(std::shared_ptr<dht::crypto::Certificate> serverC
jsonBuilder_["indentation"] = "";
if (logger_) {
if (serverCertificate_)
logger_->d("[proxy:client] using ca certificate for ssl:\n%s",
serverCertificate_->toString(false /*chain*/).c_str());
logger_->debug("[proxy:client] using ca certificate for ssl:\n{}",
serverCertificate_->toString(false /*chain*/));
if (clientIdentity_.first and clientIdentity_.second)
logger_->d("[proxy:client] using client certificate for ssl:\n%s",
clientIdentity_.second->toString(false /*chain*/).c_str());
logger_->debug("[proxy:client] using client certificate for ssl:\n{}",
clientIdentity_.second->toString(false /*chain*/));
}
// run http client
httpClientThread_ = std::thread([this]() {
try {
if (logger_)
logger_->d("[proxy:client] starting io_context");
logger_->debug("[proxy:client] starting io_context");
// Ensures the httpContext_ won't run out of work
auto work = asio::make_work_guard(httpContext_);
httpContext_.run();
if (logger_)
logger_->d("[proxy:client] http client io_context stopped");
logger_->debug("[proxy:client] http client io_context stopped");
} catch (const std::exception& ex) {
if (logger_)
logger_->e("[proxy:client] run error: %s", ex.what());
logger_->error("[proxy:client] run error: {}", ex.what());
}
});
if (!proxyUrl_.empty())
@@ -159,7 +159,7 @@ DhtProxyClient::startProxy()
return;
if (logger_)
logger_->d("[proxy:client] start proxy with %s", proxyUrl_.c_str());
logger_->debug("[proxy:client] start proxy with {}", proxyUrl_);
nextProxyConfirmationTimer_ = std::make_unique<asio::steady_timer>(httpContext_, std::chrono::steady_clock::now());
nextProxyConfirmationTimer_->async_wait(std::bind(&DhtProxyClient::handleProxyConfirm, this, std::placeholders::_1));
@@ -176,7 +176,7 @@ DhtProxyClient::handleProxyConfirm(const asio::error_code& ec)
return;
else if (ec) {
if (logger_)
logger_->e("[proxy:client] confirm error: %s", ec.message().c_str());
logger_->error("[proxy:client] confirm error: {}", ec.message());
return;
}
if (proxyUrl_.empty())
@@ -195,7 +195,7 @@ DhtProxyClient::stop()
if (not isDestroying_.exchange(true)) {
std::shared_ptr<http::Resolver> resolver;
{
std::lock_guard<std::mutex> l(resolverLock_);
std::lock_guard l(resolverLock_);
resolver = std::move(resolver_);
}
if (resolver)
@@ -204,7 +204,7 @@ DhtProxyClient::stop()
if (infoState_)
infoState_->cancel = true;
{
std::lock_guard<std::mutex> lock(requestLock_);
std::lock_guard lock(requestLock_);
for (auto& request : requests_)
request.second->cancel();
}
@@ -219,7 +219,7 @@ DhtProxyClient::stop()
std::vector<Sp<Value>>
DhtProxyClient::getLocal(const InfoHash& k, const Value::Filter& filter) const
{
std::lock_guard<std::mutex> lock(searchLock_);
std::lock_guard lock(searchLock_);
auto s = searches_.find(k);
if (s == searches_.end())
return {};
@@ -229,7 +229,7 @@ DhtProxyClient::getLocal(const InfoHash& k, const Value::Filter& filter) const
Sp<Value>
DhtProxyClient::getLocalById(const InfoHash& k, Value::Id id) const
{
std::lock_guard<std::mutex> lock(searchLock_);
std::lock_guard lock(searchLock_);
auto s = searches_.find(k);
if (s == searches_.end())
return {};
@@ -239,9 +239,9 @@ DhtProxyClient::getLocalById(const InfoHash& k, Value::Id id) const
void
DhtProxyClient::cancelAllListeners()
{
std::lock_guard<std::mutex> lock(searchLock_);
std::lock_guard lock(searchLock_);
if (logger_)
logger_->d("[proxy:client] [listeners] [%zu searches] cancel all", searches_.size());
logger_->debug("[proxy:client] [listeners] [{} searches] cancel all", searches_.size());
for (auto& s : searches_) {
s.second.ops.cancelAll([&](size_t token) {
auto l = s.second.listeners.find(token);
@@ -266,7 +266,7 @@ DhtProxyClient::shutdown(ShutdownCallback cb, bool)
NodeStatus
DhtProxyClient::getStatus(sa_family_t af) const
{
std::lock_guard<std::mutex> l(lockCurrentProxyInfos_);
std::lock_guard l(lockCurrentProxyInfos_);
switch (af) {
case AF_INET:
return statusIpv4_;
@@ -280,7 +280,7 @@ DhtProxyClient::getStatus(sa_family_t af) const
bool
DhtProxyClient::isRunning(sa_family_t af) const
{
std::lock_guard<std::mutex> l(lockCurrentProxyInfos_);
std::lock_guard l(lockCurrentProxyInfos_);
switch (af) {
case AF_INET:
return statusIpv4_ != NodeStatus::Disconnected;
@@ -297,7 +297,7 @@ DhtProxyClient::periodic(const uint8_t*, size_t, SockAddr, const time_point& /*n
// Exec all currently stored callbacks
decltype(callbacks_) callbacks;
{
std::lock_guard<std::mutex> lock(lockCallbacks_);
std::lock_guard lock(lockCallbacks_);
callbacks = std::move(callbacks_);
}
for (auto& callback : callbacks)
@@ -317,7 +317,7 @@ void
DhtProxyClient::get(const InfoHash& key, GetCallback cb, DoneCallback donecb, Value::Filter&& f, Where&& w)
{
if (logger_)
logger_->d("[proxy:client] [get] [search %s]", key.to_c_str());
logger_->debug("[proxy:client] [get] [search {}]", key.to_view());
if (isDestroying_) {
if (donecb)
@@ -357,7 +357,7 @@ DhtProxyClient::get(const InfoHash& key, GetCallback cb, DoneCallback donecb, Va
}
if (not values.empty() and cb) {
{
std::lock_guard<std::mutex> lock(lockCallbacks_);
std::lock_guard lock(lockCallbacks_);
callbacks_.emplace_back([opstate, cb, values = std::move(values)]() {
if (not opstate->stop.load() and not cb(values)) {
opstate->stop.store(true);
@@ -368,21 +368,21 @@ DhtProxyClient::get(const InfoHash& key, GetCallback cb, DoneCallback donecb, Va
}
} catch (const std::exception& e) {
if (logger_)
logger_->e("[proxy:client] [get %s] body parsing error: %s", key.to_c_str(), e.what());
logger_->error("[proxy:client] [get {}] body parsing error: {}", key.to_view(), e.what());
opstate->ok.store(false);
}
});
request->add_on_done_callback([this, reqid, opstate, donecb, key](const http::Response& response) {
if (response.status_code != 200) {
if (logger_)
logger_->e("[proxy:client] [get %s] failed with code=%i", key.to_c_str(), response.status_code);
logger_->error("[proxy:client] [get {}] failed with code={}", key.to_view(), response.status_code);
opstate->ok.store(false);
if (not response.aborted and response.status_code == 0)
opFailed();
}
if (donecb) {
{
std::lock_guard<std::mutex> lock(lockCallbacks_);
std::lock_guard lock(lockCallbacks_);
callbacks_.emplace_back([donecb, opstate]() {
donecb(opstate->ok, {});
opstate->stop.store(true);
@@ -391,18 +391,18 @@ DhtProxyClient::get(const InfoHash& key, GetCallback cb, DoneCallback donecb, Va
loopSignal_();
}
if (not isDestroying_) {
std::lock_guard<std::mutex> l(requestLock_);
std::lock_guard l(requestLock_);
requests_.erase(reqid);
}
});
{
std::lock_guard<std::mutex> l(requestLock_);
std::lock_guard l(requestLock_);
requests_[reqid] = request;
}
request->send();
} catch (const std::exception& e) {
if (logger_)
logger_->e("[proxy:client] [get %s] error: %s", key.to_c_str(), e.what());
logger_->error("[proxy:client] [get {}] error: {}", key.to_view(), e.what());
}
}
@@ -415,16 +415,16 @@ DhtProxyClient::put(const InfoHash& key, Sp<Value> val, DoneCallback cb, time_po
return;
}
if (logger_)
logger_->d("[proxy:client] [put] [search %s]", key.to_c_str());
logger_->debug("[proxy:client] [put] [search {}]", key.to_view());
std::shared_ptr<std::atomic_bool> ok;
if (permanent) {
std::lock_guard<std::mutex> lock(searchLock_);
std::lock_guard lock(searchLock_);
ok = std::make_shared<std::atomic_bool>(true);
auto& search = searches_[key];
if (val->id) {
auto id = val->id;
auto refreshPutTimer = std::make_unique<asio::steady_timer>(httpContext_,
auto refreshPutTimer = std::make_unique<asio::system_timer>(httpContext_,
proxy::OP_TIMEOUT - proxy::OP_MARGIN);
refreshPutTimer->async_wait(
std::bind(&DhtProxyClient::handleRefreshPut, this, std::placeholders::_1, key, id));
@@ -443,7 +443,7 @@ DhtProxyClient::put(const InfoHash& key, Sp<Value> val, DoneCallback cb, time_po
if (ok)
*ok = result;
if (cb) {
std::lock_guard<std::mutex> lock(lockCallbacks_);
std::lock_guard lock(lockCallbacks_);
callbacks_.emplace_back([cb, result]() { cb(result, {}); });
}
loopSignal_();
@@ -459,12 +459,12 @@ DhtProxyClient::handleRefreshPut(const asio::error_code& ec, InfoHash key, Value
return;
else if (ec) {
if (logger_)
logger_->e("[proxy:client] [put] [refresh %s] %s", key.toString().c_str(), ec.message().c_str());
logger_->error("[proxy:client] [put] [refresh {}] {}", key.to_view(), ec.message());
return;
}
if (logger_)
logger_->d("[proxy:client] [put] [refresh %s]", key.to_c_str());
std::lock_guard<std::mutex> lock(searchLock_);
logger_->debug("[proxy:client] [put] [refresh {}]", key.to_view());
std::lock_guard lock(searchLock_);
auto search = searches_.find(key);
if (search != searches_.end()) {
auto p = search->second.puts.find(id);
@@ -480,7 +480,7 @@ DhtProxyClient::handleRefreshPut(const asio::error_code& ec, InfoHash key, Value
std::shared_ptr<http::Request>
DhtProxyClient::buildRequest(const std::string& target)
{
std::unique_lock<std::mutex> l(resolverLock_);
std::unique_lock l(resolverLock_);
auto resolver = resolver_;
l.unlock();
if (not resolver)
@@ -535,12 +535,12 @@ DhtProxyClient::doPut(const InfoHash& key, Sp<Value> val, DoneCallbackSimple cb,
auto id = dht::Value(parsedValue).id;
val->id = id;
if (permanent) {
std::lock_guard<std::mutex> lock(searchLock_);
std::lock_guard lock(searchLock_);
auto& search = searches_[key];
auto it = search.pendingPuts.find(val);
if (it != search.pendingPuts.end()) {
auto sok = std::make_shared<std::atomic_bool>(ok);
auto refreshPutTimer = std::make_unique<asio::steady_timer>(httpContext_,
auto refreshPutTimer = std::make_unique<asio::system_timer>(httpContext_,
proxy::OP_TIMEOUT
- proxy::OP_MARGIN);
refreshPutTimer->async_wait(
@@ -565,12 +565,12 @@ DhtProxyClient::doPut(const InfoHash& key, Sp<Value> val, DoneCallbackSimple cb,
if (cb)
cb(ok);
if (not isDestroying_) {
std::lock_guard<std::mutex> l(requestLock_);
std::lock_guard l(requestLock_);
requests_.erase(reqid);
}
});
{
std::lock_guard<std::mutex> l(requestLock_);
std::lock_guard l(requestLock_);
requests_[reqid] = request;
}
request->send();
@@ -623,7 +623,7 @@ DhtProxyClient::cancelPut(const InfoHash& key, const Value::Id& id)
if (search == searches_.end())
return false;
if (logger_)
logger_->d("[proxy:client] [put] [search %s] cancel", key.to_c_str());
logger_->debug("[proxy:client] [put] [search {}] cancel", key.to_view());
return search->second.puts.erase(id) > 0;
}
@@ -637,10 +637,10 @@ void
DhtProxyClient::getProxyInfos()
{
if (logger_)
logger_->d("[proxy:client] [info] requesting proxy server node information");
logger_->debug("[proxy:client] [info] requesting proxy server node information");
auto infoState = std::make_shared<InfoState>();
{
std::lock_guard<std::mutex> l(lockCurrentProxyInfos_);
std::lock_guard l(lockCurrentProxyInfos_);
if (infoState_)
infoState_->cancel = true;
infoState_ = infoState;
@@ -650,12 +650,12 @@ DhtProxyClient::getProxyInfos()
statusIpv6_ = NodeStatus::Connecting;
}
if (logger_)
logger_->d("[proxy:client] [status] sending request");
logger_->debug("[proxy:client] [status] sending request");
auto resolver = std::make_shared<http::Resolver>(httpContext_, proxyUrl_, logger_);
queryProxyInfo(infoState, resolver, AF_INET);
queryProxyInfo(infoState, resolver, AF_INET6);
std::lock_guard<std::mutex> l(resolverLock_);
std::lock_guard l(resolverLock_);
resolver_ = resolver;
}
@@ -663,7 +663,7 @@ void
DhtProxyClient::queryProxyInfo(const Sp<InfoState>& infoState, const Sp<http::Resolver>& resolver, sa_family_t family)
{
if (logger_)
logger_->d("[proxy:client] [status] query ipv%i info", family == AF_INET ? 4 : 6);
logger_->debug("[proxy:client] [status] query ipv{} info", family == AF_INET ? 4 : 6);
try {
auto request = std::make_shared<http::Request>(httpContext_, resolver, family);
if (serverCertificate_)
@@ -676,9 +676,9 @@ DhtProxyClient::queryProxyInfo(const Sp<InfoState>& infoState, const Sp<http::Re
return;
if (response.status_code != 200) {
if (logger_)
logger_->e("[proxy:client] [status] ipv%i failed with code=%i",
family == AF_INET ? 4 : 6,
response.status_code);
logger_->error("[proxy:client] [status] ipv{} failed with code={}",
family == AF_INET ? 4 : 6,
response.status_code);
// pass along the failures
if ((family == AF_INET and infoState->ipv4 == 0) or (family == AF_INET6 and infoState->ipv6 == 0))
onProxyInfos(Json::Value {}, family);
@@ -701,7 +701,7 @@ DhtProxyClient::queryProxyInfo(const Sp<InfoState>& infoState, const Sp<http::Re
}
}
if (not isDestroying_) {
std::lock_guard<std::mutex> l(requestLock_);
std::lock_guard l(requestLock_);
requests_.erase(reqid);
}
});
@@ -709,13 +709,13 @@ DhtProxyClient::queryProxyInfo(const Sp<InfoState>& infoState, const Sp<http::Re
if (infoState->cancel.load())
return;
{
std::lock_guard<std::mutex> l(requestLock_);
std::lock_guard l(requestLock_);
requests_[reqid] = request;
}
request->send();
} catch (const std::exception& e) {
if (logger_)
logger_->e("[proxy:client] [status] error sending request: %s", e.what());
logger_->error("[proxy:client] [status] error sending request: {}", e.what());
}
}
@@ -724,19 +724,17 @@ DhtProxyClient::onProxyInfos(const Json::Value& proxyInfos, const sa_family_t fa
{
if (isDestroying_)
return;
std::unique_lock<std::mutex> l(lockCurrentProxyInfos_);
std::unique_lock l(lockCurrentProxyInfos_);
auto oldStatus = std::max(statusIpv4_, statusIpv6_);
auto& status = family == AF_INET ? statusIpv4_ : statusIpv6_;
auto ipChanged = false;
auto& pubAddress = family == AF_INET ? publicAddressV4_ : publicAddressV6_;
auto& localAddress = family == AF_INET ? localAddrv4_ : localAddrv6_;
if (not proxyInfos.isMember("node_id")) {
if (logger_)
logger_->e("[proxy:client] [info] request failed for %s", family == AF_INET ? "ipv4" : "ipv6");
logger_->error("[proxy:client] [info] request failed for {}", family == AF_INET ? "ipv4" : "ipv6");
status = NodeStatus::Disconnected;
if (pubAddress) {
pubAddress = {};
ipChanged = true;
}
} else {
if (logger_)
@@ -746,15 +744,14 @@ DhtProxyClient::onProxyInfos(const Json::Value& proxyInfos, const sa_family_t fa
stats4_ = NodeStats(proxyInfos["ipv4"]);
stats6_ = NodeStats(proxyInfos["ipv6"]);
auto publicIp = parsePublicAddress(proxyInfos["public_ip"]);
ipChanged = pubAddress && pubAddress.toString() != publicIp.toString();
bool pub = ipChanged || !pubAddress;
bool pub = pubAddress != publicIp;
pubAddress = publicIp;
if (proxyInfos.isMember("local_ip")) {
std::string localIp = proxyInfos["local_ip"].asString();
if (localAddress.toString() != localIp) {
localAddress.setAddress(localIp.c_str());
ipChanged = (bool) localAddress;
pub = (bool) localAddress;
}
}
@@ -764,21 +761,21 @@ DhtProxyClient::onProxyInfos(const Json::Value& proxyInfos, const sa_family_t fa
addresses.emplace_back(publicAddressV4_);
if (publicAddressV6_)
addresses.emplace_back(publicAddressV6_);
std::lock_guard<std::mutex> lock(lockCallbacks_);
std::lock_guard lock(lockCallbacks_);
callbacks_.emplace_back(
[cb = publicAddressChangedCb_, addresses = std::move(addresses)]() { cb(std::move(addresses)); });
}
if (!ipChanged && stats4_.good_nodes + stats6_.good_nodes)
if (stats4_.good_nodes + stats6_.good_nodes)
status = NodeStatus::Connected;
else if (!ipChanged && stats4_.dubious_nodes + stats6_.dubious_nodes)
else if (stats4_.dubious_nodes + stats6_.dubious_nodes)
status = NodeStatus::Connecting;
else
status = NodeStatus::Disconnected;
} catch (const std::exception& e) {
if (logger_)
logger_->e("[proxy:client] [info] error processing: %s", e.what());
logger_->error("[proxy:client] [info] error processing: {}", e.what());
}
}
auto newStatus = std::max(statusIpv4_, statusIpv6_);
@@ -788,7 +785,7 @@ DhtProxyClient::onProxyInfos(const Json::Value& proxyInfos, const sa_family_t fa
listenerRestartTimer_->expires_at(std::chrono::steady_clock::now());
listenerRestartTimer_->async_wait(std::bind(&DhtProxyClient::restartListeners, this, std::placeholders::_1));
if (not onConnectCallbacks_.empty()) {
std::lock_guard<std::mutex> lock(lockCallbacks_);
std::lock_guard lock(lockCallbacks_);
callbacks_.emplace_back([cbs = std::move(onConnectCallbacks_)]() mutable {
while (not cbs.empty()) {
cbs.front()();
@@ -801,10 +798,7 @@ DhtProxyClient::onProxyInfos(const Json::Value& proxyInfos, const sa_family_t fa
nextProxyConfirmationTimer_->async_wait(
std::bind(&DhtProxyClient::handleProxyConfirm, this, std::placeholders::_1));
} else if (newStatus == NodeStatus::Disconnected) {
auto next = std::chrono::steady_clock::now();
if (!ipChanged)
next += std::chrono::minutes(1);
nextProxyConfirmationTimer_->expires_at(next);
nextProxyConfirmationTimer_->expires_at(std::chrono::steady_clock::now() + std::chrono::minutes(1));
nextProxyConfirmationTimer_->async_wait(
std::bind(&DhtProxyClient::handleProxyConfirm, this, std::placeholders::_1));
}
@@ -826,7 +820,7 @@ DhtProxyClient::parsePublicAddress(const Json::Value& val)
std::vector<SockAddr>
DhtProxyClient::getPublicAddress(sa_family_t family)
{
std::lock_guard<std::mutex> l(lockCurrentProxyInfos_);
std::lock_guard l(lockCurrentProxyInfos_);
std::vector<SockAddr> result;
if (publicAddressV6_ && family != AF_INET)
result.emplace_back(publicAddressV6_);
@@ -839,11 +833,11 @@ size_t
DhtProxyClient::listen(const InfoHash& key, ValueCallback cb, Value::Filter filter, Where where)
{
if (logger_)
logger_->d("[proxy:client] [listen] [search %s]", key.to_c_str());
logger_->debug("[proxy:client] [listen] [search {}]", key.to_view());
if (isDestroying_)
return 0;
std::lock_guard<std::mutex> lock(searchLock_);
std::lock_guard lock(searchLock_);
auto& search = searches_[key];
auto query = std::make_shared<Query>(Select {}, std::move(where));
return search.ops
@@ -852,13 +846,13 @@ DhtProxyClient::listen(const InfoHash& key, ValueCallback cb, Value::Filter filt
auto search = searches_.find(key);
if (search == searches_.end()) {
if (logger_)
logger_->e("[proxy:client] [listen] [search %s] search not found", key.to_c_str());
logger_->error("[proxy:client] [listen] [search {}] search not found", key.to_view());
return 0;
}
if (logger_)
logger_->d("[proxy:client] [listen] [search %s] sending %s",
key.to_c_str(),
deviceKey_.empty() ? "listen" : "subscribe");
logger_->debug("[proxy:client] [listen] [search {}] sending {}",
key.to_view(),
deviceKey_.empty() ? "listen" : "subscribe");
// Add listener
auto token = ++listenerToken_;
auto l = search->second.listeners.find(token);
@@ -880,7 +874,7 @@ DhtProxyClient::listen(const InfoHash& key, ValueCallback cb, Value::Filter filt
system_clock::time_point t) {
if (opstate->stop)
return false;
std::lock_guard<std::mutex> lock(searchLock_);
std::lock_guard lock(searchLock_);
auto s = searches_.find(key);
if (s != searches_.end()) {
auto l = s->second.listeners.find(token);
@@ -897,8 +891,8 @@ DhtProxyClient::listen(const InfoHash& key, ValueCallback cb, Value::Filter filt
* (if the proxy crash for any reason)
*/
if (!l->second.refreshSubscriberTimer)
l->second.refreshSubscriberTimer = std::make_unique<asio::steady_timer>(httpContext_);
l->second.refreshSubscriberTimer->expires_at(std::chrono::steady_clock::now() + proxy::OP_TIMEOUT
l->second.refreshSubscriberTimer = std::make_unique<asio::system_timer>(httpContext_);
l->second.refreshSubscriberTimer->expires_at(std::chrono::system_clock::now() + proxy::OP_TIMEOUT
- proxy::OP_MARGIN);
l->second.refreshSubscriberTimer->async_wait(
std::bind(&DhtProxyClient::handleResubscribe, this, std::placeholders::_1, key, token, opstate));
@@ -909,11 +903,11 @@ DhtProxyClient::listen(const InfoHash& key, ValueCallback cb, Value::Filter filt
if (deviceKey_.empty()) { // listen
method = ListenMethod::LISTEN;
header.method(restinio::http_method_get());
header.request_target("/key/" + key.toString() + "/listen");
header.request_target(fmt::format("/key/{}/listen", key.to_view()));
} else {
method = ListenMethod::SUBSCRIBE;
header.method(restinio::http_method_subscribe());
header.request_target("/key/" + key.toString());
header.request_target(fmt::format("/key/{}", key.to_view()));
}
sendListen(header, l->second.cb, opstate, l->second, method);
return token;
@@ -930,12 +924,12 @@ DhtProxyClient::handleResubscribe(const asio::error_code& ec,
return;
else if (ec) {
if (logger_)
logger_->e("[proxy:client] [resubscribe %s] %s", key.toString().c_str(), ec.message().c_str());
logger_->error("[proxy:client] [resubscribe {}] {}", key.to_view(), ec.message());
return;
}
if (opstate->stop)
return;
std::lock_guard<std::mutex> lock(searchLock_);
std::lock_guard lock(searchLock_);
auto s = searches_.find(key);
if (s != searches_.end()) {
auto l = s->second.listeners.find(token);
@@ -943,7 +937,7 @@ DhtProxyClient::handleResubscribe(const asio::error_code& ec,
resubscribe(key, token, l->second);
} else {
if (logger_)
logger_->e("[proxy:client] [resubscribe %s] token not found", key.toString().c_str());
logger_->error("[proxy:client] [resubscribe {}] token not found", key.to_view());
}
}
}
@@ -952,9 +946,9 @@ bool
DhtProxyClient::cancelListen(const InfoHash& key, size_t gtoken)
{
if (logger_)
logger_->d(key, "[proxy:client] [search %s] cancel listen %zu", key.to_c_str(), gtoken);
logger_->debug("[proxy:client] [search {}] cancel listen {}", key.to_view(), gtoken);
std::lock_guard<std::mutex> lock(searchLock_);
std::lock_guard lock(searchLock_);
// find the listener in cache
auto it = searches_.find(key);
if (it == searches_.end())
@@ -979,13 +973,13 @@ DhtProxyClient::handleExpireListener(const asio::error_code& ec, const InfoHash&
return;
else if (ec) {
if (logger_)
logger_->e("[proxy:client] [listen %s] error in cancel: %s", key.toString().c_str(), ec.message().c_str());
logger_->error("[proxy:client] [listen {}] error in cancel: {}", key.to_view(), ec.message());
return;
}
if (logger_)
logger_->d("[proxy:client] [listen %s] expire listener", key.toString().c_str());
logger_->debug("[proxy:client] [listen {}] expire listener", key.to_view());
std::lock_guard<std::mutex> lock(searchLock_);
std::lock_guard lock(searchLock_);
auto search = searches_.find(key);
if (search == searches_.end())
return;
@@ -1014,25 +1008,25 @@ DhtProxyClient::handleExpireListener(const asio::error_code& ec, const InfoHash&
request->add_on_done_callback([this, reqid, key](const http::Response& response) {
if (response.status_code != 200) {
if (logger_)
logger_->e("[proxy:client] [unsubscribe %s] failed with code=%i",
key.to_c_str(),
response.status_code);
logger_->error("[proxy:client] [unsubscribe {}] failed with code={}",
key.to_view(),
response.status_code);
if (not response.aborted and response.status_code == 0)
opFailed();
}
if (not isDestroying_) {
std::lock_guard<std::mutex> l(requestLock_);
std::lock_guard l(requestLock_);
requests_.erase(reqid);
}
});
{
std::lock_guard<std::mutex> l(requestLock_);
std::lock_guard l(requestLock_);
requests_[reqid] = request;
}
request->send();
} catch (const std::exception& e) {
if (logger_)
logger_->e("[proxy:client] [unsubscribe %s] failed: %s", key.to_c_str(), e.what());
logger_->error("[proxy:client] [unsubscribe {}] failed: {}", key.to_view(), e.what());
}
} else {
// stop the request
@@ -1043,9 +1037,9 @@ DhtProxyClient::handleExpireListener(const asio::error_code& ec, const InfoHash&
}
search->second.listeners.erase(it);
if (logger_)
logger_->d("[proxy:client] [listen:cancel] [search %s] %zu listener remaining",
key.to_c_str(),
search->second.listeners.size());
logger_->debug("[proxy:client] [listen:cancel] [search {}] {} listener remaining",
key.to_view(),
search->second.listeners.size());
});
if (next != time_point::max()) {
search->second.opExpirationTimer->expires_at(next);
@@ -1065,7 +1059,7 @@ DhtProxyClient::sendListen(const restinio::http_request_header_t& header,
ListenMethod method)
{
if (logger_)
logger_->e("[proxy:client] [listen] sendListen: %d", (int) method);
logger_->error("[proxy:client] [listen] sendListen: {}", (int) method);
try {
auto request = buildRequest();
listener.request = request;
@@ -1103,7 +1097,7 @@ DhtProxyClient::sendListen(const restinio::http_request_header_t& header,
if (cb) {
auto expired = json.get("expired", Json::Value(false)).asBool();
{
std::lock_guard<std::mutex> lock(lockCallbacks_);
std::lock_guard lock(lockCallbacks_);
callbacks_.emplace_back([cb, value, opstate, expired]() {
if (not opstate->stop.load()
and not cb({value}, expired, system_clock::time_point::min()))
@@ -1115,27 +1109,27 @@ DhtProxyClient::sendListen(const restinio::http_request_header_t& header,
}
} catch (const std::exception& e) {
if (logger_)
logger_->e("[proxy:client] [listen] request #%i error in parsing: %s", reqid, e.what());
logger_->error("[proxy:client] [listen] request #{} error in parsing: {}", reqid, e.what());
opstate->ok.store(false);
}
});
request->add_on_done_callback([this, opstate, reqid](const http::Response& response) {
if (response.status_code != 200) {
if (logger_)
logger_->e("[proxy:client] [listen] send request #%i failed with code=%i",
reqid,
response.status_code);
logger_->error("[proxy:client] [listen] send request #{} failed with code={}",
reqid,
response.status_code);
opstate->ok.store(false);
if (not response.aborted and response.status_code == 0)
opFailed();
}
if (not isDestroying_) {
std::lock_guard<std::mutex> l(requestLock_);
std::lock_guard l(requestLock_);
requests_.erase(reqid);
}
});
{
std::lock_guard<std::mutex> l(requestLock_);
std::lock_guard l(requestLock_);
requests_[reqid] = request;
}
request->add_on_status_callback([request, seconds = this->listenKeepIdle()](unsigned status_code) {
@@ -1147,7 +1141,7 @@ DhtProxyClient::sendListen(const restinio::http_request_header_t& header,
request->send();
} catch (const std::exception& e) {
if (logger_)
logger_->e("[proxy:client] [listen] request failed: %s", e.what());
logger_->error("[proxy:client] [listen] request failed: {}", e.what());
}
}
@@ -1157,9 +1151,9 @@ DhtProxyClient::opFailed()
if (isDestroying_)
return;
if (logger_)
logger_->e("[proxy:client] proxy request failed");
logger_->error("[proxy:client] proxy request failed");
{
std::lock_guard<std::mutex> l(lockCurrentProxyInfos_);
std::lock_guard l(lockCurrentProxyInfos_);
statusIpv4_ = NodeStatus::Disconnected;
statusIpv6_ = NodeStatus::Disconnected;
}
@@ -1192,15 +1186,15 @@ DhtProxyClient::restartListeners(const asio::error_code& ec)
if (logger_)
logger_->d("[proxy:client] [listeners] refresh permanent puts");
std::lock_guard<std::mutex> lock(searchLock_);
std::lock_guard lock(searchLock_);
for (auto& search : searches_) {
const auto& key = search.first;
for (auto& put : search.second.puts) {
doPut(key, put.second.value, [ok = put.second.ok](bool result) { *ok = result; }, time_point::max(), true);
if (!put.second.refreshPutTimer) {
put.second.refreshPutTimer = std::make_unique<asio::steady_timer>(httpContext_);
put.second.refreshPutTimer = std::make_unique<asio::system_timer>(httpContext_);
}
put.second.refreshPutTimer->expires_at(std::chrono::steady_clock::now() + proxy::OP_TIMEOUT
put.second.refreshPutTimer->expires_at(std::chrono::system_clock::now() + proxy::OP_TIMEOUT
- proxy::OP_MARGIN);
put.second.refreshPutTimer->async_wait(
std::bind(&DhtProxyClient::handleRefreshPut, this, std::placeholders::_1, key, put.first));
@@ -1212,16 +1206,15 @@ DhtProxyClient::restartListeners(const asio::error_code& ec)
}
if (not deviceKey_.empty()) {
if (logger_)
logger_->d("[proxy:client] [listeners] resubscribe due to a connectivity change");
logger_->debug("[proxy:client] [listeners] resubscribe due to a connectivity change");
// Connectivity changed, refresh all subscribe
for (auto& search : searches_)
for (auto& listener : search.second.listeners)
if (!listener.second.opstate->ok)
resubscribe(search.first, listener.first, listener.second);
resubscribe(search.first, listener.first, listener.second);
return;
}
if (logger_)
logger_->d("[proxy:client] [listeners] restarting listeners");
logger_->debug("[proxy:client] [listeners] restarting listeners");
for (auto& search : searches_) {
for (auto& l : search.second.listeners) {
auto& listener = l.second;
@@ -1255,7 +1248,7 @@ DhtProxyClient::pushNotificationReceived([[maybe_unused]] const std::map<std::st
#ifdef OPENDHT_PUSH_NOTIFICATIONS
{
// If a push notification is received, the proxy is up and running
std::lock_guard<std::mutex> l(lockCurrentProxyInfos_);
std::lock_guard l(lockCurrentProxyInfos_);
auto oldStatus = std::max(statusIpv4_, statusIpv6_);
if (oldStatus != NodeStatus::Connected)
launchConnectedCbs_ = true;
@@ -1267,10 +1260,10 @@ DhtProxyClient::pushNotificationReceived([[maybe_unused]] const std::map<std::st
auto sessionId = notification.find("s");
if (sessionId != notification.end() and sessionId->second != pushSessionId_) {
if (logger_)
logger_->d("[proxy:client] [push] ignoring push for other session");
logger_->debug("[proxy:client] [push] ignoring push for other session");
return PushNotificationResult::IgnoredWrongSession;
}
std::lock_guard<std::mutex> lock(searchLock_);
std::lock_guard lock(searchLock_);
auto timeout = notification.find("timeout");
if (timeout != notification.cend()) {
InfoHash key(timeout->second);
@@ -1281,10 +1274,10 @@ DhtProxyClient::pushNotificationReceived([[maybe_unused]] const std::map<std::st
auto vid = std::stoull(vidIt->second);
auto& put = search.puts.at(vid);
if (!put.refreshPutTimer)
put.refreshPutTimer = std::make_unique<asio::steady_timer>(httpContext_,
std::chrono::steady_clock::now());
put.refreshPutTimer = std::make_unique<asio::system_timer>(httpContext_,
std::chrono::system_clock::now());
else
put.refreshPutTimer->expires_at(std::chrono::steady_clock::now());
put.refreshPutTimer->expires_at(std::chrono::system_clock::now());
put.refreshPutTimer->async_wait(
std::bind(&DhtProxyClient::handleRefreshPut, this, std::placeholders::_1, key, vid));
ret = PushNotificationResult::PutRefresh;
@@ -1296,19 +1289,19 @@ DhtProxyClient::pushNotificationReceived([[maybe_unused]] const std::map<std::st
: PushNotificationResult::ListenRefresh;
}
} else {
auto key = InfoHash(notification.at("key"));
system_clock::time_point sendTime = system_clock::time_point::min();
try {
sendTime = system_clock::time_point(std::chrono::milliseconds(std::stoull(notification.at("t"))));
} catch (...) {
}
auto key = InfoHash(notification.at("key"));
auto& search = searches_.at(key);
auto expired = notification.find("exp");
for (auto& list : search.listeners) {
if (list.second.opstate->stop)
continue;
if (logger_)
logger_->d("[proxy:client] [push] [search %s] received", key.to_c_str());
logger_->debug("[proxy:client] [push] [search {}] received", key);
auto token = list.first;
auto opstate = list.second.opstate;
if (expired == notification.end()) {
@@ -1332,11 +1325,11 @@ DhtProxyClient::pushNotificationReceived([[maybe_unused]] const std::map<std::st
ids.emplace_back(std::stoull(substr));
}
{
std::lock_guard<std::mutex> lockCb(lockCallbacks_);
std::lock_guard lockCb(lockCallbacks_);
callbacks_.emplace_back([this, key, token, opstate, ids, sendTime]() {
if (opstate->stop)
return;
std::lock_guard<std::mutex> lock(searchLock_);
std::lock_guard lock(searchLock_);
auto s = searches_.find(key);
if (s == searches_.end())
return;
@@ -1356,7 +1349,7 @@ DhtProxyClient::pushNotificationReceived([[maybe_unused]] const std::map<std::st
ret = PushNotificationResult::IgnoredNoOp;
} catch (const std::exception& e) {
if (logger_)
logger_->e("[proxy:client] [push] receive error: %s", e.what());
logger_->error("[proxy:client] [push] receive error: {}", e.what());
ret = PushNotificationResult::Error;
}
if (launchLoop)
@@ -1374,7 +1367,7 @@ DhtProxyClient::resubscribe([[maybe_unused]] const InfoHash& key,
if (deviceKey_.empty())
return;
if (logger_)
logger_->d("[proxy:client] [resubscribe] [search %s]", key.to_c_str());
logger_->debug("[proxy:client] [resubscribe] [search {}]", key);
auto opstate = listener.opstate;
listener.request.reset(); // This will update ok to false
@@ -1382,11 +1375,11 @@ DhtProxyClient::resubscribe([[maybe_unused]] const InfoHash& key,
restinio::http_request_header_t header;
header.method(restinio::http_method_subscribe());
header.request_target("/" + key.toString());
header.request_target("/key/" + key.toString());
if (!listener.refreshSubscriberTimer) {
listener.refreshSubscriberTimer = std::make_unique<asio::steady_timer>(httpContext_);
listener.refreshSubscriberTimer = std::make_unique<asio::system_timer>(httpContext_);
}
listener.refreshSubscriberTimer->expires_at(std::chrono::steady_clock::now() + proxy::OP_TIMEOUT - proxy::OP_MARGIN);
listener.refreshSubscriberTimer->expires_at(std::chrono::system_clock::now() + proxy::OP_TIMEOUT - proxy::OP_MARGIN);
listener.refreshSubscriberTimer->async_wait(
std::bind(&DhtProxyClient::handleResubscribe, this, std::placeholders::_1, key, token, opstate));
auto vcb = listener.cb;
@@ -1428,17 +1421,20 @@ void
DhtProxyClient::setPushNotificationToken([[maybe_unused]] const std::string& token)
{
#ifdef OPENDHT_PUSH_NOTIFICATIONS
std::unique_lock<std::mutex> l(lockCurrentProxyInfos_);
if (deviceKey_ != token) {
{
std::unique_lock l(lockCurrentProxyInfos_);
if (deviceKey_ == token)
return;
deviceKey_ = token;
if (statusIpv4_ == NodeStatus::Connected || statusIpv6_ == NodeStatus::Connected) {
if (logger_)
logger_->d("[proxy:client] [push] token changed, resubscribing");
for (auto& search : searches_) {
for (auto& listener : search.second.listeners) {
resubscribe(search.first, listener.first, listener.second);
}
}
if (!(statusIpv4_ == NodeStatus::Connected || statusIpv6_ == NodeStatus::Connected))
return;
}
if (logger_)
logger_->debug("[proxy:client] [push] token changed, resubscribing");
std::lock_guard lock(searchLock_);
for (auto& search : searches_) {
for (auto& listener : search.second.listeners) {
resubscribe(search.first, listener.first, listener.second);
}
}
#endif
+93 -100
View File
@@ -72,21 +72,21 @@ public:
void info(Builder&& msg_builder)
{
if (m_logger)
m_logger->d("[proxy:server] %s", msg_builder().c_str());
m_logger->debug("[proxy:server] {}", msg_builder());
}
template<typename Builder>
void warn(Builder&& msg_builder)
{
if (m_logger)
m_logger->w("[proxy:server] %s", msg_builder().c_str());
m_logger->warn("[proxy:server] {}", msg_builder());
}
template<typename Builder>
void error(Builder&& msg_builder)
{
if (m_logger)
m_logger->e("[proxy:server] %s", msg_builder().c_str());
m_logger->error("[proxy:server] {}", msg_builder());
}
private:
@@ -126,8 +126,8 @@ DhtProxyServer::ServerStats::toJson() const
result["putCount"] = static_cast<Json::UInt64>(putCount);
result["totalPermanentPuts"] = static_cast<Json::UInt64>(totalPermanentPuts);
result["pushListenersCount"] = static_cast<Json::UInt64>(pushListenersCount);
result["serverStartTime"] = static_cast<Json::LargestInt>(to_time_t(serverStartTime));
result["lastUpdated"] = static_cast<Json::LargestInt>(to_time_t(lastUpdated));
result["serverStartTime"] = static_cast<Json::LargestInt>(std::chrono::system_clock::to_time_t(serverStartTime));
result["lastUpdated"] = static_cast<Json::LargestInt>(std::chrono::system_clock::to_time_t(lastUpdated));
result["androidPush"] = androidPush.toJson();
result["iosPush"] = iosPush.toJson();
result["unifiedPush"] = unifiedPush.toJson();
@@ -176,13 +176,15 @@ DhtProxyServer::ConnectionListener::state_changed(const restinio::connection_sta
void
DhtProxyServer::onConnectionClosed(restinio::connection_id_t id)
{
std::lock_guard<std::mutex> lock(lockListener_);
std::lock_guard lock(lockListener_);
auto it = listeners_.find(id);
if (it != listeners_.end()) {
dht_->cancelListen(it->second.hash, std::move(it->second.token));
listeners_.erase(it);
if (logger_)
logger_->d("[proxy:server] [connection:%li] listener cancelled, %li still connected", id, listeners_.size());
logger_->debug("[proxy:server] [connection:{}] listener cancelled, {} still connected",
id,
listeners_.size());
}
}
@@ -208,7 +210,7 @@ DhtProxyServer::PermanentPut::msgpack_unpack(const msgpack::object& o)
clientId = cid->as<std::string>();
}
if (auto exp = findMapValue(o, "exp"sv)) {
expiration = from_time_t(exp->as<time_t>());
expiration = std::chrono::system_clock::from_time_t(exp->as<time_t>());
}
if (auto token = findMapValue(o, "token"sv)) {
pushToken = token->as<std::string>();
@@ -238,7 +240,7 @@ DhtProxyServer::Listener::msgpack_unpack(const msgpack::object& o)
clientId = cid->as<std::string>();
}
if (auto exp = findMapValue(o, "exp"sv)) {
expiration = from_time_t(exp->as<time_t>());
expiration = std::chrono::system_clock::from_time_t(exp->as<time_t>());
}
if (auto sid = findMapValue(o, "sid"sv)) {
if (not sessionCtx)
@@ -322,7 +324,7 @@ DhtProxyServer::DhtProxyServer(const std::shared_ptr<DhtRunner>& dht,
if (ec)
throw std::runtime_error("Error setting certificate chain: " + ec.message());
if (logger_)
logger_->d("[proxy:server] using certificate chain for ssl:\n%s", certchain.c_str());
logger_->debug("[proxy:server] using certificate chain for ssl:\n{}", certchain);
// build http server
auto settings = restinio::run_on_this_thread_settings_t<RestRouterTraitsTls>();
addServerSettings(settings);
@@ -362,15 +364,12 @@ DhtProxyServer::DhtProxyServer(const std::shared_ptr<DhtRunner>& dht,
std::streamsize size = stateFile.tellg();
stateFile.seekg(0, std::ios::beg);
if (logger_)
logger_->d("Loading proxy state from %.*s (%td bytes)",
(int) persistPath_.size(),
persistPath_.c_str(),
size);
logger_->debug("[proxy:server] Loading proxy state from {} ({} bytes)", persistPath_, size);
loadState(stateFile, size);
}
} catch (const std::exception& e) {
if (logger_)
logger_->e("Error loading state from file: %s", e.what());
logger_->error("[proxy:server] Error loading state from file: {}", e.what());
}
}
}
@@ -382,13 +381,13 @@ DhtProxyServer::saveState(Os& stream)
msgpack::packer<Os> pk(&stream);
pk.pack_map(2);
{
std::lock_guard<std::mutex> lock(lockSearchPuts_);
std::lock_guard lock(lockSearchPuts_);
pk.pack("puts");
pk.pack(puts_);
}
#ifdef OPENDHT_PUSH_NOTIFICATIONS
{
std::lock_guard<std::mutex> lock(lockListener_);
std::lock_guard lock(lockListener_);
pk.pack("pushListeners");
pk.pack(pushListeners_);
}
@@ -409,13 +408,13 @@ DhtProxyServer::loadState(Is& is, size_t size)
if (oh.get().type != msgpack::type::MAP)
continue;
if (auto puts = findMapValue(oh.get(), "puts"sv)) {
std::lock_guard<std::mutex> lock(lockSearchPuts_);
std::lock_guard lock(lockSearchPuts_);
puts_ = puts->as<decltype(puts_)>();
if (logger_)
logger_->d("Loading %zu persistent puts", puts_.size());
logger_->debug("[proxy:server] Loading {} persistent puts", puts_.size());
for (auto& put : puts_) {
for (auto& pput : put.second.puts) {
pput.second.expireTimer = std::make_unique<asio::steady_timer>(io_context(),
pput.second.expireTimer = std::make_unique<asio::system_timer>(io_context(),
pput.second.expiration);
pput.second.expireTimer->async_wait(std::bind(&DhtProxyServer::handleCancelPermamentPut,
this,
@@ -433,12 +432,12 @@ DhtProxyServer::loadState(Is& is, size_t size)
json["to"] = clientId;
json["vid"] = std::to_string(vid);
if (sessionCtx) {
std::lock_guard<std::mutex> l(sessionCtx->lock);
std::lock_guard l(sessionCtx->lock);
json["s"] = sessionCtx->sessionId;
}
return json;
};
pput.second.expireNotifyTimer = std::make_unique<asio::steady_timer>(io_context(),
pput.second.expireNotifyTimer = std::make_unique<asio::system_timer>(io_context(),
pput.second.expiration
- proxy::OP_MARGIN);
pput.second.expireNotifyTimer->async_wait(
@@ -451,19 +450,19 @@ DhtProxyServer::loadState(Is& is, size_t size)
pput.second.topic));
}
#endif
dht_->put(put.first, pput.second.value, DoneCallbackSimple {}, time_point::max(), true);
dht_->put(put.first, pput.second.value, DoneCallbackSimple {}, dht::time_point::max(), true);
}
}
} else {
if (logger_)
logger_->d("No persistent puts in state");
logger_->debug("No persistent puts in state");
}
#ifdef OPENDHT_PUSH_NOTIFICATIONS
if (auto pushListeners = findMapValue(oh.get(), "pushListeners"sv)) {
std::lock_guard<std::mutex> lock(lockListener_);
std::lock_guard lock(lockListener_);
pushListeners_ = pushListeners->as<decltype(pushListeners_)>();
if (logger_)
logger_->d("Loading %zu push listeners", pushListeners_.size());
logger_->debug("[proxy:server] Loading {} push listeners", pushListeners_.size());
for (auto& pushListener : pushListeners_) {
for (auto& listeners : pushListener.second.listeners) {
for (auto& listener : listeners.second) {
@@ -488,7 +487,7 @@ DhtProxyServer::loadState(Is& is, size_t size)
expired);
});
// expire notify
listener.expireNotifyTimer = std::make_unique<asio::steady_timer>(io_context(),
listener.expireNotifyTimer = std::make_unique<asio::system_timer>(io_context(),
listener.expiration
- proxy::OP_MARGIN);
auto jsonProvider = [infoHash = listeners.first.toString(),
@@ -497,7 +496,7 @@ DhtProxyServer::loadState(Is& is, size_t size)
Json::Value json;
json["timeout"] = infoHash;
json["to"] = clientId;
std::lock_guard<std::mutex> l(sessionCtx->lock);
std::lock_guard l(sessionCtx->lock);
json["s"] = sessionCtx->sessionId;
return json;
};
@@ -510,7 +509,7 @@ DhtProxyServer::loadState(Is& is, size_t size)
listener.type,
listener.topic));
// cancel push listen
listener.expireTimer = std::make_unique<asio::steady_timer>(io_context(),
listener.expireTimer = std::make_unique<asio::system_timer>(io_context(),
listener.expiration);
listener.expireTimer->async_wait(std::bind(&DhtProxyServer::handleCancelPushListen,
this,
@@ -523,12 +522,12 @@ DhtProxyServer::loadState(Is& is, size_t size)
}
} else {
if (logger_)
logger_->d("No push listeners in state");
logger_->debug("[proxy:server] No push listeners in state");
}
#endif
}
if (logger_)
logger_->d("loading ended");
logger_->debug("[proxy:server] loading ended");
}
}
@@ -542,12 +541,12 @@ DhtProxyServer::~DhtProxyServer()
{
if (not persistPath_.empty()) {
if (logger_)
logger_->d("Saving proxy state to %.*s", (int) persistPath_.size(), persistPath_.c_str());
logger_->debug("[proxy:server] Saving proxy state to {}", persistPath_);
std::ofstream stateFile(persistPath_, std::ios::binary);
saveState(stateFile);
}
if (dht_) {
std::lock_guard<std::mutex> lock(lockListener_);
std::lock_guard lock(lockListener_);
for (auto& l : listeners_) {
dht_->cancelListen(l.second.hash, std::move(l.second.token));
if (l.second.response)
@@ -568,12 +567,12 @@ DhtProxyServer::~DhtProxyServer()
#endif
}
if (logger_)
logger_->d("[proxy:server] closing http server");
logger_->debug("[proxy:server] closing http server");
ioContext_->stop();
if (serverThread_.joinable())
serverThread_.join();
if (logger_)
logger_->d("[proxy:server] http server closed");
logger_->debug("[proxy:server] http server closed");
}
template<typename ServerSettings>
@@ -647,7 +646,7 @@ DhtProxyServer::updateStats()
nodeInfo_ = newInfo;
if (logger_) {
auto str = Json::writeString(jsonBuilder_, newInfo->toJson());
logger_->d("[proxy:server] [stats] %s", str.c_str());
logger_->debug("[proxy:server] [stats] {}", str);
}
});
}
@@ -818,7 +817,7 @@ DhtProxyServer::listen(restinio::request_handle_t request, restinio::router::rou
auto response = std::make_shared<ResponseByPartsBuilder>(
initHttpResponse(request->create_response<ResponseByParts>()));
response->flush();
std::lock_guard<std::mutex> lock(lockListener_);
std::lock_guard lock(lockListener_);
// save the listener to handle a disconnect
auto& session = listeners_[request->connection_id()];
session.hash = infoHash;
@@ -908,6 +907,7 @@ DhtProxyServer::subscribe(restinio::request_handle_t request, restinio::router::
Json::Value r;
auto* char_data = reinterpret_cast<const char*>(request->body().data());
auto reader = std::unique_ptr<Json::CharReader>(jsonReaderBuilder_.newCharReader());
if (!reader->parse(char_data, char_data + request->body().size(), &r, &err)) {
auto response = initHttpResponse(request->create_response(restinio::status_bad_request()));
response.set_body(RESP_MSG_JSON_INCORRECT);
@@ -929,13 +929,10 @@ DhtProxyServer::subscribe(restinio::request_handle_t request, restinio::router::
auto sessionId = root["session_id"].asString();
if (logger_)
logger_->d("[proxy:server] [subscribe %s] [client %s] [session %s]",
infoHash.toString().c_str(),
clientId.c_str(),
sessionId.c_str());
logger_->debug("[proxy:server] [subscribe {}] [client {}] [session {}]", infoHash, clientId, sessionId);
// Insert new or return existing push listeners of a token
std::lock_guard<std::mutex> lock(lockListener_);
std::lock_guard lock(lockListener_);
auto& pushListener = pushListeners_[pushToken];
auto& pushListeners = pushListener.listeners[infoHash];
@@ -949,25 +946,25 @@ DhtProxyServer::subscribe(restinio::request_handle_t request, restinio::router::
listIt->clientId = clientId;
listIt->sessionCtx = std::make_shared<PushSessionContext>(sessionId);
} else {
std::lock_guard<std::mutex> l(listIt->sessionCtx->lock);
std::lock_guard l(listIt->sessionCtx->lock);
listIt->sessionCtx->sessionId = sessionId;
}
auto& listener = *listIt;
// Expiration
auto timeout = std::chrono::steady_clock::now() + proxy::OP_TIMEOUT;
auto timeout = clock::now() + proxy::OP_TIMEOUT;
listener.expiration = timeout;
listener.type = type;
listener.topic = topic;
if (listener.expireNotifyTimer)
listener.expireNotifyTimer->expires_at(timeout - proxy::OP_MARGIN);
else
listener.expireNotifyTimer = std::make_unique<asio::steady_timer>(io_context(), timeout - proxy::OP_MARGIN);
listener.expireNotifyTimer = std::make_unique<asio::system_timer>(io_context(), timeout - proxy::OP_MARGIN);
auto jsonProvider = [h = infoHash.toString(), clientId, sessionCtx = listener.sessionCtx]() {
Json::Value json;
json["timeout"] = h;
json["to"] = clientId;
std::lock_guard<std::mutex> l(sessionCtx->lock);
std::lock_guard l(sessionCtx->lock);
json["s"] = sessionCtx->sessionId;
return json;
};
@@ -979,7 +976,7 @@ DhtProxyServer::subscribe(restinio::request_handle_t request, restinio::router::
listener.type,
listener.topic));
if (!listener.expireTimer)
listener.expireTimer = std::make_unique<asio::steady_timer>(io_context(), timeout);
listener.expireTimer = std::make_unique<asio::system_timer>(io_context(), timeout);
else
listener.expireTimer->expires_at(timeout);
listener.expireTimer->async_wait(std::bind(&DhtProxyServer::handleCancelPushListen,
@@ -992,9 +989,7 @@ DhtProxyServer::subscribe(restinio::request_handle_t request, restinio::router::
// Send response
if (not newListener) {
if (logger_)
logger_->d("[proxy:server] [subscribe %s] found [client %s]",
infoHash.toString().c_str(),
listener.clientId.c_str());
logger_->debug("[proxy:server] [subscribe {}] found [client {}]", infoHash, listener.clientId);
// Send response header
auto response = std::make_shared<ResponseByPartsBuilder>(
initHttpResponse(request->create_response<ResponseByParts>()));
@@ -1019,7 +1014,7 @@ DhtProxyServer::subscribe(restinio::request_handle_t request, restinio::router::
// =========== No existing listener for an infoHash ============
// Add listen on dht
if (logger_)
logger_->d("[proxy:server] [subscribe %s] new", infoHash.toString().c_str());
logger_->debug("[proxy:server] [subscribe {}] new", infoHash);
listener.internalToken = dht_->listen(infoHash,
[this,
infoHash,
@@ -1058,7 +1053,7 @@ DhtProxyServer::unsubscribe(restinio::request_handle_t request, restinio::router
infoHash = InfoHash::get(params["hash"]);
if (logger_)
logger_->d("[proxy:server] [unsubscribe %s]", infoHash.toString().c_str());
logger_->debug("[proxy:server] [unsubscribe {}]", infoHash);
try {
std::string err;
@@ -1095,10 +1090,10 @@ DhtProxyServer::handleNotifyPushListenExpire(const asio::error_code& ec,
return;
else if (ec) {
if (logger_)
logger_->e("[proxy:server] [subscribe] error sending put refresh: %s", ec.message().c_str());
logger_->error("[proxy:server] [subscribe] error sending put refresh: {}", ec.message());
}
if (logger_)
logger_->d("[proxy:server] [subscribe] sending refresh to %s token", pushToken.c_str());
logger_->debug("[proxy:server] [subscribe] sending refresh to {} token", pushToken);
sendPushNotification(pushToken, jsonProvider(), type, false, topic);
}
@@ -1112,11 +1107,11 @@ DhtProxyServer::handleCancelPushListen(const asio::error_code& ec,
return;
else if (ec) {
if (logger_)
logger_->e("[proxy:server] [listen:push %s] error cancel: %s", key.toString().c_str(), ec.message().c_str());
logger_->error("[proxy:server] [listen:push {}] error cancel: {}", key, ec.message());
}
if (logger_)
logger_->d("[proxy:server] [listen:push %s] cancelled for %s", key.toString().c_str(), clientId.c_str());
std::lock_guard<std::mutex> lock(lockListener_);
logger_->debug("[proxy:server] [listen:push {}] cancelled for {}", key, clientId);
std::lock_guard lock(lockListener_);
auto pushListener = pushListeners_.find(pushToken);
if (pushListener == pushListeners_.end())
@@ -1157,19 +1152,24 @@ DhtProxyServer::handlePushListen(const InfoHash& infoHash,
json["t"] = Json::Value::Int64(duration_cast<milliseconds>(system_clock::now().time_since_epoch()).count());
{
std::lock_guard<std::mutex> l(sessionCtx->lock);
std::lock_guard l(sessionCtx->lock);
json["s"] = sessionCtx->sessionId;
}
// Build a comma-separated list of ids from the values. This is intended to be used when
// streaming from the iOS notification extension in order to filter out unwanted values.
std::string ids;
std::string pushTypes;
for (const auto& value : values) {
if (!ids.empty())
if (!ids.empty()) {
ids += ",";
pushTypes += ",";
}
pushTypes += value->pushType;
ids += std::to_string(value->id);
}
json["ids"] = ids;
json["pt"] = pushTypes;
// If message is expired copy the value ID to the `exp` field.
// This is presumably used by the Android notification system.
@@ -1182,13 +1182,13 @@ DhtProxyServer::handlePushListen(const InfoHash& infoHash,
minPriority = std::min(minPriority, v->priority);
if (logger_)
logger_->d("[proxy:server] [listen %s] [client %s] [session %s] [expired %i] [priority %i] [values %zu]",
infoHash.toString().c_str(),
clientId.c_str(),
sessionCtx->sessionId.c_str(),
expired,
minPriority,
values.size());
logger_->debug("[proxy:server] [listen {}] [client {}] [session {}] [expired {}] [priority {}] [values {}]",
infoHash,
clientId,
sessionCtx->sessionId,
expired,
minPriority,
values.size());
sendPushNotification(pushToken, std::move(json), type, !expired and minPriority == 0, topic);
@@ -1206,14 +1206,7 @@ DhtProxyServer::sendPushNotification(
try {
std::shared_ptr<http::Request> request;
if (type == PushType::UnifiedPush) {
http::Url tokenUrl(token);
request = std::make_shared<http::Request>(io_context(),
concat(tokenUrl.protocol, "://"sv, tokenUrl.host),
tokenUrl.service,
tokenUrl.protocol.find("https") == 0,
logger_);
request->set_target(tokenUrl.target);
request->set_header_field(restinio::http_field_t::host, std::string(tokenUrl.host));
request = std::make_shared<http::Request>(io_context(), token, logger_);
} else {
request = std::make_shared<http::Request>(io_context(),
pushHostPort_.first,
@@ -1232,8 +1225,8 @@ DhtProxyServer::sendPushNotification(
Blob auth; ///< Auth secret
auto delim = topicView.find('|');
if (delim != std::string_view::npos) {
pubKey = base64_decode(topicView.substr(0, delim));
auth = base64_decode(topicView.substr(delim + 1));
pubKey = base64_default_or_url_decode(topicView.substr(0, delim));
auth = base64_default_or_url_decode(topicView.substr(delim + 1));
}
request->set_header_field(restinio::http_field_t::ttl, "86400");
@@ -1251,7 +1244,7 @@ DhtProxyServer::sendPushNotification(
request->set_body(std::string(encrypted.begin(), encrypted.end()));
} catch (const std::exception& e) {
if (logger_)
logger_->e("[proxy:server] [notification] encryption failed: %s", e.what());
logger_->error("[proxy:server] [notification] encryption failed: {}", e.what());
return;
}
} else {
@@ -1307,13 +1300,13 @@ DhtProxyServer::sendPushNotification(
request->add_on_state_change_callback([this, reqid](http::Request::State state, const http::Response& response) {
if (state == http::Request::State::DONE) {
if (logger_ and response.status_code != 200)
logger_->e("[proxy:server] [notification] push failed: %i", response.status_code);
std::lock_guard<std::mutex> l(requestLock_);
logger_->error("[proxy:server] [notification] push failed: {}", response.status_code);
std::lock_guard l(requestLock_);
requests_.erase(reqid);
}
});
{
std::lock_guard<std::mutex> l(requestLock_);
std::lock_guard l(requestLock_);
requests_[reqid] = request;
}
request->send();
@@ -1334,9 +1327,9 @@ DhtProxyServer::sendPushNotification(
}
} catch (const std::exception& e) {
if (logger_)
logger_->e("[proxy:server] [notification] error send push: %s", e.what());
logger_->error("[proxy:server] [notification] error send push: {}", e.what());
if (reqid) {
std::lock_guard<std::mutex> l(requestLock_);
std::lock_guard l(requestLock_);
requests_.erase(reqid);
}
}
@@ -1351,11 +1344,11 @@ DhtProxyServer::handleCancelPermamentPut(const asio::error_code& ec, const InfoH
return;
else if (ec) {
if (logger_)
logger_->e("[proxy:server] [put:permament] error sending put refresh: %s", ec.message().c_str());
logger_->error("[proxy:server] [put:permament] error sending put refresh: {}", ec.message());
}
if (logger_)
logger_->d("[proxy:server] [put %s] cancel permament put %i", key.toString().c_str(), vid);
std::lock_guard<std::mutex> lock(lockSearchPuts_);
logger_->debug("[proxy:server] [put {}] cancel permament put {}", key, vid);
std::lock_guard lock(lockSearchPuts_);
auto sPuts = puts_.find(key);
if (sPuts == puts_.end())
return;
@@ -1398,10 +1391,10 @@ DhtProxyServer::put(restinio::request_handle_t request, restinio::router::route_
auto value = std::make_shared<Value>(root);
bool permanent = root.isMember("permanent");
if (logger_)
logger_->d("[proxy:server] [put %s] %s %s",
infoHash.toString().c_str(),
value->toString().c_str(),
(permanent ? "permanent" : ""));
logger_->debug("[proxy:server] [put {}] {} {}",
infoHash,
value->toString(),
(permanent ? "permanent" : ""));
if (permanent) {
std::string pushToken, clientId, sessionId, platform, topic;
auto& pVal = root["permanent"];
@@ -1412,8 +1405,8 @@ DhtProxyServer::put(restinio::request_handle_t request, restinio::router::route_
sessionId = pVal["session_id"].asString();
topic = pVal["topic"].asString();
}
std::lock_guard<std::mutex> lock(lockSearchPuts_);
auto timeout = std::chrono::steady_clock::now() + proxy::OP_TIMEOUT;
std::lock_guard lock(lockSearchPuts_);
auto timeout = clock::now() + proxy::OP_TIMEOUT;
auto& sPuts = puts_[infoHash];
if (value->id == Value::INVALID_ID) {
for (auto& pp : sPuts.puts) {
@@ -1429,7 +1422,7 @@ DhtProxyServer::put(restinio::request_handle_t request, restinio::router::route_
if (not pp.second.sessionCtx)
pp.second.sessionCtx = std::make_shared<PushSessionContext>(sessionId);
else {
std::lock_guard<std::mutex> l(pp.second.sessionCtx->lock);
std::lock_guard l(pp.second.sessionCtx->lock);
pp.second.sessionCtx->sessionId = sessionId;
}
}
@@ -1447,7 +1440,7 @@ DhtProxyServer::put(restinio::request_handle_t request, restinio::router::route_
pput.expiration = timeout;
if (not pput.expireTimer) {
// cancel permanent put
pput.expireTimer = std::make_unique<asio::steady_timer>(io_context(), timeout);
pput.expireTimer = std::make_unique<asio::system_timer>(io_context(), timeout);
#ifdef OPENDHT_PUSH_NOTIFICATIONS
if (not pushToken.empty()) {
pput.pushToken = pushToken;
@@ -1464,7 +1457,7 @@ DhtProxyServer::put(restinio::request_handle_t request, restinio::router::route_
if (not pput.sessionCtx)
pput.sessionCtx = std::make_shared<PushSessionContext>(sessionId);
else {
std::lock_guard<std::mutex> l(pput.sessionCtx->lock);
std::lock_guard l(pput.sessionCtx->lock);
pput.sessionCtx->sessionId = sessionId;
}
}
@@ -1481,12 +1474,12 @@ DhtProxyServer::put(restinio::request_handle_t request, restinio::router::route_
json["timeout"] = infoHash.toString();
json["to"] = clientId;
json["vid"] = std::to_string(vid);
std::lock_guard<std::mutex> l(sessionCtx->lock);
std::lock_guard l(sessionCtx->lock);
json["s"] = sessionCtx->sessionId;
return json;
};
if (!pput.expireNotifyTimer)
pput.expireNotifyTimer = std::make_unique<asio::steady_timer>(io_context(),
pput.expireNotifyTimer = std::make_unique<asio::system_timer>(io_context(),
timeout - proxy::OP_MARGIN);
else
pput.expireNotifyTimer->expires_at(timeout - proxy::OP_MARGIN);
@@ -1515,7 +1508,7 @@ DhtProxyServer::put(restinio::request_handle_t request, restinio::router::route_
response.done();
}
},
time_point::max(),
dht::time_point::max(),
permanent);
return restinio::request_handling_status_t::accepted;
} else {
@@ -1525,7 +1518,7 @@ DhtProxyServer::put(restinio::request_handle_t request, restinio::router::route_
}
} catch (const std::exception& e) {
if (logger_)
logger_->d("[proxy:server] error in put: %s", e.what());
logger_->error("[proxy:server] error in put: {}", e.what());
return serverError(*request);
}
}
@@ -1575,7 +1568,7 @@ DhtProxyServer::putSigned(restinio::request_handle_t request, restinio::router::
}
} catch (const std::exception& e) {
if (logger_)
logger_->d("[proxy:server] error in putSigned: %s", e.what());
logger_->error("[proxy:server] error in putSigned: {}", e.what());
return serverError(*request);
}
}
@@ -1627,7 +1620,7 @@ DhtProxyServer::putEncrypted(restinio::request_handle_t request, restinio::route
}
} catch (const std::exception& e) {
if (logger_)
logger_->d("[proxy:server] error in put: %s", e.what());
logger_->error("[proxy:server] error in put: {}", e.what());
return serverError(*request);
}
}
+128 -154
View File
@@ -76,7 +76,7 @@ DhtRunner::run(const char* ip4, const char* ip6, const char* service, Config& co
void
DhtRunner::run(const Config& config, Context&& context)
{
std::lock_guard<std::mutex> lck(dht_mtx);
std::lock_guard lck(dht_mtx);
auto expected = State::Idle;
if (not running.compare_exchange_strong(expected, State::Running)) {
if (context.logger)
@@ -99,6 +99,11 @@ DhtRunner::run(const Config& config, Context&& context)
auto state_path = config.dht_config.node_config.persist_path;
if (not state_path.empty())
state_path += "_port.txt";
// Ports loaded from the persisted state are a best-effort hint to keep a
// stable identity across restarts; remember which ones we set so that we
// can fall back to random ports if they are already in use (e.g. a fast
// restart while the previous instance still holds them).
bool triedSavedPort = false;
if (not state_path.empty() && (local4.getPort() == 0 || local6.getPort() == 0)) {
std::ifstream inConfig(state_path);
if (inConfig.is_open()) {
@@ -110,6 +115,7 @@ DhtRunner::run(const Config& config, Context&& context)
fmt::ptr(this),
port);
local4.setPort(port);
triedSavedPort = true;
}
}
if (inConfig >> port) {
@@ -119,6 +125,7 @@ DhtRunner::run(const Config& config, Context&& context)
fmt::ptr(this),
port);
local6.setPort(port);
triedSavedPort = true;
}
}
}
@@ -136,12 +143,25 @@ DhtRunner::run(const Config& config, Context&& context)
if (config.proxy_server.empty()) {
if (not context.sock) {
context.sock.reset(new net::UdpSocket(local4, local6, context.logger));
try {
context.sock.reset(new net::UdpSocket(local4, local6, context.logger));
} catch (const DhtException&) {
if (not triedSavedPort)
throw;
// The saved ports are unavailable (typically still held by a
// previous instance): fall back to OS-assigned ports.
if (context.logger)
context.logger->w("[runner %p] Saved ports unavailable, falling back to random ports",
fmt::ptr(this));
local4.setPort(0);
local6.setPort(0);
context.sock.reset(new net::UdpSocket(local4, local6, context.logger));
}
}
context.sock->setOnReceive([&](net::PacketList&& pkts) {
net::PacketList ret;
{
std::lock_guard<std::mutex> lck(sock_mtx);
std::lock_guard lck(sock_mtx);
rcv.splice(rcv.end(), std::move(pkts));
size_t dropped = 0;
while (rcv.size() > net::RX_QUEUE_MAX_SIZE) {
@@ -186,9 +206,6 @@ DhtRunner::run(const Config& config, Context&& context)
if (context.certificateStore) {
dht_->setLocalCertificateStore(std::move(context.certificateStore));
}
if (context.certificateStorePkId) {
dht_->setLocalCertificateStore(std::move(context.certificateStorePkId));
}
if (context.publicAddressChangedCb) {
dht_->setOnPublicAddressChanged(std::move(context.publicAddressChangedCb));
}
@@ -197,19 +214,19 @@ DhtRunner::run(const Config& config, Context&& context)
return;
dht_thread = std::thread([this]() {
while (running != State::Idle) {
std::unique_lock<std::mutex> lk(dht_mtx);
std::unique_lock lk(dht_mtx);
time_point wakeup = loop_();
auto hasJobToDo = [this]() {
if (running == State::Idle)
return true;
{
std::lock_guard<std::mutex> lck(sock_mtx);
std::lock_guard lck(sock_mtx);
if (not rcv.empty())
return true;
}
{
std::lock_guard<std::mutex> lck(storage_mtx);
std::lock_guard lck(storage_mtx);
if (not pending_ops_prio.empty())
return true;
auto s = getStatus();
@@ -284,7 +301,7 @@ DhtRunner::run(const Config& config, Context&& context)
void
DhtRunner::shutdown(ShutdownCallback cb, bool stop)
{
std::unique_lock<std::mutex> lck(storage_mtx);
std::unique_lock lck(storage_mtx);
auto expected = State::Running;
if (not running.compare_exchange_strong(expected, State::Stopping)) {
if (expected == State::Stopping and ongoing_ops) {
@@ -344,7 +361,7 @@ DhtRunner::checkShutdown()
{
decltype(shutdownCallbacks_) cbs;
{
std::lock_guard<std::mutex> lck(storage_mtx);
std::lock_guard lck(storage_mtx);
if (running != State::Stopping or ongoing_ops)
return false;
cbs = std::move(shutdownCallbacks_);
@@ -359,7 +376,7 @@ void
DhtRunner::join()
{
{
std::lock_guard<std::mutex> lck(dht_mtx);
std::lock_guard lck(dht_mtx);
if (running.exchange(State::Idle) == State::Idle)
return;
cv.notify_all();
@@ -378,7 +395,7 @@ DhtRunner::join()
dht_thread.join();
{
std::lock_guard<std::mutex> lck(storage_mtx);
std::lock_guard lck(storage_mtx);
if (ongoing_ops and logger_) {
logger_->warn("[runner {:p}] stopping with {:d} remaining ops", fmt::ptr(this), ongoing_ops.load());
}
@@ -388,7 +405,7 @@ DhtRunner::join()
shutdownCallbacks_.clear();
}
{
std::lock_guard<std::mutex> lck(dht_mtx);
std::lock_guard lck(dht_mtx);
resetDht();
status4 = NodeStatus::Disconnected;
status6 = NodeStatus::Disconnected;
@@ -398,7 +415,7 @@ DhtRunner::join()
SockAddr
DhtRunner::getBound(sa_family_t af) const
{
std::lock_guard<std::mutex> lck(dht_mtx);
std::lock_guard lck(dht_mtx);
if (dht_)
if (auto sock = dht_->getSocket())
return sock->getBound(af);
@@ -408,7 +425,7 @@ DhtRunner::getBound(sa_family_t af) const
in_port_t
DhtRunner::getBoundPort(sa_family_t af) const
{
std::lock_guard<std::mutex> lck(dht_mtx);
std::lock_guard lck(dht_mtx);
if (dht_)
if (auto sock = dht_->getSocket())
return sock->getPort(af);
@@ -418,7 +435,7 @@ DhtRunner::getBoundPort(sa_family_t af) const
void
DhtRunner::dumpTables() const
{
std::lock_guard<std::mutex> lck(dht_mtx);
std::lock_guard lck(dht_mtx);
dht_->dumpTables();
}
@@ -443,7 +460,7 @@ DhtRunner::getNodeId() const
std::pair<size_t, size_t>
DhtRunner::getStoreSize() const
{
std::lock_guard<std::mutex> lck(dht_mtx);
std::lock_guard lck(dht_mtx);
if (!dht_)
return {};
return dht_->getStoreSize();
@@ -452,16 +469,43 @@ DhtRunner::getStoreSize() const
void
DhtRunner::setStorageLimit(size_t limit)
{
std::lock_guard<std::mutex> lck(dht_mtx);
std::lock_guard lck(dht_mtx);
if (!dht_)
throw std::runtime_error("dht is not running");
return dht_->setStorageLimit(limit);
}
size_t
DhtRunner::getStorageLimit() const
{
std::lock_guard lck(dht_mtx);
if (!dht_)
throw std::runtime_error("dht is not running");
return dht_->getStorageLimit();
}
void
DhtRunner::setLocalStorageLimit(size_t limit)
{
std::lock_guard lck(dht_mtx);
if (!dht_)
throw std::runtime_error("dht is not running");
return dht_->setLocalStorageLimit(limit);
}
size_t
DhtRunner::getLocalStorageLimit() const
{
std::lock_guard lck(dht_mtx);
if (!dht_)
throw std::runtime_error("dht is not running");
return dht_->getLocalStorageLimit();
}
std::vector<NodeExport>
DhtRunner::exportNodes() const
{
std::lock_guard<std::mutex> lck(dht_mtx);
std::lock_guard lck(dht_mtx);
if (!dht_)
return {};
return dht_->exportNodes();
@@ -470,7 +514,7 @@ DhtRunner::exportNodes() const
std::vector<ValuesExport>
DhtRunner::exportValues() const
{
std::lock_guard<std::mutex> lck(dht_mtx);
std::lock_guard lck(dht_mtx);
if (!dht_)
return {};
return dht_->exportValues();
@@ -479,7 +523,7 @@ DhtRunner::exportValues() const
void
DhtRunner::setLogger(const Sp<Logger>& logger)
{
std::lock_guard<std::mutex> lck(dht_mtx);
std::lock_guard lck(dht_mtx);
logger_ = logger;
if (dht_)
dht_->setLogger(logger);
@@ -488,7 +532,7 @@ DhtRunner::setLogger(const Sp<Logger>& logger)
void
DhtRunner::setLogFilter(const InfoHash& f)
{
std::lock_guard<std::mutex> lck(dht_mtx);
std::lock_guard lck(dht_mtx);
if (dht_)
dht_->setLogFilter(f);
}
@@ -496,22 +540,28 @@ DhtRunner::setLogFilter(const InfoHash& f)
void
DhtRunner::registerType(const ValueType& type)
{
std::lock_guard<std::mutex> lck(dht_mtx);
std::lock_guard lck(dht_mtx);
dht_->registerType(type);
}
void
DhtRunner::registerInsecureType(const ValueType& type)
{
std::lock_guard<std::mutex> lck(dht_mtx);
std::lock_guard lck(dht_mtx);
dht_->registerInsecureType(type);
}
void
DhtRunner::importValues(const std::vector<ValuesExport>& values)
{
std::lock_guard<std::mutex> lck(dht_mtx);
dht_->importValues(values);
{
std::lock_guard lck(dht_mtx);
dht_->importValues(values);
}
// Recompute the runner wakeup after import schedules value expirations.
std::lock_guard lck(storage_mtx);
pending_ops_prio.emplace([](SecureDht&) {});
cv.notify_all();
}
unsigned
@@ -521,7 +571,7 @@ DhtRunner::getNodesStats(sa_family_t af,
unsigned* cached_return,
unsigned* incoming_return) const
{
std::lock_guard<std::mutex> lck(dht_mtx);
std::lock_guard lck(dht_mtx);
const auto stats = dht_->getNodesStats(af);
if (good_return)
*good_return = stats.good_nodes;
@@ -537,25 +587,17 @@ DhtRunner::getNodesStats(sa_family_t af,
NodeStats
DhtRunner::getNodesStats(sa_family_t af) const
{
std::lock_guard<std::mutex> lck(dht_mtx);
std::lock_guard lck(dht_mtx);
return dht_->getNodesStats(af);
}
NodeInfo
DhtRunner::getNodeInfo() const
{
std::lock_guard<std::mutex> lck(dht_mtx);
NodeInfo info {};
if (dht_) {
info.id = dht_->getId();
info.node_id = dht_->getNodeId();
info.ipv4 = dht_->getNodesStats(AF_INET);
info.ipv6 = dht_->getNodesStats(AF_INET6);
if (auto sock = dht_->getSocket()) {
info.bound4 = sock->getBoundRef(AF_INET).getPort();
info.bound6 = sock->getBoundRef(AF_INET6).getPort();
}
}
std::lock_guard lck(dht_mtx);
if (dht_)
info = dht_->getNodeInfo();
info.ongoing_ops = ongoing_ops;
return info;
}
@@ -563,21 +605,12 @@ DhtRunner::getNodeInfo() const
void
DhtRunner::getNodeInfo(std::function<void(std::shared_ptr<NodeInfo>)> cb)
{
std::lock_guard<std::mutex> lck(storage_mtx);
std::lock_guard lck(storage_mtx);
ongoing_ops++;
pending_ops_prio.emplace([cb = std::move(cb), this](SecureDht& dht) {
auto sinfo = std::make_shared<NodeInfo>();
auto& info = *sinfo;
info.id = dht.getId();
info.node_id = dht.getNodeId();
info.ipv4 = dht.getNodesStats(AF_INET);
info.ipv6 = dht.getNodesStats(AF_INET6);
std::tie(info.storage_size, info.storage_values) = dht.getStoreSize();
if (auto sock = dht.getSocket()) {
info.bound4 = sock->getBoundRef(AF_INET).getPort();
info.bound6 = sock->getBoundRef(AF_INET6).getPort();
}
info.ongoing_ops = ongoing_ops;
*sinfo = dht.getNodeInfo();
sinfo->ongoing_ops = ongoing_ops;
cb(std::move(sinfo));
opEnded();
});
@@ -587,44 +620,44 @@ DhtRunner::getNodeInfo(std::function<void(std::shared_ptr<NodeInfo>)> cb)
std::vector<unsigned>
DhtRunner::getNodeMessageStats(bool in) const
{
std::lock_guard<std::mutex> lck(dht_mtx);
std::lock_guard lck(dht_mtx);
return dht_->getNodeMessageStats(in);
}
std::string
DhtRunner::getStorageLog() const
{
std::lock_guard<std::mutex> lck(dht_mtx);
std::lock_guard lck(dht_mtx);
return dht_->getStorageLog();
}
std::string
DhtRunner::getStorageLog(const InfoHash& f) const
{
std::lock_guard<std::mutex> lck(dht_mtx);
std::lock_guard lck(dht_mtx);
return dht_->getStorageLog(f);
}
std::string
DhtRunner::getRoutingTablesLog(sa_family_t af) const
{
std::lock_guard<std::mutex> lck(dht_mtx);
std::lock_guard lck(dht_mtx);
return dht_->getRoutingTablesLog(af);
}
std::string
DhtRunner::getSearchesLog(sa_family_t af) const
{
std::lock_guard<std::mutex> lck(dht_mtx);
std::lock_guard lck(dht_mtx);
return dht_->getSearchesLog(af);
}
std::string
DhtRunner::getSearchLog(const InfoHash& f, sa_family_t af) const
{
std::lock_guard<std::mutex> lck(dht_mtx);
std::lock_guard lck(dht_mtx);
return dht_->getSearchLog(f, af);
}
std::vector<SockAddr>
DhtRunner::getPublicAddress(sa_family_t af) const
{
std::lock_guard<std::mutex> lck(dht_mtx);
std::lock_guard lck(dht_mtx);
if (dht_)
return dht_->getPublicAddress(af);
return {};
@@ -641,7 +674,7 @@ DhtRunner::getPublicAddressStr(sa_family_t af) const
void
DhtRunner::getPublicAddress(std::function<void(std::vector<SockAddr>&&)> cb, sa_family_t af)
{
std::lock_guard<std::mutex> lck(storage_mtx);
std::lock_guard lck(storage_mtx);
ongoing_ops++;
pending_ops_prio.emplace([cb = std::move(cb), this, af](SecureDht& dht) {
cb(dht.getPublicAddress(af));
@@ -653,22 +686,14 @@ DhtRunner::getPublicAddress(std::function<void(std::vector<SockAddr>&&)> cb, sa_
void
DhtRunner::registerCertificate(const std::shared_ptr<crypto::Certificate>& cert)
{
std::lock_guard<std::mutex> lck(dht_mtx);
std::lock_guard lck(dht_mtx);
dht_->registerCertificate(cert);
}
void
DhtRunner::setLocalCertificateStore(CertificateStoreQueryLegacy&& query_method)
{
std::lock_guard<std::mutex> lck(dht_mtx);
if (dht_)
dht_->setLocalCertificateStore(std::forward<CertificateStoreQueryLegacy>(query_method));
}
void
DhtRunner::setLocalCertificateStore(CertificateStoreQuery&& query_method)
{
std::lock_guard<std::mutex> lck(dht_mtx);
std::lock_guard lck(dht_mtx);
if (dht_)
dht_->setLocalCertificateStore(std::forward<CertificateStoreQuery>(query_method));
}
@@ -681,7 +706,7 @@ DhtRunner::loop_()
decltype(pending_ops) ops {};
{
std::lock_guard<std::mutex> lck(storage_mtx);
std::lock_guard lck(storage_mtx);
auto s = getStatus();
ops = (pending_ops_prio.empty()
&& (s == NodeStatus::Connected or s == NodeStatus::Disconnected or running == State::Stopping))
@@ -697,7 +722,7 @@ DhtRunner::loop_()
decltype(rcv) received {};
decltype(rcv) received_treated {};
{
std::lock_guard<std::mutex> lck(sock_mtx);
std::lock_guard lck(sock_mtx);
// move to stack
received = std::move(rcv);
}
@@ -732,7 +757,7 @@ DhtRunner::loop_()
}
if (not received_treated.empty()) {
std::lock_guard<std::mutex> lck(sock_mtx);
std::lock_guard lck(sock_mtx);
if (rcv_free.size() < net::RX_QUEUE_MAX_SIZE)
rcv_free.splice(rcv_free.end(), std::move(received_treated));
}
@@ -756,7 +781,7 @@ DhtRunner::loop_()
void
DhtRunner::get(InfoHash hash, GetCallback vcb, DoneCallback dcb, Value::Filter f, Where w)
{
std::unique_lock<std::mutex> lck(storage_mtx);
std::unique_lock lck(storage_mtx);
if (running != State::Running) {
lck.unlock();
if (dcb)
@@ -778,7 +803,7 @@ DhtRunner::get(const std::string& key, GetCallback vcb, DoneCallbackSimple dcb,
void
DhtRunner::query(const InfoHash& hash, QueryCallback cb, DoneCallback done_cb, Query q)
{
std::unique_lock<std::mutex> lck(storage_mtx);
std::unique_lock lck(storage_mtx);
if (running != State::Running) {
lck.unlock();
if (done_cb)
@@ -796,7 +821,7 @@ std::future<size_t>
DhtRunner::listen(InfoHash hash, ValueCallback vcb, Value::Filter f, Where w)
{
auto ret_token = std::make_shared<std::promise<size_t>>();
std::unique_lock<std::mutex> lck(storage_mtx);
std::unique_lock lck(storage_mtx);
if (running != State::Running) {
lck.unlock();
ret_token->set_value(0);
@@ -818,7 +843,7 @@ DhtRunner::listen(const std::string& key, GetCallback vcb, Value::Filter f, Wher
void
DhtRunner::cancelListen(InfoHash h, size_t token)
{
std::lock_guard<std::mutex> lck(storage_mtx);
std::lock_guard lck(storage_mtx);
if (running != State::Running)
return;
ongoing_ops++;
@@ -832,7 +857,7 @@ DhtRunner::cancelListen(InfoHash h, size_t token)
void
DhtRunner::cancelListen(InfoHash h, std::shared_future<size_t> ftoken)
{
std::lock_guard<std::mutex> lck(storage_mtx);
std::lock_guard lck(storage_mtx);
if (running != State::Running)
return;
ongoing_ops++;
@@ -846,7 +871,7 @@ DhtRunner::cancelListen(InfoHash h, std::shared_future<size_t> ftoken)
void
DhtRunner::put(InfoHash hash, Value&& value, DoneCallback cb, time_point created, bool permanent)
{
std::unique_lock<std::mutex> lck(storage_mtx);
std::unique_lock lck(storage_mtx);
if (running != State::Running) {
lck.unlock();
if (cb)
@@ -863,7 +888,7 @@ DhtRunner::put(InfoHash hash, Value&& value, DoneCallback cb, time_point created
void
DhtRunner::put(InfoHash hash, std::shared_ptr<Value> value, DoneCallback cb, time_point created, bool permanent)
{
std::unique_lock<std::mutex> lck(storage_mtx);
std::unique_lock lck(storage_mtx);
if (running != State::Running) {
lck.unlock();
if (cb)
@@ -886,7 +911,7 @@ DhtRunner::put(const std::string& key, Value&& value, DoneCallbackSimple cb, tim
void
DhtRunner::cancelPut(const InfoHash& h, Value::Id id)
{
std::lock_guard<std::mutex> lck(storage_mtx);
std::lock_guard lck(storage_mtx);
pending_ops.emplace([=](SecureDht& dht) { dht.cancelPut(h, id); });
cv.notify_all();
}
@@ -894,7 +919,7 @@ DhtRunner::cancelPut(const InfoHash& h, Value::Id id)
void
DhtRunner::cancelPut(const InfoHash& h, const std::shared_ptr<Value>& value)
{
std::lock_guard<std::mutex> lck(storage_mtx);
std::lock_guard lck(storage_mtx);
pending_ops.emplace([=](SecureDht& dht) { dht.cancelPut(h, value->id); });
cv.notify_all();
}
@@ -902,7 +927,7 @@ DhtRunner::cancelPut(const InfoHash& h, const std::shared_ptr<Value>& value)
void
DhtRunner::putSigned(InfoHash hash, std::shared_ptr<Value> value, DoneCallback cb, bool permanent)
{
std::unique_lock<std::mutex> lck(storage_mtx);
std::unique_lock lck(storage_mtx);
if (running != State::Running) {
lck.unlock();
if (cb)
@@ -931,7 +956,7 @@ DhtRunner::putSigned(const std::string& key, Value&& value, DoneCallbackSimple c
void
DhtRunner::putEncrypted(InfoHash hash, const PkId& to, std::shared_ptr<Value> value, DoneCallback cb, bool permanent)
{
std::unique_lock<std::mutex> lck(storage_mtx);
std::unique_lock lck(storage_mtx);
if (running != State::Running) {
lck.unlock();
if (cb)
@@ -945,35 +970,6 @@ DhtRunner::putEncrypted(InfoHash hash, const PkId& to, std::shared_ptr<Value> va
cv.notify_all();
}
void
DhtRunner::putEncrypted(InfoHash hash, InfoHash to, std::shared_ptr<Value> value, DoneCallback cb, bool permanent)
{
std::unique_lock<std::mutex> lck(storage_mtx);
if (running != State::Running) {
lck.unlock();
if (cb)
cb(false, {});
return;
}
ongoing_ops++;
pending_ops.emplace([=, cb = std::move(cb), value = std::move(value)](SecureDht& dht) mutable {
dht.putEncrypted(hash, to, value, bindOpDoneCallback(std::move(cb)), permanent);
});
cv.notify_all();
}
void
DhtRunner::putEncrypted(InfoHash hash, InfoHash to, Value&& value, DoneCallback cb, bool permanent)
{
putEncrypted(hash, to, std::make_shared<Value>(std::move(value)), std::move(cb), permanent);
}
void
DhtRunner::putEncrypted(const std::string& key, InfoHash to, Value&& value, DoneCallback cb, bool permanent)
{
putEncrypted(InfoHash::get(key), to, std::forward<Value>(value), std::move(cb), permanent);
}
void
DhtRunner::putEncrypted(InfoHash hash,
const std::shared_ptr<crypto::PublicKey>& to,
@@ -981,7 +977,7 @@ DhtRunner::putEncrypted(InfoHash hash,
DoneCallback cb,
bool permanent)
{
std::unique_lock<std::mutex> lck(storage_mtx);
std::unique_lock lck(storage_mtx);
if (running != State::Running) {
lck.unlock();
if (cb)
@@ -1005,7 +1001,7 @@ DhtRunner::putEncrypted(
void
DhtRunner::bootstrap(const std::string& host, const std::string& service)
{
std::lock_guard<std::mutex> lck(storage_mtx);
std::lock_guard lck(storage_mtx);
pending_ops_prio.emplace([host, service](SecureDht& dht) mutable { dht.addBootstrap(host, service); });
cv.notify_all();
}
@@ -1013,7 +1009,7 @@ DhtRunner::bootstrap(const std::string& host, const std::string& service)
void
DhtRunner::bootstrap(const std::string& hostService)
{
std::lock_guard<std::mutex> lck(storage_mtx);
std::lock_guard lck(storage_mtx);
auto [h, s] = splitPort(hostService);
pending_ops_prio.emplace(
[host = std::string(h), service = std::string(s)](SecureDht& dht) { dht.addBootstrap(host, service); });
@@ -1023,7 +1019,7 @@ DhtRunner::bootstrap(const std::string& hostService)
void
DhtRunner::clearBootstrap()
{
std::lock_guard<std::mutex> lck(storage_mtx);
std::lock_guard lck(storage_mtx);
pending_ops_prio.emplace([](SecureDht& dht) mutable { dht.clearBootstrap(); });
cv.notify_all();
}
@@ -1035,7 +1031,7 @@ DhtRunner::bootstrap(std::vector<SockAddr> nodes, DoneCallbackSimple cb)
cb(false);
return;
}
std::lock_guard<std::mutex> lck(storage_mtx);
std::lock_guard lck(storage_mtx);
ongoing_ops++;
pending_ops_prio.emplace([cb = bindOpDoneCallback(std::move(cb)), nodes = std::move(nodes)](SecureDht& dht) mutable {
auto rem = cb ? std::make_shared<std::pair<size_t, bool>>(nodes.size(), false) : nullptr;
@@ -1058,7 +1054,7 @@ DhtRunner::bootstrap(std::vector<SockAddr> nodes, DoneCallbackSimple cb)
void
DhtRunner::bootstrap(SockAddr addr, DoneCallbackSimple cb)
{
std::unique_lock<std::mutex> lck(storage_mtx);
std::unique_lock lck(storage_mtx);
if (running != State::Running) {
lck.unlock();
if (cb)
@@ -1075,7 +1071,7 @@ DhtRunner::bootstrap(SockAddr addr, DoneCallbackSimple cb)
void
DhtRunner::bootstrap(const InfoHash& id, const SockAddr& address)
{
std::lock_guard<std::mutex> lck(storage_mtx);
std::lock_guard lck(storage_mtx);
if (running != State::Running)
return;
pending_ops_prio.emplace([id, address](SecureDht& dht) mutable { dht.insertNode(id, address); });
@@ -1085,7 +1081,7 @@ DhtRunner::bootstrap(const InfoHash& id, const SockAddr& address)
void
DhtRunner::bootstrap(std::vector<NodeExport> nodes)
{
std::lock_guard<std::mutex> lck(storage_mtx);
std::lock_guard lck(storage_mtx);
if (running != State::Running)
return;
pending_ops_prio.emplace([nodes = std::move(nodes)](SecureDht& dht) {
@@ -1098,7 +1094,7 @@ DhtRunner::bootstrap(std::vector<NodeExport> nodes)
void
DhtRunner::connectivityChanged()
{
std::lock_guard<std::mutex> lck(storage_mtx);
std::lock_guard lck(storage_mtx);
pending_ops_prio.emplace([=](SecureDht& dht) {
dht.connectivityChanged();
#ifdef OPENDHT_PEER_DISCOVERY
@@ -1109,29 +1105,10 @@ DhtRunner::connectivityChanged()
cv.notify_all();
}
void
DhtRunner::findCertificate(InfoHash hash, std::function<void(const Sp<crypto::Certificate>&)> cb)
{
std::unique_lock<std::mutex> lck(storage_mtx);
if (running != State::Running) {
lck.unlock();
cb({});
return;
}
ongoing_ops++;
pending_ops.emplace([this, hash, cb = std::move(cb)](SecureDht& dht) {
dht.findCertificate(hash, [this, cb = std::move(cb)](const Sp<crypto::Certificate>& crt) {
cb(crt);
opEnded();
});
});
cv.notify_all();
}
void
DhtRunner::findCertificate(PkId hash, std::function<void(const std::shared_ptr<crypto::Certificate>&)> cb)
{
std::unique_lock<std::mutex> lck(storage_mtx);
std::unique_lock lck(storage_mtx);
if (running != State::Running) {
lck.unlock();
return;
@@ -1157,7 +1134,7 @@ void
DhtRunner::setProxyServer(const std::string& proxy, const std::string& pushNodeId)
{
#ifdef OPENDHT_PROXY_CLIENT
std::lock_guard<std::mutex> lck(dht_mtx);
std::lock_guard lck(dht_mtx);
if (config_.proxy_server == proxy and config_.push_node_id == pushNodeId)
return;
config_.proxy_server = proxy;
@@ -1184,8 +1161,8 @@ DhtRunner::enableProxy(bool proxify)
config_.client_identity,
[this] {
if (config_.threaded) {
std::lock_guard<std::mutex> lck(storage_mtx);
pending_ops_prio.emplace([=](SecureDht&) mutable {});
std::lock_guard lck(storage_mtx);
pending_ops_prio.emplace([](SecureDht&) {});
cv.notify_all();
}
},
@@ -1197,11 +1174,8 @@ DhtRunner::enableProxy(bool proxify)
config_.push_platform,
logger_);
dht_ = std::make_unique<SecureDht>(std::move(dht_via_proxy), config_.dht_config, identityAnnouncedCb_, logger_);
// and use it
use_proxy = proxify;
} else {
use_proxy = proxify;
}
use_proxy = proxify;
#else
if (proxify)
throw std::runtime_error("DHT proxy requested but OpenDHT built without proxy support.");
@@ -1211,7 +1185,7 @@ DhtRunner::enableProxy(bool proxify)
void
DhtRunner::forwardAllMessages(bool forward)
{
std::lock_guard<std::mutex> lck(dht_mtx);
std::lock_guard lck(dht_mtx);
if (dht_)
dht_->forwardAllMessages(forward);
}
@@ -1222,7 +1196,7 @@ DhtRunner::forwardAllMessages(bool forward)
void
DhtRunner::setPushNotificationToken(const std::string& token)
{
std::lock_guard<std::mutex> lck(dht_mtx);
std::lock_guard lck(dht_mtx);
#if defined(OPENDHT_PROXY_CLIENT) && defined(OPENDHT_PUSH_NOTIFICATIONS)
config_.push_token = token;
if (dht_)
@@ -1235,7 +1209,7 @@ DhtRunner::setPushNotificationToken(const std::string& token)
void
DhtRunner::setPushNotificationTopic(const std::string& topic)
{
std::lock_guard<std::mutex> lck(dht_mtx);
std::lock_guard lck(dht_mtx);
#if defined(OPENDHT_PROXY_CLIENT) && defined(OPENDHT_PUSH_NOTIFICATIONS)
config_.push_topic = topic;
if (dht_)
@@ -1248,7 +1222,7 @@ DhtRunner::setPushNotificationTopic(const std::string& topic)
void
DhtRunner::setPushNotificationPlatform(const std::string& platform)
{
std::lock_guard<std::mutex> lck(dht_mtx);
std::lock_guard lck(dht_mtx);
#if defined(OPENDHT_PROXY_CLIENT) && defined(OPENDHT_PUSH_NOTIFICATIONS)
config_.push_platform = platform;
if (dht_)
@@ -1264,7 +1238,7 @@ DhtRunner::pushNotificationReceived([[maybe_unused]] const std::map<std::string,
#if defined(OPENDHT_PROXY_CLIENT) && defined(OPENDHT_PUSH_NOTIFICATIONS)
auto ret_token = std::make_shared<std::promise<PushNotificationResult>>();
auto future = ret_token->get_future();
std::lock_guard<std::mutex> lck(storage_mtx);
std::lock_guard lck(storage_mtx);
pending_ops_prio.emplace([ret_token, this, data](SecureDht&) {
if (dht_)
ret_token->set_value(dht_->pushNotificationReceived(data));
+59 -47
View File
@@ -42,8 +42,8 @@ constexpr std::string_view ORIGIN_PROTOCOL {"//"sv};
constexpr const char HTTP_HEADER_DELIM[] = "\r\n\r\n";
constexpr unsigned MAX_REDIRECTS {5};
Url::Url(std::string_view url_str)
: url(url_str)
Url::Url(std::string url_str)
: url(std::move(url_str))
{
std::string_view u = url;
if (u.empty())
@@ -239,7 +239,7 @@ Connection::~Connection()
void
Connection::close()
{
std::lock_guard<std::mutex> lock(mutex_);
std::lock_guard lock(mutex_);
asio::error_code ec;
if (ssl_socket_) {
if (ssl_socket_->is_open())
@@ -302,13 +302,13 @@ static X509*
issuer_from_chain(STACK_OF(X509) * fullchain)
{
X509 *cert, *issuer;
X509_NAME* issuer_name;
const X509_NAME* issuer_name;
cert = cert_from_chain(fullchain);
if ((issuer_name = X509_get_issuer_name(cert)) == nullptr)
return nullptr;
issuer = X509_find_by_subject(fullchain, issuer_name);
issuer = X509_find_by_subject(fullchain, const_cast<X509_NAME*>(issuer_name));
return issuer;
}
@@ -490,7 +490,7 @@ ocspValidateResponse(const OscpRequestInfo& info,
void
Connection::set_ssl_verification(const std::string& hostname, const asio::ssl::verify_mode verify_mode)
{
std::lock_guard<std::mutex> lock(mutex_);
std::lock_guard lock(mutex_);
if (ssl_socket_) {
// Set SNI Hostname (many hosts need this to handshake successfully)
SSL_set_tlsext_host_name(ssl_socket_->asio_ssl_stream().native_handle(), hostname.c_str());
@@ -499,9 +499,9 @@ Connection::set_ssl_verification(const std::string& hostname, const asio::ssl::v
ssl_socket_->asio_ssl_stream().set_verify_callback(
[id = id_, logger = logger_, hostname, checkOcsp = checkOcsp_](bool preverified,
asio::ssl::verify_context& ctx) -> bool {
X509* cert = X509_STORE_CTX_get_current_cert(ctx.native_handle());
if (logger) {
char subject_name[1024];
X509* cert = X509_STORE_CTX_get_current_cert(ctx.native_handle());
X509_NAME_oneline(X509_get_subject_name(cert), subject_name, 1024);
logger->debug("[connection:{:d}] verify {:s} compliance to RFC 2818:\n{:s}",
id,
@@ -513,8 +513,13 @@ Connection::set_ssl_verification(const std::string& hostname, const asio::ssl::v
auto verifier = asio::ssl::host_name_verification(hostname);
bool verified = verifier(preverified, ctx);
auto verify_ec = X509_STORE_CTX_get_error(ctx.native_handle());
if (verify_ec != 0 /*X509_V_OK*/ and logger)
logger->error("[http::connection:{:d}] ssl verification error={:d} {}", id, verify_ec, verified);
if (verify_ec != 0 /*X509_V_OK*/ and logger) {
logger->error("[http::connection:{:d}] ssl verification error {:d}: {}. verified: {}",
id,
verify_ec,
X509_verify_cert_error_string(verify_ec),
verified);
}
if (verified and checkOcsp) {
std::unique_ptr<stack_st_X509, void (*)(stack_st_X509*)> chain(X509_STORE_CTX_get1_chain(
ctx.native_handle()),
@@ -571,7 +576,7 @@ Connection::input()
std::string
Connection::read_bytes(size_t bytes)
{
std::lock_guard<std::mutex> lock(mutex_);
std::lock_guard lock(mutex_);
if (bytes == 0)
bytes = read_buf_.in_avail();
std::string content;
@@ -592,7 +597,7 @@ Connection::read_until(const char delim)
void
Connection::async_connect(std::vector<asio::ip::tcp::endpoint>&& endpoints, ConnectHandlerCb cb)
{
std::lock_guard<std::mutex> lock(mutex_);
std::lock_guard lock(mutex_);
if (!ssl_socket_ && !socket_) {
cb(asio::error::operation_aborted, {});
return;
@@ -624,7 +629,7 @@ Connection::async_connect(std::vector<asio::ip::tcp::endpoint>&& endpoints, Conn
void
Connection::async_handshake(HandlerCb cb)
{
std::lock_guard<std::mutex> lock(mutex_);
std::lock_guard lock(mutex_);
if (ssl_socket_) {
std::weak_ptr<Connection> wthis = shared_from_this();
ssl_socket_
@@ -658,7 +663,7 @@ Connection::async_handshake(HandlerCb cb)
void
Connection::async_write(BytesHandlerCb cb)
{
std::lock_guard<std::mutex> lock(mutex_);
std::lock_guard lock(mutex_);
if (!is_open()) {
if (cb)
asio::post(ctx_, [cb]() { cb(asio::error::broken_pipe, 0); });
@@ -675,7 +680,7 @@ Connection::async_write(BytesHandlerCb cb)
void
Connection::async_read_until(const char* delim, BytesHandlerCb cb)
{
std::lock_guard<std::mutex> lock(mutex_);
std::lock_guard lock(mutex_);
if (!is_open()) {
if (cb)
asio::post(ctx_, [cb]() { cb(asio::error::broken_pipe, 0); });
@@ -692,7 +697,7 @@ Connection::async_read_until(const char* delim, BytesHandlerCb cb)
void
Connection::async_read_until(char delim, BytesHandlerCb cb)
{
std::lock_guard<std::mutex> lock(mutex_);
std::lock_guard lock(mutex_);
if (!is_open()) {
if (cb)
asio::post(ctx_, [cb]() { cb(asio::error::broken_pipe, 0); });
@@ -709,7 +714,7 @@ Connection::async_read_until(char delim, BytesHandlerCb cb)
void
Connection::async_read(size_t bytes, BytesHandlerCb cb)
{
std::lock_guard<std::mutex> lock(mutex_);
std::lock_guard lock(mutex_);
if (!is_open()) {
if (cb)
asio::post(ctx_, [cb]() { cb(asio::error::broken_pipe, 0); });
@@ -726,7 +731,7 @@ Connection::async_read(size_t bytes, BytesHandlerCb cb)
void
Connection::async_read_some(size_t bytes, BytesHandlerCb cb)
{
std::lock_guard<std::mutex> lock(mutex_);
std::lock_guard lock(mutex_);
if (!is_open()) {
if (cb)
asio::post(ctx_, [cb]() { cb(asio::error::broken_pipe, 0); });
@@ -746,7 +751,7 @@ Connection::async_read_some(size_t bytes, BytesHandlerCb cb)
void
Connection::set_keepalive(uint32_t seconds)
{
std::lock_guard<std::mutex> lock(mutex_);
std::lock_guard lock(mutex_);
if (!ssl_socket_ && !socket_)
return;
@@ -825,8 +830,8 @@ Connection::timeout(const std::chrono::seconds& timeout, HandlerCb cb)
// Resolver
Resolver::Resolver(asio::io_context& ctx, const std::string& url, std::shared_ptr<dht::Logger> logger)
: url_(url)
Resolver::Resolver(asio::io_context& ctx, std::string url, std::shared_ptr<dht::Logger> logger)
: url_(std::move(url))
, resolver_(ctx)
, destroyed_(std::make_shared<bool>(false))
, logger_(logger)
@@ -866,7 +871,7 @@ Resolver::~Resolver()
{
decltype(cbs_) cbs;
{
std::lock_guard<std::mutex> lock(mutex_);
std::lock_guard lock(mutex_);
cbs = std::move(cbs_);
}
while (not cbs.empty()) {
@@ -902,7 +907,7 @@ filter(const std::vector<asio::ip::tcp::endpoint>& epts, sa_family_t family)
void
Resolver::add_callback(ResolverCb cb, sa_family_t family)
{
std::lock_guard<std::mutex> lock(mutex_);
std::lock_guard lock(mutex_);
if (!completed_)
cbs_.emplace(family == AF_UNSPEC ? std::move(cb)
: [cb, family](const asio::error_code& ec,
@@ -912,8 +917,16 @@ Resolver::add_callback(ResolverCb cb, sa_family_t family)
else
cb(ec, filter(endpoints, family));
});
else
cb(ec_, family == AF_UNSPEC ? endpoints_ : filter(endpoints_, family));
else {
// Always dispatch the callback on the io_context thread. Invoking it
// synchronously would run the caller's continuation (e.g. the whole
// connect chain of Request::send) on the calling thread, mutating asio
// objects concurrently with the io thread running the io_context.
asio::post(resolver_.get_executor(),
[cb = std::move(cb),
ec = ec_,
endpoints = family == AF_UNSPEC ? endpoints_ : filter(endpoints_, family)] { cb(ec, endpoints); });
}
}
void
@@ -941,7 +954,7 @@ Resolver::resolve(std::string_view host, std::string_view service)
}
decltype(cbs_) cbs;
{
std::lock_guard<std::mutex> lock(mutex_);
std::lock_guard lock(mutex_);
ec_ = ec;
endpoints_ = std::vector<asio::ip::tcp::endpoint> {endpoints.begin(),
endpoints.end()};
@@ -962,14 +975,14 @@ Resolver::resolve(std::string_view host, std::string_view service)
std::atomic_uint Request::ids_ {1};
Request::Request(asio::io_context& ctx,
const std::string& url,
std::string url,
const Json::Value& json,
OnJsonCb jsoncb,
std::shared_ptr<dht::Logger> logger)
: logger_(std::move(logger))
, id_(Request::ids_++)
, ctx_(ctx)
, resolver_(std::make_shared<Resolver>(ctx, url, logger))
, resolver_(std::make_shared<Resolver>(ctx, std::move(url), logger))
{
init_default_headers();
set_header_field(restinio::http_field_t::content_type, std::string(HTTP_HEADER_CONTENT_TYPE_JSON));
@@ -992,11 +1005,11 @@ Request::Request(asio::io_context& ctx,
});
}
Request::Request(asio::io_context& ctx, const std::string& url, OnJsonCb jsoncb, std::shared_ptr<dht::Logger> logger)
Request::Request(asio::io_context& ctx, std::string url, OnJsonCb jsoncb, std::shared_ptr<dht::Logger> logger)
: logger_(std::move(logger))
, id_(Request::ids_++)
, ctx_(ctx)
, resolver_(std::make_shared<Resolver>(ctx, url, logger))
, resolver_(std::make_shared<Resolver>(ctx, std::move(url), logger))
{
init_default_headers();
set_header_field(restinio::http_field_t::accept, std::string(HTTP_HEADER_CONTENT_TYPE_JSON));
@@ -1017,20 +1030,20 @@ Request::Request(asio::io_context& ctx, const std::string& url, OnJsonCb jsoncb,
});
}
Request::Request(asio::io_context& ctx, const std::string& url, std::shared_ptr<dht::Logger> logger)
Request::Request(asio::io_context& ctx, std::string url, std::shared_ptr<dht::Logger> logger)
: logger_(logger)
, id_(Request::ids_++)
, ctx_(ctx)
, resolver_(std::make_shared<Resolver>(ctx, url, logger))
, resolver_(std::make_shared<Resolver>(ctx, std::move(url), logger))
{
init_default_headers();
}
Request::Request(asio::io_context& ctx, const std::string& url, OnDoneCb onDone, std::shared_ptr<dht::Logger> logger)
Request::Request(asio::io_context& ctx, std::string url, OnDoneCb onDone, std::shared_ptr<dht::Logger> logger)
: logger_(logger)
, id_(Request::ids_++)
, ctx_(ctx)
, resolver_(std::make_shared<Resolver>(ctx, url, logger))
, resolver_(std::make_shared<Resolver>(ctx, std::move(url), logger))
{
init_default_headers();
add_on_done_callback(std::move(onDone));
@@ -1081,7 +1094,6 @@ Request::Request(asio::io_context& ctx,
, family_(family)
, resolver_(resolver)
{
set_header_field(restinio::http_field_t::host, concat(get_url().host, ":", get_url().service));
set_target(Url(target).target);
}
@@ -1385,7 +1397,7 @@ Request::connect(std::vector<asio::ip::tcp::endpoint>&& endpoints, HandlerCb cb)
if (not sthis)
return;
auto& this_ = *sthis;
std::lock_guard<std::mutex> lock(this_.mutex_);
std::lock_guard lock(this_.mutex_);
if (ec == asio::error::operation_aborted) {
this_.terminate(ec);
return;
@@ -1438,7 +1450,7 @@ Request::send()
[wthis](const asio::error_code& ec, std::vector<asio::ip::tcp::endpoint> endpoints) {
if (auto sthis = wthis.lock()) {
auto& this_ = *sthis;
std::lock_guard<std::mutex> lock(this_.mutex_);
std::lock_guard lock(this_.mutex_);
if (ec) {
if (this_.logger_)
this_.logger_->error("[http:request:{:d}] resolve error: {:s}", this_.id_, ec.message());
@@ -1513,7 +1525,7 @@ Request::terminate(const asio::error_code& ec)
void
Request::handle_request(const asio::error_code& ec)
{
std::lock_guard<std::mutex> lock(mutex_);
std::lock_guard lock(mutex_);
if (ec and ec != asio::error::eof) {
terminate(ec);
return;
@@ -1537,7 +1549,7 @@ Request::handle_request(const asio::error_code& ec)
void
Request::handle_response(const asio::error_code& ec, size_t /* n_bytes */)
{
std::lock_guard<std::mutex> lock(mutex_);
std::lock_guard lock(mutex_);
if (ec && ec != asio::error::eof) {
terminate(ec);
return;
@@ -1546,10 +1558,10 @@ Request::handle_response(const asio::error_code& ec, size_t /* n_bytes */)
enum llhttp_errno ret = llhttp_execute(parser_.get(), request.c_str(), request.size());
if (ret != HPE_OK && ret != HPE_PAUSED) {
if (logger_)
logger_->e("Error parsing HTTP: %zu %s %d",
(int) ret,
llhttp_errno_name(ret),
llhttp_get_error_reason(parser_.get()));
logger_->error("Error parsing HTTP: {} {} {}",
(int) ret,
llhttp_errno_name(ret),
llhttp_get_error_reason(parser_.get()));
terminate(asio::error::basic_errors::broken_pipe);
return;
}
@@ -1589,14 +1601,14 @@ Request::onHeadersComplete()
auto location_it = response_.headers.find(restinio::field_to_string(restinio::http_field_t::location));
if (location_it == response_.headers.end()) {
if (logger_)
logger_->e("[http:client] [request:%i] got redirect without location", id_);
logger_->error("[http:client] [request:{}] got redirect without location", id_);
terminate(asio::error::connection_aborted);
}
if (follow_redirect and num_redirect < MAX_REDIRECTS) {
auto newUrl = getRelativePath(get_url(), location_it->second);
if (logger_)
logger_->w("[http:client] [request:%i] redirect to %s", id_, newUrl.c_str());
logger_->warn("[http:client] [request:{}] redirect to {}", id_, newUrl);
auto next = std::make_shared<Request>(ctx_, newUrl, logger_);
next->set_method(header_.method());
next->headers_ = std::move(headers_);
@@ -1608,7 +1620,7 @@ Request::onHeadersComplete()
next->send();
} else {
if (logger_)
logger_->e("[http:client] [request:%i] got redirect without location", id_);
logger_->error("[http:client] [request:{}] got redirect without location", id_);
terminate(asio::error::connection_aborted);
}
} else {
@@ -1655,11 +1667,11 @@ const Response&
Request::await()
{
std::mutex mtx;
std::unique_lock<std::mutex> lock(mtx);
std::unique_lock lock(mtx);
std::condition_variable cv;
bool ok {false};
add_on_done_callback([&](const Response&) {
std::lock_guard<std::mutex> lk(mtx);
std::lock_guard lk(mtx);
ok = true;
cv.notify_all();
});
+35 -14
View File
@@ -49,27 +49,36 @@ constexpr const Color::Modifier def(Color::FG_DEFAULT);
constexpr const Color::Modifier red(Color::FG_RED);
constexpr const Color::Modifier yellow(Color::FG_YELLOW);
using namespace std::chrono;
using log_precision = microseconds;
constexpr auto den = log_precision::period::den;
/**
* Print va_list to std::ostream (used for logging).
*/
void
printfLog(std::ostream& s, const std::string& message)
printfLog(std::ostream& s, source_loc loc, std::string_view prefix, const std::string& message)
{
using namespace std::chrono;
using log_precision = microseconds;
auto num = duration_cast<log_precision>(steady_clock::now().time_since_epoch()).count();
constexpr auto den = log_precision::period::den;
fmt::print(s, "[{:06d}.{:06d}] ", num / den, num % den);
if (!loc.file.empty())
fmt::print(s,
"[{:06d}.{:06d} {:>20}:{:<5} {:<24}] {}",
num / den,
num % den,
loc.file,
loc.line,
loc.function,
prefix);
else
fmt::print(s, "[{:06d}.{:06d}] {}", num / den, num % den, prefix);
s << message << std::endl;
}
void
printLog(std::ostream& s, fmt::string_view format, fmt::format_args args)
printLog(std::ostream& s, source_loc loc, std::string_view prefix, fmt::string_view format, fmt::format_args args)
{
using namespace std::chrono;
using log_precision = microseconds;
auto num = duration_cast<log_precision>(steady_clock::now().time_since_epoch()).count();
constexpr auto den = log_precision::period::den;
fmt::print(s, "[{:06d}.{:06d}] ", num / den, num % den);
fmt::print(s, "[{:06d}.{:06d}] [{:>16}:{:<5}] {}", num / den, num % den, loc.file, loc.line, prefix);
fmt::vprint(s, format, args);
s << std::endl;
}
@@ -77,12 +86,12 @@ printLog(std::ostream& s, fmt::string_view format, fmt::format_args args)
std::shared_ptr<Logger>
getStdLogger()
{
return std::make_shared<Logger>([](LogLevel level, std::string&& message) {
return std::make_shared<Logger>([](source_loc loc, LogLevel level, std::string_view prefix, std::string&& message) {
if (level == LogLevel::error)
std::cerr << red;
else if (level == LogLevel::warning)
std::cerr << yellow;
printfLog(std::cerr, message);
printfLog(std::cerr, loc, prefix, message);
std::cerr << def;
});
}
@@ -93,7 +102,9 @@ getFileLogger(const std::string& path)
auto logfile = std::make_shared<std::ofstream>();
logfile->open(path, std::ios::out);
return std::make_shared<Logger>(
[logfile](LogLevel /*level*/, std::string&& message) { printfLog(*logfile, message); });
[logfile](source_loc loc, LogLevel /*level*/, std::string_view prefix, std::string&& message) {
printfLog(*logfile, loc, prefix, message);
});
}
#ifndef _WIN32
@@ -129,7 +140,17 @@ getSyslogLogger(const char* name)
opened_logfile = logfile;
}
return std::make_shared<Logger>(
[logfile](LogLevel level, std::string&& message) { syslog(syslogLevel(level), "%s", message.c_str()); });
[logfile](source_loc loc, LogLevel level, std::string_view prefix, std::string&& message) {
syslog(syslogLevel(level),
"%.*s:%u %.*s%.*s",
(int) loc.file.size(),
loc.file.data(),
loc.line,
(int) prefix.size(),
prefix.data(),
(int) message.size(),
message.data());
});
#else
return getStdLogger();
#endif
+194 -124
View File
@@ -40,8 +40,44 @@ struct NetworkEngine::PartialMessage
time_point start;
time_point last_part;
std::unique_ptr<ParsedMessage> msg;
std::vector<std::unique_ptr<ParsedMessage>> early_parts;
size_t early_bytes {};
bool request_quota_consumed {};
};
bool
NetworkEngine::isExpectedReply(const ParsedMessage& msg, const SockAddr& from)
{
auto request = requests.find(msg.tid);
if (request != requests.end() && request->second->pending() && !request->second->node->id
&& request->second->node->getAddr() == from)
return true;
auto node = cache.getNode(msg.id, from.getFamily());
if (!node || node->getAddr() != from)
return false;
auto pending = node->getRequest(msg.tid);
return (pending && pending->pending()) || node->getSocket(msg.tid);
}
bool
NetworkEngine::tryCompleteMessage(PartialMessage& partial, const SockAddr& from)
{
try {
if (!partial.msg->complete())
return false;
process(std::move(partial.msg), from);
} catch (const std::exception& e) {
if (logger_)
logger_->warn("Error while processing partial message: {}", e.what());
} catch (...) {
if (logger_)
logger_->warn("Unknown error while processing partial message");
}
return true;
}
std::vector<Blob>
serializeValues(const std::vector<Sp<Value>>& st)
{
@@ -129,7 +165,7 @@ NetworkEngine::tellListener(const Sp<Node>& node,
}
} catch (const std::overflow_error& e) {
if (logger_)
logger_->e("Unable to send value: buffer not large enough.");
logger_->error("Unable to send value: buffer not large enough.");
}
}
@@ -162,7 +198,7 @@ NetworkEngine::tellListenerRefreshed(const Sp<Node>& n,
pk.pack(KEY_REQ_REFRESHED);
pk.pack(values);
if (logger_)
logger_->d(n->id, "[node %s] sending %zu refreshed values", n->toString().c_str(), values.size());
logger_->debug("[node {}] sending {} refreshed values", n->toString(), values.size());
}
pk.pack(KEY_Y);
@@ -233,7 +269,7 @@ NetworkEngine::tellListenerExpired(const Sp<Node>& n,
pk.pack(KEY_REQ_EXPIRED);
pk.pack(values);
if (logger_)
logger_->d(n->id, "[node %s] sending %zu expired values", n->toString().c_str(), values.size());
logger_->debug("[node {}] sending {} expired values", n->toString(), values.size());
}
pk.pack(KEY_Y);
@@ -298,6 +334,7 @@ NetworkEngine::clear()
request.second->node->setExpired();
}
requests.clear();
partial_messages.clear();
}
void
@@ -433,13 +470,13 @@ NetworkEngine::processMessage(const uint8_t* buf, size_t buflen, SockAddr f)
auto from = f.getMappedIPv4();
if (isMartian(from)) {
if (logger_)
logger_->w("Received packet from martian node %s", from.toString().c_str());
logger_->warn("Received packet from martian node {}", from.toString());
return;
}
if (isNodeBlacklisted(from)) {
if (logger_)
logger_->w("Received packet from blacklisted node %s", from.toString().c_str());
logger_->warn("Received packet from blacklisted node {}", from.toString());
return;
}
@@ -449,7 +486,7 @@ NetworkEngine::processMessage(const uint8_t* buf, size_t buflen, SockAddr f)
msg->msgpack_unpack(msg_res.get());
} catch (const std::exception& e) {
if (logger_)
logger_->w("Unable to parse message of size %lu: %s", buflen, e.what());
logger_->warn("Unable to parse message of size {}: {}", buflen, e.what());
// if (logger_)
// logger_->DBG.logPrintable(buf, buflen);
return;
@@ -457,7 +494,7 @@ NetworkEngine::processMessage(const uint8_t* buf, size_t buflen, SockAddr f)
if (msg->network != config.network) {
if (logger_)
logger_->d("Received message from other config.network %u", msg->network);
logger_->debug("Received message from other config.network {}", msg->network);
return;
}
@@ -465,73 +502,124 @@ NetworkEngine::processMessage(const uint8_t* buf, size_t buflen, SockAddr f)
// partial value data
if (msg->type == MessageType::ValueData) {
auto pmsg_it = partial_messages.find(msg->tid);
if (pmsg_it == partial_messages.end()) {
if (logIncoming_)
if (logger_)
logger_->d("Unable to find partial message");
if (msg->fragment_parts.empty()
|| std::any_of(msg->fragment_parts.begin(), msg->fragment_parts.end(), [](const auto& fragment) {
return fragment.second.second.empty();
})) {
rateLimit(from);
return;
}
if (!pmsg_it->second.from.equals(from)) {
if (logger_)
logger_->d("Received partial message data from unexpected IP address");
rateLimit(from);
PartialMessageKey key {from, msg->tid};
auto pmsg_it = partial_messages.find(key);
if (pmsg_it == partial_messages.end()) {
if (!rateLimit(from))
return;
auto inserted = partial_messages.emplace(key, PartialMessage {});
pmsg_it = inserted.first;
pmsg_it->second.from = from;
pmsg_it->second.start = now;
pmsg_it->second.last_part = now;
pmsg_it->second.request_quota_consumed = true;
scheduler.add(now + RX_MAX_PACKET_TIME, std::bind(&NetworkEngine::maintainRxBuffer, this, key));
}
auto& pmsg = pmsg_it->second;
if (!pmsg.msg) {
if (pmsg.early_parts.size() >= RX_MAX_EARLY_PARTS) {
partial_messages.erase(pmsg_it);
return;
}
size_t fragment_bytes {};
for (const auto& [index, fragment] : msg->fragment_parts) {
(void) index;
if (fragment.second.size() > RX_MAX_EARLY_BYTES - fragment_bytes) {
partial_messages.erase(pmsg_it);
return;
}
fragment_bytes += fragment.second.size();
}
if (fragment_bytes > RX_MAX_EARLY_BYTES - pmsg.early_bytes) {
partial_messages.erase(pmsg_it);
return;
}
pmsg.early_bytes += fragment_bytes;
pmsg.early_parts.emplace_back(std::move(msg));
pmsg.last_part = now;
scheduler.add(now + RX_TIMEOUT, std::bind(&NetworkEngine::maintainRxBuffer, this, key));
return;
}
// append data block
if (pmsg_it->second.msg->append(*msg)) {
pmsg_it->second.last_part = now;
// check data completion
if (pmsg_it->second.msg->complete()) {
try {
// process the full message
process(std::move(pmsg_it->second.msg), from);
partial_messages.erase(pmsg_it);
} catch (...) {
return;
}
if (pmsg.msg->append(*msg)) {
pmsg.last_part = now;
if (tryCompleteMessage(pmsg, from)) {
partial_messages.erase(pmsg_it);
} else
scheduler.add(now + RX_TIMEOUT, std::bind(&NetworkEngine::maintainRxBuffer, this, msg->tid));
scheduler.add(now + RX_TIMEOUT, std::bind(&NetworkEngine::maintainRxBuffer, this, key));
}
return;
}
if (msg->id == myid or not msg->id) {
if (logger_)
logger_->d("Received message from self");
logger_->debug("Received message from self");
return;
}
if (msg->type > MessageType::Reply) {
if (msg->value_parts.empty()) {
if (msg->type > MessageType::Reply) {
/* Rate limit requests. */
if (!rateLimit(from)) {
if (logger_)
logger_->warn("Dropping request due to rate limiting");
return;
}
}
try {
process(std::move(msg), from);
} catch (const std::exception& e) {
if (logger_)
logger_->warn("Error while processing message: {}", e.what());
}
return;
}
PartialMessageKey partial_key {from, msg->tid};
if (msg->type == MessageType::Reply && !isExpectedReply(*msg, from))
return;
const auto partial = partial_messages.find(partial_key);
const bool request_quota_consumed = partial != partial_messages.end() && !partial->second.msg
&& partial->second.request_quota_consumed;
if (msg->type > MessageType::Reply && !request_quota_consumed) {
/* Rate limit requests. */
if (!rateLimit(from)) {
if (logger_)
logger_->w("Dropping request due to rate limiting");
logger_->warn("Dropping request due to rate limiting");
return;
}
}
if (msg->value_parts.empty()) {
try {
process(std::move(msg), from);
} catch (...) {
return;
}
} else {
// starting partial message session
auto k = msg->tid;
auto& pmsg = partial_messages[k];
if (not pmsg.msg) {
// starting partial message session
auto inserted = partial_messages.emplace(partial_key, PartialMessage {});
auto& pmsg = inserted.first->second;
if (not pmsg.msg) {
if (inserted.second) {
pmsg.from = from;
pmsg.msg = std::move(msg);
pmsg.start = now;
pmsg.last_part = now;
scheduler.add(now + RX_MAX_PACKET_TIME, std::bind(&NetworkEngine::maintainRxBuffer, this, k));
scheduler.add(now + RX_TIMEOUT, std::bind(&NetworkEngine::maintainRxBuffer, this, k));
} else if (logger_)
logger_->e("Partial message with given TID %u already exists", k);
}
scheduler.add(now + RX_MAX_PACKET_TIME, std::bind(&NetworkEngine::maintainRxBuffer, this, partial_key));
}
pmsg.msg = std::move(msg);
pmsg.last_part = now;
for (const auto& part : pmsg.early_parts)
pmsg.msg->append(*part);
pmsg.early_parts.clear();
pmsg.early_bytes = 0;
if (tryCompleteMessage(pmsg, from)) {
partial_messages.erase(inserted.first);
} else {
scheduler.add(now + RX_TIMEOUT, std::bind(&NetworkEngine::maintainRxBuffer, this, partial_key));
}
} else if (logger_)
logger_->error("Partial message from {} with given TID {} already exists", from.toString(), msg->tid);
}
void
@@ -551,7 +639,7 @@ NetworkEngine::process(std::unique_ptr<ParsedMessage>&& msg, const SockAddr& fro
rsocket->on_receive(node, std::move(*msg));
} catch (const DhtProtocolException& e) {
if (logger_)
logger_->w("Unable to deserialize nodes %s", e.what());
logger_->warn("Unable to deserialize nodes {}", e.what());
}
} else if (msg->type == MessageType::Error or msg->type == MessageType::Reply) {
auto rsocket = node->getSocket(msg->tid);
@@ -569,10 +657,7 @@ NetworkEngine::process(std::unique_ptr<ParsedMessage>&& msg, const SockAddr& fro
if (not node->isClient())
onNewNode(node, 1);
if (logger_)
logger_->d(node->id,
"[node %s] Unable to find transaction with id %u",
node->toString().c_str(),
msg->tid);
logger_->debug("[node {}] Unable to find transaction with id {}", node->toString(), msg->tid);
return;
}
}
@@ -585,9 +670,7 @@ NetworkEngine::process(std::unique_ptr<ParsedMessage>&& msg, const SockAddr& fro
if (req and (req->cancelled() or req->expired() or req->completed())) {
if (logger_)
logger_->w(node->id,
"[node %s] response to expired, cancelled or completed request",
node->toString().c_str());
logger_->warn("[node {}] response to expired, cancelled or completed request", node->toString());
return;
}
@@ -604,11 +687,10 @@ NetworkEngine::process(std::unique_ptr<ParsedMessage>&& msg, const SockAddr& fro
} else {
if (logIncoming_)
if (logger_)
logger_->w(msg->id,
"[node %s %s] received unknown error message %u",
msg->id.toString().c_str(),
from.toString().c_str(),
msg->error_code);
logger_->warn("[node {} {}] received unknown error message {}",
msg->id.toString(),
from.toString(),
msg->error_code);
}
break;
}
@@ -625,7 +707,7 @@ NetworkEngine::process(std::unique_ptr<ParsedMessage>&& msg, const SockAddr& fro
r.setDone(std::move(*msg));
} catch (const DhtProtocolException& e) {
if (logger_)
logger_->w("Unable to deserialize nodes %s", e.what());
logger_->warn("Unable to deserialize nodes {}", e.what());
}
break;
} else { /* request socket data */
@@ -634,7 +716,7 @@ NetworkEngine::process(std::unique_ptr<ParsedMessage>&& msg, const SockAddr& fro
rsocket->on_receive(node, std::move(*msg));
} catch (const DhtProtocolException& e) {
if (logger_)
logger_->w("Unable to deserialize nodes %s", e.what());
logger_->warn("Unable to deserialize nodes {}", e.what());
}
}
break;
@@ -651,13 +733,13 @@ NetworkEngine::process(std::unique_ptr<ParsedMessage>&& msg, const SockAddr& fro
++in_stats.ping;
if (logIncoming_)
if (logger_)
logger_->d(node->id, "[node %s] sending pong", node->toString().c_str());
logger_->debug("[node {}] sending pong", node->toString());
onPing(node);
sendPong(from, msg->tid);
break;
case MessageType::FindNode: {
// if (logger_)
// logger_->d(msg->target, node->id, "[node %s] got 'find' request for %s (%d)",
// logger_->d("[node %s] got 'find' request for %s (%d)",
// node->toString().c_str(), msg->target.toString().c_str(), msg->want);
++in_stats.find;
RequestAnswer answer = onFindNode(node, msg->target, msg->want);
@@ -667,7 +749,7 @@ NetworkEngine::process(std::unique_ptr<ParsedMessage>&& msg, const SockAddr& fro
}
case MessageType::GetValues: {
// if (logger_)
// logger_->d(msg->info_hash, node->id, "[node %s] got 'get' request for %s",
// logger_->d("[node %s] got 'get' request for %s",
// node->toString().c_str(), msg->info_hash.toString().c_str());
++in_stats.get;
RequestAnswer answer = onGetValues(node, msg->info_hash, msg->want, msg->query);
@@ -677,11 +759,7 @@ NetworkEngine::process(std::unique_ptr<ParsedMessage>&& msg, const SockAddr& fro
}
case MessageType::AnnounceValue: {
if (logIncoming_ and logger_)
logger_->d(msg->info_hash,
node->id,
"[node %s] got 'put' request for %s",
node->toString().c_str(),
msg->info_hash.toString().c_str());
logger_->debug("[node {}] got 'put' request for {}", node->toString(), msg->info_hash.toString());
++in_stats.put;
onAnnounce(node, msg->info_hash, msg->token, msg->values, msg->created);
@@ -695,45 +773,33 @@ NetworkEngine::process(std::unique_ptr<ParsedMessage>&& msg, const SockAddr& fro
}
case MessageType::Refresh:
if (logIncoming_ and logger_)
logger_->d(msg->info_hash,
node->id,
"[node %s] got 'refresh' request for %s",
node->toString().c_str(),
msg->info_hash.toString().c_str());
logger_->debug("[node {}] got 'refresh' request for {}",
node->toString(),
msg->info_hash.toString());
onRefresh(node, msg->info_hash, msg->token, msg->value_id);
/* Same note as above in MessageType::AnnounceValue applies. */
sendValueAnnounced(from, msg->tid, msg->value_id);
break;
case MessageType::Listen: {
if (logIncoming_ and logger_)
logger_->d(msg->info_hash,
node->id,
"[node %s] got 'listen' request for %s",
node->toString().c_str(),
msg->info_hash.toString().c_str());
logger_->debug("[node {}] got 'listen' request for {}", node->toString(), msg->info_hash.toString());
++in_stats.listen;
RequestAnswer answer
= onListen(node, msg->info_hash, msg->token, msg->socket_id, std::move(msg->query), msg->version);
auto nnodes = bufferNodes(from.getFamily(), msg->info_hash, msg->want, answer.nodes4, answer.nodes6);
sendListenConfirmation(from, msg->tid);
sendListenConfirmation(from, msg->tid, answer.ntoken);
break;
}
case MessageType::UpdateValue: {
if (logIncoming_ and logger_)
logger_->d(msg->info_hash,
node->id,
"[node %s] got 'update' request for %s",
node->toString().c_str(),
msg->info_hash.toString().c_str());
logger_->debug("[node {}] got 'update' request for {}", node->toString(), msg->info_hash.toString());
++in_stats.updateValue;
if (auto rsocket = node->getSocket(msg->socket_id))
rsocket->on_receive(node, std::move(*msg));
else if (logger_)
logger_->e(msg->info_hash,
node->id,
"[node %s] 'update' request without socket for %s",
node->toString().c_str(),
msg->info_hash.toString().c_str());
logger_->error("[node {}] 'update' request without socket for {}",
node->toString(),
msg->info_hash.toString());
sendListenConfirmation(from, msg->tid);
break;
}
@@ -802,9 +868,9 @@ NetworkEngine::sendPing(const Sp<Node>& node, RequestCb&& on_done, RequestExpire
tid,
node,
Blob(buffer.data(), buffer.data() + buffer.size()),
[=](const Request& req_status, ParsedMessage&&) {
[this, on_done = std::move(on_done)](const Request& req_status, ParsedMessage&&) {
if (logger_)
logger_->d(req_status.node->id, "[node %s] got pong.", req_status.node->toString().c_str());
logger_->debug("[node {}] got pong.", req_status.node->toString());
if (on_done) {
on_done(req_status, {});
}
@@ -1300,17 +1366,21 @@ NetworkEngine::sendListen(const Sp<Node>& n,
}
void
NetworkEngine::sendListenConfirmation(const SockAddr& addr, Tid tid)
NetworkEngine::sendListenConfirmation(const SockAddr& addr, Tid tid, const Blob& token)
{
msgpack::sbuffer buffer;
msgpack::packer<msgpack::sbuffer> pk(&buffer);
pk.pack_map(4 + (config.network ? 1 : 0) + (config.is_client ? 1 : 0));
pk.pack(KEY_R);
pk.pack_map(2);
pk.pack_map(2 + (not token.empty() ? 1 : 0));
pk.pack(KEY_REQ_ID);
pk.pack(myid);
insertAddr(pk, addr);
if (not token.empty()) {
pk.pack(KEY_REQ_TOKEN);
packToken(pk, token);
}
pk.pack(KEY_TID);
pk.pack(tid);
@@ -1381,18 +1451,18 @@ NetworkEngine::sendAnnounceValue(const Sp<Node>& n,
tid,
n,
Blob(buffer.data(), buffer.data() + buffer.size()),
[=](const Request& req_status, ParsedMessage&& msg) { /* on done */
if (msg.value_id == Value::INVALID_ID) {
if (logger_)
logger_->d(infohash,
"Unknown search or announce!");
} else {
if (on_done) {
RequestAnswer answer {};
answer.vid = msg.value_id;
on_done(req_status, std::move(answer));
}
}
[this, on_done = std::move(on_done)](const Request& req_status, ParsedMessage&& msg) {
/* on done */
if (msg.value_id == Value::INVALID_ID) {
if (logger_)
logger_->debug("[node {}] Unknown search or announce!", req_status.node->toString());
} else {
if (on_done) {
RequestAnswer answer {};
answer.vid = msg.value_id;
on_done(req_status, std::move(answer));
}
}
},
std::move(on_expired));
req->parts = std::move(v);
@@ -1537,18 +1607,18 @@ NetworkEngine::sendRefreshValue(const Sp<Node>& n,
tid,
n,
Blob(buffer.data(), buffer.data() + buffer.size()),
[=](const Request& req_status, ParsedMessage&& msg) { /* on done */
if (msg.value_id == Value::INVALID_ID) {
if (logger_)
logger_->d(infohash,
"Unknown search or announce!");
} else {
if (on_done) {
RequestAnswer answer {};
answer.vid = msg.value_id;
on_done(req_status, std::move(answer));
}
}
[this, on_done = std::move(on_done)](const Request& req_status, ParsedMessage&& msg) {
/* on done */
if (msg.value_id == Value::INVALID_ID) {
if (logger_)
logger_->debug("[node {}] Unknown search or announce!", req_status.node->toString());
} else {
if (on_done) {
RequestAnswer answer {};
answer.vid = msg.value_id;
on_done(req_status, std::move(answer));
}
}
},
std::move(on_error),
std::move(on_expired));
@@ -1628,14 +1698,14 @@ NetworkEngine::sendError(const SockAddr& addr, Tid tid, uint16_t code, const std
}
void
NetworkEngine::maintainRxBuffer(Tid tid)
NetworkEngine::maintainRxBuffer(const PartialMessageKey& key)
{
auto msg = partial_messages.find(tid);
auto msg = partial_messages.find(key);
if (msg != partial_messages.end()) {
const auto& now = scheduler.time();
if (msg->second.start + RX_MAX_PACKET_TIME < now || msg->second.last_part + RX_TIMEOUT < now) {
if (logger_)
logger_->w("Dropping expired partial message from %s", msg->second.from.toString().c_str());
logger_->warn("Dropping expired partial message from {}", msg->second.from.toString());
partial_messages.erase(msg);
}
}
+7 -7
View File
@@ -107,14 +107,14 @@ UdpSocket::UdpSocket(in_port_t port, const std::shared_ptr<Logger>& l)
SockAddr bind6;
bind6.setFamily(AF_INET6);
bind6.setPort(port);
std::lock_guard<std::mutex> lk(lock);
std::lock_guard lk(lock);
openSockets(bind4, bind6);
}
UdpSocket::UdpSocket(const SockAddr& bind4, const SockAddr& bind6, const std::shared_ptr<Logger>& l)
: logger(l)
{
std::lock_guard<std::mutex> lk(lock);
std::lock_guard lk(lock);
openSockets(bind4, bind6);
}
@@ -161,7 +161,7 @@ UdpSocket::sendTo(const SockAddr& dest, const uint8_t* data, size_t size, [[mayb
if (logger)
logger->d("Can't send message to %s: %s", dest.toString().c_str(), strerror(err));
if (err == EPIPE || err == ENOTCONN || err == ECONNRESET) {
std::lock_guard<std::mutex> lk(lock);
std::lock_guard lk(lock);
auto bind4 = std::move(bound4), bind6 = std::move(bound6);
openSockets(bind4, bind6);
return sendTo(dest, data, size, false);
@@ -286,13 +286,13 @@ UdpSocket::openSockets(const SockAddr& bind4, const SockAddr& bind6)
pkt.received = clock::now();
onReceived(std::move(pkts));
} else if (rc == -1) {
if (logger)
logger->e("Error receiving packet: %s", strerror(errno));
int err = errno;
if (logger)
logger->e("Error receiving packet: %s", strerror(err));
if (err == EPIPE || err == ENOTCONN || err == ECONNRESET) {
if (not running)
break;
std::unique_lock<std::mutex> lk(lock, std::try_to_lock);
std::unique_lock lk(lock, std::try_to_lock);
if (lk.owns_lock()) {
if (not running)
break;
@@ -337,7 +337,7 @@ UdpSocket::openSockets(const SockAddr& bind4, const SockAddr& bind6)
close(stop_readfd);
if (stopfd != -1)
close(stopfd);
std::unique_lock<std::mutex> lk(lock, std::try_to_lock);
std::unique_lock lk(lock, std::try_to_lock);
if (lk.owns_lock()) {
s4 = -1;
s6 = -1;
+1 -3
View File
@@ -147,9 +147,7 @@ Node::closeSocket(Tid id)
std::string
Node::toString() const
{
std::ostringstream ss;
ss << (*this);
return ss.str();
return fmt::format("{} {:<21}", id, addr.toString());
}
std::ostream&
+8 -1
View File
@@ -210,6 +210,13 @@ SearchCache::cancelAll(const std::function<void(size_t)>& onCancel)
ops.clear();
}
void
SearchCache::clear()
{
ops.clear();
nextExpiration_ = time_point::max();
}
time_point
SearchCache::expire(const time_point& now, const std::function<void(size_t)>& onCancel)
{
@@ -217,7 +224,7 @@ SearchCache::expire(const time_point& now, const std::function<void(size_t)>& on
auto ret = nextExpiration_;
for (auto it = ops.begin(); it != ops.end();) {
auto expiration = it->second->getExpiration();
if (expiration < now) {
if (expiration <= now) {
auto cache = std::move(it->second);
it = ops.erase(it);
onCancel(cache->searchToken);
+1
View File
@@ -164,6 +164,7 @@ public:
bool cancelListen(size_t gtoken, const time_point& now);
void cancelAll(const std::function<void(size_t)>& onCancel);
void clear();
time_point expire(const time_point& now, const std::function<void(size_t)>& onCancel);
time_point getExpiration() const { return nextExpiration_; }
+69 -25
View File
@@ -6,6 +6,7 @@
#include "sockaddr.h"
#include "net.h"
#include <algorithm>
#include <map>
#include <string_view>
@@ -58,8 +59,12 @@ unpackTid(const msgpack::object& o)
switch (o.type) {
case msgpack::type::POSITIVE_INTEGER:
return o.as<Tid>();
default:
return ntohl(*reinterpret_cast<const uint32_t*>(o.as<std::array<char, 4>>().data()));
default: {
auto arr = o.as<std::array<char, 4>>();
uint32_t val;
std::memcpy(&val, arr.data(), sizeof(val));
return ntohl(val);
}
}
}
@@ -95,9 +100,25 @@ struct ParsedMessage
std::vector<Value::Id> expired_values {};
/* index for fields values */
std::vector<Sp<FieldValueIndex>> fields;
/** When part of the message header: {index -> (total size, {})}
* When part of partial value data: {index -> (offset, part_data)} */
std::map<unsigned, std::pair<unsigned, Blob>> value_parts;
/** Tracks partial value reassembly for fragmented messages.
* Populated from message header with total expected sizes. */
struct PartialValue
{
size_t total_size {0};
size_t received_bytes {0};
std::map<size_t, Blob> fragments {}; // offset -> fragment data
bool isComplete() const { return total_size > 0 && received_bytes == total_size; }
Blob reassemble() const
{
Blob result(total_size);
for (const auto& [offset, data] : fragments)
std::copy(data.begin(), data.end(), result.begin() + offset);
return result;
}
};
std::map<unsigned, PartialValue> value_parts;
/** When part of partial value data: {index -> (offset, part_data)} */
std::map<unsigned, std::pair<size_t, Blob>> fragment_parts;
/* query describing a filter to apply on values. */
Query query;
/* states if ipv4 or ipv6 request */
@@ -118,17 +139,32 @@ bool
ParsedMessage::append(const ParsedMessage& block)
{
bool ret(false);
for (const auto& ve : block.value_parts) {
auto part_val = value_parts.find(ve.first);
if (part_val == value_parts.end() || part_val->second.second.size() >= part_val->second.first)
for (const auto& [index, fragment] : block.fragment_parts) {
auto part_it = value_parts.find(index);
if (part_it == value_parts.end())
continue;
// TODO: handle out-of-order packets
if (ve.second.first != part_val->second.second.size()) {
// std::cout << "skipping out-of-order packet" << std::endl;
auto& pv = part_it->second;
if (pv.isComplete())
continue;
auto offset = fragment.first;
const auto& data = fragment.second;
if (data.empty() || offset > pv.total_size || data.size() > pv.total_size - offset)
continue;
// Check for overlap with existing fragments
auto it = pv.fragments.lower_bound(offset);
if (it != pv.fragments.end()) {
auto distance = it->first - offset;
if (distance < data.size())
continue;
}
if (it != pv.fragments.begin()) {
auto prev = std::prev(it);
if (prev->second.size() > offset - prev->first)
continue;
}
pv.fragments.emplace(offset, data);
pv.received_bytes += data.size();
ret = true;
part_val->second.second.insert(part_val->second.second.end(), ve.second.second.begin(), ve.second.second.end());
}
return ret;
}
@@ -136,16 +172,14 @@ ParsedMessage::append(const ParsedMessage& block)
bool
ParsedMessage::complete()
{
for (auto& e : value_parts) {
if (e.second.first > e.second.second.size()) {
// std::cout << "uncomplete part " << e.first << ": " << e.second.second.size() << "/" << e.second.first <<
// std::endl;
for (const auto& [index, pv] : value_parts) {
if (!pv.isComplete())
return false;
}
}
for (auto& e : value_parts) {
for (auto& [index, pv] : value_parts) {
auto assembled = pv.reassemble();
msgpack::unpacked msg;
msgpack::unpack(msg, (const char*) e.second.second.data(), e.second.second.size());
msgpack::unpack(msg, (const char*) assembled.data(), assembled.size());
values.emplace_back(std::make_shared<Value>(msg.get()));
}
return true;
@@ -233,21 +267,28 @@ ParsedMessage::msgpack_unpack(const msgpack::object& msg)
auto d = findMapValue(vdat.val, "d"sv);
if (not o or not d)
continue;
value_parts.emplace(vdat.key.as<unsigned>(), std::pair<size_t, Blob>(o->as<size_t>(), unpackBlob(*d)));
fragment_parts.emplace(vdat.key.as<unsigned>(), std::pair<size_t, Blob>(o->as<size_t>(), unpackBlob(*d)));
}
return;
}
if (!parsed.a && !parsed.r && !parsed.e && !parsed.u)
throw msgpack::type_error();
auto& req = parsed.a ? *parsed.a : (parsed.r ? *parsed.r : (parsed.u ? *parsed.u : *parsed.e));
if (parsed.e) {
if (parsed.e->type != msgpack::type::ARRAY)
if (parsed.e->type != msgpack::type::ARRAY or parsed.e->via.array.size < 1)
throw msgpack::type_error();
error_code = parsed.e->via.array.ptr[0].as<uint16_t>();
}
/* An error-only message carries no request map. The error array must never
be reached through the map view of the msgpack object union, and a
request field that is not a map must be rejected rather than
reinterpreted. */
const msgpack::object* req = parsed.a ? parsed.a : (parsed.r ? parsed.r : parsed.u);
if (req and req->type != msgpack::type::MAP)
throw msgpack::type_error();
struct ParsedReq
{
msgpack::object* values;
@@ -256,8 +297,9 @@ ParsedMessage::msgpack_unpack(const msgpack::object& msg)
msgpack::object* want;
} parsedReq {};
for (unsigned i = 0; i < req.via.map.size; i++) {
auto& o = req.via.map.ptr[i];
const uint32_t req_size = req ? req->via.map.size : 0;
for (unsigned i = 0; i < req_size; i++) {
auto& o = req->via.map.ptr[i];
if (o.key.type != msgpack::type::STR)
continue;
auto key = o.key.as<std::string_view>();
@@ -324,7 +366,7 @@ ParsedMessage::msgpack_unpack(const msgpack::object& msg)
// Skip oversize values with a small margin for header overhead
if (packed_v.via.u64 > MAX_VALUE_SIZE + 32)
continue;
value_parts.emplace(i, std::make_pair(packed_v.via.u64, Blob {}));
value_parts[i].total_size = packed_v.via.u64;
} else {
try {
values.emplace_back(std::make_shared<Value>(parsedReq.values->via.array.ptr[i]));
@@ -336,6 +378,8 @@ ParsedMessage::msgpack_unpack(const msgpack::object& msg)
} else if (parsedReq.fields) {
if (auto rfields = findMapValue(*parsedReq.fields, "f"sv)) {
auto vfields = rfields->as<std::set<Value::Field>>();
if (vfields.empty())
throw msgpack::type_error();
if (auto rvalues = findMapValue(*parsedReq.fields, "v"sv)) {
if (rvalues->type != msgpack::type::ARRAY)
throw msgpack::type_error();
+20 -20
View File
@@ -123,7 +123,7 @@ PeerDiscovery::DomainPeerDiscovery::~DomainPeerDiscovery()
void
PeerDiscovery::DomainPeerDiscovery::startDiscovery(std::string_view type, ServiceDiscoveredCallback callback)
{
std::lock_guard<std::mutex> lck(dmtx_);
std::lock_guard lck(dmtx_);
callbackmap_[std::string(type)] = callback;
if (not drunning_) {
drunning_ = true;
@@ -137,7 +137,7 @@ PeerDiscovery::DomainPeerDiscovery::startDiscovery(std::string_view type, Servic
void
PeerDiscovery::DomainPeerDiscovery::loopListener()
{
std::lock_guard<std::mutex> lck(dmtx_);
std::lock_guard lck(dmtx_);
if (not drunning_)
return;
sockFd_.async_receive_from(asio::buffer(receiveBuf_),
@@ -147,7 +147,7 @@ PeerDiscovery::DomainPeerDiscovery::loopListener()
return;
if (error) {
if (logger_)
logger_->e("Error receiving message: %s", error.message().c_str());
logger_->error("Error receiving message: {}", error.message());
}
try {
auto rcv = msgpack::unpack(receiveBuf_.data(), bytes);
@@ -163,7 +163,7 @@ PeerDiscovery::DomainPeerDiscovery::loopListener()
continue;
ServiceDiscoveredCallback cb;
{
std::lock_guard<std::mutex> lck(dmtx_);
std::lock_guard lck(dmtx_);
if (drunning_) {
auto callback = callbackmap_.find(o.key.as<std::string_view>());
if (callback != callbackmap_.end())
@@ -180,7 +180,7 @@ PeerDiscovery::DomainPeerDiscovery::loopListener()
}
} catch (const std::exception& e) {
if (logger_)
logger_->e("Error receiving packet: %s", e.what());
logger_->error("Error receiving packet: {}", e.what());
}
loopListener();
});
@@ -189,7 +189,7 @@ PeerDiscovery::DomainPeerDiscovery::loopListener()
void
PeerDiscovery::DomainPeerDiscovery::query(const asio::ip::udp::endpoint& peer)
{
std::lock_guard<std::mutex> lck(dmtx_);
std::lock_guard lck(dmtx_);
if (not drunning_)
return;
@@ -200,16 +200,16 @@ PeerDiscovery::DomainPeerDiscovery::query(const asio::ip::udp::endpoint& peer)
peer,
[logger = logger_, peer](const asio::error_code& ec, size_t) {
if (ec and (ec != asio::error::operation_aborted) and logger)
logger->w("Error sending packet to: %s with err: %s",
peer.address().to_string().c_str(),
ec.message().c_str());
logger->warn("Error sending packet to: {} with err: {}",
peer.address().to_string(),
ec.message());
});
}
void
PeerDiscovery::DomainPeerDiscovery::publish(const asio::ip::udp::endpoint& peer)
{
std::lock_guard<std::mutex> lck(mtx_);
std::lock_guard lck(mtx_);
if (not lrunning_)
return;
@@ -217,9 +217,9 @@ PeerDiscovery::DomainPeerDiscovery::publish(const asio::ip::udp::endpoint& peer)
peer,
[logger = logger_, peer](const asio::error_code& ec, size_t) {
if (ec and (ec != asio::error::operation_aborted) and logger)
logger->w("Error sending packet to: %s with err: %s",
peer.address().to_string().c_str(),
ec.message().c_str());
logger->warn("Error sending packet to: {} with err: {}",
peer.address().to_string(),
ec.message());
});
}
@@ -229,7 +229,7 @@ PeerDiscovery::DomainPeerDiscovery::startPublish(std::string_view type, const ms
msgpack::sbuffer pack_buf_c(pack_buf.size());
pack_buf_c.write(pack_buf.data(), pack_buf.size());
std::lock_guard<std::mutex> lck(mtx_);
std::lock_guard lck(mtx_);
messages_[std::string(type)] = std::move(pack_buf_c);
reloadMessages();
lrunning_ = true;
@@ -239,7 +239,7 @@ PeerDiscovery::DomainPeerDiscovery::startPublish(std::string_view type, const ms
bool
PeerDiscovery::DomainPeerDiscovery::stopDiscovery(std::string_view type)
{
std::lock_guard<std::mutex> lck(dmtx_);
std::lock_guard lck(dmtx_);
auto it = callbackmap_.find(type);
if (it != callbackmap_.end()) {
callbackmap_.erase(it);
@@ -253,7 +253,7 @@ PeerDiscovery::DomainPeerDiscovery::stopDiscovery(std::string_view type)
bool
PeerDiscovery::DomainPeerDiscovery::stopPublish(std::string_view type)
{
std::lock_guard<std::mutex> lck(mtx_);
std::lock_guard lck(mtx_);
auto it = messages_.find(type);
if (it != messages_.end()) {
messages_.erase(it);
@@ -282,11 +282,11 @@ void
PeerDiscovery::DomainPeerDiscovery::stop()
{
{
std::lock_guard<std::mutex> lck(dmtx_);
std::lock_guard lck(dmtx_);
stopDiscovery();
}
{
std::lock_guard<std::mutex> lck(mtx_);
std::lock_guard lck(mtx_);
stopPublish();
}
}
@@ -310,7 +310,7 @@ PeerDiscovery::DomainPeerDiscovery::reDiscover()
sockFd_.set_option(asio::ip::multicast::join_group(sockAddrSend_.address()), ec);
if (ec and logger_)
logger_->w("Unable to multicast on %s: %s", sockAddrSend_.address().to_string().c_str(), ec.message().c_str());
logger_->warn("Unable to multicast on {}: {}", sockAddrSend_.address().to_string(), ec.message());
query(sockAddrSend_);
}
@@ -322,7 +322,7 @@ PeerDiscovery::DomainPeerDiscovery::connectivityChanged()
publish(sockAddrSend_);
});
if (logger_)
logger_->d("PeerDiscovery: connectivity changed");
logger_->debug("PeerDiscovery: connectivity changed");
if (peerDiscovery_period == PeerDiscovery_PERIOD_MAX) {
peerDiscovery_period = PeerDiscovery_PERIOD;
+4 -5
View File
@@ -97,19 +97,18 @@ std::vector<Sp<Node>>
RoutingTable::findClosestNodes(const InfoHash id, time_point now, size_t count) const
{
std::vector<Sp<Node>> nodes;
nodes.reserve(count);
auto bucket = findBucket(id);
if (bucket == end()) {
return nodes;
}
nodes.reserve(count);
auto sortedBucketInsert = [&](const Bucket& b) {
for (auto n : b.nodes) {
for (const auto& n : b.nodes) {
if (not n->isGood(now))
continue;
auto here = std::find_if(nodes.begin(), nodes.end(), [&id, &n](Sp<Node>& node) {
auto here = std::find_if(nodes.begin(), nodes.end(), [&id, &n](const Sp<Node>& node) {
return id.xorCmp(n->id, node->id) < 0;
});
nodes.insert(here, n);
@@ -199,7 +198,7 @@ RoutingTable::onNewNode(
if (confirm == 2)
b->time = now;
for (auto& n : b->nodes) {
for (const auto& n : b->nodes) {
if (n == node)
return false;
}
+11
View File
@@ -133,6 +133,16 @@ struct Dht::SearchNode
return last_get_reply + Node::NODE_EXPIRE_TIME;
}
/**
* Update the token without refreshing last_get_reply,
* so that periodic find_node/get requests still occur.
*/
void refreshToken(const Blob& tok)
{
if (not tok.empty() and tok.size() <= 64)
token = tok;
}
friend std::ostream& operator<<(std::ostream& s, const SearchNode& node)
{
s << "getStatus:" << node.getStatus.size() << " listenStatus:" << node.listenStatus.size()
@@ -836,6 +846,7 @@ struct Dht::Search
listeners.clear();
nodes.clear();
nextSearchStep.reset();
cache.clear();
}
};
+22 -167
View File
@@ -40,26 +40,12 @@ SecureDht::SecureDht(std::unique_ptr<DhtInterface> dht,
registerInsecureType(CERTIFICATE_TYPE);
if (certificate_) {
auto certId = certificate_->getId();
auto certLongId = certificate_->getLongId();
if (key_ and (certId != key_->getPublicKey().getId() or certLongId != key_->getPublicKey().getLongId()))
if (key_ and (certLongId != key_->getPublicKey().getLongId()))
throw DhtException("SecureDht: provided certificate doesn't match private key.");
auto cb = iacb ? [
this,
completedState = std::make_shared<std::pair<unsigned,bool>>(2, true),
iacb = std::move(iacb)
](bool ok) {
if (logger_)
logger_->d("SecureDht: certificate announcement %s", ok ? "succeeded" : "failed");
completedState->second = completedState->second and ok;
if (--completedState->first == 0)
iacb(completedState->second);
} : iacb;
dht_->addOnConnectedCallback([this, certId, certLongId, cb = std::move(cb)] {
dht_->put(certId, Value {CERTIFICATE_TYPE, *certificate_, 1}, cb, {}, true);
dht_->put(InfoHash::get(certLongId), Value {CERTIFICATE_TYPE, *certificate_, 1}, cb, {}, true);
dht_->addOnConnectedCallback([this, certLongId, cb = std::move(iacb)]() mutable {
dht_->put(InfoHash::get(certLongId), Value {CERTIFICATE_TYPE, *certificate_, 1}, std::move(cb), {}, true);
});
}
}
@@ -110,37 +96,13 @@ SecureDht::secureType(ValueType&& type)
return type;
}
Sp<crypto::Certificate>
SecureDht::getCertificate(const InfoHash& node) const
{
if (node == getId())
return certificate_;
auto it = nodesCertificates_.find(node);
if (it == nodesCertificates_.end())
return nullptr;
else
return it->second;
}
Sp<crypto::PublicKey>
SecureDht::getPublicKey(const InfoHash& node) const
{
if (node == getId())
return certificate_->getSharedPublicKey();
auto it = nodesPubKeys_.find(node);
if (it == nodesPubKeys_.end())
return nullptr;
else
return it->second;
}
Sp<crypto::Certificate>
SecureDht::getCertificate(const PkId& node) const
{
if (node == getLongId())
return certificate_;
auto it = nodesCertificatesLong_.find(node);
if (it == nodesCertificatesLong_.end())
auto it = nodesCertificates_.find(node);
if (it == nodesCertificates_.end())
return nullptr;
else
return it->second;
@@ -151,8 +113,8 @@ SecureDht::getPublicKey(const PkId& node) const
{
if (node == getLongId())
return certificate_->getSharedPublicKey();
auto it = nodesPubKeysLong_.find(node);
if (it == nodesPubKeysLong_.end())
auto it = nodesPubKeys_.find(node);
if (it == nodesPubKeys_.end())
return nullptr;
else
return it->second;
@@ -175,7 +137,7 @@ SecureDht::registerCertificate(const InfoHash& node, const Blob& data)
return crt;
} else {
if (logger_)
logger_->w("Certificate {} for node {} does not match node id !", h.toString(), node.toString());
logger_->warn("Certificate {} for node {} does not match node id !", h.toString(), node.toString());
return nullptr;
}
}
@@ -196,7 +158,7 @@ SecureDht::registerCertificate(const PkId& node, const Blob& data)
return crt;
} else {
if (logger_)
logger_->w("Certificate {} for node {} does not match node id !", h.toString(), node.toString());
logger_->warn("Certificate {} for node {} does not match node id !", h.toString(), node.toString());
return nullptr;
}
}
@@ -205,85 +167,10 @@ void
SecureDht::registerCertificate(const Sp<crypto::Certificate>& cert)
{
if (cert) {
nodesCertificates_[cert->getId()] = cert;
nodesCertificatesLong_[cert->getLongId()] = cert;
nodesCertificates_[cert->getLongId()] = cert;
}
}
void
SecureDht::findCertificate(const InfoHash& node, const std::function<void(const Sp<crypto::Certificate>)>& cb)
{
Sp<crypto::Certificate> b = getCertificate(node);
if (b && *b) {
if (logger_)
logger_->d("Using certificate from cache for %s", node.to_c_str());
if (cb)
cb(b);
return;
}
if (localQueryMethodLegacy_) {
auto res = localQueryMethodLegacy_(node);
if (not res.empty()) {
if (logger_)
logger_->d("Registering certificate from local store for %s", node.to_c_str());
nodesCertificates_.emplace(node, res.front());
nodesCertificatesLong_.emplace(res.front()->getLongId(), res.front());
if (cb)
cb(res.front());
return;
}
}
auto found = std::make_shared<bool>(false);
dht_->get(
node,
[cb, node, found, this](const std::vector<Sp<Value>>& vals) {
for (const auto& v : vals) {
if (auto cert = registerCertificate(node, v->data)) {
*found = true;
if (logger_)
logger_->d(node, "Found certificate for %s", node.to_c_str());
if (cb)
cb(cert);
return false;
}
}
return !*found;
},
[cb, found](bool) {
if (!*found and cb)
cb(nullptr);
},
Value::TypeFilter(CERTIFICATE_TYPE));
}
void
SecureDht::findPublicKey(const InfoHash& node, const std::function<void(const Sp<crypto::PublicKey>)>& cb)
{
auto pk = getPublicKey(node);
if (pk && *pk) {
if (logger_)
logger_->d(node, "Found public key from cache for %s", node.to_c_str());
if (cb)
cb(pk);
return;
}
findCertificate(node, [=](const Sp<crypto::Certificate>& crt) {
if (crt && *crt) {
auto pk = crt->getSharedPublicKey();
if (*pk) {
nodesPubKeys_[pk->getId()] = pk;
nodesPubKeysLong_[pk->getLongId()] = pk;
if (cb)
cb(pk);
return;
}
}
if (cb)
cb(nullptr);
});
}
void
SecureDht::findCertificate(const PkId& node, const std::function<void(const Sp<crypto::Certificate>)>& cb)
{
@@ -301,8 +188,7 @@ SecureDht::findCertificate(const PkId& node, const std::function<void(const Sp<c
if (not res.empty()) {
if (logger_)
logger_->debug("Registering certificate from local store for {}", node.to_c_str());
nodesCertificates_.emplace(res.front()->getId(), res.front());
nodesCertificatesLong_.emplace(node, res.front());
nodesCertificates_.emplace(node, res.front());
if (cb)
cb(res.front());
return;
@@ -347,8 +233,7 @@ SecureDht::findPublicKey(const PkId& node, const std::function<void(const Sp<cry
if (crt && *crt) {
auto pk = crt->getSharedPublicKey();
if (*pk) {
nodesPubKeys_[pk->getId()] = pk;
nodesPubKeysLong_[pk->getLongId()] = pk;
nodesPubKeys_[pk->getLongId()] = pk;
if (cb)
cb(pk);
return;
@@ -376,14 +261,13 @@ SecureDht::checkValue(const Sp<Value>& v)
if (auto decrypted_val = v->decrypt(*key_)) {
auto cacheValue = not isDecrypted and decrypted_val->owner;
if (cacheValue) {
nodesPubKeys_[decrypted_val->owner->getId()] = decrypted_val->owner;
nodesPubKeysLong_[decrypted_val->owner->getLongId()] = decrypted_val->owner;
nodesPubKeys_[decrypted_val->owner->getLongId()] = decrypted_val->owner;
}
return decrypted_val;
}
} catch (const std::exception& e) {
if (logger_)
logger_->w("Could not decrypt value %s : %s", v->toString().c_str(), e.what());
logger_->warn("Could not decrypt value {} : {}", v->toString(), e.what());
}
}
// Check signed values
@@ -391,12 +275,11 @@ SecureDht::checkValue(const Sp<Value>& v)
auto cacheValue = not v->isSignatureChecked() and enableCache_ and v->owner;
if (v->checkSignature()) {
if (cacheValue) {
nodesPubKeys_[v->owner->getId()] = v->owner;
nodesPubKeysLong_[v->owner->getLongId()] = v->owner;
nodesPubKeys_[v->owner->getLongId()] = v->owner;
}
return v;
} else if (logger_)
logger_->w("Signature verification failed for %s", v->toString().c_str());
logger_->warn("Signature verification failed for {}", v->toString());
}
// Forward normal values
else {
@@ -484,14 +367,15 @@ SecureDht::putSigned(const InfoHash& hash, Sp<Value> val, DoneCallback callback,
hash,
[val, this](const std::vector<Sp<Value>>& vals) {
if (logger_)
logger_->d("Found online previous value being announced.");
logger_->debug("Found online previous value being announced.");
for (const auto& v : vals) {
if (!v->isSigned()) {
if (logger_)
logger_->e("Existing non-signed value seems to exists at this location.");
logger_->error("Existing non-signed value seems to exists at this location.");
} else if (not v->owner or v->owner->getId() != getId()) {
if (logger_)
logger_->e("Existing signed value belonging to someone else seems to exists at this location.");
logger_->error(
"Existing signed value belonging to someone else seems to exists at this location.");
} else if (val->seq <= v->seq)
val->seq = v->seq + 1;
}
@@ -505,35 +389,6 @@ SecureDht::putSigned(const InfoHash& hash, Sp<Value> val, DoneCallback callback,
Where().id(val->id));
}
void
SecureDht::putEncrypted(const InfoHash& hash, const InfoHash& to, Sp<Value> val, DoneCallback callback, bool permanent)
{
if (not key_) {
if (callback)
callback(false, {});
return;
}
findPublicKey(to,
[this, hash, val = std::move(val), callback = std::move(callback), permanent](
const Sp<crypto::PublicKey>& pk) {
if (!pk || !*pk) {
if (callback)
callback(false, {});
return;
}
if (logger_)
logger_->w("Encrypting data for PK: %s", pk->getId().toString().c_str());
try {
dht_->put(hash, encrypt(*val, *pk), callback, time_point::max(), permanent);
} catch (const std::exception& e) {
if (logger_)
logger_->e("Error putting encrypted data: %s", e.what());
if (callback)
callback(false, {});
}
});
}
void
SecureDht::putEncrypted(const InfoHash& hash, const PkId& to, Sp<Value> val, DoneCallback callback, bool permanent)
{
@@ -556,7 +411,7 @@ SecureDht::putEncrypted(const InfoHash& hash, const PkId& to, Sp<Value> val, Don
dht_->put(hash, encrypt(*val, *pk), callback, time_point::max(), permanent);
} catch (const std::exception& e) {
if (logger_)
logger_->e("Error putting encrypted data: %s", e.what());
logger_->error("Error putting encrypted data: {}", e.what());
if (callback)
callback(false, {});
}
@@ -578,7 +433,7 @@ SecureDht::putEncrypted(
dht_->put(hash, encrypt(*val, pk), callback, time_point::max(), permanent);
} catch (const std::exception& e) {
if (logger_)
logger_->e("Error putting encrypted data: %s", e.what());
logger_->error("Error putting encrypted data: {}", e.what());
if (callback)
callback(false, {});
}
+26 -5
View File
@@ -3,8 +3,9 @@
#pragma once
#include "infohash.h"
#include "value.h"
#include "listener.h"
#include "sockaddr.h"
#include "value.h"
#include <map>
#include <utility>
@@ -17,6 +18,11 @@ namespace dht {
class StorageBucket
{
public:
StorageBucket() = default;
explicit StorageBucket(SockAddr addr)
: addr_(std::move(addr))
{}
void insert(const InfoHash& id, const Value& value, time_point expiration)
{
totalSize_ += value.size();
@@ -50,12 +56,16 @@ public:
insert(id, value, expiration);
}
size_t size() const { return totalSize_; }
size_t valueCount() const { return storedValues_.size(); }
bool empty() const { return storedValues_.empty(); }
std::pair<InfoHash, Value::Id> getOldest() const
{
return storedValues_.empty() ? std::pair<InfoHash, Value::Id> {} : storedValues_.begin()->second;
}
const SockAddr& getAddr() const { return addr_; }
private:
SockAddr addr_ {};
std::multimap<time_point, std::pair<InfoHash, Value::Id>> storedValues_;
size_t totalSize_ {0};
};
@@ -110,7 +120,7 @@ struct Storage
bool empty() const { return values.empty(); }
StoreDiff clear();
StoreDiff clear(const InfoHash& id);
size_t valueCount() const { return values.size(); }
@@ -179,7 +189,12 @@ struct Storage
Sp<Value> remove(const InfoHash& id, Value::Id);
std::pair<ssize_t, std::vector<Sp<Value>>> expire(const InfoHash& id, time_point now);
struct ExpireResult
{
ssize_t size_diff;
std::vector<Sp<Value>> expired_values;
};
ExpireResult expire(const InfoHash& id, time_point now);
private:
Storage(const Storage&) = delete;
@@ -262,16 +277,22 @@ Storage::remove(const InfoHash& id, Value::Id vid)
}
Storage::StoreDiff
Storage::clear()
Storage::clear(const InfoHash& id)
{
ssize_t num_values = values.size();
ssize_t tot_size = total_size;
for (auto& v : values) {
if (v.store_bucket)
v.store_bucket->erase(id, *v.data, v.expiration);
if (v.expiration_job)
v.expiration_job->cancel();
}
values.clear();
total_size = 0;
return {-tot_size, -num_values, 0, 0};
}
std::pair<ssize_t, std::vector<Sp<Value>>>
Storage::ExpireResult
Storage::expire(const InfoHash& id, time_point now)
{
// expire listeners
+9 -3
View File
@@ -6,8 +6,10 @@
#include <atomic>
#include <thread>
#include <iostream>
#include <ciso646> // fix windows compiler bug
#include <cmath> // std::pow
#include <cmath> // std::pow
#if __cplusplus < 202002L
#include <ciso646>
#endif
namespace dht {
@@ -78,6 +80,8 @@ ThreadPool::run(std::function<void()>&& cb)
break;
task = std::move(tasks_.front());
tasks_.pop();
if (tasks_.empty() && stopping_)
stopCv_.notify_one();
}
// run task
@@ -135,7 +139,9 @@ ThreadPool::stop(bool wait)
{
std::unique_lock l(lock_);
if (wait) {
cv_.wait(l, [&]() { return tasks_.empty(); });
stopping_ = true;
stopCv_.wait(l, [&]() { return tasks_.empty(); });
stopping_ = false;
}
running_ = false;
tasks_ = {};
+69 -21
View File
@@ -71,7 +71,7 @@ SockAddr::resolve(const std::string& host, const std::string& service)
addrinfo* info = nullptr;
int rc = getaddrinfo(host.c_str(), service.empty() ? nullptr : service.c_str(), &hints, &info);
if (rc != 0)
throw std::invalid_argument(std::string("Error: `") + host + ":" + service + "`: " + gai_strerror(rc));
throw std::invalid_argument(fmt::format("Error: `{}:{}`: {}", host, service, gai_strerror(rc)));
for (addrinfo* infop = info; infop; infop = infop->ai_next)
ips.emplace_back(infop->ai_addr, infop->ai_addrlen);
@@ -238,28 +238,67 @@ operator==(const SockAddr& a, const SockAddr& b)
return a.equals(b);
}
time_point
from_system_time(system_clock::time_point t, time_point steady_now, system_clock::time_point system_now)
{
if (t == system_clock::time_point::min())
return time_point::min();
if (t == system_clock::time_point::max())
return time_point::max();
// Saturate before the cast: the steady clock duration may hold a narrower range than the system one.
constexpr auto max_dt = std::chrono::duration_cast<system_clock::duration>(duration::max());
constexpr auto min_dt = std::chrono::duration_cast<system_clock::duration>(duration::min());
auto system_dt = t - system_now;
if (system_dt >= max_dt)
return time_point::max();
if (system_dt <= min_dt)
return time_point::min();
auto dt = std::chrono::duration_cast<duration>(system_dt);
if (dt > duration(0) and steady_now > time_point::max() - dt)
return time_point::max();
else if (dt < duration(0) and steady_now < time_point::min() - dt)
return time_point::min();
return steady_now + dt;
}
time_point
from_system_time(system_clock::time_point t)
{
return from_system_time(t, clock::now(), system_clock::now());
}
system_clock::time_point
to_system_time(time_point t, time_point steady_now, system_clock::time_point system_now)
{
if (t == time_point::min())
return system_clock::time_point::min();
if (t == time_point::max())
return system_clock::time_point::max();
// Cast first: mixing clock durations of different periods would overflow the bound computations below.
auto dt = std::chrono::duration_cast<system_clock::duration>(t - steady_now);
if (dt > system_clock::duration(0) and system_now >= system_clock::time_point::max() - dt)
return system_clock::time_point::max();
else if (dt < system_clock::duration(0) and system_now <= system_clock::time_point::min() - dt)
return system_clock::time_point::min();
return system_now + dt;
}
system_clock::time_point
to_system_time(time_point t)
{
return to_system_time(t, clock::now(), system_clock::now());
}
time_point
from_time_t(std::time_t t)
{
auto dt = system_clock::from_time_t(t) - system_clock::now();
auto now = clock::now();
if (dt > system_clock::duration(0) and now > time_point::max() - dt)
return time_point::max();
else if (dt < system_clock::duration(0) and now < time_point::min() - dt)
return time_point::min();
return now + dt;
return from_system_time(system_clock::from_time_t(t));
}
std::time_t
to_time_t(time_point t)
{
auto dt = t - clock::now();
auto now = system_clock::now();
if (dt > duration(0) and now >= system_clock::time_point::max() - dt)
return system_clock::to_time_t(system_clock::time_point::max());
else if (dt < duration(0) and now <= system_clock::time_point::min() - dt)
return system_clock::to_time_t(system_clock::time_point::min());
return system_clock::to_time_t(now + std::chrono::duration_cast<system_clock::duration>(dt));
return system_clock::to_time_t(to_system_time(t));
}
Blob
@@ -282,12 +321,6 @@ unpackBlob(const msgpack::object& o)
}
}
msgpack::unpacked
unpackMsg(Blob b)
{
return msgpack::unpack((const char*) b.data(), b.size());
}
msgpack::object*
findMapValue(const msgpack::object& map, const char* key, size_t key_length)
{
@@ -302,4 +335,19 @@ findMapValue(const msgpack::object& map, const char* key, size_t key_length)
return nullptr;
}
std::string
printByteCount(size_t bytes)
{
static constexpr std::string_view suffixes[] = {"B", "KB", "MB", "GB", "TB"};
size_t suffixIndex = 0;
double count = static_cast<double>(bytes);
while (count >= 1024 && suffixIndex < std::size(suffixes) - 1) {
count /= 1024;
suffixIndex++;
}
if (suffixIndex == 0)
return fmt::format("{} {}", bytes, suffixes[suffixIndex]);
return fmt::format("{:.2f} {} ({} bytes)", count, suffixes[suffixIndex], bytes);
}
} // namespace dht
+12 -1
View File
@@ -85,7 +85,7 @@ ValueType::DEFAULT_STORE_POLICY(InfoHash, const std::shared_ptr<Value>& v, const
size_t
Value::size() const
{
return cypher.size() + data.size() + signature.size() + user_type.size();
return cypher.size() + data.size() + signature.size() + user_type.size() + pushType.size();
}
void
@@ -104,9 +104,14 @@ Value::msgpack_unpack(const msgpack::object& o)
} else
throw msgpack::type_error();
priority = 0;
pushType.clear();
if (auto rprio = findMapValue(o, VALUE_KEY_PRIO)) {
priority = rprio->as<unsigned>();
}
if (auto rpt = findMapValue(o, VALUE_KEY_PUSHTYPE)) {
pushType = rpt->as<std::string>();
}
}
void
@@ -197,6 +202,9 @@ Value::Value(const Json::Value& json)
const auto& jprio = json["prio"];
if (jprio.isIntegral())
priority = jprio.asUInt();
const auto& jpt = json[VALUE_KEY_PUSHTYPE];
if (jpt.isString())
pushType = jpt.asString();
}
Json::Value
@@ -223,6 +231,8 @@ Value::toJson() const
}
if (priority)
val["prio"] = priority;
if (not pushType.empty())
val[VALUE_KEY_PUSHTYPE] = pushType;
return val;
}
@@ -261,6 +271,7 @@ Value::encrypt(const crypto::PrivateKey& from, const crypto::PublicKey& to)
sign(from);
Value nv {id};
nv.priority = priority;
nv.pushType = pushType;
nv.setCypher(to.encrypt(getToEncrypt()));
return nv;
}
-9
View File
@@ -1,9 +0,0 @@
if ENABLE_TESTS
bin_PROGRAMS = opendht_unit_tests
AM_CPPFLAGS = -I../include -DOPENDHT_JSONCPP
nobase_include_HEADERS = infohashtester.h valuetester.h cryptotester.h dhtrunnertester.h httptester.h dhtproxytester.h
opendht_unit_tests_SOURCES = tests_runner.cpp cryptotester.cpp infohashtester.cpp valuetester.cpp dhtrunnertester.cpp httptester.cpp dhtproxytester.cpp
opendht_unit_tests_LDFLAGS = -lopendht -lcppunit -ljsoncpp -L@top_builddir@/src/.libs @GnuTLS_LIBS@
endif
-473
View File
@@ -1,473 +0,0 @@
// Copyright (c) 2014-2026 Savoir-faire Linux Inc.
// SPDX-License-Identifier: MIT
#include "dhtrunnertester.h"
#include <opendht/thread_pool.h>
#include <chrono>
#include <mutex>
#include <condition_variable>
using namespace std::chrono_literals;
using namespace std::literals;
namespace test {
CPPUNIT_TEST_SUITE_REGISTRATION(DhtRunnerTester);
void
DhtRunnerTester::setUp()
{
dht::DhtRunner::Config config;
config.dht_config.node_config.max_peer_req_per_sec = -1;
config.dht_config.node_config.max_req_per_sec = -1;
config.dht_config.node_config.max_store_size = -1;
config.dht_config.node_config.max_store_keys = -1;
node1.run(0, config);
node2.run(0, config);
auto bound = node1.getBound();
if (bound.isUnspecified())
bound.setLoopback();
node2.bootstrap(bound);
}
void
DhtRunnerTester::tearDown()
{
unsigned done {0};
std::condition_variable cv;
std::mutex cv_m;
auto shutdown = [&] {
std::lock_guard lk(cv_m);
done++;
cv.notify_all();
};
node1.shutdown(shutdown);
node2.shutdown(shutdown);
std::unique_lock lk(cv_m);
CPPUNIT_ASSERT(cv.wait_for(lk, 30s, [&] { return done == 2u; }));
node1.join();
node2.join();
}
void
DhtRunnerTester::testConstructors()
{
CPPUNIT_ASSERT(node1.getBoundPort());
CPPUNIT_ASSERT_EQUAL(node1.getBoundPort(), node1.getBound().getPort());
CPPUNIT_ASSERT(node2.getBoundPort());
CPPUNIT_ASSERT_EQUAL(node2.getBoundPort(), node2.getBound().getPort());
dht::DhtRunner::Config config {};
dht::DhtRunner::Context context {};
dht::DhtRunner testNode;
testNode.run(config, std::move(context));
CPPUNIT_ASSERT(testNode.getBoundPort());
}
void
DhtRunnerTester::testGetPut()
{
auto key = dht::InfoHash::get("123");
dht::Value val {"hey"};
auto val_data = val.data;
std::promise<bool> p;
node2.put(key, std::move(val), [&](bool ok) { p.set_value(ok); });
CPPUNIT_ASSERT(p.get_future().get());
auto vals = node1.get(key).get();
CPPUNIT_ASSERT(not vals.empty());
CPPUNIT_ASSERT(vals.front()->data == val_data);
}
void
DhtRunnerTester::testPutDuplicate()
{
auto key = dht::InfoHash::get("123");
auto val = std::make_shared<dht::Value>("hey");
val->id = 42;
auto val_data = val->data;
std::promise<bool> p1;
std::promise<bool> p2;
node2.put(key, val, [&](bool ok) { p1.set_value(ok); });
node2.put(key, val, [&](bool ok) { p2.set_value(ok); });
auto p1ret = p1.get_future().get();
auto p2ret = p2.get_future().get();
CPPUNIT_ASSERT(p1ret);
CPPUNIT_ASSERT(p2ret);
auto vals = node1.get(key).get();
CPPUNIT_ASSERT(not vals.empty());
CPPUNIT_ASSERT(vals.size() == 1);
CPPUNIT_ASSERT(vals.front()->data == val_data);
}
void
DhtRunnerTester::testPutOverride()
{
auto key = dht::InfoHash::get("123");
auto val = std::make_shared<dht::Value>("meh");
val->id = 42;
auto val2 = std::make_shared<dht::Value>("hey");
val2->id = 42;
CPPUNIT_ASSERT_EQUAL(val->id, val2->id);
auto val_data = val2->data;
std::promise<bool> p1;
std::promise<bool> p2;
node2.put(key, val, [&](bool ok) { p1.set_value(ok); });
node2.put(key, val2, [&](bool ok) { p2.set_value(ok); });
auto p1ret = p1.get_future().get();
auto p2ret = p2.get_future().get();
CPPUNIT_ASSERT(!p1ret);
CPPUNIT_ASSERT(p2ret);
auto vals = node1.get(key).get();
CPPUNIT_ASSERT(not vals.empty());
CPPUNIT_ASSERT(vals.size() == 1);
CPPUNIT_ASSERT(vals.front()->data == val_data);
}
void
DhtRunnerTester::testListen()
{
std::mutex mutex;
std::condition_variable cv;
std::atomic_uint valueCounta(0);
std::atomic_uint valueCountb(0);
std::atomic_uint valueCountc(0);
std::atomic_uint valueCountd(0);
unsigned putCount(0);
unsigned putOkCount1(0);
unsigned putOkCount2(0);
unsigned putOkCount3(0);
auto a = dht::InfoHash::get("234");
auto b = dht::InfoHash::get("2345");
auto c = dht::InfoHash::get("23456");
auto d = dht::InfoHash::get("234567");
constexpr unsigned N = 256;
constexpr unsigned SZ = 56 * 1024;
auto ftokena = node1.listen(a, [&](const std::vector<std::shared_ptr<dht::Value>>& values, bool expired) {
if (expired)
valueCounta -= values.size();
else
valueCounta += values.size();
return true;
});
auto ftokenb = node1.listen(b, [&](const std::shared_ptr<dht::Value>&) {
/*if (expired)
valueCountb -= values.size();
else
valueCountb += values.size();*/
valueCountb++;
return false;
});
auto ftokenc = node1.listen(c, [&](const std::vector<std::shared_ptr<dht::Value>>& values, bool expired) {
if (expired)
valueCountc -= values.size();
else
valueCountc += values.size();
return true;
});
auto ftokend = node1.listen(d, [&](const std::vector<std::shared_ptr<dht::Value>>& values, bool expired) {
if (expired)
valueCountd -= values.size();
else
valueCountd += values.size();
return true;
});
std::vector<uint8_t> mtu;
mtu.reserve(SZ);
for (size_t i = 0; i < SZ; i++)
mtu.emplace_back((i % 2) ? 'T' : 'M');
for (unsigned i = 0; i < N; i++) {
node2.put(a, dht::Value("v1"), [&](bool ok) {
std::lock_guard lock(mutex);
putCount++;
if (ok)
putOkCount1++;
cv.notify_all();
});
node2.put(b, dht::Value("v2"), [&](bool ok) {
std::lock_guard lock(mutex);
putCount++;
if (ok)
putOkCount2++;
cv.notify_all();
});
auto bigVal = std::make_shared<dht::Value>();
bigVal->data = mtu;
node2.put(c, std::move(bigVal), [&](bool ok) {
std::lock_guard lock(mutex);
putCount++;
if (ok)
putOkCount3++;
cv.notify_all();
});
}
{
std::unique_lock lk(mutex);
CPPUNIT_ASSERT(cv.wait_for(lk, 30s, [&] { return putCount == N * 3u; }));
CPPUNIT_ASSERT_EQUAL(N, putOkCount1);
CPPUNIT_ASSERT_EQUAL(N, putOkCount2);
CPPUNIT_ASSERT_EQUAL(N, putOkCount3);
}
CPPUNIT_ASSERT(ftokena.valid());
CPPUNIT_ASSERT(ftokenb.valid());
CPPUNIT_ASSERT(ftokenc.valid());
CPPUNIT_ASSERT(ftokend.valid());
auto tokena = ftokena.get();
auto tokenc = ftokenc.get();
auto tokend = ftokend.get();
// tokenb might be 0 since the callback returns false.
CPPUNIT_ASSERT(tokena);
CPPUNIT_ASSERT(tokenc);
CPPUNIT_ASSERT(tokend);
CPPUNIT_ASSERT_EQUAL(N, valueCounta.load());
CPPUNIT_ASSERT_EQUAL(1u, valueCountb.load());
CPPUNIT_ASSERT_EQUAL(N, valueCountc.load());
CPPUNIT_ASSERT_EQUAL(0u, valueCountd.load());
node1.cancelListen(a, tokena);
node1.cancelListen(b, std::move(ftokenb));
node1.cancelListen(c, tokenc);
node1.cancelListen(d, tokend);
}
void
DhtRunnerTester::testIdOps()
{
std::mutex mutex;
std::condition_variable cv;
unsigned identityCount(0);
unsigned valueCount(0);
unsigned valueCountEdit(0);
dht::DhtRunner::Config config2;
config2.dht_config.node_config.max_peer_req_per_sec = -1;
config2.dht_config.node_config.max_req_per_sec = -1;
config2.dht_config.id = dht::crypto::generateIdentity();
dht::DhtRunner::Context context2;
context2.identityAnnouncedCb = [&](bool ok) {
CPPUNIT_ASSERT(ok);
std::lock_guard lk(mutex);
identityCount++;
cv.notify_all();
};
node2.join();
node2.run(42232, config2, std::move(context2));
auto bound = node1.getBound();
if (bound.isUnspecified())
bound.setLoopback();
node2.bootstrap(bound);
{
std::unique_lock lk(mutex);
CPPUNIT_ASSERT(cv.wait_for(lk, 20s, [&] { return identityCount == 1; }));
}
node1.findCertificate(node2.getId(), [&](const std::shared_ptr<dht::crypto::Certificate>& crt) {
CPPUNIT_ASSERT(crt);
std::lock_guard lk(mutex);
valueCount++;
cv.notify_all();
});
{
std::unique_lock lk(mutex);
CPPUNIT_ASSERT(cv.wait_for(lk, 20s, [&] { return valueCount == 1u; }));
CPPUNIT_ASSERT_EQUAL(1u, identityCount);
}
dht::DhtRunner::Context context1;
context1.identityAnnouncedCb = [&](bool ok) {
CPPUNIT_ASSERT(ok);
std::lock_guard lk(mutex);
identityCount++;
cv.notify_all();
};
config2.dht_config.id = dht::crypto::generateIdentity();
node1.join();
node1.run(42222, config2, std::move(context1));
bound = node2.getBound();
if (bound.isUnspecified())
bound.setLoopback();
node1.bootstrap(bound);
auto key = dht::InfoHash::get("key");
node1.putEncrypted(key, node2.getId(), dht::Value("yo"), [&](bool ok) {
CPPUNIT_ASSERT(ok);
std::lock_guard lk(mutex);
valueCount++;
cv.notify_all();
});
node1.putEncrypted(key, node2.getPublicKey(), std::make_shared<dht::Value>("yo"), [&](bool ok) {
CPPUNIT_ASSERT(ok);
std::lock_guard lk(mutex);
valueCount++;
cv.notify_all();
});
node2.listen<std::string>(key, [&](std::string&& value) {
CPPUNIT_ASSERT_EQUAL("yo"s, value);
std::lock_guard lk(mutex);
valueCount++;
cv.notify_all();
return true;
});
auto key2 = dht::InfoHash::get("key2");
auto editValue = std::make_shared<dht::Value>("v1");
node1.putSigned(key2, editValue, [&](bool ok) {
CPPUNIT_ASSERT(ok);
std::lock_guard lk(mutex);
valueCountEdit++;
cv.notify_all();
});
node2.listen(key2, [&](const std::vector<std::shared_ptr<dht::Value>>& values, bool /*expired*/) {
for (const auto& v : values) {
if (v->seq == 0)
CPPUNIT_ASSERT_EQUAL("v1"s, dht::unpackMsg<std::string>(v->data));
else if (v->seq == 1)
CPPUNIT_ASSERT_EQUAL("v2"s, dht::unpackMsg<std::string>(v->data));
CPPUNIT_ASSERT_EQUAL(v->owner->getLongId(), node1.getPublicKey()->getLongId());
}
std::lock_guard lk(mutex);
valueCountEdit += values.size();
cv.notify_all();
return true;
});
{
std::unique_lock lk(mutex);
CPPUNIT_ASSERT(cv.wait_for(lk, 20s, [&] { return identityCount == 2u; }));
CPPUNIT_ASSERT(cv.wait_for(lk, 20s, [&] { return valueCount == 5u; }));
CPPUNIT_ASSERT(cv.wait_for(lk, 20s, [&] { return valueCountEdit == 2u; }));
}
node2.findCertificate(node1.getPublicKey()->getLongId(), [&](const std::shared_ptr<dht::crypto::Certificate>& crt) {
CPPUNIT_ASSERT(crt);
std::lock_guard lk(mutex);
valueCount++;
cv.notify_all();
});
{
std::unique_lock lk(mutex);
CPPUNIT_ASSERT(cv.wait_for(lk, 20s, [&] { return valueCount == 6u; }));
}
// editValue->data = dht::packMsg("v2");
editValue = std::make_shared<dht::Value>(editValue->id);
editValue->data = dht::packMsg("v2");
node1.putSigned(key2, editValue, [&](bool ok) {
CPPUNIT_ASSERT(ok);
std::lock_guard lk(mutex);
valueCountEdit++;
cv.notify_all();
});
std::unique_lock lk(mutex);
CPPUNIT_ASSERT(cv.wait_for(lk, 20s, [&] { return valueCountEdit == 4u; }));
}
void
DhtRunnerTester::testListenLotOfBytes()
{
std::mutex mutex;
std::condition_variable cv;
std::atomic_uint valueCount(0);
unsigned putCount(0);
unsigned putOkCount(0);
std::string data(10000, 'a');
auto foo = dht::InfoHash::get("foo");
constexpr unsigned N = 1024;
for (unsigned i = 0; i < N; i++) {
node2.put(foo, data, [&](bool ok) {
std::lock_guard lock(mutex);
putCount++;
if (ok)
putOkCount++;
cv.notify_all();
});
}
{
std::unique_lock lk(mutex);
CPPUNIT_ASSERT(cv.wait_for(lk, 30s, [&] { return putCount == N; }));
}
dht::DhtRunner node3 {};
dht::DhtRunner::Config config;
config.dht_config.node_config.max_peer_req_per_sec = -1;
config.dht_config.node_config.max_req_per_sec = -1;
node3.run(42242, config);
auto bound = node1.getBound();
if (bound.isUnspecified())
bound.setLoopback();
node3.bootstrap(bound);
auto ftokenfoo = node3.listen(foo, [&](const std::shared_ptr<dht::Value>&) {
valueCount++;
cv.notify_all();
return true;
});
{
std::unique_lock lk(mutex);
CPPUNIT_ASSERT(cv.wait_for(lk, 30s, [&] { return valueCount == N; }));
}
node3.cancelListen(foo, ftokenfoo.get());
}
void
DhtRunnerTester::testMultithread()
{
std::mutex mutex;
std::condition_variable cv;
unsigned putCount(0);
unsigned putOkCount(0);
constexpr unsigned N = 2048;
for (unsigned i = 0; i < N; i++) {
dht::ThreadPool::computation().run([&] {
node2.put(dht::InfoHash::get("123" + std::to_string(i)), "hehe", [&](bool ok) {
std::lock_guard lock(mutex);
putCount++;
if (ok)
putOkCount++;
cv.notify_all();
});
node2.get(
dht::InfoHash::get("123" + std::to_string(N - i - 1)),
[](const std::shared_ptr<dht::Value>&) { return true; },
[&](bool ok) {
std::lock_guard lock(mutex);
putCount++;
if (ok)
putOkCount++;
cv.notify_all();
});
});
}
std::unique_lock lk(mutex);
CPPUNIT_ASSERT(cv.wait_for(lk, 30s, [&] { return putCount == 2 * N; }));
CPPUNIT_ASSERT_EQUAL(2 * N, putOkCount);
}
} // namespace test
@@ -1,7 +1,7 @@
// Copyright (c) 2014-2026 Savoir-faire Linux Inc.
// SPDX-License-Identifier: MIT
#include "cryptotester.h"
#include "test_crypto.h"
#include <opendht/crypto.h>
@@ -59,6 +59,24 @@ CryptoTester::testSignatureEncryption()
}
}
void
CryptoTester::testPublicKeyPemIsPlainText()
{
auto key = dht::crypto::PrivateKey::generate();
auto pem = key.getPublicKey().toString();
// The exported key is PEM text: it must not carry the terminating null
// byte GnuTLS counts in the exported size, which would otherwise be
// embedded in anything the string is serialized into.
CPPUNIT_ASSERT(not pem.empty());
CPPUNIT_ASSERT_EQUAL(std::string::npos, pem.find('\0'));
CPPUNIT_ASSERT_EQUAL('\n', pem.back());
// The exported text still reads back as the same key.
dht::crypto::PublicKey reloaded(dht::Blob(pem.begin(), pem.end()));
CPPUNIT_ASSERT_EQUAL(key.getPublicKey().getId(), reloaded.getId());
}
void
CryptoTester::testCertificateRevocation()
{
@@ -95,10 +113,159 @@ CryptoTester::testCertificateRevocation()
CPPUNIT_ASSERT_MESSAGE(v.toString(), v);
}
namespace {
/** Read a CRL Number extension blob (big-endian, variable length) as an integer. */
uint64_t
crlNumberValue(const dht::Blob& number)
{
uint64_t value {0};
for (uint8_t byte : number)
value = (value << 8) | byte;
return value;
}
} // namespace
void
CryptoTester::testRevocationListNumber()
{
auto ca = dht::crypto::generateIdentity("ca");
auto device = dht::crypto::generateIdentity("dev", ca);
dht::crypto::RevocationList crl;
CPPUNIT_ASSERT_EQUAL(uint64_t(0), crlNumberValue(crl.getNumber()));
crl.revoke(*device.second);
crl.sign(ca);
auto first = crl.getNumber();
// An unnumbered list gets a random number on its first signature.
CPPUNIT_ASSERT(not first.empty());
CPPUNIT_ASSERT(crlNumberValue(first) != 0);
// Re-signing keeps the same lineage and moves the number forward.
crl.sign(ca);
auto second = crl.getNumber();
CPPUNIT_ASSERT_EQUAL(crlNumberValue(first) + 1, crlNumberValue(second));
crl.sign(ca);
CPPUNIT_ASSERT_EQUAL(crlNumberValue(first) + 2, crlNumberValue(crl.getNumber()));
// The number is a DER integer, so it is stored with leading zeros stripped.
// Incrementing must still work for numbers shorter than 8 bytes.
for (uint64_t start : {uint64_t(1), uint64_t(0x7f), uint64_t(0x80), uint64_t(0xffff)}) {
dht::crypto::RevocationList small;
small.revoke(*device.second);
small.sign(ca, {}, start);
CPPUNIT_ASSERT_EQUAL(start, crlNumberValue(small.getNumber()));
small.sign(ca);
CPPUNIT_ASSERT_EQUAL(start + 1, crlNumberValue(small.getNumber()));
small.sign(ca);
CPPUNIT_ASSERT_EQUAL(start + 2, crlNumberValue(small.getNumber()));
}
}
void
CryptoTester::testRevocationListExplicitNumber()
{
auto ca = dht::crypto::generateIdentity("ca");
auto device = dht::crypto::generateIdentity("dev", ca);
// Values chosen to cover leading zero bytes, a full-width number and the largest
// value that still encodes as a positive 8-byte DER integer.
const std::vector<uint64_t> numbers {1,
42,
0xff,
0x100,
0x0000'0001'0000'0000ull,
0x0000'02a1'0000'0001ull,
0x7fff'ffff'ffff'ffffull};
for (auto number : numbers) {
dht::crypto::RevocationList crl;
crl.revoke(*device.second);
crl.sign(ca, {}, number);
CPPUNIT_ASSERT_EQUAL(number, crlNumberValue(crl.getNumber()));
CPPUNIT_ASSERT(crl.isSignedBy(*ca.second));
CPPUNIT_ASSERT(crl.isRevoked(*device.second));
// The number must survive a pack/unpack round-trip.
dht::crypto::RevocationList unpacked(crl.getPacked());
CPPUNIT_ASSERT_EQUAL(number, crlNumberValue(unpacked.getNumber()));
}
// An explicit number overrides the increment, in both directions.
dht::crypto::RevocationList crl;
crl.revoke(*device.second);
crl.sign(ca, {}, 1000);
CPPUNIT_ASSERT_EQUAL(uint64_t(1000), crlNumberValue(crl.getNumber()));
crl.sign(ca, {}, 5);
CPPUNIT_ASSERT_EQUAL(uint64_t(5), crlNumberValue(crl.getNumber()));
// Passing 0 falls back to the default behavior: increment the current number.
crl.sign(ca, {}, 0);
CPPUNIT_ASSERT_EQUAL(uint64_t(6), crlNumberValue(crl.getNumber()));
}
void
CryptoTester::testRevocationListPartitionedNumber()
{
// Two devices holding the same authority issue revocation lists independently.
// RFC 5280 5.2.3 requires CRL numbers to increase monotonically for a given CRL
// scope and issuer; partitioning the low bits by device keeps each sequence
// monotonic while making collisions between devices impossible.
constexpr unsigned TAG_BITS = 24;
constexpr uint64_t TAG_A = 0x0a0b0c;
constexpr uint64_t TAG_B = 0x0a0b0d;
auto ca = dht::crypto::generateIdentity("ca");
auto account = dht::crypto::generateIdentity("acc", ca, 4096, true);
auto deviceA = dht::crypto::generateIdentity("devA", account);
auto deviceB = dht::crypto::generateIdentity("devB", account);
uint64_t previous {0};
dht::crypto::RevocationList crlA;
for (unsigned i = 1; i <= 4; i++) {
crlA.revoke(*deviceA.second);
crlA.sign(account, {}, (uint64_t(i) << TAG_BITS) | TAG_A);
auto number = crlNumberValue(crlA.getNumber());
CPPUNIT_ASSERT_EQUAL(TAG_A, number & ((uint64_t(1) << TAG_BITS) - 1));
CPPUNIT_ASSERT(number > previous);
CPPUNIT_ASSERT_EQUAL(uint64_t(0), number >> 63);
previous = number;
}
// Both devices branch from the same counter but never mint the same number.
dht::crypto::RevocationList branchA;
branchA.revoke(*deviceA.second);
branchA.sign(account, {}, (uint64_t(9) << TAG_BITS) | TAG_A);
dht::crypto::RevocationList branchB;
branchB.revoke(*deviceB.second);
branchB.sign(account, {}, (uint64_t(9) << TAG_BITS) | TAG_B);
CPPUNIT_ASSERT(crlNumberValue(branchA.getNumber()) != crlNumberValue(branchB.getNumber()));
// Both are valid and revoke only their own target.
CPPUNIT_ASSERT(branchA.isSignedBy(*account.second));
CPPUNIT_ASSERT(branchB.isSignedBy(*account.second));
CPPUNIT_ASSERT(branchA.isRevoked(*deviceA.second));
CPPUNIT_ASSERT(not branchA.isRevoked(*deviceB.second));
CPPUNIT_ASSERT(branchB.isRevoked(*deviceB.second));
CPPUNIT_ASSERT(not branchB.isRevoked(*deviceA.second));
// A number minted by an older client is fully random, so bit 63 may be set. Masking
// it out and taking the counter part still yields a strictly greater number.
constexpr uint64_t legacy = 0xdead'beef'0123'4567ull;
dht::crypto::RevocationList legacyCrl;
legacyCrl.revoke(*deviceA.second);
legacyCrl.sign(account, {}, legacy);
auto counter = ((crlNumberValue(legacyCrl.getNumber()) & ~(uint64_t(1) << 63)) >> TAG_BITS) + 1;
legacyCrl.sign(account, {}, (counter << TAG_BITS) | TAG_A);
CPPUNIT_ASSERT(crlNumberValue(legacyCrl.getNumber()) > (legacy & ~(uint64_t(1) << 63)));
CPPUNIT_ASSERT_EQUAL(uint64_t(0), crlNumberValue(legacyCrl.getNumber()) >> 63);
}
void
CryptoTester::testCertificateRequest()
{
// Generate CA
{ // Generate CA
auto ca = dht::crypto::generateIdentity("Test CA");
// Generate signed request
@@ -149,6 +316,25 @@ CryptoTester::testCertificateSerialNumber()
};
auto serial = dht::crypto::Certificate(cert_pem).getSerialNumber();
CPPUNIT_ASSERT(std::equal(SERIAL.begin(), SERIAL.end(), serial.begin(), serial.end()));
auto key = dht::crypto::PrivateKey::generateEC();
for (unsigned i = 0; i < 32; i++) {
auto generated = dht::crypto::Certificate::generate(key, "generated", {});
auto generatedSerial = generated.getSerialNumber();
CPPUNIT_ASSERT(not generatedSerial.empty());
CPPUNIT_ASSERT_EQUAL(uint8_t {0}, uint8_t(generatedSerial.front() & 0x80));
CPPUNIT_ASSERT(std::any_of(generatedSerial.begin(), generatedSerial.end(), [](uint8_t byte) { return byte != 0; }));
}
}
void
CryptoTester::testEcKeyCurve()
{
auto key = dht::crypto::PrivateKey::generateEC(GNUTLS_ECC_CURVE_SECP256R1);
unsigned int bits = 0;
CPPUNIT_ASSERT_EQUAL(static_cast<int>(GNUTLS_PK_EC),
gnutls_x509_privkey_get_pk_algorithm2(key.x509_key, &bits));
CPPUNIT_ASSERT_EQUAL(256u, bits);
}
void
@@ -12,9 +12,14 @@ class CryptoTester : public CppUnit::TestFixture
{
CPPUNIT_TEST_SUITE(CryptoTester);
CPPUNIT_TEST(testSignatureEncryption);
CPPUNIT_TEST(testPublicKeyPemIsPlainText);
CPPUNIT_TEST(testCertificateRevocation);
CPPUNIT_TEST(testRevocationListNumber);
CPPUNIT_TEST(testRevocationListExplicitNumber);
CPPUNIT_TEST(testRevocationListPartitionedNumber);
CPPUNIT_TEST(testCertificateRequest);
CPPUNIT_TEST(testCertificateSerialNumber);
CPPUNIT_TEST(testEcKeyCurve);
CPPUNIT_TEST(testOcsp);
CPPUNIT_TEST(testAesEncryption);
CPPUNIT_TEST(testAesEncryptionWithMultipleKeySizes);
@@ -36,10 +41,26 @@ public:
* Test data signature, encryption and decryption
*/
void testSignatureEncryption();
/**
* Test that an exported public key is text only, with no embedded null byte
*/
void testPublicKeyPemIsPlainText();
/**
* Test certificate generation, validation and revocation
*/
void testCertificateRevocation();
/**
* Test the default CRL Number behavior: random on first signature, incremented after
*/
void testRevocationListNumber();
/**
* Test signing a revocation list with a caller-provided CRL Number
*/
void testRevocationListExplicitNumber();
/**
* Test that issuers sharing an authority can partition the CRL Number space
*/
void testRevocationListPartitionedNumber();
/**
* Test certificate requests
*/
@@ -48,6 +69,10 @@ public:
* Test certificate serial number extraction
*/
void testCertificateSerialNumber();
/**
* Test EC private key generation with an explicit curve
*/
void testEcKeyCurve();
/**
* Test OCSP
*/
@@ -1,7 +1,7 @@
// Copyright (c) 2014-2026 Savoir-faire Linux Inc.
// SPDX-License-Identifier: MIT
#include "dhtproxytester.h"
#include "test_dhtproxy.h"
// std
#include <iostream>
@@ -57,11 +57,11 @@ DhtProxyTester::tearDown()
std::condition_variable cv;
std::mutex cv_m;
nodeProxy->shutdown([&] {
std::lock_guard<std::mutex> lk(cv_m);
std::lock_guard lk(cv_m);
done = true;
cv.notify_all();
});
std::unique_lock<std::mutex> lk(cv_m);
std::unique_lock lk(cv_m);
CPPUNIT_ASSERT(cv.wait_for(lk, 15s, [&] { return done; }));
serverProxy.reset();
nodeProxy.reset();
@@ -81,11 +81,11 @@ DhtProxyTester::testGetPut()
auto val_data = val.data;
{
nodePeer.put(key, std::move(val), [&](bool) {
std::lock_guard<std::mutex> lk(cv_m);
std::lock_guard lk(cv_m);
done = true;
cv.notify_all();
});
std::unique_lock<std::mutex> lk(cv_m);
std::unique_lock lk(cv_m);
CPPUNIT_ASSERT(cv.wait_for(lk, 10s, [&] { return done; }));
}
@@ -101,7 +101,7 @@ DhtProxyTester::testListen()
std::condition_variable cv;
std::mutex cv_m;
std::unique_lock<std::mutex> lk(cv_m);
std::unique_lock lk(cv_m);
auto key = dht::InfoHash::get("GLaDOs");
bool done = false;
@@ -111,7 +111,7 @@ DhtProxyTester::testListen()
auto firstVal_data = firstVal.data;
nodePeer.put(key, std::move(firstVal), [&](bool ok) {
CPPUNIT_ASSERT(ok);
std::lock_guard<std::mutex> lk(cv_m);
std::lock_guard lk(cv_m);
done = true;
cv.notify_all();
});
@@ -121,7 +121,7 @@ DhtProxyTester::testListen()
std::vector<dht::Blob> values;
auto token = nodeClient.listen(key, [&](const std::vector<std::shared_ptr<dht::Value>>& v, bool expired) {
if (not expired) {
std::lock_guard<std::mutex> lk(cv_m);
std::lock_guard lk(cv_m);
for (const auto& value : v)
values.emplace_back(value->data);
done = true;
@@ -156,7 +156,7 @@ DhtProxyTester::testResubscribeGetValues()
bool done = false;
std::condition_variable cv;
std::mutex cv_m;
std::unique_lock<std::mutex> lk(cv_m);
std::unique_lock lk(cv_m);
auto key = dht::InfoHash::get("GLaDOs");
// If a peer sent a value, the listen operation from the client
@@ -164,7 +164,7 @@ DhtProxyTester::testResubscribeGetValues()
dht::Value firstVal {"Hey! It's been a long time. How have you been?"};
auto firstVal_data = firstVal.data;
nodePeer.put(key, std::move(firstVal), [&](bool ok) {
std::lock_guard<std::mutex> lk(cv_m);
std::lock_guard lk(cv_m);
CPPUNIT_ASSERT(ok);
done = true;
cv.notify_all();
@@ -186,7 +186,7 @@ DhtProxyTester::testResubscribeGetValues()
std::vector<std::shared_ptr<dht::Value>> values;
auto ftoken = nodeClient.listen(key, [&](const std::vector<std::shared_ptr<dht::Value>>& v, bool expired) {
if (not expired) {
std::lock_guard<std::mutex> lk(cv_m);
std::lock_guard lk(cv_m);
values.insert(values.end(), v.begin(), v.end());
done = true;
cv.notify_all();
@@ -218,14 +218,14 @@ DhtProxyTester::testPutGet40KChars()
mtu.emplace_back((i % 2) ? 'T' : 'M');
std::condition_variable cv;
std::mutex cv_m;
std::unique_lock<std::mutex> lk(cv_m);
std::unique_lock lk(cv_m);
bool done_put = false;
bool done_get = false;
// Act
dht::Value val {mtu};
nodePeer.put(key, std::move(val), [&](bool ok) {
std::lock_guard<std::mutex> lk(cv_m);
std::lock_guard lk(cv_m);
done_put = ok;
cv.notify_all();
});
@@ -238,7 +238,7 @@ DhtProxyTester::testPutGet40KChars()
return true;
},
[&](bool ok) {
std::lock_guard<std::mutex> lk(cv_m);
std::lock_guard lk(cv_m);
done_get = ok;
cv.notify_all();
});
@@ -316,12 +316,12 @@ DhtProxyTester::testShutdownStop()
std::mutex cv_m;
nodeTest->shutdown(
[&] {
std::lock_guard<std::mutex> lk(cv_m);
std::lock_guard lk(cv_m);
done = true;
cv.notify_all();
},
true);
std::unique_lock<std::mutex> lk(cv_m);
std::unique_lock lk(cv_m);
CPPUNIT_ASSERT(cv.wait_for(lk, 10s, [&] { return done; }));
}
CPPUNIT_ASSERT_EQUAL(2 * C, callback_count.load());
File renamed without changes.
+93
View File
@@ -0,0 +1,93 @@
// Copyright (c) 2014-2026 Savoir-faire Linux Inc.
// SPDX-License-Identifier: MIT
#include "test_dhtproxy_client.h"
#include <any>
#include <mutex>
#if defined(__clang__)
#pragma clang diagnostic push
#pragma clang diagnostic ignored "-Wkeyword-macro"
#elif defined(__GNUC__)
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wpedantic"
#endif
#define private public
#include "opendht/http.h"
#include "opendht/dht_proxy_client.h"
#undef private
#if defined(__clang__)
#pragma clang diagnostic pop
#elif defined(__GNUC__)
#pragma GCC diagnostic pop
#endif
#include "../src/op_cache.cpp"
#include "../src/dht_proxy_client.cpp"
namespace test {
CPPUNIT_TEST_SUITE_REGISTRATION(DhtProxyClientTester);
using namespace dht;
void
DhtProxyClientTester::testResubscribeUsesKeyRoute()
{
#ifdef OPENDHT_PUSH_NOTIFICATIONS
DhtProxyClient client({}, {}, [] {}, "http://127.0.0.1:8080", "OpenDHT-Test", "client-id", "push-token");
DhtProxyClient::Listener listener {OpValueCache([](const std::vector<Sp<Value>>&, bool) { return true; })};
listener.opstate = std::make_shared<DhtProxyClient::OperationState>();
listener.cb = [](const std::vector<Sp<Value>>&, bool, system_clock::time_point) {
return true;
};
auto key = InfoHash::get("proxy-client-resubscribe");
client.resubscribe(key, 1, listener);
CPPUNIT_ASSERT(listener.request);
CPPUNIT_ASSERT_EQUAL(std::string("/key/") + key.toString(), std::string(listener.request->header_.request_target()));
CPPUNIT_ASSERT(listener.request->body_.find("\"refresh\"") != std::string::npos);
CPPUNIT_ASSERT(listener.request->body_.find("true") != std::string::npos);
#else
CPPUNIT_ASSERT(true);
#endif
}
void
DhtProxyClientTester::testSetPushNotificationTokenResubscribesWithNewToken()
{
#ifdef OPENDHT_PUSH_NOTIFICATIONS
DhtProxyClient client({}, {}, [] {}, "http://127.0.0.1:8080", "OpenDHT-Test", "client-id", "old-token");
client.statusIpv4_ = NodeStatus::Connected;
auto key = InfoHash::get("proxy-client-token-rotation");
auto& search = client.searches_[key];
auto [it, inserted] = search.listeners.emplace(std::piecewise_construct,
std::forward_as_tuple(1),
std::forward_as_tuple(OpValueCache(
[](const std::vector<Sp<Value>>&, bool) { return true; })));
CPPUNIT_ASSERT(inserted);
auto& listener = it->second;
listener.opstate = std::make_shared<DhtProxyClient::OperationState>();
listener.cb = [](const std::vector<Sp<Value>>&, bool, system_clock::time_point) {
return true;
};
client.setPushNotificationToken("new-token");
CPPUNIT_ASSERT(listener.request);
CPPUNIT_ASSERT_EQUAL(std::string("/key/") + key.toString(), std::string(listener.request->header_.request_target()));
CPPUNIT_ASSERT(listener.request->body_.find("new-token") != std::string::npos);
CPPUNIT_ASSERT(listener.request->body_.find("old-token") == std::string::npos);
CPPUNIT_ASSERT(listener.request->body_.find("\"refresh\"") != std::string::npos);
CPPUNIT_ASSERT(listener.request->body_.find("true") != std::string::npos);
#else
CPPUNIT_ASSERT(true);
#endif
}
} // namespace test
+22
View File
@@ -0,0 +1,22 @@
// Copyright (c) 2014-2026 Savoir-faire Linux Inc.
// SPDX-License-Identifier: MIT
#pragma once
#include <cppunit/TestFixture.h>
#include <cppunit/extensions/HelperMacros.h>
namespace test {
class DhtProxyClientTester : public CppUnit::TestFixture
{
CPPUNIT_TEST_SUITE(DhtProxyClientTester);
CPPUNIT_TEST(testResubscribeUsesKeyRoute);
CPPUNIT_TEST(testSetPushNotificationTokenResubscribesWithNewToken);
CPPUNIT_TEST_SUITE_END();
public:
void testResubscribeUsesKeyRoute();
void testSetPushNotificationTokenResubscribesWithNewToken();
};
} // namespace test
@@ -1,7 +1,7 @@
// Copyright (c) 2014-2026 Savoir-faire Linux Inc.
// SPDX-License-Identifier: MIT
#include "dhtproxy_stress_tester.h"
#include "test_dhtproxy_stress.h"
#include <iostream>
#include <string>
@@ -51,11 +51,11 @@ DhtProxyStressTester::tearDown()
std::condition_variable cv;
std::mutex cv_m;
nodeProxy->shutdown([&] {
std::lock_guard<std::mutex> lk(cv_m);
std::lock_guard lk(cv_m);
done = true;
cv.notify_all();
});
std::unique_lock<std::mutex> lk(cv_m);
std::unique_lock lk(cv_m);
CPPUNIT_ASSERT(cv.wait_for(lk, 15s, [&] { return done; }));
serverProxy.reset();
nodeProxy.reset();
@@ -70,7 +70,7 @@ DhtProxyStressTester::testRepeatValues()
std::condition_variable cv;
std::mutex cv_m;
std::unique_lock<std::mutex> lk(cv_m);
std::unique_lock lk(cv_m);
auto key = dht::InfoHash::get("GLaDOs");
bool done = false;
@@ -80,7 +80,7 @@ DhtProxyStressTester::testRepeatValues()
auto firstVal_data = firstVal.data;
nodePeer.put(key, std::move(firstVal), [&](bool ok) {
CPPUNIT_ASSERT(ok);
std::lock_guard<std::mutex> lk(cv_m);
std::lock_guard lk(cv_m);
done = true;
cv.notify_all();
});
@@ -90,7 +90,7 @@ DhtProxyStressTester::testRepeatValues()
std::vector<dht::Blob> values;
auto token = nodeClient.listen(key, [&](const std::vector<std::shared_ptr<dht::Value>>& v, bool expired) {
if (not expired) {
std::lock_guard<std::mutex> lk(cv_m);
std::lock_guard lk(cv_m);
for (const auto& value : v)
values.emplace_back(value->data);
done = true;
@@ -132,7 +132,7 @@ DhtProxyStressTester::testRepeatValues()
val->id = id++;
nodePeer.put(key, val, [&](bool ok) {
CPPUNIT_ASSERT(ok);
std::lock_guard<std::mutex> lk(cv_m);
std::lock_guard lk(cv_m);
done = true;
cv.notify_all();
});
@@ -145,7 +145,7 @@ DhtProxyStressTester::testRepeatValues()
if (not expired) {
for (const auto& value : v) {
if (value->id == val->id) {
std::lock_guard<std::mutex> lk(cv_m);
std::lock_guard lk(cv_m);
done = true;
cv.notify_all();
break;
File renamed without changes.
+936
View File
@@ -0,0 +1,936 @@
// Copyright (c) 2014-2026 Savoir-faire Linux Inc.
// SPDX-License-Identifier: MIT
#include "test_dhtrunner.h"
#include <opendht/thread_pool.h>
#include <chrono>
#include <future>
#include <mutex>
#include <condition_variable>
#include <fstream>
#include <cstdio>
using namespace std::chrono_literals;
using namespace std::literals;
namespace test {
CPPUNIT_TEST_SUITE_REGISTRATION(DhtRunnerTester);
template<typename T>
T
getFutureValue(std::future<T> future, std::chrono::steady_clock::duration timeout = 30s)
{
CPPUNIT_ASSERT(std::future_status::ready == future.wait_for(timeout));
return future.get();
}
void
DhtRunnerTester::setUp()
{
dht::DhtRunner::Config config;
config.dht_config.node_config.max_peer_req_per_sec = -1;
config.dht_config.node_config.max_req_per_sec = -1;
config.dht_config.node_config.max_store_size = -1;
config.dht_config.node_config.max_store_keys = -1;
node1.run(0, config);
node2.run(0, config);
auto bound = node1.getBound();
if (bound.isUnspecified())
bound.setLoopback();
node2.bootstrap(bound);
}
void
DhtRunnerTester::tearDown()
{
unsigned done {0};
std::condition_variable cv;
std::mutex cv_m;
auto shutdown = [&] {
std::lock_guard lk(cv_m);
done++;
cv.notify_all();
};
node1.shutdown(shutdown);
node2.shutdown(shutdown);
std::unique_lock lk(cv_m);
CPPUNIT_ASSERT(cv.wait_for(lk, 30s, [&] { return done == 2u; }));
node1.join();
node2.join();
}
void
DhtRunnerTester::testConstructors()
{
CPPUNIT_ASSERT(node1.getBoundPort());
CPPUNIT_ASSERT_EQUAL(node1.getBoundPort(), node1.getBound().getPort());
CPPUNIT_ASSERT(node2.getBoundPort());
CPPUNIT_ASSERT_EQUAL(node2.getBoundPort(), node2.getBound().getPort());
dht::DhtRunner::Config config {};
dht::DhtRunner::Context context {};
dht::DhtRunner testNode;
testNode.run(config, std::move(context));
CPPUNIT_ASSERT(testNode.getBoundPort());
}
void
DhtRunnerTester::testGetPut()
{
auto key = dht::InfoHash::get("123");
dht::Value val {"hey"};
auto val_data = val.data;
std::promise<bool> p;
auto future = p.get_future();
node2.put(key, std::move(val), [&](bool ok) { p.set_value(ok); });
CPPUNIT_ASSERT(getFutureValue(std::move(future)));
auto vals = getFutureValue(node1.get(key));
CPPUNIT_ASSERT(not vals.empty());
CPPUNIT_ASSERT(vals.front()->data == val_data);
}
void
DhtRunnerTester::testPutDuplicate()
{
auto key = dht::InfoHash::get("123");
auto val = std::make_shared<dht::Value>("hey");
val->id = 42;
auto val_data = val->data;
std::promise<bool> p1;
std::promise<bool> p2;
auto p1future = p1.get_future();
auto p2future = p2.get_future();
node2.put(key, val, [&](bool ok) { p1.set_value(ok); });
node2.put(key, val, [&](bool ok) { p2.set_value(ok); });
auto p1ret = getFutureValue(std::move(p1future));
auto p2ret = getFutureValue(std::move(p2future));
CPPUNIT_ASSERT(p1ret);
CPPUNIT_ASSERT(p2ret);
auto vals = getFutureValue(node1.get(key));
CPPUNIT_ASSERT(not vals.empty());
CPPUNIT_ASSERT(vals.size() == 1);
CPPUNIT_ASSERT(vals.front()->data == val_data);
}
void
DhtRunnerTester::testPutOverride()
{
auto key = dht::InfoHash::get("123");
auto val = std::make_shared<dht::Value>("meh");
val->id = 42;
auto val2 = std::make_shared<dht::Value>("hey");
val2->id = 42;
CPPUNIT_ASSERT_EQUAL(val->id, val2->id);
auto val_data = val2->data;
std::promise<bool> p1;
std::promise<bool> p2;
auto p1future = p1.get_future();
auto p2future = p2.get_future();
node2.put(key, val, [&](bool ok) { p1.set_value(ok); });
node2.put(key, val2, [&](bool ok) { p2.set_value(ok); });
auto p1ret = getFutureValue(std::move(p1future));
auto p2ret = getFutureValue(std::move(p2future));
CPPUNIT_ASSERT(!p1ret);
CPPUNIT_ASSERT(p2ret);
auto vals = getFutureValue(node1.get(key));
CPPUNIT_ASSERT(not vals.empty());
CPPUNIT_ASSERT(vals.size() == 1);
CPPUNIT_ASSERT(vals.front()->data == val_data);
}
void
DhtRunnerTester::testImportValuesPreservesRemoteQuota()
{
node1.shutdown();
node1.join();
node2.shutdown();
node2.join();
dht::DhtRunner::Config config;
config.dht_config.node_config.max_peer_req_per_sec = -1;
config.dht_config.node_config.max_req_per_sec = -1;
config.dht_config.node_config.max_store_size = 1536;
config.dht_config.node_config.max_local_store_size = -1;
config.dht_config.node_config.max_store_keys = -1;
node1.run(0, config);
node2.run(0, config);
auto bound = node1.getBound();
if (bound.isUnspecified())
bound.setLoopback();
node2.bootstrap(bound);
std::this_thread::sleep_for(1s);
auto key1 = dht::InfoHash::get("import-quota-1");
auto key2 = dht::InfoHash::get("import-quota-2");
auto value1 = std::make_shared<dht::Value>(std::string(1024, 'a'));
auto value2 = std::make_shared<dht::Value>(std::string(1024, 'b'));
std::promise<bool> p1;
auto p1future = p1.get_future();
node2.put(key1, value1, [&](bool ok) { p1.set_value(ok); });
CPPUNIT_ASSERT(getFutureValue(std::move(p1future)));
std::this_thread::sleep_for(2s);
auto exported = node1.exportValues();
CPPUNIT_ASSERT(!exported.empty());
node1.shutdown();
node1.join();
dht::DhtRunner restored;
restored.run(0, config);
auto restoredBound = restored.getBound();
if (restoredBound.isUnspecified())
restoredBound.setLoopback();
node2.shutdown();
node2.join();
node2.run(0, config);
node2.bootstrap(restoredBound);
std::this_thread::sleep_for(1s);
restored.importValues(exported);
std::this_thread::sleep_for(500ms);
std::promise<bool> p2;
auto p2future = p2.get_future();
node2.put(key2, value2, [&](bool ok) { p2.set_value(ok); });
CPPUNIT_ASSERT(getFutureValue(std::move(p2future)));
std::this_thread::sleep_for(2s);
auto vals1 = getFutureValue(restored.get(key1));
auto vals2 = getFutureValue(restored.get(key2));
CPPUNIT_ASSERT(vals1.size() + vals2.size() <= 1);
restored.shutdown();
restored.join();
}
void
DhtRunnerTester::testExportValuesUsesSystemTime()
{
auto key = dht::InfoHash::get("export-system-time");
const auto created = dht::clock::now();
msgpack::sbuffer importBuffer;
msgpack::packer<msgpack::sbuffer> pk(&importBuffer);
pk.pack_array(1);
pk.pack_array(2);
pk.pack(created.time_since_epoch().count());
dht::Value {std::string("persisted")}.msgpack_pack(pk);
node1.importValues({
dht::ValuesExport {key, {importBuffer.data(), importBuffer.data() + importBuffer.size()}}
});
auto exported = node1.exportValues();
CPPUNIT_ASSERT_EQUAL(size_t(1), exported.size());
msgpack::unpacked msg;
msgpack::unpack(msg, reinterpret_cast<const char*>(exported.front().second.data()), exported.front().second.size());
const auto values = msg.get();
CPPUNIT_ASSERT(values.type == msgpack::type::ARRAY);
CPPUNIT_ASSERT_EQUAL(uint32_t(1), values.via.array.size);
const auto& persistedValue = values.via.array.ptr[0];
CPPUNIT_ASSERT_EQUAL(uint32_t(4), persistedValue.via.array.size);
CPPUNIT_ASSERT(persistedValue.via.array.ptr[0].type == msgpack::type::EXT);
CPPUNIT_ASSERT(persistedValue.via.array.ptr[3].type == msgpack::type::EXT);
const auto persistedCreated = persistedValue.via.array.ptr[0].as<std::chrono::system_clock::time_point>();
const auto persistedExpiration = persistedValue.via.array.ptr[3].as<std::chrono::system_clock::time_point>();
const auto delta = std::chrono::system_clock::now() - persistedCreated;
CPPUNIT_ASSERT(delta >= std::chrono::system_clock::duration::zero());
CPPUNIT_ASSERT(delta < 5s);
CPPUNIT_ASSERT(persistedExpiration > persistedCreated);
}
void
DhtRunnerTester::testImportValuesPreservesStoredExpiration()
{
static const dht::ValueType SHORT_LIVED_TYPE {200, "Short lived", 2s};
node1.registerInsecureType(SHORT_LIVED_TYPE);
auto key = dht::InfoHash::get("import-expiration");
auto now = dht::clock::now();
auto created = now;
auto expiration = now + SHORT_LIVED_TYPE.expiration;
msgpack::sbuffer buffer;
msgpack::packer<msgpack::sbuffer> pk(&buffer);
pk.pack_array(1);
pk.pack_array(4);
pk.pack(created.time_since_epoch().count());
dht::Value {SHORT_LIVED_TYPE.id, std::string("short-lived")}.msgpack_pack(pk);
pk.pack_bin(0);
pk.pack(expiration.time_since_epoch().count());
node1.importValues({
dht::ValuesExport {key, {buffer.data(), buffer.data() + buffer.size()}}
});
auto exported = node1.exportValues();
CPPUNIT_ASSERT(!exported.empty());
dht::DhtRunner restored;
dht::DhtRunner::Config config;
config.dht_config.node_config.max_peer_req_per_sec = -1;
config.dht_config.node_config.max_req_per_sec = -1;
config.dht_config.node_config.max_store_size = -1;
config.dht_config.node_config.max_store_keys = -1;
restored.run(0, config);
restored.importValues(exported);
CPPUNIT_ASSERT(!restored.exportValues().empty());
std::this_thread::sleep_for(3s);
auto restoredValues = restored.exportValues();
CPPUNIT_ASSERT_EQUAL(size_t(1), restoredValues.size());
msgpack::unpacked msg;
msgpack::unpack(msg, (const char*) restoredValues.front().second.data(), restoredValues.front().second.size());
auto valarr = msg.get();
CPPUNIT_ASSERT(valarr.type == msgpack::type::ARRAY);
CPPUNIT_ASSERT_EQUAL(uint32_t(0), valarr.via.array.size);
restored.shutdown();
restored.join();
}
void
DhtRunnerTester::testImportPermanentValueExpires()
{
static const dht::ValueType SHORT_TYPE {201, "Short perm", 500ms};
dht::DhtRunner::Config config;
config.dht_config.node_config.max_peer_req_per_sec = -1;
config.dht_config.node_config.max_req_per_sec = -1;
config.dht_config.node_config.max_store_size = -1;
config.dht_config.node_config.max_store_keys = -1;
dht::DhtRunner restored;
restored.run(0, config);
restored.registerInsecureType(SHORT_TYPE);
// Ensure the runner is waiting on a previously computed scheduler deadline.
std::promise<void> loopCompleted;
auto loopCompletedFuture = loopCompleted.get_future();
restored.getNodeInfo([&loopCompleted](auto) { loopCompleted.set_value(); });
getFutureValue(std::move(loopCompletedFuture));
auto key = dht::InfoHash::get("import-permanent-expiration");
auto now = dht::clock::now();
msgpack::sbuffer buffer;
msgpack::packer<msgpack::sbuffer> pk(&buffer);
pk.pack_array(1);
pk.pack_array(4);
pk.pack(now.time_since_epoch().count());
dht::Value {SHORT_TYPE.id, std::string("was-permanent")}.msgpack_pack(pk);
pk.pack_bin(0);
pk.pack(dht::clock::time_point::max().time_since_epoch().count());
restored.importValues({
dht::ValuesExport {key, {buffer.data(), buffer.data() + buffer.size()}}
});
auto containsValue = [&] {
for (const auto& [exportedKey, data] : restored.exportValues()) {
if (exportedKey != key)
continue;
msgpack::unpacked msg;
msgpack::unpack(msg, reinterpret_cast<const char*>(data.data()), data.size());
auto valarr = msg.get();
CPPUNIT_ASSERT(valarr.type == msgpack::type::ARRAY);
return valarr.via.array.size != 0;
}
return false;
};
CPPUNIT_ASSERT(containsValue());
auto timeout = dht::clock::now() + 2s;
while (containsValue() and dht::clock::now() < timeout)
std::this_thread::sleep_for(10ms);
CPPUNIT_ASSERT(!containsValue());
restored.shutdown();
restored.join();
}
void
DhtRunnerTester::testBootstrapSetsConnectingState()
{
node1.shutdown();
node1.join();
node2.shutdown();
node2.join();
dht::DhtRunner::Config config;
config.dht_config.node_config.max_peer_req_per_sec = -1;
config.dht_config.node_config.max_req_per_sec = -1;
config.dht_config.node_config.max_store_size = -1;
config.dht_config.node_config.max_store_keys = -1;
std::mutex mutex;
std::condition_variable cv;
bool sawConnecting {false};
dht::DhtRunner::Context context;
context.statusChangedCallback = [&](dht::NodeStatus status4, dht::NodeStatus status6) {
std::lock_guard lock(mutex);
sawConnecting = sawConnecting || status4 == dht::NodeStatus::Connecting
|| status6 == dht::NodeStatus::Connecting;
cv.notify_all();
};
dht::DhtRunner bootstrapNode;
dht::DhtRunner clientNode;
bootstrapNode.run(0, config);
clientNode.run(config, std::move(context));
auto bound = bootstrapNode.getBound();
if (bound.isUnspecified())
bound.setLoopback();
clientNode.bootstrap("127.0.0.1", std::to_string(bound.getPort()));
{
std::unique_lock lock(mutex);
CPPUNIT_ASSERT(cv.wait_for(lock, 5s, [&] { return sawConnecting; }));
}
bootstrapNode.shutdown();
bootstrapNode.join();
clientNode.shutdown();
clientNode.join();
}
void
DhtRunnerTester::testBootstrapThenPutNoRace()
{
node1.shutdown();
node1.join();
node2.shutdown();
node2.join();
dht::DhtRunner::Config config;
config.dht_config.node_config.max_peer_req_per_sec = -1;
config.dht_config.node_config.max_req_per_sec = -1;
config.dht_config.node_config.max_store_size = -1;
config.dht_config.node_config.max_store_keys = -1;
dht::DhtRunner bootstrapNode;
dht::DhtRunner clientNode;
bootstrapNode.run(0, config);
clientNode.run(0, config);
auto bound = bootstrapNode.getBound();
if (bound.isUnspecified())
bound.setLoopback();
clientNode.bootstrap(bound);
std::promise<bool> putDone;
auto putFuture = putDone.get_future();
clientNode.put(dht::InfoHash::get("bootstrap-then-put"), dht::Value {"value"}, [&](bool ok) {
putDone.set_value(ok);
});
CPPUNIT_ASSERT(getFutureValue(std::move(putFuture)));
auto vals = getFutureValue(bootstrapNode.get(dht::InfoHash::get("bootstrap-then-put")));
CPPUNIT_ASSERT(!vals.empty());
bootstrapNode.shutdown();
bootstrapNode.join();
clientNode.shutdown();
clientNode.join();
}
void
DhtRunnerTester::testBootstrapMissingNodeThenPutFails()
{
node1.shutdown();
node1.join();
node2.shutdown();
node2.join();
dht::DhtRunner::Config config;
config.dht_config.node_config.max_peer_req_per_sec = -1;
config.dht_config.node_config.max_req_per_sec = -1;
config.dht_config.node_config.max_store_size = -1;
config.dht_config.node_config.max_store_keys = -1;
auto getUnusedPort = [&]() {
dht::DhtRunner tmpNode;
tmpNode.run(0, config);
auto port = tmpNode.getBoundPort();
tmpNode.shutdown();
tmpNode.join();
return port;
};
auto missingPort = getUnusedPort();
std::mutex mutex;
std::condition_variable cv;
bool sawConnecting {false};
bool sawDisconnectedAfterConnecting {false};
bool putReturnedBeforeDisconnect {false};
dht::DhtRunner::Context context;
context.statusChangedCallback = [&](dht::NodeStatus status4, dht::NodeStatus status6) {
std::lock_guard lock(mutex);
if (status4 == dht::NodeStatus::Connecting || status6 == dht::NodeStatus::Connecting)
sawConnecting = true;
if (sawConnecting && status4 == dht::NodeStatus::Disconnected && status6 == dht::NodeStatus::Disconnected)
sawDisconnectedAfterConnecting = true;
cv.notify_all();
};
dht::DhtRunner clientNode;
clientNode.run(0, config, std::move(context));
while (clientNode.getBoundPort() == missingPort)
missingPort = getUnusedPort();
std::promise<bool> putDone;
auto putFuture = putDone.get_future();
clientNode.bootstrap("127.0.0.1", std::to_string(missingPort));
clientNode.put(dht::InfoHash::get("bootstrap-missing-put"), dht::Value {"value"}, [&](bool ok) {
std::lock_guard lock(mutex);
putReturnedBeforeDisconnect = !sawDisconnectedAfterConnecting;
putDone.set_value(ok);
cv.notify_all();
});
{
std::unique_lock lock(mutex);
CPPUNIT_ASSERT(cv.wait_for(lock, 10s, [&] { return sawConnecting; }));
CPPUNIT_ASSERT(cv.wait_for(lock, 20s, [&] { return sawDisconnectedAfterConnecting; }));
}
CPPUNIT_ASSERT(std::future_status::ready == putFuture.wait_for(20s));
CPPUNIT_ASSERT(!putFuture.get());
CPPUNIT_ASSERT(!putReturnedBeforeDisconnect);
clientNode.shutdown();
clientNode.join();
}
void
DhtRunnerTester::testShutdownCompletesWithPendingPut()
{
node1.shutdown();
node1.join();
node2.shutdown();
node2.join();
dht::DhtRunner::Config config;
config.dht_config.node_config.max_peer_req_per_sec = -1;
config.dht_config.node_config.max_req_per_sec = -1;
config.dht_config.node_config.max_store_size = -1;
config.dht_config.node_config.max_store_keys = -1;
auto getUnusedPort = [&]() {
dht::DhtRunner tmpNode;
tmpNode.run(0, config);
auto port = tmpNode.getBoundPort();
tmpNode.shutdown();
tmpNode.join();
return port;
};
auto missingPort = getUnusedPort();
dht::DhtRunner clientNode;
clientNode.run(0, config);
while (clientNode.getBoundPort() == missingPort)
missingPort = getUnusedPort();
clientNode.bootstrap("127.0.0.1", std::to_string(missingPort));
clientNode.put(dht::InfoHash::get("shutdown-pending-put"), dht::Value {"value"}, [](bool) {});
std::promise<void> shutdownDone;
auto shutdownFuture = shutdownDone.get_future();
clientNode.shutdown([&]() { shutdownDone.set_value(); });
CPPUNIT_ASSERT(std::future_status::ready == shutdownFuture.wait_for(5s));
clientNode.join();
}
void
DhtRunnerTester::testSavedPortFallback()
{
// A saved port that is already taken by another process (e.g. fast
// application restart while the old instance still holds it) must not
// prevent the node from starting: it should fall back to a random port.
dht::DhtRunner::Config config;
config.dht_config.node_config.persist_path = "/tmp/opendht_test_port_fallback";
std::string portFile = config.dht_config.node_config.persist_path + "_port.txt";
// node1 (from setUp) holds a port; save it as our persisted port.
auto takenPort = node1.getBoundPort();
{
std::ofstream out(portFile);
out << takenPort << std::endl << takenPort << std::endl;
}
dht::DhtRunner testNode;
CPPUNIT_ASSERT_NO_THROW(testNode.run(0, config));
CPPUNIT_ASSERT(testNode.getBoundPort());
CPPUNIT_ASSERT(testNode.getBoundPort() != takenPort);
testNode.shutdown();
testNode.join();
std::remove(portFile.c_str());
std::remove((config.dht_config.node_config.persist_path + "_port.txt").c_str());
}
void
DhtRunnerTester::testListen()
{
std::mutex mutex;
std::condition_variable cv;
std::atomic_uint valueCounta(0);
std::atomic_uint valueCountb(0);
std::atomic_uint valueCountc(0);
std::atomic_uint valueCountd(0);
unsigned putCount(0);
unsigned putOkCount1(0);
unsigned putOkCount2(0);
unsigned putOkCount3(0);
auto a = dht::InfoHash::get("234");
auto b = dht::InfoHash::get("2345");
auto c = dht::InfoHash::get("23456");
auto d = dht::InfoHash::get("234567");
constexpr unsigned N = 256;
constexpr unsigned SZ = 56 * 1024;
auto ftokena = node1.listen(a, [&](const std::vector<std::shared_ptr<dht::Value>>& values, bool expired) {
if (expired)
valueCounta -= values.size();
else
valueCounta += values.size();
return true;
});
auto ftokenb = node1.listen(b, [&](const std::shared_ptr<dht::Value>&) {
/*if (expired)
valueCountb -= values.size();
else
valueCountb += values.size();*/
valueCountb++;
return false;
});
auto ftokenc = node1.listen(c, [&](const std::vector<std::shared_ptr<dht::Value>>& values, bool expired) {
if (expired)
valueCountc -= values.size();
else
valueCountc += values.size();
return true;
});
auto ftokend = node1.listen(d, [&](const std::vector<std::shared_ptr<dht::Value>>& values, bool expired) {
if (expired)
valueCountd -= values.size();
else
valueCountd += values.size();
return true;
});
std::vector<uint8_t> mtu;
mtu.reserve(SZ);
for (size_t i = 0; i < SZ; i++)
mtu.emplace_back((i % 2) ? 'T' : 'M');
for (unsigned i = 0; i < N; i++) {
node2.put(a, dht::Value("v1"), [&](bool ok) {
std::lock_guard lock(mutex);
putCount++;
if (ok)
putOkCount1++;
cv.notify_all();
});
node2.put(b, dht::Value("v2"), [&](bool ok) {
std::lock_guard lock(mutex);
putCount++;
if (ok)
putOkCount2++;
cv.notify_all();
});
auto bigVal = std::make_shared<dht::Value>();
bigVal->data = mtu;
node2.put(c, std::move(bigVal), [&](bool ok) {
std::lock_guard lock(mutex);
putCount++;
if (ok)
putOkCount3++;
cv.notify_all();
});
}
{
std::unique_lock lk(mutex);
CPPUNIT_ASSERT(cv.wait_for(lk, 30s, [&] { return putCount == N * 3u; }));
CPPUNIT_ASSERT_EQUAL(N, putOkCount1);
CPPUNIT_ASSERT_EQUAL(N, putOkCount2);
CPPUNIT_ASSERT_EQUAL(N, putOkCount3);
}
CPPUNIT_ASSERT(ftokena.valid());
CPPUNIT_ASSERT(ftokenb.valid());
CPPUNIT_ASSERT(ftokenc.valid());
CPPUNIT_ASSERT(ftokend.valid());
auto tokena = getFutureValue(std::move(ftokena));
auto tokenc = getFutureValue(std::move(ftokenc));
auto tokend = getFutureValue(std::move(ftokend));
// tokenb might be 0 since the callback returns false.
CPPUNIT_ASSERT(tokena);
CPPUNIT_ASSERT(tokenc);
CPPUNIT_ASSERT(tokend);
CPPUNIT_ASSERT_EQUAL(N, valueCounta.load());
CPPUNIT_ASSERT_EQUAL(1u, valueCountb.load());
CPPUNIT_ASSERT_EQUAL(N, valueCountc.load());
CPPUNIT_ASSERT_EQUAL(0u, valueCountd.load());
node1.cancelListen(a, tokena);
node1.cancelListen(b, std::move(ftokenb));
node1.cancelListen(c, tokenc);
node1.cancelListen(d, tokend);
}
void
DhtRunnerTester::testIdOps()
{
std::mutex mutex;
std::condition_variable cv;
unsigned identityCount(0);
unsigned valueCount(0);
unsigned valueCountEdit(0);
dht::DhtRunner::Config config2;
config2.dht_config.node_config.max_peer_req_per_sec = -1;
config2.dht_config.node_config.max_req_per_sec = -1;
config2.dht_config.id = dht::crypto::generateIdentity();
dht::DhtRunner::Context context2;
context2.identityAnnouncedCb = [&](bool ok) {
CPPUNIT_ASSERT(ok);
std::lock_guard lk(mutex);
identityCount++;
cv.notify_all();
};
node2.join();
node2.run(0, config2, std::move(context2));
auto bound = node1.getBound();
if (bound.isUnspecified())
bound.setLoopback();
node2.bootstrap(bound);
{
std::unique_lock lk(mutex);
CPPUNIT_ASSERT(cv.wait_for(lk, 20s, [&] { return identityCount == 1; }));
}
node1.findCertificate(node2.getPublicKey()->getLongId(), [&](const std::shared_ptr<dht::crypto::Certificate>& crt) {
CPPUNIT_ASSERT(crt);
std::lock_guard lk(mutex);
valueCount++;
cv.notify_all();
});
{
std::unique_lock lk(mutex);
CPPUNIT_ASSERT(cv.wait_for(lk, 20s, [&] { return valueCount == 1u; }));
CPPUNIT_ASSERT_EQUAL(1u, identityCount);
}
dht::DhtRunner::Context context1;
context1.identityAnnouncedCb = [&](bool ok) {
CPPUNIT_ASSERT(ok);
std::lock_guard lk(mutex);
identityCount++;
cv.notify_all();
};
config2.dht_config.id = dht::crypto::generateIdentity();
node1.join();
node1.run(0, config2, std::move(context1));
bound = node2.getBound();
if (bound.isUnspecified())
bound.setLoopback();
node1.bootstrap(bound);
auto key = dht::InfoHash::get("key");
node1.putEncrypted(key, node2.getPublicKey()->getLongId(), std::make_shared<dht::Value>("yo"), [&](bool ok) {
CPPUNIT_ASSERT(ok);
std::lock_guard lk(mutex);
valueCount++;
cv.notify_all();
});
node1.putEncrypted(key, node2.getPublicKey(), dht::Value("yo"), [&](bool ok) {
CPPUNIT_ASSERT(ok);
std::lock_guard lk(mutex);
valueCount++;
cv.notify_all();
});
node2.listen<std::string>(key, [&](std::string&& value) {
CPPUNIT_ASSERT_EQUAL("yo"s, value);
std::lock_guard lk(mutex);
valueCount++;
cv.notify_all();
return true;
});
auto key2 = dht::InfoHash::get("key2");
auto editValue = std::make_shared<dht::Value>("v1");
node1.putSigned(key2, editValue, [&](bool ok) {
CPPUNIT_ASSERT(ok);
std::lock_guard lk(mutex);
valueCountEdit++;
cv.notify_all();
});
node2.listen(key2, [&](const std::vector<std::shared_ptr<dht::Value>>& values, bool /*expired*/) {
for (const auto& v : values) {
if (v->seq == 0)
CPPUNIT_ASSERT_EQUAL("v1"s, dht::unpackMsg<std::string>(v->data));
else if (v->seq == 1)
CPPUNIT_ASSERT_EQUAL("v2"s, dht::unpackMsg<std::string>(v->data));
CPPUNIT_ASSERT_EQUAL(v->owner->getLongId(), node1.getPublicKey()->getLongId());
}
std::lock_guard lk(mutex);
valueCountEdit += values.size();
cv.notify_all();
return true;
});
{
std::unique_lock lk(mutex);
CPPUNIT_ASSERT(cv.wait_for(lk, 20s, [&] { return identityCount == 2u; }));
CPPUNIT_ASSERT(cv.wait_for(lk, 20s, [&] { return valueCount == 5u; }));
CPPUNIT_ASSERT(cv.wait_for(lk, 20s, [&] { return valueCountEdit == 2u; }));
}
node2.findCertificate(node1.getPublicKey()->getLongId(), [&](const std::shared_ptr<dht::crypto::Certificate>& crt) {
CPPUNIT_ASSERT(crt);
std::lock_guard lk(mutex);
valueCount++;
cv.notify_all();
});
{
std::unique_lock lk(mutex);
CPPUNIT_ASSERT(cv.wait_for(lk, 20s, [&] { return valueCount == 6u; }));
}
// editValue->data = dht::packMsg("v2");
editValue = std::make_shared<dht::Value>(editValue->id);
editValue->data = dht::packMsg("v2");
node1.putSigned(key2, editValue, [&](bool ok) {
CPPUNIT_ASSERT(ok);
std::lock_guard lk(mutex);
valueCountEdit++;
cv.notify_all();
});
std::unique_lock lk(mutex);
CPPUNIT_ASSERT(cv.wait_for(lk, 20s, [&] { return valueCountEdit == 4u; }));
}
void
DhtRunnerTester::testListenLotOfBytes()
{
std::mutex mutex;
std::condition_variable cv;
std::atomic_uint valueCount(0);
unsigned putCount(0);
unsigned putOkCount(0);
std::string data(10000, 'a');
auto foo = dht::InfoHash::get("foo");
constexpr unsigned N = 1024;
for (unsigned i = 0; i < N; i++) {
node2.put(foo, data, [&](bool ok) {
std::lock_guard lock(mutex);
putCount++;
if (ok)
putOkCount++;
cv.notify_all();
});
}
{
std::unique_lock lk(mutex);
CPPUNIT_ASSERT(cv.wait_for(lk, 30s, [&] { return putCount == N; }));
}
dht::DhtRunner node3 {};
dht::DhtRunner::Config config;
config.dht_config.node_config.max_peer_req_per_sec = -1;
config.dht_config.node_config.max_req_per_sec = -1;
node3.run(42242, config);
auto bound = node1.getBound();
if (bound.isUnspecified())
bound.setLoopback();
node3.bootstrap(bound);
auto ftokenfoo = node3.listen(foo, [&](const std::shared_ptr<dht::Value>&) {
valueCount++;
cv.notify_all();
return true;
});
{
std::unique_lock lk(mutex);
CPPUNIT_ASSERT(cv.wait_for(lk, 30s, [&] { return valueCount == N; }));
}
node3.cancelListen(foo, getFutureValue(std::move(ftokenfoo)));
}
void
DhtRunnerTester::testMultithread()
{
std::mutex mutex;
std::condition_variable cv;
unsigned putCount(0);
unsigned putOkCount(0);
constexpr unsigned N = 2048;
for (unsigned i = 0; i < N; i++) {
dht::ThreadPool::computation().run([&] {
node2.put(dht::InfoHash::get("123" + std::to_string(i)), "hehe", [&](bool ok) {
std::lock_guard lock(mutex);
putCount++;
if (ok)
putOkCount++;
cv.notify_all();
});
node2.get(
dht::InfoHash::get("123" + std::to_string(N - i - 1)),
[](const std::shared_ptr<dht::Value>&) { return true; },
[&](bool ok) {
std::lock_guard lock(mutex);
putCount++;
if (ok)
putOkCount++;
cv.notify_all();
});
});
}
std::unique_lock lk(mutex);
CPPUNIT_ASSERT(cv.wait_for(lk, 30s, [&] { return putCount == 2 * N; }));
CPPUNIT_ASSERT_EQUAL(2 * N, putOkCount);
}
} // namespace test
@@ -20,6 +20,15 @@ class DhtRunnerTester : public CppUnit::TestFixture
CPPUNIT_TEST(testListen);
CPPUNIT_TEST(testListenLotOfBytes);
CPPUNIT_TEST(testIdOps);
CPPUNIT_TEST(testImportValuesPreservesRemoteQuota);
CPPUNIT_TEST(testExportValuesUsesSystemTime);
CPPUNIT_TEST(testImportValuesPreservesStoredExpiration);
CPPUNIT_TEST(testImportPermanentValueExpires);
CPPUNIT_TEST(testBootstrapSetsConnectingState);
CPPUNIT_TEST(testBootstrapThenPutNoRace);
CPPUNIT_TEST(testBootstrapMissingNodeThenPutFails);
CPPUNIT_TEST(testShutdownCompletesWithPendingPut);
CPPUNIT_TEST(testSavedPortFallback);
CPPUNIT_TEST_SUITE_END();
dht::DhtRunner node1 {};
@@ -58,6 +67,15 @@ public:
* Test methods requiring a node identity
*/
void testIdOps();
void testImportValuesPreservesRemoteQuota();
void testExportValuesUsesSystemTime();
void testImportValuesPreservesStoredExpiration();
void testImportPermanentValueExpires();
void testBootstrapSetsConnectingState();
void testBootstrapThenPutNoRace();
void testBootstrapMissingNodeThenPutFails();
void testShutdownCompletesWithPendingPut();
void testSavedPortFallback();
/**
* Test listen method with lot of datas
*/
+37 -6
View File
@@ -1,13 +1,14 @@
// Copyright (c) 2014-2026 Savoir-faire Linux Inc.
// SPDX-License-Identifier: MIT
#include "httptester.h"
#include "test_http.h"
#include <opendht/value.h>
#include <opendht/dhtrunner.h>
#include <iostream>
#include <string>
#include <atomic>
#include <chrono>
#include <condition_variable>
@@ -332,13 +333,43 @@ HttpTester::test_parse_url_just_slash()
CPPUNIT_ASSERT_EQUAL(std::string("/"), parsed.target);
}
void
HttpTester::test_completed_resolver_callback_uses_io_thread()
{
asio::io_context ioContext;
const asio::ip::tcp::endpoint endpoint(asio::ip::address_v4::loopback(), 80);
dht::http::Resolver resolver(ioContext, {endpoint});
const auto callerThread = std::this_thread::get_id();
std::thread::id callbackThread;
asio::error_code callbackError;
std::vector<asio::ip::tcp::endpoint> resolvedEndpoints;
std::atomic_bool callbackCalled {false};
resolver.add_callback([&](const asio::error_code& ec, const auto& endpoints) {
callbackError = ec;
callbackThread = std::this_thread::get_id();
resolvedEndpoints = endpoints;
callbackCalled = true;
});
CPPUNIT_ASSERT(!callbackCalled);
std::thread ioThread([&] { ioContext.run(); });
ioThread.join();
CPPUNIT_ASSERT(callbackCalled);
CPPUNIT_ASSERT(!callbackError);
CPPUNIT_ASSERT(callbackThread != callerThread);
CPPUNIT_ASSERT_EQUAL(size_t(1), resolvedEndpoints.size());
CPPUNIT_ASSERT_EQUAL(endpoint, resolvedEndpoints.front());
}
void
HttpTester::test_send_json()
{
// Arrange
std::condition_variable cv;
std::mutex cv_m;
std::unique_lock<std::mutex> lk(cv_m);
std::unique_lock lk(cv_m);
bool done = false;
unsigned int status = 0;
@@ -350,7 +381,7 @@ HttpTester::test_send_json()
"http://127.0.0.1:8080/key/key",
json,
[&](Json::Value value, const dht::http::Response& response) {
std::lock_guard<std::mutex> lk(cv_m);
std::lock_guard lk(cv_m);
resp_val = std::move(value);
status = response.status_code;
done = true;
@@ -368,7 +399,7 @@ HttpTester::test_send_json()
request = std::make_shared<dht::http::Request>(serverProxy->io_context(),
"http://google.ca",
[&](const dht::http::Response& response) {
std::lock_guard<std::mutex> lk(cv_m);
std::lock_guard lk(cv_m);
status = response.status_code;
done = true;
cv.notify_all();
@@ -382,7 +413,7 @@ HttpTester::test_send_json()
request = std::make_shared<dht::http::Request>(serverProxy->io_context(),
"https://google.ca",
[&](const dht::http::Response& response) {
std::lock_guard<std::mutex> lk(cv_m);
std::lock_guard lk(cv_m);
status = response.status_code;
done = true;
cv.notify_all();
@@ -396,7 +427,7 @@ HttpTester::test_send_json()
request = std::make_shared<dht::http::Request>(serverProxy->io_context(),
"https://google.ca/sdbjklwGBIP",
[&](const dht::http::Response& response) {
std::lock_guard<std::mutex> lk(cv_m);
std::lock_guard lk(cv_m);
status = response.status_code;
done = true;
cv.notify_all();
+2
View File
@@ -39,6 +39,7 @@ class HttpTester : public CppUnit::TestFixture
CPPUNIT_TEST(test_parse_url_complex_path_query_fragment);
CPPUNIT_TEST(test_parse_url_empty);
CPPUNIT_TEST(test_parse_url_just_slash);
CPPUNIT_TEST(test_completed_resolver_callback_uses_io_thread);
// send
CPPUNIT_TEST(test_send_json);
CPPUNIT_TEST_SUITE_END();
@@ -82,6 +83,7 @@ public:
void test_parse_url_complex_path_query_fragment();
void test_parse_url_empty();
void test_parse_url_just_slash();
void test_completed_resolver_callback_uses_io_thread();
/**
* Test send(json)
*/
@@ -1,11 +1,12 @@
// Copyright (c) 2014-2026 Savoir-faire Linux Inc.
// SPDX-License-Identifier: MIT
#include "infohashtester.h"
#include "test_infohash.h"
// std
#include <iostream>
#include <string>
#include <string_view>
// opendht
#include "opendht/infohash.h"
@@ -21,47 +22,60 @@ void
InfoHashTester::testConstructors()
{
// Default constructor creates a null infohash
auto nullHash = dht::InfoHash();
constexpr auto nullHash = dht::InfoHash();
CPPUNIT_ASSERT_EQUAL((size_t) 20u, nullHash.size());
CPPUNIT_ASSERT(!nullHash);
// Build from a uint8_t. if length to short, should get a null infohash
uint8_t to_short[] = {0x0, 0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7, 0x8};
auto infohash = dht::InfoHash(to_short, 8);
constexpr uint8_t too_short[] = {0x0, 0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7, 0x8};
auto infohash = dht::InfoHash(too_short, 8);
CPPUNIT_ASSERT_EQUAL((size_t) 20u, infohash.size());
CPPUNIT_ASSERT_EQUAL(std::string("0000000000000000000000000000000000000000"), infohash.toString());
// Build from a uint8_t. if length is enough, data should contains the uint8_t
uint8_t enough[] = {0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7, 0x8, 0x9, 0xa,
0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7, 0x8, 0x9, 0xa};
constexpr uint8_t enough[] = {0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7, 0x8, 0x9, 0xa,
0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7, 0x8, 0x9, 0xa};
infohash = dht::InfoHash(enough, 20);
CPPUNIT_ASSERT(infohash.size() == 20);
static_assert(dht::InfoHash(enough, 20) != nullHash);
CPPUNIT_ASSERT_EQUAL((size_t) 20u, infohash.size());
const auto* data = infohash.data();
for (auto i = 0; i < 20; ++i) {
CPPUNIT_ASSERT_EQUAL(enough[i], data[i]);
}
// if too long, should be cutted to 20
uint8_t tooLong[] = {0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7, 0x8, 0x9, 0xa, 0x1,
0x2, 0x3, 0x4, 0x5, 0x6, 0x7, 0x8, 0x9, 0xa, 0xb0};
constexpr uint8_t tooLong[] = {0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7, 0x8, 0x9, 0xa, 0x1,
0x2, 0x3, 0x4, 0x5, 0x6, 0x7, 0x8, 0x9, 0xa, 0xb0};
infohash = dht::InfoHash(tooLong, 21);
CPPUNIT_ASSERT(infohash.size() == 20);
const auto* data2 = infohash.data();
for (auto i = 0; i < 20; ++i) {
CPPUNIT_ASSERT_EQUAL(enough[i], data2[i]);
}
// Build from string
auto infohashFromStr = dht::InfoHash("0102030405060708090A0102030405060708090A");
constexpr auto infohashFromStr = dht::InfoHash("0102030405060708090A0102030405060708090A");
CPPUNIT_ASSERT_EQUAL((size_t) 20u, infohashFromStr.size());
const auto* dataStr = infohashFromStr.data();
for (auto i = 0; i < 20; ++i) {
CPPUNIT_ASSERT_EQUAL((int) dataStr[i], (int) data[i]);
}
// invalid strings
constexpr auto invalidHash = dht::InfoHash("invalid_string0102030405060708090A010203");
CPPUNIT_ASSERT_EQUAL((size_t) 20u, invalidHash.size());
CPPUNIT_ASSERT(!invalidHash);
CPPUNIT_ASSERT_EQUAL(nullHash, invalidHash);
static_assert(invalidHash == nullHash, "Invalid hash string should result in null hash");
constexpr auto tooShortHash = dht::InfoHash("0123456789abcdef0123456789abcdef012345");
CPPUNIT_ASSERT_EQUAL((size_t) 20u, tooShortHash.size());
CPPUNIT_ASSERT(!tooShortHash);
CPPUNIT_ASSERT_EQUAL(nullHash, tooShortHash);
static_assert(tooShortHash == nullHash, "Too short hash string should result in null hash");
}
void
InfoHashTester::testComparators()
{
auto nullHash = dht::InfoHash();
auto minHash = dht::InfoHash("0000000000000000000000000000000000111110");
auto maxHash = dht::InfoHash("0111110000000000000000000000000000000000");
constexpr auto nullHash = dht::InfoHash();
constexpr auto minHash = dht::InfoHash("0000000000000000000000000000000000111110");
constexpr auto maxHash = dht::InfoHash("0111110000000000000000000000000000000000");
// operator ==
CPPUNIT_ASSERT_EQUAL(minHash, minHash);
CPPUNIT_ASSERT_EQUAL(minHash, dht::InfoHash("0000000000000000000000000000000000111110"));
@@ -126,10 +140,21 @@ InfoHashTester::testXorCmp()
void
InfoHashTester::testHex()
{
const std::string TEST_HASH_STR("01b20304d5060708090a010203e05060708090ae");
using namespace std::literals;
static constexpr auto H = "0123456789abcdef0123456789abcdef01234567"sv;
const std::string TEST_HASH_STR(H);
dht::InfoHash TEST_HASH(TEST_HASH_STR);
static constexpr dht::InfoHash TEST_HASH_CONST(H);
CPPUNIT_ASSERT_EQUAL(TEST_HASH_STR, TEST_HASH.toString());
CPPUNIT_ASSERT_EQUAL(H, TEST_HASH.to_view());
CPPUNIT_ASSERT_EQUAL(TEST_HASH_STR, dht::toHex(TEST_HASH.data(), TEST_HASH.size()));
CPPUNIT_ASSERT_EQUAL(TEST_HASH_CONST.to_view(), H);
static constexpr auto hexArray = dht::toHexArray(TEST_HASH_CONST);
static constexpr auto hexArrayView = std::string_view(hexArray.data(), hexArray.size());
static_assert(hexArrayView == H, "Hex array does not match");
CPPUNIT_ASSERT_EQUAL(std::string_view(TEST_HASH_STR), hexArrayView);
}
void
File renamed without changes.
+672
View File
@@ -0,0 +1,672 @@
// Copyright (c) 2014-2026 Savoir-faire Linux Inc.
// SPDX-License-Identifier: MIT
#include "test_networkengine.h"
#ifdef _MSC_VER
namespace test {
CPPUNIT_TEST_SUITE_REGISTRATION(NetworkEngineTester);
void
NetworkEngineTester::setUp()
{}
void
NetworkEngineTester::tearDown()
{}
void
NetworkEngineTester::testDisabledOnMsvc()
{
CPPUNIT_ASSERT(true);
}
} // namespace test
#else
#include <any>
#include <mutex>
#ifdef _WIN32
#ifdef opendht_EXPORTS
#undef opendht_EXPORTS
#endif
#endif
#include "opendht/node.h"
#include "opendht/network_engine.h"
#include "opendht/network_utils.h"
#include "opendht/utils.h"
#include "opendht/value.h"
#if defined(__clang__)
#pragma clang diagnostic push
#pragma clang diagnostic ignored "-Wkeyword-macro"
#elif defined(__GNUC__)
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wpedantic"
#endif
#define private public
#include "opendht/dht.h"
#undef private
#if defined(__clang__)
#pragma clang diagnostic pop
#elif defined(__GNUC__)
#pragma GCC diagnostic pop
#endif
#include "../src/parsed_message.h"
#include "../src/search.h"
// Hack to test internal classes
#include "../src/op_cache.cpp"
#include "../src/routing_table.cpp"
#include "../src/dht.cpp"
#include "../src/node.cpp"
#include "../src/node_cache.cpp"
#include "../src/network_engine.cpp"
#include <string>
namespace test {
CPPUNIT_TEST_SUITE_REGISTRATION(NetworkEngineTester);
using namespace dht;
using namespace dht::net;
static constexpr size_t TEST_MTU {1280};
class TestDatagramSocket : public net::DatagramSocket
{
public:
TestDatagramSocket()
{
bound_.setFamily(AF_INET);
bound_.setLoopback();
bound_.setPort(4222);
}
int sendTo(const SockAddr& dest, const uint8_t* data, size_t size, bool replied) override
{
sends.emplace_back(SentPacket {dest, Blob(data, data + size), replied});
return 0;
}
bool hasIPv4() const override { return true; }
bool hasIPv6() const override { return false; }
const SockAddr& getBoundRef(sa_family_t family = AF_UNSPEC) const override
{
(void) family;
return bound_;
}
void stop() override {}
struct SentPacket
{
SockAddr dest;
Blob data;
bool replied;
};
SockAddr bound_;
std::vector<SentPacket> sends {};
};
static Blob
serializeValue(const std::string& data)
{
auto val = std::make_shared<Value>((const uint8_t*) data.data(), data.size());
return packMsg(val);
}
static Blob
makeReplyHeaderPacketBlob(const InfoHash& id, Tid tid, const std::vector<size_t>& valueSizes)
{
msgpack::sbuffer buffer;
msgpack::packer<msgpack::sbuffer> pk(&buffer);
pk.pack_map(3);
pk.pack(KEY_R);
pk.pack_map(2);
pk.pack(KEY_REQ_ID);
pk.pack(id);
pk.pack(KEY_REQ_VALUES);
pk.pack_array(valueSizes.size());
for (const auto& size : valueSizes)
pk.pack(size);
pk.pack(KEY_TID);
pk.pack(tid);
pk.pack(KEY_Y);
pk.pack(KEY_R);
return {buffer.data(), buffer.data() + buffer.size()};
}
static Blob
makeValueDataPacketBlob(Tid tid, unsigned index, size_t offset, const Blob& data)
{
msgpack::sbuffer buffer;
msgpack::packer<msgpack::sbuffer> pk(&buffer);
pk.pack_map(3);
pk.pack(KEY_Y);
pk.pack(KEY_V);
pk.pack(KEY_TID);
pk.pack(tid);
pk.pack(KEY_V);
pk.pack_map(1);
pk.pack(index);
pk.pack_map(2);
pk.pack("o");
pk.pack(offset);
pk.pack("d");
pk.pack_bin(data.size());
pk.pack_bin_body((const char*) data.data(), data.size());
return {buffer.data(), buffer.data() + buffer.size()};
}
static Blob
makeListenPacketBlob(const InfoHash& id, const InfoHash& hash, Tid tid, Tid socketId, const Blob& token)
{
msgpack::sbuffer buffer;
msgpack::packer<msgpack::sbuffer> pk(&buffer);
pk.pack_map(4);
pk.pack(KEY_A);
pk.pack_map(5);
pk.pack(KEY_REQ_ID);
pk.pack(id);
pk.pack(KEY_VERSION);
pk.pack(1);
pk.pack(KEY_REQ_H);
pk.pack(hash);
pk.pack(KEY_REQ_TOKEN);
packToken(pk, token);
pk.pack(KEY_REQ_SID);
pk.pack(socketId);
pk.pack(KEY_Q);
pk.pack(QUERY_LISTEN);
pk.pack(KEY_TID);
pk.pack(tid);
pk.pack(KEY_Y);
pk.pack(KEY_Q);
return {buffer.data(), buffer.data() + buffer.size()};
}
static SockAddr
makeIPv4(const char* ip, in_port_t port)
{
SockAddr addr;
addr.setFamily(AF_INET);
addr.setAddress(ip);
addr.setPort(port);
return addr;
}
static net::NetworkEngine
makeEngine(std::unique_ptr<TestDatagramSocket>&& socket,
Scheduler& scheduler,
InfoHash& myid,
std::mt19937_64& rd,
int& onNewNodeCalls)
{
net::NetworkConfig config {};
config.max_req_per_sec = -1;
config.max_peer_req_per_sec = -1;
return net::NetworkEngine(
myid,
config,
std::move(socket),
{},
rd,
scheduler,
[](Sp<net::Request>, net::DhtProtocolException) {},
[&onNewNodeCalls](const Sp<Node>&, int) { ++onNewNodeCalls; },
[](const InfoHash&, const SockAddr&) {},
[](Sp<Node>) { return net::RequestAnswer {}; },
[](Sp<Node>, const InfoHash&, want_t) { return net::RequestAnswer {}; },
[](Sp<Node>, const InfoHash&, want_t, const Query&) { return net::RequestAnswer {}; },
[](Sp<Node>, const InfoHash&, const Blob&, Tid, const Query&, int) { return net::RequestAnswer {}; },
[](Sp<Node>, const InfoHash&, const Blob&, const std::vector<Sp<Value>>&, const time_point&) {
return net::RequestAnswer {};
},
[](Sp<Node>, const InfoHash&, const Blob&, const Value::Id&) { return net::RequestAnswer {}; });
}
static Sp<Node>
addPendingGetRequest(net::NetworkEngine& engine, const InfoHash& id, const SockAddr& addr, Tid tid)
{
auto node = engine.insertNode(id, addr);
auto request = std::make_shared<net::Request>(
MessageType::GetValues,
tid,
node,
Blob {},
[](const net::Request&, ParsedMessage&&) {},
[](const net::Request&, bool) {});
node->requested(request);
return node;
}
void
NetworkEngineTester::setUp()
{}
void
NetworkEngineTester::tearDown()
{}
void
NetworkEngineTester::testCompletesPartialSessionWithDataBeforeHeader()
{
Scheduler scheduler;
std::mt19937_64 rd(0);
InfoHash myid = InfoHash::getRandom(rd);
InfoHash remoteId = InfoHash::getRandom(rd);
int onNewNodeCalls = 0;
auto engine = makeEngine(std::make_unique<TestDatagramSocket>(), scheduler, myid, rd, onNewNodeCalls);
auto serialized = serializeValue("fragment before header");
auto from = makeIPv4("127.0.0.2", 5000);
addPendingGetRequest(engine, remoteId, from, 99);
auto initialOnNewNodeCalls = onNewNodeCalls;
auto fragment = makeValueDataPacketBlob(99, 0, 0, serialized);
auto header = makeReplyHeaderPacketBlob(remoteId, 99, {serialized.size()});
engine.processMessage(fragment.data(), fragment.size(), from);
CPPUNIT_ASSERT_EQUAL((size_t) 1, engine.getPartialCount());
CPPUNIT_ASSERT_EQUAL(initialOnNewNodeCalls, onNewNodeCalls);
engine.processMessage(header.data(), header.size(), from);
CPPUNIT_ASSERT_EQUAL((size_t) 0, engine.getPartialCount());
CPPUNIT_ASSERT(onNewNodeCalls > initialOnNewNodeCalls);
}
void
NetworkEngineTester::testBuffersLargeValueBeforeHeader()
{
Scheduler scheduler;
std::mt19937_64 rd(8);
InfoHash myid = InfoHash::getRandom(rd);
InfoHash remoteId = InfoHash::getRandom(rd);
int onNewNodeCalls = 0;
auto engine = makeEngine(std::make_unique<TestDatagramSocket>(), scheduler, myid, rd, onNewNodeCalls);
auto serialized = serializeValue(std::string(57 * 1024, 'l'));
auto from = makeIPv4("127.0.0.2", 5008);
addPendingGetRequest(engine, remoteId, from, 101);
auto initialOnNewNodeCalls = onNewNodeCalls;
for (size_t offset = 0; offset < serialized.size(); offset += TEST_MTU) {
auto end = std::min(offset + TEST_MTU, serialized.size());
Blob data(serialized.begin() + offset, serialized.begin() + end);
auto fragment = makeValueDataPacketBlob(101, 0, offset, data);
engine.processMessage(fragment.data(), fragment.size(), from);
}
CPPUNIT_ASSERT_EQUAL((size_t) 1, engine.getPartialCount());
auto header = makeReplyHeaderPacketBlob(remoteId, 101, {serialized.size()});
engine.processMessage(header.data(), header.size(), from);
CPPUNIT_ASSERT_EQUAL((size_t) 0, engine.getPartialCount());
CPPUNIT_ASSERT(onNewNodeCalls > initialOnNewNodeCalls);
}
void
NetworkEngineTester::testIgnoresEmptyPartialData()
{
Scheduler scheduler;
std::mt19937_64 rd(7);
InfoHash myid = InfoHash::getRandom(rd);
int onNewNodeCalls = 0;
auto engine = makeEngine(std::make_unique<TestDatagramSocket>(), scheduler, myid, rd, onNewNodeCalls);
auto packet = makeValueDataPacketBlob(100, 0, 0, {});
auto from = makeIPv4("127.0.0.2", 5007);
engine.processMessage(packet.data(), packet.size(), from);
CPPUNIT_ASSERT_EQUAL((size_t) 0, engine.getPartialCount());
CPPUNIT_ASSERT_EQUAL(0, onNewNodeCalls);
}
void
NetworkEngineTester::testRejectsUnsolicitedPartialReply()
{
Scheduler scheduler;
std::mt19937_64 rd(9);
InfoHash myid = InfoHash::getRandom(rd);
InfoHash remoteId = InfoHash::getRandom(rd);
int onNewNodeCalls = 0;
auto engine = makeEngine(std::make_unique<TestDatagramSocket>(), scheduler, myid, rd, onNewNodeCalls);
auto from = makeIPv4("127.0.0.2", 5009);
auto header = makeReplyHeaderPacketBlob(remoteId, 102, {TEST_MTU});
engine.processMessage(header.data(), header.size(), from);
CPPUNIT_ASSERT_EQUAL((size_t) 0, engine.getPartialCount());
CPPUNIT_ASSERT_EQUAL((size_t) 0, engine.getNodeCacheSize());
}
void
NetworkEngineTester::testDropsMalformedCompletedPartialMessage()
{
Scheduler scheduler;
std::mt19937_64 rd(10);
InfoHash myid = InfoHash::getRandom(rd);
InfoHash remoteId = InfoHash::getRandom(rd);
int onNewNodeCalls = 0;
auto engine = makeEngine(std::make_unique<TestDatagramSocket>(), scheduler, myid, rd, onNewNodeCalls);
auto from = makeIPv4("127.0.0.2", 5010);
addPendingGetRequest(engine, remoteId, from, 103);
Blob malformed {0xc1};
auto header = makeReplyHeaderPacketBlob(remoteId, 103, {malformed.size()});
auto fragment = makeValueDataPacketBlob(103, 0, 0, malformed);
engine.processMessage(header.data(), header.size(), from);
CPPUNIT_ASSERT_EQUAL((size_t) 1, engine.getPartialCount());
engine.processMessage(fragment.data(), fragment.size(), from);
CPPUNIT_ASSERT_EQUAL((size_t) 0, engine.getPartialCount());
auto earlyFrom = makeIPv4("127.0.0.2", 5011);
addPendingGetRequest(engine, remoteId, earlyFrom, 104);
auto earlyHeader = makeReplyHeaderPacketBlob(remoteId, 104, {malformed.size()});
auto earlyFragment = makeValueDataPacketBlob(104, 0, 0, malformed);
engine.processMessage(earlyFragment.data(), earlyFragment.size(), earlyFrom);
CPPUNIT_ASSERT_EQUAL((size_t) 1, engine.getPartialCount());
engine.processMessage(earlyHeader.data(), earlyHeader.size(), earlyFrom);
CPPUNIT_ASSERT_EQUAL((size_t) 0, engine.getPartialCount());
}
void
NetworkEngineTester::testCompletesPartialSessionFromSameSource()
{
Scheduler scheduler;
std::mt19937_64 rd(1);
InfoHash myid = InfoHash::getRandom(rd);
InfoHash remoteId = InfoHash::getRandom(rd);
int onNewNodeCalls = 0;
auto engine = makeEngine(std::make_unique<TestDatagramSocket>(), scheduler, myid, rd, onNewNodeCalls);
std::string data(TEST_MTU * 2 + 64, 'z');
auto serialized = serializeValue(data);
auto from = makeIPv4("127.0.0.2", 5001);
addPendingGetRequest(engine, remoteId, from, 7);
auto initialOnNewNodeCalls = onNewNodeCalls;
auto header = makeReplyHeaderPacketBlob(remoteId, 7, {serialized.size()});
engine.processMessage(header.data(), header.size(), from);
CPPUNIT_ASSERT_EQUAL((size_t) 1, engine.getPartialCount());
for (size_t offset = 0; offset < serialized.size(); offset += TEST_MTU) {
auto end = std::min(offset + TEST_MTU, serialized.size());
Blob fragment(serialized.begin() + offset, serialized.begin() + end);
auto packet = makeValueDataPacketBlob(7, 0, offset, fragment);
engine.processMessage(packet.data(), packet.size(), from);
}
CPPUNIT_ASSERT_EQUAL((size_t) 0, engine.getPartialCount());
CPPUNIT_ASSERT(onNewNodeCalls > initialOnNewNodeCalls);
}
void
NetworkEngineTester::testSeparatesPartialSessionsBySource()
{
Scheduler scheduler;
std::mt19937_64 rd(2);
InfoHash myid = InfoHash::getRandom(rd);
InfoHash firstRemoteId = InfoHash::getRandom(rd);
InfoHash secondRemoteId = InfoHash::getRandom(rd);
int onNewNodeCalls = 0;
auto engine = makeEngine(std::make_unique<TestDatagramSocket>(), scheduler, myid, rd, onNewNodeCalls);
auto serialized = serializeValue("same transaction, different peers");
auto firstFrom = makeIPv4("127.0.0.2", 5002);
auto secondFrom = makeIPv4("127.0.0.3", 5002);
addPendingGetRequest(engine, firstRemoteId, firstFrom, 11);
addPendingGetRequest(engine, secondRemoteId, secondFrom, 11);
auto initialOnNewNodeCalls = onNewNodeCalls;
auto firstHeader = makeReplyHeaderPacketBlob(firstRemoteId, 11, {serialized.size()});
auto secondHeader = makeReplyHeaderPacketBlob(secondRemoteId, 11, {serialized.size()});
engine.processMessage(firstHeader.data(), firstHeader.size(), firstFrom);
engine.processMessage(secondHeader.data(), secondHeader.size(), secondFrom);
CPPUNIT_ASSERT_EQUAL((size_t) 2, engine.getPartialCount());
auto fragment = makeValueDataPacketBlob(11, 0, 0, serialized);
engine.processMessage(fragment.data(), fragment.size(), firstFrom);
CPPUNIT_ASSERT_EQUAL((size_t) 1, engine.getPartialCount());
engine.processMessage(fragment.data(), fragment.size(), secondFrom);
CPPUNIT_ASSERT_EQUAL((size_t) 0, engine.getPartialCount());
CPPUNIT_ASSERT(onNewNodeCalls > initialOnNewNodeCalls);
}
void
NetworkEngineTester::testListenConfirmationCarriesToken()
{
Scheduler scheduler;
std::mt19937_64 rd(3);
InfoHash myid = InfoHash::getRandom(rd);
InfoHash remoteId = InfoHash::getRandom(rd);
InfoHash hash = InfoHash::getRandom(rd);
Blob requestToken {0x01, 0x02, 0x03, 0x04};
Blob responseToken {0xaa, 0xbb, 0xcc, 0xdd, 0xee};
int onNewNodeCalls = 0;
net::NetworkConfig config {};
config.max_req_per_sec = -1;
config.max_peer_req_per_sec = -1;
auto socket = std::make_unique<TestDatagramSocket>();
auto* socketPtr = socket.get();
auto engine = net::NetworkEngine(
myid,
config,
std::move(socket),
{},
rd,
scheduler,
[](Sp<net::Request>, net::DhtProtocolException) {},
[&onNewNodeCalls](const Sp<Node>&, int) { ++onNewNodeCalls; },
[](const InfoHash&, const SockAddr&) {},
[](Sp<Node>) { return net::RequestAnswer {}; },
[](Sp<Node>, const InfoHash&, want_t) { return net::RequestAnswer {}; },
[](Sp<Node>, const InfoHash&, want_t, const Query&) { return net::RequestAnswer {}; },
[responseToken](Sp<Node>, const InfoHash&, const Blob&, Tid, const Query&, int) {
net::RequestAnswer answer {};
answer.ntoken = responseToken;
return answer;
},
[](Sp<Node>, const InfoHash&, const Blob&, const std::vector<Sp<Value>>&, const time_point&) {
return net::RequestAnswer {};
},
[](Sp<Node>, const InfoHash&, const Blob&, const Value::Id&) { return net::RequestAnswer {}; });
auto from = makeIPv4("127.0.0.2", 5003);
auto packet = makeListenPacketBlob(remoteId, hash, 17, 23, requestToken);
engine.processMessage(packet.data(), packet.size(), from);
CPPUNIT_ASSERT_EQUAL((size_t) 1, socketPtr->sends.size());
const auto& sent = socketPtr->sends.front();
CPPUNIT_ASSERT(sent.dest == from);
auto replyObject = msgpack::unpack((const char*) sent.data.data(), sent.data.size());
ParsedMessage reply;
reply.msgpack_unpack(replyObject.get());
CPPUNIT_ASSERT(reply.type == MessageType::Reply);
CPPUNIT_ASSERT_EQUAL(myid, reply.id);
CPPUNIT_ASSERT(reply.token == responseToken);
}
void
NetworkEngineTester::testListenConfirmationUpdatesSearchNodeToken()
{
Config config {};
config.max_req_per_sec = -1;
config.max_peer_req_per_sec = -1;
auto rd = std::make_unique<std::mt19937_64>(6);
Dht localDht(std::make_unique<TestDatagramSocket>(), config, {}, std::move(rd));
auto node = std::make_shared<Node>(InfoHash::getRandom(localDht.rd),
makeIPv4("127.0.0.2", 5006),
localDht.rd,
false);
auto sr = std::make_shared<Dht::Search>();
auto query = std::make_shared<Query>();
ValueCallback valueCallback = [](const std::vector<std::shared_ptr<Value>>&, bool) {
return true;
};
sr->id = InfoHash::getRandom(localDht.rd);
sr->af = AF_INET;
sr->nextSearchStep = localDht.scheduler.add(time_point::max(), [] {});
sr->listeners.emplace(1, Dht::Search::SearchListener {query, valueCallback, {}});
auto searchNode = std::make_unique<Dht::SearchNode>(node);
auto socketId = node->openSocket([](const Sp<Node>&, net::RequestAnswer&&) {});
searchNode->listenStatus.emplace(std::piecewise_construct,
std::forward_as_tuple(query),
std::forward_as_tuple(valueCallback, dht::SyncCallback {}, socketId));
auto* searchNodePtr = sr->nodes.emplace_back(std::move(searchNode)).get();
CPPUNIT_ASSERT(searchNodePtr->token.empty());
CPPUNIT_ASSERT(!searchNodePtr->isSynced(localDht.scheduler.time()));
net::RequestAnswer answer {};
answer.ntoken = Blob {0xaa, 0xbb, 0xcc, 0xdd};
localDht.onListenDone(node, answer, sr);
CPPUNIT_ASSERT(searchNodePtr->token == answer.ntoken);
/* last_get_reply should NOT be updated by listen confirmation,
so that periodic find_node requests still refresh the search table. */
CPPUNIT_ASSERT(searchNodePtr->last_get_reply == time_point::min());
CPPUNIT_ASSERT(!searchNodePtr->isSynced(localDht.scheduler.time()));
}
void
NetworkEngineTester::testListenReopensSocketAfterNodeExpiration()
{
Config config {};
config.max_req_per_sec = -1;
config.max_peer_req_per_sec = -1;
auto rd = std::make_unique<std::mt19937_64>(4);
Dht localDht(std::make_unique<TestDatagramSocket>(), config, {}, std::move(rd));
auto node = std::make_shared<Node>(InfoHash::getRandom(localDht.rd),
makeIPv4("127.0.0.2", 5004),
localDht.rd,
false);
auto sr = std::make_shared<Dht::Search>();
auto query = std::make_shared<Query>();
ValueCallback valueCallback = [](const std::vector<std::shared_ptr<Value>>&, bool) {
return true;
};
dht::SyncCallback syncCallback {};
sr->id = InfoHash::getRandom(localDht.rd);
sr->af = AF_INET;
sr->listeners.emplace(1, Dht::Search::SearchListener {query, valueCallback, {}});
auto searchNode = std::make_unique<Dht::SearchNode>(node);
auto staleSocketId = node->openSocket([](const Sp<Node>&, net::RequestAnswer&&) {});
auto status = searchNode->listenStatus
.emplace(std::piecewise_construct,
std::forward_as_tuple(query),
std::forward_as_tuple(valueCallback, syncCallback, staleSocketId))
.first;
status->second.req = std::make_shared<net::Request>(net::Request::State::COMPLETED);
status->second.req->reply_time = time_point::min();
node->setExpired();
CPPUNIT_ASSERT(!node->getSocket(staleSocketId));
auto* searchNodePtr = sr->nodes.emplace_back(std::move(searchNode)).get();
searchNodePtr->token = Blob {0x42};
searchNodePtr->last_get_reply = localDht.scheduler.time();
localDht.searchSynchedNodeListen(sr, *searchNodePtr);
auto listenIt = searchNodePtr->listenStatus.find(query);
CPPUNIT_ASSERT(listenIt != searchNodePtr->listenStatus.end());
CPPUNIT_ASSERT(listenIt->second.req);
CPPUNIT_ASSERT(listenIt->second.socketId);
CPPUNIT_ASSERT(listenIt->second.socketId != staleSocketId);
CPPUNIT_ASSERT(node->getSocket(listenIt->second.socketId));
}
void
NetworkEngineTester::testUnauthorizedListenFlushClearsListenState()
{
Config config {};
config.max_req_per_sec = -1;
config.max_peer_req_per_sec = -1;
auto rd = std::make_unique<std::mt19937_64>(5);
Dht localDht(std::make_unique<TestDatagramSocket>(), config, {}, std::move(rd));
auto node = std::make_shared<Node>(InfoHash::getRandom(localDht.rd),
makeIPv4("127.0.0.2", 5005),
localDht.rd,
false);
auto sr = std::make_shared<Dht::Search>();
auto query = std::make_shared<Query>();
ValueCallback valueCallback = [](const std::vector<std::shared_ptr<Value>>&, bool) {
return true;
};
dht::SyncCallback syncCallback {};
sr->id = InfoHash::getRandom(localDht.rd);
sr->af = AF_INET;
auto searchNode = std::make_unique<Dht::SearchNode>(node);
auto socketId = node->openSocket([](const Sp<Node>&, net::RequestAnswer&&) {});
auto listenIt = searchNode->listenStatus
.emplace(std::piecewise_construct,
std::forward_as_tuple(query),
std::forward_as_tuple(valueCallback, syncCallback, socketId))
.first;
listenIt->second.req = std::make_shared<net::Request>();
listenIt->second.refresh = localDht.scheduler.add(localDht.scheduler.time() + std::chrono::seconds(1), [] {});
searchNode->token = Blob {0x10, 0x11, 0x12};
searchNode->last_get_reply = localDht.scheduler.time();
auto* searchNodePtr = sr->nodes.emplace_back(std::move(searchNode)).get();
localDht.searches(AF_INET).emplace(sr->id, sr);
auto failingReq = std::make_shared<net::Request>(
MessageType::Listen,
node->getNewTid(),
node,
Blob {},
[](const net::Request&, ParsedMessage&&) {},
[](const net::Request&, bool) {});
localDht.onError(failingReq,
net::DhtProtocolException {net::DhtProtocolException::UNAUTHORIZED,
net::DhtProtocolException::LISTEN_WRONG_TOKEN});
CPPUNIT_ASSERT(searchNodePtr->token.empty());
CPPUNIT_ASSERT(searchNodePtr->last_get_reply == time_point::min());
auto listenStatusIt = searchNodePtr->listenStatus.find(query);
CPPUNIT_ASSERT(listenStatusIt != searchNodePtr->listenStatus.end());
CPPUNIT_ASSERT(!listenStatusIt->second.req);
CPPUNIT_ASSERT(!listenStatusIt->second.refresh);
}
} // namespace test
#endif
+51
View File
@@ -0,0 +1,51 @@
// Copyright (c) 2014-2026 Savoir-faire Linux Inc.
// SPDX-License-Identifier: MIT
#pragma once
#include <cppunit/TestFixture.h>
#include <cppunit/extensions/HelperMacros.h>
namespace test {
class NetworkEngineTester : public CppUnit::TestFixture
{
CPPUNIT_TEST_SUITE(NetworkEngineTester);
#ifdef _MSC_VER
CPPUNIT_TEST(testDisabledOnMsvc);
#else
CPPUNIT_TEST(testCompletesPartialSessionWithDataBeforeHeader);
CPPUNIT_TEST(testBuffersLargeValueBeforeHeader);
CPPUNIT_TEST(testIgnoresEmptyPartialData);
CPPUNIT_TEST(testRejectsUnsolicitedPartialReply);
CPPUNIT_TEST(testDropsMalformedCompletedPartialMessage);
CPPUNIT_TEST(testCompletesPartialSessionFromSameSource);
CPPUNIT_TEST(testSeparatesPartialSessionsBySource);
CPPUNIT_TEST(testListenConfirmationCarriesToken);
CPPUNIT_TEST(testListenConfirmationUpdatesSearchNodeToken);
CPPUNIT_TEST(testListenReopensSocketAfterNodeExpiration);
CPPUNIT_TEST(testUnauthorizedListenFlushClearsListenState);
#endif
CPPUNIT_TEST_SUITE_END();
public:
void setUp();
void tearDown();
#ifdef _MSC_VER
void testDisabledOnMsvc();
#else
void testCompletesPartialSessionWithDataBeforeHeader();
void testBuffersLargeValueBeforeHeader();
void testIgnoresEmptyPartialData();
void testRejectsUnsolicitedPartialReply();
void testDropsMalformedCompletedPartialMessage();
void testCompletesPartialSessionFromSameSource();
void testSeparatesPartialSessionsBySource();
void testListenConfirmationCarriesToken();
void testListenConfirmationUpdatesSearchNodeToken();
void testListenReopensSocketAfterNodeExpiration();
void testUnauthorizedListenFlushClearsListenState();
#endif
};
} // namespace test
+638
View File
@@ -0,0 +1,638 @@
// Copyright (c) 2014-2026 Savoir-faire Linux Inc.
// SPDX-License-Identifier: MIT
#include "test_parsedmessage.h"
#include "opendht/value.h"
#include "opendht/node.h"
#include "opendht/utils.h"
#include "../src/parsed_message.h"
#include <iostream>
#include <limits>
#include <string>
namespace test {
CPPUNIT_TEST_SUITE_REGISTRATION(ParsedMessageTester);
using namespace dht;
using namespace dht::net;
static constexpr size_t TEST_MTU {1280};
// Helper: create a ParsedMessage with a single value_parts entry of given total_size
static ParsedMessage
makeHeader(unsigned index, size_t total_size)
{
ParsedMessage msg;
msg.value_parts[index].total_size = total_size;
return msg;
}
// Helper: create a ParsedMessage fragment (as if parsed from ValueData)
static ParsedMessage
makeFragment(unsigned index, size_t offset, const Blob& data)
{
ParsedMessage frag;
frag.fragment_parts[index] = {offset, data};
return frag;
}
// Helper: serialize a Value to a Blob
static Blob
serializeValue(const std::string& data)
{
auto val = std::make_shared<Value>((const uint8_t*) data.data(), data.size());
return packMsg(val);
}
static ParsedMessage
parsePackedMessage(msgpack::sbuffer& buffer)
{
auto msg = msgpack::unpack(buffer.data(), buffer.size());
ParsedMessage parsed;
parsed.msgpack_unpack(msg.get());
return parsed;
}
static ParsedMessage
makeReplyHeaderPacket(Tid tid, const std::vector<size_t>& valueSizes)
{
msgpack::sbuffer buffer;
msgpack::packer<msgpack::sbuffer> pk(&buffer);
pk.pack_map(3);
pk.pack(KEY_R);
pk.pack_map(1);
pk.pack(KEY_REQ_VALUES);
pk.pack_array(valueSizes.size());
for (const auto& size : valueSizes)
pk.pack(size);
pk.pack(KEY_TID);
pk.pack(tid);
pk.pack(KEY_Y);
pk.pack(KEY_R);
return parsePackedMessage(buffer);
}
static ParsedMessage
makeValueDataPacket(Tid tid, unsigned index, size_t offset, const Blob& data)
{
msgpack::sbuffer buffer;
msgpack::packer<msgpack::sbuffer> pk(&buffer);
pk.pack_map(3);
pk.pack(KEY_Y);
pk.pack(KEY_V);
pk.pack(KEY_TID);
pk.pack(tid);
pk.pack(KEY_V);
pk.pack_map(1);
pk.pack(index);
pk.pack_map(2);
pk.pack("o");
pk.pack(offset);
pk.pack("d");
pk.pack_bin(data.size());
pk.pack_bin_body((const char*) data.data(), data.size());
return parsePackedMessage(buffer);
}
void
ParsedMessageTester::setUp()
{}
void
ParsedMessageTester::tearDown()
{}
// --- append tests ---
void
ParsedMessageTester::testAppendInOrder()
{
Blob full = {1, 2, 3, 4, 5, 6};
auto msg = makeHeader(0, full.size());
// Send first half, then second half
Blob part1(full.begin(), full.begin() + 3);
Blob part2(full.begin() + 3, full.end());
auto frag1 = makeFragment(0, 0, part1);
auto frag2 = makeFragment(0, 3, part2);
CPPUNIT_ASSERT(msg.append(frag1));
CPPUNIT_ASSERT(msg.append(frag2));
auto& pv = msg.value_parts[0];
CPPUNIT_ASSERT(pv.isComplete());
CPPUNIT_ASSERT_EQUAL(full.size(), pv.received_bytes);
auto reassembled = pv.reassemble();
CPPUNIT_ASSERT(reassembled == full);
}
void
ParsedMessageTester::testAppendOutOfOrder()
{
Blob full = {10, 20, 30, 40, 50, 60};
auto msg = makeHeader(0, full.size());
Blob part1(full.begin(), full.begin() + 2); // [0..2)
Blob part2(full.begin() + 2, full.begin() + 4); // [2..4)
Blob part3(full.begin() + 4, full.end()); // [4..6)
// Send in order: part3, part1, part2
CPPUNIT_ASSERT(msg.append(makeFragment(0, 4, part3)));
CPPUNIT_ASSERT(!msg.value_parts[0].isComplete());
CPPUNIT_ASSERT(msg.append(makeFragment(0, 0, part1)));
CPPUNIT_ASSERT(!msg.value_parts[0].isComplete());
CPPUNIT_ASSERT(msg.append(makeFragment(0, 2, part2)));
CPPUNIT_ASSERT(msg.value_parts[0].isComplete());
auto reassembled = msg.value_parts[0].reassemble();
CPPUNIT_ASSERT(reassembled == full);
}
void
ParsedMessageTester::testAppendReverseOrder()
{
Blob full = {1, 2, 3, 4, 5, 6, 7, 8};
auto msg = makeHeader(0, full.size());
// 4 fragments of 2 bytes each, sent in reverse
for (int i = 3; i >= 0; --i) {
size_t offset = i * 2;
Blob part(full.begin() + offset, full.begin() + offset + 2);
CPPUNIT_ASSERT(msg.append(makeFragment(0, offset, part)));
}
CPPUNIT_ASSERT(msg.value_parts[0].isComplete());
CPPUNIT_ASSERT(msg.value_parts[0].reassemble() == full);
}
void
ParsedMessageTester::testAppendDuplicateFragment()
{
Blob full = {1, 2, 3, 4};
auto msg = makeHeader(0, full.size());
Blob part1(full.begin(), full.begin() + 2);
Blob part2(full.begin() + 2, full.end());
CPPUNIT_ASSERT(msg.append(makeFragment(0, 0, part1)));
// Duplicate of first fragment should be rejected
CPPUNIT_ASSERT(!msg.append(makeFragment(0, 0, part1)));
CPPUNIT_ASSERT(msg.append(makeFragment(0, 2, part2)));
CPPUNIT_ASSERT(msg.value_parts[0].isComplete());
CPPUNIT_ASSERT_EQUAL((size_t) 4, msg.value_parts[0].received_bytes);
}
void
ParsedMessageTester::testAppendOverlappingFragments()
{
Blob full = {1, 2, 3, 4, 5, 6};
auto msg = makeHeader(0, full.size());
// Fragment [0..3)
Blob part1(full.begin(), full.begin() + 3);
CPPUNIT_ASSERT(msg.append(makeFragment(0, 0, part1)));
// Overlapping fragment [2..5) should be rejected
Blob overlap(full.begin() + 2, full.begin() + 5);
CPPUNIT_ASSERT(!msg.append(makeFragment(0, 2, overlap)));
// Non-overlapping fragment [3..6) should succeed
Blob part2(full.begin() + 3, full.end());
CPPUNIT_ASSERT(msg.append(makeFragment(0, 3, part2)));
CPPUNIT_ASSERT(msg.value_parts[0].isComplete());
CPPUNIT_ASSERT(msg.value_parts[0].reassemble() == full);
}
void
ParsedMessageTester::testAppendFragmentBeyondTotal()
{
auto msg = makeHeader(0, 4);
// Fragment at offset 3 with 2 bytes goes beyond total_size of 4
Blob data = {1, 2};
CPPUNIT_ASSERT(!msg.append(makeFragment(0, 3, data)));
CPPUNIT_ASSERT_EQUAL((size_t) 0, msg.value_parts[0].received_bytes);
}
void
ParsedMessageTester::testAppendEmptyFragment()
{
auto msg = makeHeader(0, 4);
Blob empty;
CPPUNIT_ASSERT(!msg.append(makeFragment(0, 0, empty)));
CPPUNIT_ASSERT_EQUAL((size_t) 0, msg.value_parts[0].received_bytes);
}
void
ParsedMessageTester::testAppendUnknownIndex()
{
auto msg = makeHeader(0, 10);
// Fragment for index 1 which doesn't exist in value_parts
Blob data = {1, 2, 3};
CPPUNIT_ASSERT(!msg.append(makeFragment(1, 0, data)));
// Index 0 should be untouched
CPPUNIT_ASSERT_EQUAL((size_t) 0, msg.value_parts[0].received_bytes);
}
void
ParsedMessageTester::testAppendAlreadyComplete()
{
Blob full = {1, 2, 3, 4};
auto msg = makeHeader(0, full.size());
// Complete the value
CPPUNIT_ASSERT(msg.append(makeFragment(0, 0, full)));
CPPUNIT_ASSERT(msg.value_parts[0].isComplete());
// Further fragment should be rejected
Blob extra = {5, 6};
CPPUNIT_ASSERT(!msg.append(makeFragment(0, 0, extra)));
}
void
ParsedMessageTester::testAppendOverflowingOffset()
{
auto msg = makeHeader(0, 4);
Blob data = {1};
CPPUNIT_ASSERT(!msg.append(makeFragment(0, std::numeric_limits<size_t>::max(), data)));
CPPUNIT_ASSERT_EQUAL((size_t) 0, msg.value_parts[0].received_bytes);
}
void
ParsedMessageTester::testParseRejectsNonMapPacket()
{
msgpack::sbuffer buffer;
msgpack::packer<msgpack::sbuffer> pk(&buffer);
pk.pack_array(1);
pk.pack(42);
auto msg = msgpack::unpack(buffer.data(), buffer.size());
ParsedMessage parsed;
CPPUNIT_ASSERT_THROW(parsed.msgpack_unpack(msg.get()), msgpack::type_error);
}
void
ParsedMessageTester::testParseRejectsInvalidValueDataPayload()
{
msgpack::sbuffer buffer;
msgpack::packer<msgpack::sbuffer> pk(&buffer);
pk.pack_map(3);
pk.pack(KEY_Y);
pk.pack(KEY_V);
pk.pack(KEY_TID);
pk.pack(1u);
pk.pack(KEY_V);
pk.pack_array(1);
pk.pack(42);
auto msg = msgpack::unpack(buffer.data(), buffer.size());
ParsedMessage parsed;
CPPUNIT_ASSERT_THROW(parsed.msgpack_unpack(msg.get()), msgpack::type_error);
}
void
ParsedMessageTester::testParseRejectsInvalidValuesField()
{
msgpack::sbuffer buffer;
msgpack::packer<msgpack::sbuffer> pk(&buffer);
pk.pack_map(3);
pk.pack(KEY_R);
pk.pack_map(1);
pk.pack(KEY_REQ_VALUES);
pk.pack("invalid");
pk.pack(KEY_TID);
pk.pack(1u);
pk.pack(KEY_Y);
pk.pack(KEY_R);
auto msg = msgpack::unpack(buffer.data(), buffer.size());
ParsedMessage parsed;
CPPUNIT_ASSERT_THROW(parsed.msgpack_unpack(msg.get()), msgpack::type_error);
}
void
ParsedMessageTester::testParseRejectsEmptyFieldSelection()
{
msgpack::sbuffer buffer;
msgpack::packer<msgpack::sbuffer> pk(&buffer);
pk.pack_map(3);
pk.pack(KEY_R);
pk.pack_map(1);
pk.pack(KEY_REQ_FIELDS);
pk.pack_map(2);
pk.pack("f");
pk.pack_array(0);
pk.pack("v");
pk.pack_array(0);
pk.pack(KEY_TID);
pk.pack(1u);
pk.pack(KEY_Y);
pk.pack(KEY_R);
auto msg = msgpack::unpack(buffer.data(), buffer.size());
ParsedMessage parsed;
CPPUNIT_ASSERT_THROW(parsed.msgpack_unpack(msg.get()), msgpack::type_error);
}
void
ParsedMessageTester::testParseRejectsEmptyErrorArray()
{
msgpack::sbuffer buffer;
msgpack::packer<msgpack::sbuffer> pk(&buffer);
pk.pack_map(3);
pk.pack(KEY_E);
pk.pack_array(0);
pk.pack(KEY_TID);
pk.pack(1u);
pk.pack(KEY_Y);
pk.pack(KEY_E);
auto msg = msgpack::unpack(buffer.data(), buffer.size());
ParsedMessage parsed;
CPPUNIT_ASSERT_THROW(parsed.msgpack_unpack(msg.get()), msgpack::type_error);
}
void
ParsedMessageTester::testParseRejectsNonMapRequest()
{
msgpack::sbuffer buffer;
msgpack::packer<msgpack::sbuffer> pk(&buffer);
pk.pack_map(4);
pk.pack(KEY_A);
pk.pack_array(2);
pk.pack(1u);
pk.pack(2u);
pk.pack(KEY_Q);
pk.pack(QUERY_PING);
pk.pack(KEY_TID);
pk.pack(1u);
pk.pack(KEY_Y);
pk.pack("q");
auto msg = msgpack::unpack(buffer.data(), buffer.size());
ParsedMessage parsed;
CPPUNIT_ASSERT_THROW(parsed.msgpack_unpack(msg.get()), msgpack::type_error);
}
void
ParsedMessageTester::testParseErrorOnlyMessageKeepsRequestFieldsEmpty()
{
msgpack::sbuffer buffer;
msgpack::packer<msgpack::sbuffer> pk(&buffer);
// An error message with no request map: the error array must not be walked
// as if it were a sequence of key/value pairs.
pk.pack_map(3);
pk.pack(KEY_E);
pk.pack_array(2);
pk.pack(401u);
pk.pack("unauthorized");
pk.pack(KEY_TID);
pk.pack(9u);
pk.pack(KEY_Y);
pk.pack(KEY_E);
auto msg = msgpack::unpack(buffer.data(), buffer.size());
ParsedMessage parsed;
parsed.msgpack_unpack(msg.get());
CPPUNIT_ASSERT(parsed.type == MessageType::Error);
CPPUNIT_ASSERT_EQUAL((uint16_t) 401, parsed.error_code);
CPPUNIT_ASSERT(not parsed.id);
CPPUNIT_ASSERT(parsed.token.empty());
CPPUNIT_ASSERT(parsed.values.empty());
CPPUNIT_ASSERT(parsed.nodes4_raw.empty());
CPPUNIT_ASSERT(parsed.nodes6_raw.empty());
}
void
ParsedMessageTester::testParseIgnoresIncompleteValueDataEntry()
{
msgpack::sbuffer buffer;
msgpack::packer<msgpack::sbuffer> pk(&buffer);
pk.pack_map(3);
pk.pack(KEY_Y);
pk.pack(KEY_V);
pk.pack(KEY_TID);
pk.pack(1u);
pk.pack(KEY_V);
pk.pack_map(1);
pk.pack(0u);
pk.pack_map(1);
pk.pack("o");
pk.pack(0u);
auto msg = msgpack::unpack(buffer.data(), buffer.size());
ParsedMessage parsed;
parsed.msgpack_unpack(msg.get());
CPPUNIT_ASSERT(parsed.fragment_parts.empty());
}
// --- complete tests ---
void
ParsedMessageTester::testCompleteEmpty()
{
// No value_parts at all — should be trivially complete
ParsedMessage msg;
CPPUNIT_ASSERT(msg.complete());
CPPUNIT_ASSERT(msg.values.empty());
}
void
ParsedMessageTester::testCompleteIncomplete()
{
auto msg = makeHeader(0, 100);
Blob partial = {1, 2, 3};
msg.append(makeFragment(0, 0, partial));
CPPUNIT_ASSERT(!msg.complete());
CPPUNIT_ASSERT(msg.values.empty());
}
void
ParsedMessageTester::testCompleteZeroSizedValue()
{
auto msg = makeHeader(0, 0);
CPPUNIT_ASSERT(!msg.value_parts[0].isComplete());
CPPUNIT_ASSERT(!msg.complete());
CPPUNIT_ASSERT(msg.values.empty());
}
void
ParsedMessageTester::testCompleteSingleValue()
{
std::string data = "hello world";
auto serialized = serializeValue(data);
auto msg = makeHeader(0, serialized.size());
msg.append(makeFragment(0, 0, serialized));
CPPUNIT_ASSERT(msg.value_parts[0].isComplete());
CPPUNIT_ASSERT(msg.complete());
CPPUNIT_ASSERT_EQUAL((size_t) 1, msg.values.size());
std::string recovered(msg.values[0]->data.begin(), msg.values[0]->data.end());
CPPUNIT_ASSERT_EQUAL(data, recovered);
}
void
ParsedMessageTester::testCompleteMultipleValues()
{
std::string data1 = "first value";
std::string data2 = "second value";
auto ser1 = serializeValue(data1);
auto ser2 = serializeValue(data2);
ParsedMessage msg;
msg.value_parts[0].total_size = ser1.size();
msg.value_parts[1].total_size = ser2.size();
msg.append(makeFragment(0, 0, ser1));
msg.append(makeFragment(1, 0, ser2));
CPPUNIT_ASSERT(msg.complete());
CPPUNIT_ASSERT_EQUAL((size_t) 2, msg.values.size());
std::string r1(msg.values[0]->data.begin(), msg.values[0]->data.end());
std::string r2(msg.values[1]->data.begin(), msg.values[1]->data.end());
CPPUNIT_ASSERT_EQUAL(data1, r1);
CPPUNIT_ASSERT_EQUAL(data2, r2);
}
void
ParsedMessageTester::testCompleteReassemblyOrder()
{
// Verify that out-of-order fragments produce correct reassembly
std::string data = "test data for fragmentation";
auto serialized = serializeValue(data);
auto msg = makeHeader(0, serialized.size());
// Split into 3 fragments and send out of order
size_t chunk = serialized.size() / 3;
Blob frag1(serialized.begin(), serialized.begin() + chunk);
Blob frag2(serialized.begin() + chunk, serialized.begin() + 2 * chunk);
Blob frag3(serialized.begin() + 2 * chunk, serialized.end());
// Send: third, first, second
CPPUNIT_ASSERT(msg.append(makeFragment(0, 2 * chunk, frag3)));
CPPUNIT_ASSERT(msg.append(makeFragment(0, 0, frag1)));
CPPUNIT_ASSERT(msg.append(makeFragment(0, chunk, frag2)));
CPPUNIT_ASSERT(msg.value_parts[0].isComplete());
CPPUNIT_ASSERT(msg.complete());
CPPUNIT_ASSERT_EQUAL((size_t) 1, msg.values.size());
std::string recovered(msg.values[0]->data.begin(), msg.values[0]->data.end());
CPPUNIT_ASSERT_EQUAL(data, recovered);
}
void
ParsedMessageTester::testAppendMultipleValuesOutOfOrder()
{
// Two values, both fragmented, fragments interleaved and out of order
Blob val1 = {1, 2, 3, 4, 5, 6};
Blob val2 = {10, 20, 30, 40};
ParsedMessage msg;
msg.value_parts[0].total_size = val1.size();
msg.value_parts[1].total_size = val2.size();
// val2 second half
CPPUNIT_ASSERT(msg.append(makeFragment(1, 2, Blob {30, 40})));
// val1 last third
CPPUNIT_ASSERT(msg.append(makeFragment(0, 4, Blob {5, 6})));
// val2 first half
CPPUNIT_ASSERT(msg.append(makeFragment(1, 0, Blob {10, 20})));
// val1 first third
CPPUNIT_ASSERT(msg.append(makeFragment(0, 0, Blob {1, 2})));
// val1 middle third
CPPUNIT_ASSERT(msg.append(makeFragment(0, 2, Blob {3, 4})));
CPPUNIT_ASSERT(msg.value_parts[0].isComplete());
CPPUNIT_ASSERT(msg.value_parts[1].isComplete());
CPPUNIT_ASSERT(msg.value_parts[0].reassemble() == val1);
CPPUNIT_ASSERT(msg.value_parts[1].reassemble() == val2);
}
void
ParsedMessageTester::testReceiveLargeFragmentedValue()
{
std::string data(TEST_MTU * 3, 'x');
auto serialized = serializeValue(data);
CPPUNIT_ASSERT(serialized.size() > TEST_MTU);
auto msg = makeReplyHeaderPacket(1, {serialized.size()});
CPPUNIT_ASSERT_EQUAL((size_t) 1, msg.value_parts.size());
CPPUNIT_ASSERT_EQUAL(serialized.size(), msg.value_parts[0].total_size);
for (size_t offset = 0; offset < serialized.size(); offset += TEST_MTU) {
auto end = std::min(offset + TEST_MTU, serialized.size());
Blob fragment(serialized.begin() + offset, serialized.begin() + end);
auto packet = makeValueDataPacket(1, 0, offset, fragment);
CPPUNIT_ASSERT(msg.append(packet));
}
CPPUNIT_ASSERT(msg.value_parts[0].isComplete());
CPPUNIT_ASSERT(msg.complete());
CPPUNIT_ASSERT_EQUAL((size_t) 1, msg.values.size());
std::string recovered(msg.values[0]->data.begin(), msg.values[0]->data.end());
CPPUNIT_ASSERT_EQUAL(data, recovered);
}
void
ParsedMessageTester::testReceiveLargeFragmentedValueOutOfOrder()
{
std::string data(TEST_MTU * 3 + 117, 'y');
auto serialized = serializeValue(data);
CPPUNIT_ASSERT(serialized.size() > TEST_MTU);
auto msg = makeReplyHeaderPacket(7, {serialized.size()});
std::vector<ParsedMessage> packets;
for (size_t offset = 0; offset < serialized.size(); offset += TEST_MTU) {
auto end = std::min(offset + TEST_MTU, serialized.size());
Blob fragment(serialized.begin() + offset, serialized.begin() + end);
packets.emplace_back(makeValueDataPacket(7, 0, offset, fragment));
}
CPPUNIT_ASSERT(packets.size() >= 3);
CPPUNIT_ASSERT(msg.append(packets.back()));
CPPUNIT_ASSERT(msg.append(packets.front()));
for (size_t i = 1; i + 1 < packets.size(); ++i)
CPPUNIT_ASSERT(msg.append(packets[i]));
CPPUNIT_ASSERT(msg.value_parts[0].isComplete());
CPPUNIT_ASSERT(msg.complete());
CPPUNIT_ASSERT_EQUAL((size_t) 1, msg.values.size());
std::string recovered(msg.values[0]->data.begin(), msg.values[0]->data.end());
CPPUNIT_ASSERT_EQUAL(data, recovered);
}
} // namespace test
+76
View File
@@ -0,0 +1,76 @@
// Copyright (c) 2014-2026 Savoir-faire Linux Inc.
// SPDX-License-Identifier: MIT
#pragma once
// cppunit
#include <cppunit/TestFixture.h>
#include <cppunit/extensions/HelperMacros.h>
namespace test {
class ParsedMessageTester : public CppUnit::TestFixture
{
CPPUNIT_TEST_SUITE(ParsedMessageTester);
CPPUNIT_TEST(testAppendInOrder);
CPPUNIT_TEST(testAppendOutOfOrder);
CPPUNIT_TEST(testAppendReverseOrder);
CPPUNIT_TEST(testAppendDuplicateFragment);
CPPUNIT_TEST(testAppendOverlappingFragments);
CPPUNIT_TEST(testAppendFragmentBeyondTotal);
CPPUNIT_TEST(testAppendEmptyFragment);
CPPUNIT_TEST(testAppendUnknownIndex);
CPPUNIT_TEST(testAppendAlreadyComplete);
CPPUNIT_TEST(testAppendOverflowingOffset);
CPPUNIT_TEST(testParseRejectsNonMapPacket);
CPPUNIT_TEST(testParseRejectsInvalidValueDataPayload);
CPPUNIT_TEST(testParseRejectsInvalidValuesField);
CPPUNIT_TEST(testParseRejectsEmptyFieldSelection);
CPPUNIT_TEST(testParseRejectsEmptyErrorArray);
CPPUNIT_TEST(testParseRejectsNonMapRequest);
CPPUNIT_TEST(testParseErrorOnlyMessageKeepsRequestFieldsEmpty);
CPPUNIT_TEST(testParseIgnoresIncompleteValueDataEntry);
CPPUNIT_TEST(testCompleteEmpty);
CPPUNIT_TEST(testCompleteIncomplete);
CPPUNIT_TEST(testCompleteZeroSizedValue);
CPPUNIT_TEST(testCompleteSingleValue);
CPPUNIT_TEST(testCompleteMultipleValues);
CPPUNIT_TEST(testCompleteReassemblyOrder);
CPPUNIT_TEST(testAppendMultipleValuesOutOfOrder);
CPPUNIT_TEST(testReceiveLargeFragmentedValue);
CPPUNIT_TEST(testReceiveLargeFragmentedValueOutOfOrder);
CPPUNIT_TEST_SUITE_END();
public:
void setUp();
void tearDown();
void testAppendInOrder();
void testAppendOutOfOrder();
void testAppendReverseOrder();
void testAppendDuplicateFragment();
void testAppendOverlappingFragments();
void testAppendFragmentBeyondTotal();
void testAppendEmptyFragment();
void testAppendUnknownIndex();
void testAppendAlreadyComplete();
void testAppendOverflowingOffset();
void testParseRejectsNonMapPacket();
void testParseRejectsInvalidValueDataPayload();
void testParseRejectsInvalidValuesField();
void testParseRejectsEmptyFieldSelection();
void testParseRejectsEmptyErrorArray();
void testParseRejectsNonMapRequest();
void testParseErrorOnlyMessageKeepsRequestFieldsEmpty();
void testParseIgnoresIncompleteValueDataEntry();
void testCompleteEmpty();
void testCompleteIncomplete();
void testCompleteZeroSizedValue();
void testCompleteSingleValue();
void testCompleteMultipleValues();
void testCompleteReassemblyOrder();
void testAppendMultipleValuesOutOfOrder();
void testReceiveLargeFragmentedValue();
void testReceiveLargeFragmentedValueOutOfOrder();
};
} // namespace test
@@ -1,7 +1,7 @@
// Copyright (c) 2014-2026 Savoir-faire Linux Inc.
// SPDX-License-Identifier: MIT
#include "peerdiscoverytester.h"
#include "test_peerdiscovery.h"
#include "opendht/value.h"
#include <mutex>
@@ -54,7 +54,7 @@ PeerDiscoveryTester::testMulticastToTwoNodes()
unsigned countDht {0};
unsigned countJami {0};
{
std::unique_lock<std::mutex> l(lock);
std::unique_lock l(lock);
dht::PeerDiscovery testDht(MULTICAST_PORT);
dht::PeerDiscovery testJami(MULTICAST_PORT);
@@ -63,7 +63,7 @@ PeerDiscoveryTester::testMulticastToTwoNodes()
CPPUNIT_ASSERT_EQUAL(dhtNode.nodeid, v.nodeid);
CPPUNIT_ASSERT_EQUAL(dhtNode.nid, v.nid);
{
std::lock_guard<std::mutex> l(lock);
std::lock_guard l(lock);
countDht++;
}
cv.notify_all();
@@ -75,7 +75,7 @@ PeerDiscoveryTester::testMulticastToTwoNodes()
CPPUNIT_ASSERT_EQUAL(jamiNode.cha, v.cha);
CPPUNIT_ASSERT_EQUAL(jamiNode.str, v.str);
{
std::lock_guard<std::mutex> l(lock);
std::lock_guard l(lock);
countJami++;
}
cv.notify_all();
File renamed without changes.
+323
View File
@@ -0,0 +1,323 @@
// Copyright (c) 2014-2026 Savoir-faire Linux Inc.
// SPDX-License-Identifier: MIT
#include "test_storage.h"
#include <opendht/thread_pool.h>
#include <chrono>
#include <mutex>
#include <condition_variable>
using namespace std::chrono_literals;
using namespace std::literals;
namespace test {
CPPUNIT_TEST_SUITE_REGISTRATION(StorageTester);
static constexpr unsigned VAL_SIZE = 1024;
static dht::DhtRunner::Config
makeConfig(ssize_t maxStore = -1, ssize_t maxLocalStore = -1, ssize_t maxKeys = -1)
{
dht::DhtRunner::Config config;
config.dht_config.node_config.max_peer_req_per_sec = -1;
config.dht_config.node_config.max_req_per_sec = -1;
config.dht_config.node_config.max_store_size = maxStore;
config.dht_config.node_config.max_local_store_size = maxLocalStore;
config.dht_config.node_config.max_store_keys = maxKeys;
return config;
}
static std::shared_ptr<dht::Value>
makeValue(size_t size = VAL_SIZE)
{
return std::make_shared<dht::Value>(std::string(size, 'x'));
}
void
StorageTester::setUp()
{
// Tests configure and start nodes with specific limits as needed.
}
void
StorageTester::tearDown()
{
node1.shutdown();
node2.shutdown();
node1.join();
node2.join();
}
void
StorageTester::startNodes(dht::DhtRunner::Config config1, dht::DhtRunner::Config config2)
{
node1.run(0, config1);
node2.run(0, config2);
auto bound = node1.getBound();
if (bound.isUnspecified())
bound.setLoopback();
node2.bootstrap(bound);
std::this_thread::sleep_for(1s);
}
void
StorageTester::testRemoteStorageLimit()
{
constexpr ssize_t REMOTE_LIMIT = 10 * 1024; // 10 KB remote limit
startNodes(makeConfig(REMOTE_LIMIT, -1 /* unlimited local */), makeConfig());
// Put many values from node2 -> they are stored remotely on node1
auto key = dht::InfoHash::get("remote_test");
constexpr unsigned N = 20; // 20 * ~1KB = ~20KB, more than 10KB limit
std::mutex mtx;
std::condition_variable cv;
unsigned putCount = 0;
for (unsigned i = 0; i < N; i++) {
auto val = makeValue();
node2.put(key, val, [&](bool) {
std::lock_guard lk(mtx);
putCount++;
cv.notify_all();
});
}
{
std::unique_lock lk(mtx);
CPPUNIT_ASSERT(cv.wait_for(lk, 30s, [&] { return putCount == N; }));
}
// Wait for values to propagate
std::this_thread::sleep_for(2s);
// The combined store size on node1 should be under or near the remote limit
auto [storeSize, storeValues] = node1.getStoreSize();
CPPUNIT_ASSERT_MESSAGE("Remote store size (" + std::to_string(storeSize)
+ ") should be limited (limit: " + std::to_string(REMOTE_LIMIT) + ")",
storeSize <= static_cast<size_t>(REMOTE_LIMIT) * 2 // allow some margin for timing
);
// We should have fewer values than N since eviction happened
CPPUNIT_ASSERT_MESSAGE("Should have fewer than " + std::to_string(N) + " values stored, got "
+ std::to_string(storeValues),
storeValues < N);
}
void
StorageTester::testLocalStorageLimit()
{
constexpr ssize_t LOCAL_LIMIT = 10 * 1024; // 10 KB local limit
// Only node1 is needed - testing local puts
auto config = makeConfig(-1 /* unlimited remote */, LOCAL_LIMIT);
node1.run(0, config);
// With 1KB values and 10KB limit, ~10 should fit, rest should be rejected
constexpr unsigned N = 20; // 20 * ~1KB = ~20KB, more than 10KB limit
std::mutex mtx;
std::condition_variable cv;
unsigned putCount = 0;
for (unsigned i = 0; i < N; i++) {
auto key = dht::InfoHash::get("local_test_" + std::to_string(i));
auto val = makeValue();
node1.put(key, val, [&](bool) {
std::lock_guard lk(mtx);
putCount++;
cv.notify_all();
});
}
{
std::unique_lock lk(mtx);
CPPUNIT_ASSERT(cv.wait_for(lk, 30s, [&] { return putCount == N; }));
}
auto [storeSize, storeValues] = node1.getStoreSize();
CPPUNIT_ASSERT_MESSAGE("Local store size (" + std::to_string(storeSize) + ") should not exceed "
+ std::to_string(LOCAL_LIMIT),
storeSize <= static_cast<size_t>(LOCAL_LIMIT));
CPPUNIT_ASSERT_MESSAGE("Some values should be stored (got " + std::to_string(storeValues) + ")", storeValues > 0);
CPPUNIT_ASSERT_MESSAGE("Not all values should fit (stored " + std::to_string(storeValues) + "/" + std::to_string(N)
+ ")",
storeValues < N);
}
void
StorageTester::testIndependentLimits()
{
constexpr ssize_t REMOTE_LIMIT = 10 * 1024;
constexpr ssize_t LOCAL_LIMIT = 10 * 1024;
startNodes(makeConfig(REMOTE_LIMIT, LOCAL_LIMIT), makeConfig());
// Put local values on node1 — some will be rejected when limit is hit
constexpr unsigned N_LOCAL = 20;
std::mutex mtx;
std::condition_variable cv;
unsigned putCount = 0;
for (unsigned i = 0; i < N_LOCAL; i++) {
auto key = dht::InfoHash::get("local_indep_" + std::to_string(i));
auto val = makeValue();
node1.put(key, val, [&](bool) {
std::lock_guard lk(mtx);
putCount++;
cv.notify_all();
});
}
{
std::unique_lock lk(mtx);
CPPUNIT_ASSERT(cv.wait_for(lk, 30s, [&] { return putCount == N_LOCAL; }));
}
// Now put remote values from node2
constexpr unsigned N_REMOTE = 20;
unsigned remotePutCount = 0;
for (unsigned i = 0; i < N_REMOTE; i++) {
auto key = dht::InfoHash::get("remote_indep_" + std::to_string(i));
auto val = makeValue();
node2.put(key, val, [&](bool) {
std::lock_guard lk(mtx);
remotePutCount++;
cv.notify_all();
});
}
{
std::unique_lock lk(mtx);
CPPUNIT_ASSERT(cv.wait_for(lk, 30s, [&] { return remotePutCount == N_REMOTE; }));
}
// Wait for values to propagate
std::this_thread::sleep_for(2s);
auto [storeSize, storeValues] = node1.getStoreSize();
// Combined should be bounded by remote + local limits (with some margin for timing)
CPPUNIT_ASSERT_MESSAGE("Combined store size (" + std::to_string(storeSize) + ") should be limited",
storeSize <= static_cast<size_t>(REMOTE_LIMIT + LOCAL_LIMIT) * 2);
// Not all values should be stored (both limits should reject/evict some)
CPPUNIT_ASSERT_MESSAGE("Should have fewer than " + std::to_string(N_LOCAL + N_REMOTE) + " values, got "
+ std::to_string(storeValues),
storeValues < N_LOCAL + N_REMOTE);
}
void
StorageTester::testLocalPutNotAffectedByRemoteLimit()
{
constexpr ssize_t REMOTE_LIMIT = 1024; // 1 KB - very small
startNodes(makeConfig(REMOTE_LIMIT, -1 /* unlimited local */), makeConfig());
// Fill up remote storage from node2
auto remoteKey = dht::InfoHash::get("fill_remote");
constexpr unsigned N_REMOTE = 5;
std::mutex mtx;
std::condition_variable cv;
unsigned putCount = 0;
for (unsigned i = 0; i < N_REMOTE; i++) {
auto val = makeValue();
node2.put(remoteKey, val, [&](bool) {
std::lock_guard lk(mtx);
putCount++;
cv.notify_all();
});
}
{
std::unique_lock lk(mtx);
CPPUNIT_ASSERT(cv.wait_for(lk, 30s, [&] { return putCount == N_REMOTE; }));
}
std::this_thread::sleep_for(2s);
// Local puts on node1 should all succeed regardless of remote limit
constexpr unsigned N_LOCAL = 10;
unsigned localPutCount = 0;
unsigned localPutOk = 0;
for (unsigned i = 0; i < N_LOCAL; i++) {
auto key = dht::InfoHash::get("local_separate_" + std::to_string(i));
auto val = makeValue();
node1.put(key, val, [&](bool ok) {
std::lock_guard lk(mtx);
localPutCount++;
if (ok)
localPutOk++;
cv.notify_all();
});
}
{
std::unique_lock lk(mtx);
CPPUNIT_ASSERT(cv.wait_for(lk, 30s, [&] { return localPutCount == N_LOCAL; }));
}
// All local puts should succeed
CPPUNIT_ASSERT_EQUAL(N_LOCAL, localPutOk);
// Verify local values can be retrieved
for (unsigned i = 0; i < N_LOCAL; i++) {
auto key = dht::InfoHash::get("local_separate_" + std::to_string(i));
auto vals = node1.get(key).get();
CPPUNIT_ASSERT_MESSAGE("Local value " + std::to_string(i) + " should be retrievable", !vals.empty());
}
}
void
StorageTester::testRemotePutNotAffectedByLocalLimit()
{
constexpr ssize_t LOCAL_LIMIT = 1024; // 1 KB - very small
startNodes(makeConfig(-1 /* unlimited remote */, LOCAL_LIMIT), makeConfig());
// Fill up local storage on node1
constexpr unsigned N_LOCAL = 5;
std::mutex mtx;
std::condition_variable cv;
unsigned localPutCount = 0;
for (unsigned i = 0; i < N_LOCAL; i++) {
auto key = dht::InfoHash::get("fill_local_" + std::to_string(i));
auto val = makeValue();
node1.put(key, val, [&](bool) {
std::lock_guard lk(mtx);
localPutCount++;
cv.notify_all();
});
}
{
std::unique_lock lk(mtx);
CPPUNIT_ASSERT(cv.wait_for(lk, 30s, [&] { return localPutCount == N_LOCAL; }));
}
// Remote puts from node2 should all succeed despite full local storage
auto remoteKey = dht::InfoHash::get("remote_separate");
constexpr unsigned N_REMOTE = 10;
unsigned remotePutCount = 0;
unsigned remotePutOk = 0;
for (unsigned i = 0; i < N_REMOTE; i++) {
auto val = makeValue();
node2.put(remoteKey, val, [&](bool ok) {
std::lock_guard lk(mtx);
remotePutCount++;
if (ok)
remotePutOk++;
cv.notify_all();
});
}
{
std::unique_lock lk(mtx);
CPPUNIT_ASSERT(cv.wait_for(lk, 30s, [&] { return remotePutCount == N_REMOTE; }));
}
// All remote puts should succeed
CPPUNIT_ASSERT_EQUAL(N_REMOTE, remotePutOk);
std::this_thread::sleep_for(2s);
auto vals = node1.get(remoteKey).get();
CPPUNIT_ASSERT_MESSAGE("Remote values should be stored on node1 despite full local storage, got "
+ std::to_string(vals.size()),
!vals.empty());
}
} // namespace test
+57
View File
@@ -0,0 +1,57 @@
// Copyright (c) 2014-2026 Savoir-faire Linux Inc.
// SPDX-License-Identifier: MIT
#pragma once
// cppunit
#include <cppunit/TestFixture.h>
#include <cppunit/extensions/HelperMacros.h>
#include <opendht/dhtrunner.h>
namespace test {
class StorageTester : public CppUnit::TestFixture
{
CPPUNIT_TEST_SUITE(StorageTester);
CPPUNIT_TEST(testRemoteStorageLimit);
CPPUNIT_TEST(testLocalStorageLimit);
CPPUNIT_TEST(testIndependentLimits);
CPPUNIT_TEST(testLocalPutNotAffectedByRemoteLimit);
CPPUNIT_TEST(testRemotePutNotAffectedByLocalLimit);
CPPUNIT_TEST_SUITE_END();
dht::DhtRunner node1 {};
dht::DhtRunner node2 {};
/**
* Start both nodes with given configs, bootstrap node2 to node1
*/
void startNodes(dht::DhtRunner::Config config1, dht::DhtRunner::Config config2);
public:
void setUp();
void tearDown();
/**
* Test that remote values (received from other nodes) are limited by max_store_size
*/
void testRemoteStorageLimit();
/**
* Test that locally put values are limited by max_local_store_size
*/
void testLocalStorageLimit();
/**
* Test that remote and local limits work independently
*/
void testIndependentLimits();
/**
* Test that local puts are not blocked when remote storage is full
*/
void testLocalPutNotAffectedByRemoteLimit();
/**
* Test that remote puts are not blocked when local storage is full
*/
void testRemotePutNotAffectedByLocalLimit();
};
} // namespace test
@@ -1,7 +1,7 @@
// Copyright (c) 2014-2026 Savoir-faire Linux Inc.
// SPDX-License-Identifier: MIT
#include "threadpooltester.h"
#include "test_threadpool.h"
#include "opendht/thread_pool.h"
#include <atomic>
@@ -80,6 +80,46 @@ ThreadPoolTester::testContext()
CPPUNIT_ASSERT_EQUAL(N, count.load());
}
void
ThreadPoolTester::testJoinNonEmptyTaskQueue()
{
dht::ThreadPool pool(2);
constexpr unsigned N = 128;
std::atomic_uint count {0};
// Schedule enough work so that the task queue is non-empty when calling
// pool.join() below.
for (unsigned i = 0; i < N; i++)
pool.run([&] {
std::this_thread::sleep_for(std::chrono::milliseconds(1));
count++;
});
std::atomic_bool joined {false};
std::condition_variable cv;
std::thread joinThread([&] {
CPPUNIT_ASSERT(count.load() < N);
pool.join();
joined = true;
cv.notify_one();
});
// After calling join(), all pending tasks should complete, and then all
// threads in the pool should join.
std::mutex mtx;
std::unique_lock lock(mtx);
bool poolJoined = cv.wait_for(lock, std::chrono::seconds(10), [&] { return joined.load(); });
if (!poolJoined) {
// Detach joinThread if we timed out, otherwise the test can crash.
joinThread.detach();
} else {
joinThread.join();
}
CPPUNIT_ASSERT_EQUAL_MESSAGE("Some tasks were not completed", N, count.load());
CPPUNIT_ASSERT_MESSAGE("Thread pool failed to join before timeout", poolJoined);
}
void
ThreadPoolTester::tearDown()
{}
@@ -14,6 +14,7 @@ class ThreadPoolTester : public CppUnit::TestFixture
CPPUNIT_TEST(testThreadPool);
CPPUNIT_TEST(testExecutor);
CPPUNIT_TEST(testContext);
CPPUNIT_TEST(testJoinNonEmptyTaskQueue);
CPPUNIT_TEST_SUITE_END();
public:
@@ -29,6 +30,7 @@ public:
void testThreadPool();
void testExecutor();
void testContext();
void testJoinNonEmptyTaskQueue();
};
} // namespace test
+165
View File
@@ -0,0 +1,165 @@
// Copyright (c) 2014-2026 Savoir-faire Linux Inc.
// SPDX-License-Identifier: MIT
#include "test_value.h"
#include <iostream>
#include <string>
// opendht
#include "opendht/value.h"
#include "opendht/crypto.h"
#include <msgpack.hpp>
namespace test {
CPPUNIT_TEST_SUITE_REGISTRATION(ValueTester);
void
ValueTester::setUp()
{}
void
ValueTester::testConstructors()
{
std::string the_data {"42 cats"};
dht::Value the_dht_value {(const uint8_t*) the_data.data(), the_data.size()};
std::string from_value {the_dht_value.data.begin(), the_dht_value.data.end()};
CPPUNIT_ASSERT_EQUAL(the_data, from_value);
}
void
ValueTester::testFilter()
{
dht::Value::Filter defaultFiler {};
auto isPairSize = dht::Value::Filter([](const dht::Value& v) { return v.data.size() % 2 == 0; });
auto isUserTypeTest = dht::Value::Filter([](const dht::Value& v) { return v.user_type == "test"; });
std::string data1 {"42 cats"};
dht::Value value1 {(const uint8_t*) data1.data(), data1.size()};
value1.user_type = "test";
std::string data2 {"420 cats"};
dht::Value value2 {(const uint8_t*) data2.data(), data2.size()};
dht::Value value3 {(const uint8_t*) data2.data(), data2.size()};
value3.user_type = "test";
CPPUNIT_ASSERT(!isPairSize(value1));
CPPUNIT_ASSERT(isUserTypeTest(value1));
auto isBoth = dht::Value::Filter::chain(isPairSize, isUserTypeTest);
auto isUserTypeTest2 = dht::Value::Filter::chain(defaultFiler, isUserTypeTest);
CPPUNIT_ASSERT(isUserTypeTest2(value1));
CPPUNIT_ASSERT(!isUserTypeTest2(value2));
CPPUNIT_ASSERT(!isBoth(value1));
CPPUNIT_ASSERT(!isBoth(value2));
CPPUNIT_ASSERT(isBoth(value3));
}
void
ValueTester::tearDown()
{}
void
ValueTester::testPushTypeMsgpackRoundTrip()
{
dht::Value original {(const uint8_t*) "hello", 5};
original.id = 42;
original.priority = 3;
original.pushType = "audioCall";
msgpack::sbuffer buffer;
msgpack::packer<msgpack::sbuffer> pk(&buffer);
original.msgpack_pack(pk);
msgpack::unpacked msg;
msgpack::unpack(msg, buffer.data(), buffer.size());
dht::Value restored(msg.get());
CPPUNIT_ASSERT_EQUAL(original.id, restored.id);
CPPUNIT_ASSERT_EQUAL(original.priority, restored.priority);
CPPUNIT_ASSERT_EQUAL(original.pushType, restored.pushType);
}
void
ValueTester::testPushTypeAbsentAfterUnpack()
{
dht::Value withPt {(const uint8_t*) "data", 4};
withPt.id = 1;
withPt.pushType = "videoCall";
withPt.priority = 2;
msgpack::sbuffer buf1;
msgpack::packer<msgpack::sbuffer> pk1(&buf1);
withPt.msgpack_pack(pk1);
msgpack::unpacked msg1;
msgpack::unpack(msg1, buf1.data(), buf1.size());
dht::Value reused;
reused.msgpack_unpack(msg1.get());
CPPUNIT_ASSERT_EQUAL(std::string("videoCall"), reused.pushType);
CPPUNIT_ASSERT_EQUAL(2u, reused.priority);
dht::Value plain {(const uint8_t*) "data", 4};
plain.id = 2;
msgpack::sbuffer buf2;
msgpack::packer<msgpack::sbuffer> pk2(&buf2);
plain.msgpack_pack(pk2);
msgpack::unpacked msg2;
msgpack::unpack(msg2, buf2.data(), buf2.size());
reused.msgpack_unpack(msg2.get());
CPPUNIT_ASSERT_EQUAL(std::string(), reused.pushType);
CPPUNIT_ASSERT_EQUAL(0u, reused.priority);
}
void
ValueTester::testPushTypePreservedAfterEncrypt()
{
auto key = dht::crypto::PrivateKey::generate();
const auto& pubkey = key.getPublicKey();
dht::Value original {(const uint8_t*) "secret", 6};
original.id = 99;
original.priority = 1;
original.pushType = "audioCall";
auto encrypted = original.encrypt(key, pubkey);
CPPUNIT_ASSERT_EQUAL(original.pushType, encrypted.pushType);
CPPUNIT_ASSERT_EQUAL(original.priority, encrypted.priority);
CPPUNIT_ASSERT_EQUAL(original.id, encrypted.id);
CPPUNIT_ASSERT(encrypted.isEncrypted());
}
void
ValueTester::testPushTypeJsonRoundTrip()
{
#ifdef OPENDHT_JSONCPP
dht::Value original {(const uint8_t*) "hello", 5};
original.id = 42;
original.priority = 1;
original.pushType = "videoCall";
dht::Value restored(original.toJson());
CPPUNIT_ASSERT_EQUAL(original.id, restored.id);
CPPUNIT_ASSERT_EQUAL(original.priority, restored.priority);
CPPUNIT_ASSERT_EQUAL(original.pushType, restored.pushType);
// No pushType: the field must stay absent from the JSON and empty after parse.
dht::Value plain {(const uint8_t*) "data", 4};
plain.id = 7;
auto json = plain.toJson();
CPPUNIT_ASSERT(!json.isMember("pt"));
dht::Value plainRestored(json);
CPPUNIT_ASSERT_EQUAL(std::string(), plainRestored.pushType);
#endif
}
} // namespace test
@@ -13,6 +13,10 @@ class ValueTester : public CppUnit::TestFixture
CPPUNIT_TEST_SUITE(ValueTester);
CPPUNIT_TEST(testConstructors);
CPPUNIT_TEST(testFilter);
CPPUNIT_TEST(testPushTypeMsgpackRoundTrip);
CPPUNIT_TEST(testPushTypeAbsentAfterUnpack);
CPPUNIT_TEST(testPushTypePreservedAfterEncrypt);
CPPUNIT_TEST(testPushTypeJsonRoundTrip);
CPPUNIT_TEST_SUITE_END();
public:
@@ -32,6 +36,10 @@ public:
* Test compare operators
*/
void testFilter();
void testPushTypeMsgpackRoundTrip();
void testPushTypeAbsentAfterUnpack();
void testPushTypePreservedAfterEncrypt();
void testPushTypeJsonRoundTrip();
};
} // namespace test
+26 -2
View File
@@ -5,12 +5,29 @@
#include <cppunit/CompilerOutputter.h>
#include <iostream>
namespace {
void
listTests(CppUnit::Test* test, const std::string& prefix = {})
{
const auto name = test->getName();
const auto fullName = prefix.empty() ? name : prefix + "/" + name;
if (test->getChildTestCount() == 0) {
std::cout << fullName << std::endl;
return;
}
for (int index = 0; index < test->getChildTestCount(); ++index)
listTests(test->getChildTestAt(index), fullName);
}
} // namespace
extern "C" {
#include <gnutls/gnutls.h>
}
int
main(int /*argc*/, char** /*argv*/)
main(int argc, char** argv)
{
#ifdef _MSC_VER
if (auto err = gnutls_global_init()) {
@@ -25,8 +42,15 @@ main(int /*argc*/, char** /*argv*/)
std::cout << "No test cases specified for suite" << std::endl;
return 1;
}
if (argc > 1 && std::string(argv[1]) == "--list") {
listTests(suite);
return 0;
}
CppUnit::TextUi::TestRunner runner;
runner.addTest(suite);
if (argc > 1)
runner.addTest(suite->findTest(argv[1]));
else
runner.addTest(suite);
auto result = runner.run() ? 0 : 1;
return result;
}
-62
View File
@@ -1,62 +0,0 @@
// Copyright (c) 2014-2026 Savoir-faire Linux Inc.
// SPDX-License-Identifier: MIT
#include "valuetester.h"
#include <iostream>
#include <string>
// opendht
#include "opendht/value.h"
namespace test {
CPPUNIT_TEST_SUITE_REGISTRATION(ValueTester);
void
ValueTester::setUp()
{}
void
ValueTester::testConstructors()
{
std::string the_data {"42 cats"};
dht::Value the_dht_value {(const uint8_t*) the_data.data(), the_data.size()};
std::string from_value {the_dht_value.data.begin(), the_dht_value.data.end()};
CPPUNIT_ASSERT_EQUAL(the_data, from_value);
}
void
ValueTester::testFilter()
{
dht::Value::Filter defaultFiler {};
auto isPairSize = dht::Value::Filter([](const dht::Value& v) { return v.data.size() % 2 == 0; });
auto isUserTypeTest = dht::Value::Filter([](const dht::Value& v) { return v.user_type == "test"; });
std::string data1 {"42 cats"};
dht::Value value1 {(const uint8_t*) data1.data(), data1.size()};
value1.user_type = "test";
std::string data2 {"420 cats"};
dht::Value value2 {(const uint8_t*) data2.data(), data2.size()};
dht::Value value3 {(const uint8_t*) data2.data(), data2.size()};
value3.user_type = "test";
CPPUNIT_ASSERT(!isPairSize(value1));
CPPUNIT_ASSERT(isUserTypeTest(value1));
auto isBoth = dht::Value::Filter::chain(isPairSize, isUserTypeTest);
auto isUserTypeTest2 = dht::Value::Filter::chain(defaultFiler, isUserTypeTest);
CPPUNIT_ASSERT(isUserTypeTest2(value1));
CPPUNIT_ASSERT(!isUserTypeTest2(value2));
CPPUNIT_ASSERT(!isBoth(value1));
CPPUNIT_ASSERT(!isBoth(value2));
CPPUNIT_ASSERT(isBoth(value3));
}
void
ValueTester::tearDown()
{}
} // namespace test
-20
View File
@@ -1,20 +0,0 @@
bin_PROGRAMS = dhtnode dhtchat dhtscanner
noinst_HEADERS = tools_common.h
AM_CPPFLAGS = -isystem @top_srcdir@/include @JsonCpp_CFLAGS@ @MsgPack_CFLAGS@
dhtnode_SOURCES = dhtnode.cpp
dhtnode_LDFLAGS = -lopendht -lreadline -lfmt -L@top_builddir@/src/.libs @GnuTLS_LIBS@
dhtchat_SOURCES = dhtchat.cpp
dhtchat_LDFLAGS = -lopendht -lreadline -lfmt -L@top_builddir@/src/.libs @GnuTLS_LIBS@
dhtscanner_SOURCES = dhtscanner.cpp
dhtscanner_LDFLAGS = -lopendht -lreadline -lfmt -L@top_builddir@/src/.libs @GnuTLS_LIBS@
if ENABLE_C
bin_PROGRAMS += dhtcnode
dhtcnode_CFLAGS = -std=c11 -isystem @top_srcdir@/c -isystem @top_srcdir@/include
dhtcnode_SOURCES = dhtcnode.c
dhtcnode_LDFLAGS = -lopendht-c -lreadline -lfmt -L@top_builddir@/c/.libs
endif
+20 -14
View File
@@ -12,10 +12,10 @@ extern "C" {
}
#include <ctime>
using namespace dht;
namespace dht {
namespace tools {
static std::mt19937_64 rd {dht::crypto::getSeededRandomEngine<std::mt19937_64>()};
static std::uniform_int_distribution<dht::Value::Id> rand_id;
const std::string
printTime(const std::time_t& now)
@@ -34,12 +34,18 @@ print_usage()
std::cout << "Report bugs to: https://opendht.net" << std::endl;
}
} // namespace tools
} // namespace dht
int
main(int argc, char** argv)
{
auto params = parseArgs(argc, argv);
using namespace dht;
using IdDist = std::uniform_int_distribution<dht::Value::Id>;
auto params = tools::parseArgs(argc, argv);
if (params.help) {
print_usage();
tools::print_usage();
return 0;
}
#ifdef _MSC_VER
@@ -58,14 +64,14 @@ main(int argc, char** argv)
if (not params.bootstrap.empty())
dht.bootstrap(params.bootstrap);
print_node_info(dht.getNodeInfo());
tools::print_node_info(dht.getNodeInfo());
std::cout << " type 'c {hash}' to join a channel" << std::endl << std::endl;
bool connected {false};
InfoHash room;
std::future<size_t> token;
const InfoHash myid = dht.getId();
const auto mypk = dht.getPublicKey();
#ifndef _MSC_VER
// using the GNU History API
@@ -74,7 +80,7 @@ main(int argc, char** argv)
while (true) {
// using the GNU Readline API
std::string line = readLine(connected ? PROMPT : "> ");
std::string line = tools::readLine(connected ? tools::PROMPT : "> ");
if (!line.empty() && line[0] == '\0')
break;
if (line.empty())
@@ -95,12 +101,12 @@ main(int argc, char** argv)
}
token = dht.listen<dht::ImMessage>(room, [&](dht::ImMessage&& msg) {
if (msg.from != myid)
std::cout << msg.from.toString() << " at " << printTime(msg.date) << " (took "
if (msg.owner && msg.owner->getLongId() != mypk->getLongId())
std::cout << msg.from.toString() << " at " << tools::printTime(msg.date) << " (took "
<< print_duration(std::chrono::system_clock::now()
- std::chrono::system_clock::from_time_t(msg.date))
<< ") " << (msg.to == myid ? "ENCRYPTED " : "") << ": " << msg.id << " - "
<< msg.msg << std::endl;
<< ") " << (msg.to == mypk->getId() ? "ENCRYPTED " : "") << ": " << msg.id
<< " - " << msg.msg << std::endl;
return true;
});
connected = true;
@@ -117,15 +123,15 @@ main(int argc, char** argv)
iss >> idstr;
std::getline(iss, line);
dht.putEncrypted(room,
InfoHash(idstr),
dht::ImMessage(rand_id(rd), std::move(line), now),
PkId(idstr),
dht::ImMessage(IdDist()(tools::rd), std::move(line), now),
[](bool ok) {
// dht.cancelPut(room, id);
if (not ok)
std::cout << "Message publishing failed !" << std::endl;
});
} else {
dht.putSigned(room, dht::ImMessage(rand_id(rd), std::move(line), now), [](bool ok) {
dht.putSigned(room, dht::ImMessage(IdDist()(tools::rd), std::move(line), now), [](bool ok) {
// dht.cancelPut(room, id);
if (not ok)
std::cout << "Message publishing failed !" << std::endl;
Loaded 100 of 108 files, more files were not shown because too many files have changed in this diff. Show more