Compare commits

..
191 Commits
Author SHA1 Message Date
Adrien BeraudandCopilot dff27203a4 build: prepare 4.5.0 release candidate 1
Co-authored-by: Copilot <223556219+Copilot@users.nply.github.com>
2026-11-03 22:48:09 -05:00
Adrien BeraudandCopilot 506d9f9cbb ci: install MessagePack C++ headers on Ubuntu
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-11-03 22:44:50 -05:00
dependabot[bot] 349f4e5fc6 build(deps): bump pypa/cibuildwheel from 4.3.4 to 4.3.5
Bumps [pypa/cibuildwheel](https://github.com/pypa/cibuildwheel) from 4.3.4 to 4.3.5.
- [Release notes](https://github.com/pypa/cibuildwheel/releases)
- [Changelog](https://github.com/pypa/cibuildwheel/blob/main/docs/changelog.md)
- [Commits](https://github.com/pypa/cibuildwheel/compare/v4.3.4...v4.3.5)

---
updated-dependencies:
- dependency-name: pypa/cibuildwheel
  dependency-version: 4.3.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-11-03 22:41:04 -05:00
dependabot[bot] 11d69d02e8 build(deps): bump actions/setup-node from 6 to 7
Bumps [actions/setup-node](https://github.com/actions/setup-node) from 6 to 7.
- [Release notes](https://github.com/actions/setup-node/releases)
- [Commits](https://github.com/actions/setup-node/compare/v6...v7)

---
updated-dependencies:
- dependency-name: actions/setup-node
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-11-03 22:40:02 -05:00
Adrien Beraud 3c87991e92 docs: make Rust README precise and factual 2026-10-29 22:46:57 -04:00
Adrien Beraud 52287bf608 ci: use npm environment for publishing 2026-10-29 22:27:34 -04:00
Adrien Beraud 061ce2deb4 docker: build with RelWithDebInfo 2026-10-29 22:27:04 -04:00
Adrien Beraud 64b536fb53 ci: publish Python wheels from the tag release workflow 2026-10-29 21:48:33 -04:00
Adrien Beraud 67c76f17de README: add Node.js bindings 2026-10-29 21:20:25 -04:00
Adrien Beraud 25fde756e8 docs: add linked package and Docker badges 2026-10-29 21:12:35 -04:00
Adrien Beraud f6b3eda280 rust: rewrite README 2026-10-29 20:55:07 -04:00
Adrien Beraud 9aee407a42 rust: avoid dual-stack conflict in startup error test 2026-10-29 10:35:51 -04:00
Adrien Beraud 722ef44c7d rust: move owned Value directly through callbacks and streams 2026-10-28 23:51:46 -04:00
Adrien Beraud 6ed2720bd7 rust: test single-copy async value ownership 2026-10-28 23:51:46 -04:00
Adrien Beraud 9005f73141 rust: add optional Serde typed operations 2026-10-28 23:51:46 -04:00
Adrien Beraud 9785c0a127 rust: add typed operation regression tests 2026-10-28 23:51:41 -04:00
Adrien Beraud 5662cc36d1 rust: publish crate on release with trusted publishing 2026-10-28 22:04:57 -04:00
Adrien Beraud e116cfe548 rust: add optional runtime-independent async API 2026-10-28 21:11:43 -04:00
Adrien Beraud 1c5e8378b8 rust: add async adapter regression tests 2026-10-28 21:11:42 -04:00
Adrien Beraud c630e80345 ci: use trusted publishing for Python wheels 2026-10-29 00:26:07 +00:00
cyrilleandCopilot 17f83edce4 proxy: add Huawei (HMS) push type
Route push notifications to Huawei devices, which have no Google Play
Services and therefore cannot receive FCM. gorush, already used as the
push relay, supports HMS natively as platform 3.

HMS takes its payload as a serialized JSON string in "huawei_data" and
carries its own ttl/priority fields, so it needs its own branch rather
than the generic "data" object used for FCM and APNs.

The enum value is appended last: MSGPACK_ADD_ENUM serializes it as an
integer, so reordering would break compatibility with deployed proxies.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-10-29 00:02:02 +00:00
Adrien Beraud afab251376 nodejs: isolate native Linux dependency symbols
Use RTLD_DEEPBIND when loading the Linux addon so its llhttp calls do not resolve to Node's incompatible bundled parser. The CI image uses llhttp 9.2.1 while Node 22 exports 9.4.3, corrupting the heap when the proxy client starts. Preserve default loading on other platforms.
2026-10-27 23:04:32 -04:00
Adrien Beraud 86d452fbb6 nodejs: bundle source-built llhttp license notices
The Noble dependency image builds llhttp from source, so dpkg-query cannot identify its license package. Include the pinned upstream MIT notice when bundling Linux llhttp libraries while retaining package-owner checks for other dependencies.
2026-10-27 23:04:32 -04:00
Adrien Beraud 08ff775232 ci: rely on npm trusted publishing 2026-10-27 22:48:58 -04:00
Adrien Beraud 738e0e22bf nodejs: add streaming iterators and startup proxy configuration 2026-10-27 22:46:34 -04:00
Adrien Beraud 8ec5b47c86 nodejs: test streaming iterators and proxy configuration 2026-10-27 22:09:21 -04:00
Adrien Beraud e1c31568f9 readme: cleanup 2026-10-27 22:01:05 -04:00
Adrien Beraud 45bad769b4 nodejs: fix npm archive paths and validate release versions 2026-10-27 21:22:15 -04:00
Adrien Beraud 09780fd923 nodejs: publish verified npm archives from tag releases 2026-10-27 20:36:12 -04:00
Adrien Beraud 747e70d1ae nodejs: simplify design 2026-10-27 20:12:17 -04:00
Adrien Beraud 05abfa89bf nodejs: initialize GnuTLS before using the addon 2026-10-27 11:33:17 -04:00
Adrien Beraud b714714128 build: bump version to 4.4.99 2026-10-27 10:21:04 -04:00
Adrien Beraud 2e8b36477d ci: publish Noble OpenDHT release image 2026-10-27 09:59:17 -04:00
Adrien Beraud 14543ae477 docker: switch to ninja and add Noble Dockerfile
Use Ninja instead of Make in the main Dockerfile for faster builds
and install ninja-build in DockerfileDeps. Also, add a new
Dockerfile for Ubuntu Noble.
2026-10-27 09:57:33 -04:00
Adrien Beraud 044135732f docker: add simdutf to remaining dependency images 2026-10-27 00:31:34 -04:00
Adrien Beraud 21b7f0d757 docker: restore Noble dependency image for Node builds 2026-10-27 00:31:34 -04:00
Adrien Beraud a5bfd3d933 ci: test Node packages on Ubuntu 24.04 and 26.04 2026-10-27 00:26:08 -04:00
Adrien Beraud af38323728 nodejs: release dispatcher before environment teardown 2026-10-27 00:14:56 -04:00
Adrien Beraud 62e0ded803 ci: use Ubuntu 26.04 for Node.js packages 2026-10-27 00:14:56 -04:00
Adrien Beraud a09cfefe5e tests: require native alignment for C hashes 2026-10-27 00:03:07 -04:00
Adrien Beraud 13f340694e tools: rebuild proxy web client with concurrent live sessions 2026-10-27 00:03:07 -04:00
Adrien Beraud 3723c4acf1 nodejs: migrate runtime tooling and examples to ESM 2026-10-27 00:02:09 -04:00
Adrien Beraud f19bca0df1 tests: enforce ESM imports for Node bindings 2026-10-27 00:02:09 -04:00
Adrien Beraud fbc7f90897 tests: cover Node connectivity and concurrent CLI operations 2026-10-27 00:02:09 -04:00
Adrien Beraud 3a3c421ef9 nodejs: add an interactive dhtnode example 2026-10-27 00:02:09 -04:00
Adrien Beraud 9ec7a58ee3 tests: cover the Node dhtnode example CLI 2026-10-27 00:02:00 -04:00
Adrien Beraud ce73eb052e ci: build and verify Node-API npm packages 2026-10-27 00:02:00 -04:00
Adrien Beraud d716fae4bc nodejs: expose per-family connectivity status and events 2026-10-27 00:02:00 -04:00
Adrien Beraud a040bfeebb nodejs: add crypto APIs and relocatable npm prebuilds 2026-10-26 23:57:50 -04:00
Adrien Beraud 3223742d70 tests: cover Node crypto types and npm archive contents 2026-10-26 23:57:50 -04:00
Adrien Beraud d2d56fcc20 tests: cover Node crypto persistence and authenticated values 2026-10-26 23:57:50 -04:00
Adrien Beraud af814f027f nodejs: add Node-API runner bindings over the C++ library 2026-10-26 23:57:50 -04:00
Adrien Beraud 2016a9e2fa tests: verify Node TypeScript API and deferred garbage collection 2026-10-26 23:57:50 -04:00
Adrien Beraud 1db8abb7fc c: align hash structs with native types 2026-10-26 23:57:50 -04:00
Adrien Beraud 3f7b1bc3f5 tests: cover unaligned C hash storage 2026-10-26 23:56:59 -04:00
Adrien Beraud 019c25633e tests: initialize GnuTLS for MSVC C bindings 2026-10-26 23:56:43 -04:00
Adrien Beraud 8193f43593 tests: cover Node runner collection cancellation and queue overflow 2026-10-26 23:54:00 -04:00
Adrien Beraud 39564f14c1 tests: define Node-API binding lifecycle and loopback contracts 2026-10-26 23:54:00 -04:00
Adrien Beraud 83778d84fe rust: update edition to 2024
Update the Rust edition to 2024 and mark the FFI extern block as unsafe.
2026-10-26 09:57:02 -04:00
Adrien Beraud b61ee620e4 dht: schedule projected gets with matching reuse rules 2026-10-26 09:21:10 -04:00
Adrien Beraud 0d120eb5d4 tests: cover projected query request scheduling 2026-10-26 09:21:10 -04:00
Cyrille BéraudandCopilot 9b9056c0c8 dht: do not report unfiltered answers to filtered queries
Query::isSatisfiedBy() considers an unfiltered answer sufficient for a
filtered query, which is true for a vanilla get: onGetValuesDone() applies
the operation filter to the values before calling back.

A projected query is different. It receives the fields another query selected
and can no longer evaluate its own filters on them, all the more so as those
filters may target fields that were not selected at all. Two concurrent
queries such as "SELECT id" and "SELECT id WHERE value_type=1" therefore made
the filtered callback report identifiers whose type does not match, and
searchNodeGetDone() additionally cancelled its own request so it never got a
correctly filtered answer.

Add Query::isProjectionSatisfiedBy(), which additionally requires both queries
to apply the same filters, and use it for operations with a query callback.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-10-26 09:21:10 -04:00
Cyrille BéraudandCopilot bf8e3ea251 value: compare field values in FieldValueIndex::containedIn
containedIn() only checked that the other index declared the same field
keys, never that they held the same values, contradicting its documented
contract of containment "with the same value".

Dht::query() uses containedIn() to deduplicate projected results. Because
every result of a given query shares the same projection schema, any two
results compared equal: the first one suppressed all the later ones, and the
"lesser" branch even erased the already reported result. A query selecting a
field therefore reported a single value regardless of how many matched.

Compare the FieldValue as well, and add a regression covering distinct
values, equal indexes and a narrower projection contained in a wider one.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-10-26 09:21:10 -04:00
Adrien Beraud 34eb86a119 value: support sequence numbers in wire filters 2026-10-26 09:19:57 -04:00
Adrien Beraud 75c789ceb2 rust: cover sequence filters across loopback nodes 2026-10-26 09:15:36 -04:00
Adrien Beraud da438d1bd2 value: reproduce sequence filter serialization failure 2026-10-26 09:12:12 -04:00
Adrien Beraud 44e67e1389 rust: parse hexadecimal hashes with string length checks 2026-10-26 09:11:42 -04:00
Adrien Beraud 7f9126b3b0 rust: cover empty and short hexadecimal hashes 2026-10-26 09:10:39 -04:00
Adrien Beraud 574674d3a2 rust: own native pointers in movable handle wrappers 2026-10-26 09:10:16 -04:00
Adrien Beraud c60703bfc3 rust: cover moves across native handle types 2026-10-26 09:10:16 -04:00
Adrien Beraud a0220827cd rust: reproduce moving native values out of boxes 2026-10-26 09:08:03 -04:00
Adrien Beraud fb90211157 python: expose value state and harden runner operations 2026-10-26 01:10:34 -04:00
Adrien Beraud 9501465293 python: cover value state and binding safety regressions 2026-10-26 01:06:48 -04:00
Adrien Beraud 95bd26c3be rust: bump version to 0.2.0 and add author 2026-10-26 00:51:27 -04:00
Adrien Beraud 87463c2d6c rust: own callbacks and expose safe native binding APIs 2026-10-26 00:51:27 -04:00
Adrien Beraud baf413dd0d c: expose lifecycle queries crypto and callback ownership 2026-10-26 00:51:27 -04:00
Adrien Beraud 6bf01f3b6d runner: release status callback captures on join 2026-10-26 00:51:27 -04:00
Adrien Beraud 852403fbeb crypto: verify signatures using the public key algorithm 2026-10-26 00:51:27 -04:00
Adrien Beraud 5f6d5c769e rust: reject certificate registration on stopped runners 2026-10-26 00:51:27 -04:00
Adrien Beraud 42e4419b75 bindings: verify retained values and queued push delivery 2026-10-26 00:51:27 -04:00
Adrien Beraud 4e94d123b2 rust: cover threaded crypto and native callback teardown 2026-10-26 00:51:27 -04:00
Adrien Beraud 7f3f1b4ec4 rust: cover filtered queries and extension hooks 2026-10-26 00:51:27 -04:00
Adrien Beraud 6d462c0dbf rust: cover callback ownership and binding error paths 2026-10-26 00:51:27 -04:00
Adrien Beraud 0285acc766 c: cover lifecycle progress and crypto persistence APIs 2026-10-26 00:51:27 -04:00
Adrien Beraud 0626e82f4a c: contain crypto generation failures and run binding regressions 2026-10-26 00:51:27 -04:00
Adrien Beraud aa01f68cb5 c: cover binding ownership and invalid crypto input 2026-10-26 00:51:27 -04:00
Adrien Beraud af3fd618bf python: expose observability and projected query APIs 2026-10-26 00:51:27 -04:00
Adrien Beraud b96bdfbf04 python: cover observability and querying APIs 2026-10-26 00:51:27 -04:00
Adrien Beraud 60ca85f782 python: release native resources and guard callback cleanup 2026-10-26 00:51:27 -04:00
Adrien Beraud e4966aa457 python: cover native resource and callback cleanup regressions 2026-10-26 00:51:27 -04:00
Adrien Beraud aa9d23870b python: guard native errors, iterator lifetimes, and hash bounds 2026-10-26 00:51:27 -04:00
Adrien Beraud ac68db84eb python: cover binding exception and memory safety regressions 2026-10-26 00:51:27 -04:00
Adrien Beraud bbbd9122af python: use loopback addresses in bootstrap test 2026-10-26 00:51:27 -04:00
Adrien Beraud cf58d66282 python: expose node state persistence configuration 2026-10-26 00:51:27 -04:00
Adrien Beraud 2ddf1e5ec5 python: await shutdown before checking persisted state 2026-10-26 00:51:27 -04:00
Adrien Beraud e48bfd578a python: cover node state persistence configuration 2026-10-26 00:51:27 -04:00
Adrien Beraud 1c3fc22c31 meson: install Python bindings as opendht package 2026-10-26 00:51:27 -04:00
Adrien Beraud b6cf2fd246 python: expose key persistence and value metadata 2026-10-26 00:51:27 -04:00
Adrien Beraud 91c3f62c0c python: validate restored keys through certificate chains 2026-10-26 00:51:27 -04:00
Adrien Beraud 2703cf129d python: cover key persistence and value metadata 2026-10-26 00:51:27 -04:00
Adrien Beraud 51c2aadd4e python: handle cancelled and repeated async completions 2026-10-26 00:51:27 -04:00
Adrien Beraud 8b20e912d6 docker: use alpine 3.24 runtime 2026-10-26 00:51:27 -04:00
Adrien Beraud b66ff83fe2 python: preserve permanence in blocking puts 2026-10-26 00:51:27 -04:00
Adrien Beraud 552813e1b9 python: cover permanence forwarding and async cancellation 2026-10-26 00:51:27 -04:00
Adrien Beraud 601a8fbf80 python: release GIL while joining and destroying runners 2026-10-26 00:51:27 -04:00
Adrien Beraud 26488d18a6 python: translate crypto errors and guard empty objects 2026-10-26 00:51:27 -04:00
Adrien Beraud 53c509c54f python: cover crypto errors and GIL-safe teardown 2026-10-26 00:51:27 -04:00
Adrien Beraud def6a28d96 python: fix listener ownership and isolate run config 2026-10-26 00:51:27 -04:00
Adrien Beraud dbd5953b17 python: cover listener lifetime and config isolation 2026-10-26 00:51:27 -04:00
Cyrille BéraudandCopilot 2893823a77 securedht: forward push platform and topic
SecureDht only forwarded the push token to the DHT it wraps, so the
push platform and topic given to DhtRunner after the proxy client was
created never reached it. A device started without push platform
(e.g. a Jami account created by a build without push support) then
subscribed with an empty platform once given a token, and the proxy
server pushed it as an iOS device.

Co-authored-by: Copilot (Claude Opus 5.5) <223556219+Copilot@users.noreply.github.com>
2026-10-26 02:55:59 +00:00
Cyrille BéraudandCopilot 3ec03ccc0b proxy_server: keep push platform up to date
The DHT listen callback of a push subscription kept the push type and
topic of the request that created it. Refreshing the subscription
with another platform updated the listener, but values kept being
pushed to the first platform: an Android device that once subscribed
without platform was pushed as an iOS device forever (BadDeviceToken)
and never received values such as connection requests.

Keep the push type and topic in the session context shared with the
callback, and update them on each subscription refresh and when the
state is restored.

Co-authored-by: Copilot (Claude Opus 5.5) <223556219+Copilot@users.noreply.github.com>
2026-10-26 02:55:59 +00:00
Cyrille BéraudandCopilot 20fa751da0 tests: check push platform of proxy notifications
Move the fake push gateway of the full chain test into a reusable
helper, and add two tests showing that the proxy server pushes to the
wrong platform:
- when a device refreshes its subscription with another platform,
  new values keep being pushed with the platform of the first
  subscription;
- a platform set after the proxy client started is not used by its
  subscriptions.

Co-authored-by: Copilot (Claude Opus 5.5) <223556219+Copilot@users.noreply.github.com>
2026-10-26 02:55:59 +00:00
Adrien Beraud 0d572f0dab build: update supported Python versions
Drop Python 3.9 and 3.10 and add support for Python 3.15.
2026-10-25 18:16:26 -04:00
Adrien Beraud d9c401e1e2 docker: migrate to 26.04 for main LLVM image 2026-10-25 12:42:20 -04:00
Adrien Beraud dbdbe65a5d docker: fix local build context and missing dependencies 2026-10-25 12:39:30 -04:00
Adrien Beraud 6f5bec3af6 update version to 4.4.2 2026-10-25 12:11:46 -04:00
Adrien Beraud d07a9e6583 ci: update cibuildwheel to v4.3.4 2026-10-25 12:06:54 -04:00
Adrien Beraud fd43a329e7 deps: update RESTinio to 0.7.10 across Docker and CI 2026-10-25 11:58:53 -04:00
Adrien Beraud 93b2e8d5b9 ci: migrate vcpkg setup and binary caching 2026-10-25 13:46:35 +00:00
Adrien Beraud d66713d97e build: update vcpkg to 2026.10.16 2026-10-25 13:46:35 +00:00
Adrien Beraud 7af9868c47 docker: update to Alpine 3.24, Ubuntu 26.04 2026-10-24 23:25:08 -04:00
Adrien Beraud 05a524b038 docker: update RESTinio to 0.7.10 2026-10-24 23:25:08 -04:00
Adrien Beraud 57ed00579e pht: preserve prefix padding encoding 2026-10-25 03:03:35 +00:00
Adrien Beraud 9ffccb3956 pht: test prefix padding compatibility 2026-10-25 03:03:35 +00:00
Cyrille BéraudandCopilot d99324ebd4 pht: fix the bit arithmetic of Prefix
Prefix::size_ counts bits, but commonBits walked content_ a byte at a
time up to it. Two prefixes that agree on their whole common length are
therefore read well past their end, which AddressSanitizer reports as a
heap-buffer-overflow, and the count that comes back is eight times too
large. A lookup with a common prefix length hits this as soon as an
index entry matches the key it is looking for, and entry.prefix comes
straight off the network. The flags were indexed the same way, so a
partially known prefix was read at the wrong position too.

swapBit had the other half of the confusion: it flipped bit
(8 - bit) % 8 of the byte, while isActiveBit reads bit 7 - (bit % 8).
getSibling, which flips the last bit of a prefix to name the sibling
node in the trie, therefore returned a prefix differing in some other
bit and sharing the last one, so lookup and insertion looked up the
wrong node. Splitting a leaf and finding a canary both rely on it.

Add tests covering both, which fail before the fix: commonBits returns
193 for two identical 160 bit prefixes, and a prefix and its sibling
share 7 bits rather than all but the last.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-10-25 03:03:35 +00:00
copilot-swe-agent[bot]andaberaud 933f6cbca1 dhtcluster: import netifaces for interface selection
Co-authored-by: aberaud <101827+aberaud@users.noreply.github.com>
2026-10-25 02:45:22 +00:00
copilot-swe-agent[bot]andaberaud d5d7d6483f test: cover dhtcluster interface address selection
Co-authored-by: aberaud <101827+aberaud@users.noreply.github.com>
2026-10-25 02:45:22 +00:00
dependabot[bot] cc09a39ade build(deps): bump pypa/cibuildwheel from 4.1.1 to 4.3.1
Bumps [pypa/cibuildwheel](https://github.com/pypa/cibuildwheel) from 4.1.1 to 4.3.1.
- [Release notes](https://github.com/pypa/cibuildwheel/releases)
- [Changelog](https://github.com/pypa/cibuildwheel/blob/main/docs/changelog.md)
- [Commits](https://github.com/pypa/cibuildwheel/compare/v4.1.1...v4.3.1)

---
updated-dependencies:
- dependency-name: pypa/cibuildwheel
  dependency-version: 4.3.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-10-25 02:44:52 +00:00
Adrien Beraud 621b44c8e8 Revert "disable windows wheel for now"
This reverts commit fc6edb4d20.
2026-09-23 14:42:57 -04:00
Adrien Beraud e1c088f2e4 build: bump version to 4.4.1 2026-09-23 14:38:36 -04:00
Adrien Beraud a6e0e441ac crypto: use standard unsigned int* for gnutls_ocsp_resp_get_single 2026-09-22 11:31:45 -04:00
Adrien Beraud dcc2ba3492 crypto: use gnutls_ocsp_cert_status_t* in gnutls_ocsp_resp_get_single 2026-09-22 11:31:45 -04:00
Adrien Beraud ccf00d6b09 ports: update shiftmedia-libgnutls to f76d780a6 2026-09-22 11:31:45 -04:00
Adrien Beraud d8250574fc ports: use ForcedIncludeFiles for gnutls_msvc_fixes.h in shiftmedia-libgnutls 2026-09-22 11:31:45 -04:00
Adrien Beraud 0d698c76e8 ports: update shiftmedia-libgnutls to 9ebeb826d 2026-09-22 11:31:45 -04:00
Adrien Beraud e4e3b4e73b ports: set LanguageStandard_C to stdc11 in shiftmedia-libgnutls 2026-09-22 11:31:45 -04:00
Adrien Beraud b26d622f5d ports: update shiftmedia-libgnutls to 37840cfa5 2026-09-22 11:31:45 -04:00
Adrien Beraud 78694751bd ports: update shiftmedia-libgnutls to 4ab78d57e 2026-09-22 11:31:45 -04:00
Adrien Beraud f9b90c5c1c ports: update shiftmedia-libgnutls to 5a5a9dd95 2026-09-22 11:31:45 -04:00
Adrien Beraud 56b848bf47 ports: fix savoirfairelinux/gnutls commit hash and SHA512 in shiftmedia-libgnutls 2026-09-22 11:31:45 -04:00
Adrien Beraud dd875230c3 ports: update shiftmedia-libgnutls to 90eb686e8 2026-09-22 11:31:45 -04:00
Adrien Beraud de1d428e63 ports: use gitlab gnulib mirror archive in shiftmedia-libgnutls 2026-09-22 11:31:45 -04:00
Adrien Beraud 55e3a6514f ports: fix gnulib snapshot SHA512 hash in shiftmedia-libgnutls 2026-09-22 11:31:45 -04:00
Adrien Beraud f4bbef210c ports: fix savoirfairelinux/gnutls archive SHA512 hash in shiftmedia-libgnutls 2026-09-22 11:31:45 -04:00
Adrien Beraud 2c1470cef6 vcpkg: add overlay port for savoirfairelinux/gnutls on Windows MSVC 2026-09-22 11:31:45 -04:00
Adrien Beraud adb6aa57e6 update vcpkg 2026-09-22 11:31:45 -04:00
Adrien Beraud acd0abdb70 ci: fix vcpkg cache keys across workflows and remove manual version string 2026-09-22 11:31:45 -04:00
Cyrille BéraudandCopilot b9c8f5e0fd runner: complete pushes received while stopped
Resolve idle-runner push futures immediately under the operation queue lock. A stopped runner has no loop left to execute the queued promise, otherwise consumers can wait indefinitely.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-09-21 16:02:47 -04:00
Cyrille BéraudandCopilot 18fedf431f tests: cover pushes received by stopped runners
Require immediate completion before run and after join, including a burst of notifications. Keep active-runner coverage.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-09-21 16:02:47 -04:00
Adrien Beraud fc6edb4d20 disable windows wheel for now 2026-09-19 22:09:44 -04:00
Adrien Beraud d1093c77a3 ci: use libmsgpack-cxx-dev on Debian 13 2026-09-19 21:56:06 -04:00
Adrien Beraud b543f1e3cb update vcpkg to 2026.06.01 2026-09-19 21:10:44 -04:00
Adrien Beraud be210e27da update version to 4.4.0 2026-09-04 16:46:41 -04:00
Adrien Beraud f7243fca37 value_cache: only emit an edition when the content actually changed
ValueCache re-emitted a known value as an edition on a higher seq alone,
while OpValueCache requires the content to differ as well. Since
Value::operator== ignores seq, identical content re-announced with a
bumped seq was re-emitted as an add: a source-counting OpValueCache
then counted the same node twice and kept the value after the node
expired or left the search. Apply the same rule at both levels.
2026-09-04 15:44:35 -04:00
Adrien Beraud 1f6efd5736 tests: cover same-content higher-seq re-announce in ValueCache
Value::operator== ignores seq, so a node re-announcing identical content
with a bumped seq is the same value refreshed, not an edition. ValueCache
must not re-emit it: a source-counting OpValueCache above would count the
node twice and keep the value after the node is gone.
2026-09-04 15:44:35 -04:00
Adrien Beraud 54fbad3a7e op_cache: ignore expiry of versions that were never counted
onValuesAdded ignores a different payload that does not carry a higher
seq (e.g. the same signed content re-signed with a randomized
signature), but onValuesExpired only skipped strictly older versions.
The uncounted source could therefore decrement the refcount and expire
a value still held by others. Apply the same rule on both sides.
2026-09-04 15:18:35 -04:00
Adrien Beraud 1315489a7b tests: cover expiry of an uncounted same-seq version in OpValueCache
A source reporting a different payload with the same seq is ignored on
add and not counted; its later expiration must not release the value
still held by the counted sources.
2026-09-04 15:18:35 -04:00
Adrien Beraud c189fa0e82 proxy_server: detach pending push requests before destruction
~DhtProxyServer destroyed requests_ whose ~http::Request notifies DONE;
the handler registered by sendPushNotification then erased from the map
being destroyed, corrupting the heap (double free under ASan) whenever a
push request was still pending, e.g. an unreachable push server. Detach
the state-change callbacks and release the requests before the members
are destroyed.
2026-09-04 15:18:24 -04:00
Adrien Beraud 3eae03289e storage: re-storing identical content is a refresh, not an edition
Storage::store reported edited_values=1 whenever the stored Sp<Value>
differed from the incoming one, even with equal content. A local put()
of an already-stored value was therefore announced to local listeners
as a new value a second time, unbalancing listen caches that count how
many sources hold a value. Only report an edition when the content
actually changed; the expiration is still refreshed.
2026-09-04 15:18:24 -04:00
Adrien Béraud e5e5a45b91 op_cache, proxy_client: guard SearchCache::get and push refresh expiry
SearchCache::get called the GetCallback unconditionally; Dht::query
passes none, which threw bad_function_call as soon as a listen cache
existed for the key. Return false without a callback so the network
path runs. A callback returning false means the consumer has enough
values, not a failure: report done(true) as the network path does.

DhtProxyClient push refresh expired every previously cached value once
the refresh get completed, including values that were just re-delivered
or edited, producing phantom expire callbacks. Track the ids received by
the get and only expire cached values missing from it, and skip the
reconciliation when the get failed so a transient error does not expire
values still present on the DHT.
2026-09-04 15:18:24 -04:00
Adrien Beraud 7a71beb892 value_cache: detect editions by increasing seq, like OpValueCache
ValueCache emitted any content change of a known value as an edition
while OpValueCache only accepts a higher seq. A node reporting an older
version made ValueCache replace its copy and re-emit versions that were
then dropped or, with source counting, double counted downstream. Use
the same rule at both levels.
2026-09-04 15:18:24 -04:00
Adrien Béraud 74a894229f value_cache, op_cache: fix value accounting after edition and re-delivery
OpValueCache counted every re-delivery of an identical value as a new
source, but the proxy client is a single logical source that re-sends
values on push refresh or listen restart: the refcount grew unbounded
and expirations were never propagated. Add a countSources flag to
OpValueCache, OpCache and SearchCache; the proxy client disables it.

When a value is edited (same id, higher seq), only the reporting source
is known to hold the new version: replace it in place, reset the
refcount to 1 and emit a single add. Ignore older versions from lagging
sources on add, and ignore their expiration so they cannot release a
newer version still held by others.

ValueCache emits an edited value as an add instead of silently
refreshing its old copy, and applies a grace period of one type
expiration while the listen is synced so that a single lost refresh
notification does not drop a value the remote node still holds.

SearchCache::get only answers from the cache once the listen is synced,
so a get never returns a partial view that the network would complete.

Add the single-source OpCacheTester cases enabled by countSources.
2026-09-04 15:18:13 -04:00
Adrien Beraud c81c1f70c8 tests: add listen cache tests for OpValueCache, ValueCache, DhtRunner and proxy
OpCacheTester and ValueCacheTester exercise the listen caches in
isolation: add/edit/expire flows, per-source refcounting, stale
versions, sync status, SearchCache::get and node drop.

DhtRunnerTester and DhtProxyTester gain end-to-end listen tests for
value edition, duplicate puts, multi-source and partial expiration,
get after a synced listen, and the push refresh/expire paths.

The proxy fixture joins the proxy DhtRunner before destroying the
DhtProxyServer so no DHT callback can reach a destroyed server, and
push notifications are never awaited while holding the mutex taken by
listen callbacks: the result is produced on the DHT thread.

Register the new executables in CMake and Meson.
2026-09-04 15:17:34 -04:00
Adrien BeraudandCopilot 3a383ac873 http: close connections on the io_context thread
asio sockets are not thread-safe. http::Connection::close() used to close
the socket synchronously on the calling thread. Request::cancel() and
Request::~Request() (via terminate()) are called from arbitrary threads
(e.g. DhtProxyClient::stop(), cancelAllListeners(), cancelListen()),
so the close raced with in-flight operations on the io_context thread.

In particular, asio::async_connect's range_connect_op closes and
re-opens the socket when trying the next endpoint (common on mobile
where the IPv6 endpoint fails first). A concurrent close() from another
thread corrupts the epoll reactor's per-descriptor state, leading to
crashes in epoll_reactor::deregister_descriptor / register_descriptor
called from range_connect_op::process.

Dispatch the close to the io_context thread when not already on it.
Also keep the connection alive during the TLS handshake, like every
other asynchronous operation, so that the destructor can never run
while an operation is pending.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-09-04 14:13:20 -04:00
Adrien Beraud 3149fe60c4 proxy: increase OP_MARGIN to 8 hours 2026-09-04 11:04:42 -04:00
Adrien Beraud d5956e0778 proxy: use system_clock for subscriber expiration timeouts
Track subscribe/permanent-put expiration with std::chrono::system_clock
(wall clock) instead of steady_clock in both the proxy client and server,
so persisted expirations survive restarts and map directly to epoch time.

- DhtProxyServer::clock is now system_clock; expireTimer/expireNotifyTimer
  use asio::system_timer.
- DhtProxyClient::refreshSubscriberTimer uses asio::system_timer and
  system_clock::now() for subscriber refresh.
2026-09-04 11:04:42 -04:00
Adrien Beraud 8baf53d078 utils: fix clock conversion overflow with mismatched durations
to_system_time computed its saturation bounds by subtracting a
steady_clock duration from system_clock::time_point::max(). When the
two clocks have different periods (microseconds for system_clock and
nanoseconds for steady_clock on libc++), the common type promotes the
bound to nanoseconds and overflows, so every future time point was
converted to system_clock::time_point::max().

Cast the delta to the target duration before comparing against the
bounds, and saturate from_system_time before the reverse cast for
symmetry.
2026-09-01 22:56:16 -04:00
Adrien Beraud 82ab21dc3f crypto: support explicit EC curve generation 2026-09-01 21:34:36 -04:00
Adrien Beraud d676aef027 http: support legacy X509 lookup signature 2026-08-06 14:05:48 -04:00
Adrien Beraud a2d9fdf334 Recompute runner wakeup after import schedules value expirations. 2026-08-06 13:59:48 -04:00
Adrien Beraud 0cec4135c0 update to 4.3.0 2026-08-05 16:35:01 -04:00
Adrien Beraud 6f76686a3e ci: update to Debian 13 2026-08-05 16:19:46 -04:00
Adrien Beraud c211b3c376 dht: persist values with system time 2026-08-05 16:14:15 -04:00
Adrien Beraud 0179b6ecc7 test: cover system time state persistence 2026-08-05 16:14:15 -04:00
Adrien Beraud 44e1ad839e net: harden fragmented reply completion 2026-08-05 14:29:03 -04:00
Adrien Beraud dddd97daff tests: cover partial reply hardening 2026-08-05 14:29:03 -04:00
Adrien Beraud 632bec459e net: avoid partial key allocation for regular messages 2026-08-05 14:29:03 -04:00
Cyrille BéraudandCopilot 910ca10a5f net: reject malformed error and request fields
An error array was read at index 0 without checking its size, so a packet
carrying "e":[] dereferenced a null element pointer and crashed the process
on unauthenticated UDP input.

When a message had an error array but no "a", "r" or "u" map, the same object
was then walked through the map view of the msgpack union. A two-element array
appeared to hold two key/value pairs, reading past its allocation. A request
field that was not a map was never type-checked either.

Validate the error array size, restrict request parsing to "a", "r" and "u",
and require those to be maps. Add parser regressions for the empty error
array, the error-only message and the non-map request; the first one
segfaults without this change.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-08-05 14:29:03 -04:00
Cyrille BéraudandCopilot 0826822a4d net: harden fragmented message reassembly
Key partial messages by source and transaction ID, buffer bounded fragments received before their header, and avoid charging reordered requests twice against rate limits. Cover source collisions, early fragments, empty data and large values.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-08-05 14:29:03 -04:00
Cyrille BéraudandCopilot 8aa6b009c3 net: reject empty field selections
Reject field-compressed responses with no selected fields before computing the value count, avoiding a division by zero on malformed network input. Add a parser regression test.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-08-05 14:29:03 -04:00
Cyrille BéraudandCopilot 2caa21d65f dht: let a node that lost the network try to rejoin it
Dht::onDisconnected() only starts a bootstrap when bootstrapJob is
unset, reading a set handle as "a bootstrap is already scheduled".
That reading is wrong: Scheduler::run() erases the job from its timers
and calls it, but never clears the shared handle the caller holds, and
Scheduler::cancel() leaves the handle alone too when the job is no
longer in the timers. The handle therefore stays set for as long as the
Dht lives, once a bootstrap job has run.

That is harmless while bootstrap() reschedules itself, which it does as
long as the node is disconnected. It is not harmless on the early
return: a job that fires while a ping is still in flight, or while a
dubious contact keeps the status at Connecting, returns without
rescheduling. From then on the chain is dead and onDisconnected() will
not restart it, so a node that goes on to lose every contact stays
disconnected for good. Only an explicit addBootstrap() or a
connectivityChanged() can revive it, and a node whose network simply
went away gets neither.

Clearing the handle on the early return restores the meaning
onDisconnected() relies on.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-08-03 11:34:25 -04:00
Adrien Beraud cf64fa454c crypto: encode certificate serials in network order 2026-08-02 17:24:55 -04:00
Adrien Beraud ef9ef12195 tests: cover positive certificate serials 2026-08-02 17:24:55 -04:00
Cyrille BéraudandCopilot 74a1918a4d crypto: strip the terminating null byte from an exported public key
gnutls_pubkey_export reports the size of the exported PEM including its
terminating null byte, and PublicKey::toString hands that size straight
back to the caller. The returned string therefore ends with a null byte
that is not part of the PEM text.

Certificate::toString does not have the problem because
gnutls_x509_crt_export excludes the terminator, so the two exports
disagree on what a PEM string is.

The stray byte travels: Value::toJson stores owner->toString() in the
"owner" field, so a value published through the proxy carries a null
byte inside a JSON string, which is valid JSON but a surprise for any
consumer that treats the field as text.

Resize to the reported size and drop the terminator when it is there.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-08-02 17:20:04 -04:00
dependabot[bot] 56a60fb557 build(deps): bump pypa/cibuildwheel from 4.1.0 to 4.1.1
Bumps [pypa/cibuildwheel](https://github.com/pypa/cibuildwheel) from 4.1.0 to 4.1.1.
- [Release notes](https://github.com/pypa/cibuildwheel/releases)
- [Changelog](https://github.com/pypa/cibuildwheel/blob/main/docs/changelog.md)
- [Commits](https://github.com/pypa/cibuildwheel/compare/v4.1.0...v4.1.1)

---
updated-dependencies:
- dependency-name: pypa/cibuildwheel
  dependency-version: 4.1.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-07-30 10:17:04 -04:00
Adrien Beraud f98df1d615 crypto: allow signing a revocation list with an explicit CRL number
RevocationList::sign() always overwrote the CRL Number extension: random
on the first signature, incremented afterwards. This leaves no way for a
certificate authority replicated over several devices to keep numbers
unique, since every device branching from the same list mints the same
number for a different content.

Add an optional `number` parameter, used verbatim when non-zero, so
callers can partition the number space (RFC 5280 5.2.3 requires
monotonicity for a given CRL scope and issuer, not globally).

Also fix the default increment path: the extension holds a DER integer
with leading zeros stripped, so reading it straight into an uint64_t only
worked for numbers wide enough to occupy all 8 bytes. Signing a list
numbered 5 produced 0x0500000000000001 instead of 6. Read and write the
number as a big-endian byte string instead, which makes the `endian()`
helper unnecessary.

Add tests covering the default numbering, explicit numbers (including
values with leading zero bytes and the largest positive 8-byte integer),
pack/unpack round-trips, and a device-partitioned numbering scheme.
2026-07-29 13:26:23 -04:00
Adrien Beraud 8f779f2efb http: fix build on OpenSSL 4.0 2026-07-20 11:25:02 -04:00
Adrien Beraud 699073acd8 update version to 4.2.0 2026-07-17 14:09:54 -04:00
Cyrille 02d49b9f6a http: run completed resolver callbacks on io executor 2026-07-15 14:07:14 -04:00
Adrien Beraud 85b0bc52c8 test: reproduce completed resolver callback race 2026-07-15 14:07:14 -04:00
Adrien Beraud af02c3a117 ci: install brew packages for macOS 15 2026-07-15 12:16:28 -04:00
122 changed files with 18632 additions and 1542 deletions

No files matched your search

+12
View File
@@ -0,0 +1,12 @@
.git
.vscode
.DS_Store
build
build-*
build_*
install
vcpkg_installed
rust/target
**/__pycache__
python/dist
python/*.egg-info
+21 -9
View File
@@ -2,12 +2,12 @@ name: C/C++ CI
env:
LLHTTP_VERSION: "v9.2.1"
RESTINIO_VERSION: "0.7.8.1"
RESTINIO_VERSION: "0.7.10"
EXPECTED_LITE_URL: "https://raw.githubusercontent.com/martinmoene/expected-lite/master/include/nonstd/expected.hpp"
UBUNTU_FULL_DEPS: "libncurses5-dev libreadline-dev nettle-dev libasio-dev libgnutls28-dev libuv1-dev python3-pip python3-dev python3-setuptools libcppunit-dev libjsoncpp-dev libfmt-dev libmsgpack-dev libargon2-dev"
UBUNTU_MESON_DEPS: "ninja-build libncurses5-dev libreadline-dev nettle-dev libasio-dev libgnutls28-dev libuv1-dev python3-dev python3-setuptools python3-pip libcppunit-dev libjsoncpp-dev libfmt-dev libmsgpack-dev libargon2-dev"
UBUNTU_MINIMAL_DEPS: "libncurses5-dev libreadline-dev nettle-dev libfmt-dev libgnutls28-dev libcppunit-dev libmsgpack-dev libargon2-dev"
DEBIAN12_DEPS: "git libtool pkg-config wget cmake libncurses5-dev libreadline-dev nettle-dev libasio-dev libgnutls28-dev libuv1-dev python3-pip python3-dev python3-setuptools libcppunit-dev libjsoncpp-dev libfmt-dev libmsgpack-dev libargon2-dev"
UBUNTU_FULL_DEPS: "libncurses5-dev libreadline-dev nettle-dev libasio-dev libgnutls28-dev libuv1-dev python3-pip python3-dev python3-setuptools libcppunit-dev libjsoncpp-dev libfmt-dev libmsgpack-cxx-dev libargon2-dev"
UBUNTU_MESON_DEPS: "ninja-build libncurses5-dev libreadline-dev nettle-dev libasio-dev libgnutls28-dev libuv1-dev python3-dev python3-setuptools python3-pip libcppunit-dev libjsoncpp-dev libfmt-dev libmsgpack-cxx-dev libargon2-dev"
UBUNTU_MINIMAL_DEPS: "libncurses5-dev libreadline-dev nettle-dev libfmt-dev libgnutls28-dev libcppunit-dev libmsgpack-cxx-dev libargon2-dev"
DEBIAN13_DEPS: "git libtool pkg-config wget cmake libncurses5-dev libreadline-dev nettle-dev libasio-dev libgnutls28-dev libuv1-dev python3-pip python3-dev python3-setuptools libcppunit-dev libjsoncpp-dev libfmt-dev libmsgpack-cxx-dev libargon2-dev"
on: [push, pull_request]
@@ -33,6 +33,18 @@ jobs:
- name: Build with Ninja
run: cd build && ninja
- uses: dtolnay/rust-toolchain@stable
- name: Test Rust bindings
env:
LIBRARY_PATH: ${{ github.workspace }}/build
LD_LIBRARY_PATH: ${{ github.workspace }}/build
RUSTFLAGS: -L native=${{ github.workspace }}/build
run: |
cargo test --manifest-path rust/Cargo.toml --all-targets
cargo test --manifest-path rust/Cargo.toml --all-features --all-targets
cargo test --manifest-path rust/Cargo.toml --all-features --doc
build-ubuntu-minimal:
name: Ubuntu/GCC minimal build
runs-on: ubuntu-latest
@@ -65,12 +77,12 @@ jobs:
- name: Run Unit tests
run: ctest --output-on-failure --test-dir build
build-debian12:
name: Debian 12/GCC build
build-debian13:
name: Debian 13/GCC build
runs-on: ubuntu-latest
container:
image: debian:12
image: debian:13
volumes:
- .:/workspace
options: --workdir /workspace
@@ -83,7 +95,7 @@ jobs:
- name: Install Dependencies
run: |
apt-get update
apt-get install -y $DEBIAN12_DEPS
apt-get install -y $DEBIAN13_DEPS
- name: Build llhttp dependency
run: |
+6 -4
View File
@@ -12,7 +12,7 @@ jobs:
sudo apt update && \
sudo apt install libncurses5-dev libreadline-dev nettle-dev \
libgnutls28-dev libuv1-dev python3-dev python3-setuptools libcppunit-dev libjsoncpp-dev \
libfmt-dev libmsgpack-dev libargon2-dev libasio-dev \
libfmt-dev libmsgpack-cxx-dev libargon2-dev libasio-dev \
llvm llvm-dev clang clang-tools && \
sudo apt remove gcc g++
@@ -23,11 +23,13 @@ jobs:
-O /usr/include/nonstd/expected.hpp
- name: restinio
env:
RESTINIO_VERSION: "0.7.10"
run: |
mkdir restinio && cd restinio
wget https://github.com/Stiffstream/restinio/releases/download/v0.7.8.1/restinio-0.7.8.1.tar.bz2
ls -l && tar -xjf restinio-0.7.8.1.tar.bz2
cd restinio-0.7.8.1/dev
wget "https://github.com/Stiffstream/restinio/releases/download/v${RESTINIO_VERSION}/restinio-${RESTINIO_VERSION}.tar.bz2"
ls -l && tar -xjf "restinio-${RESTINIO_VERSION}.tar.bz2"
cd "restinio-${RESTINIO_VERSION}/dev"
cmake -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=Off -DRESTINIO_SAMPLE=Off -DRESTINIO_BENCHMARK=Off \
-DRESTINIO_WITH_SOBJECTIZER=Off -DRESTINIO_DEP_STANDALONE_ASIO=system -DRESTINIO_DEP_LLHTTP=system \
-DRESTINIO_DEP_FMT=system -DRESTINIO_DEP_EXPECTED_LITE=system .
+206
View File
@@ -0,0 +1,206 @@
name: Node.js Package
on:
push:
branches: ['**']
paths:
- 'nodejs/**'
- '.github/workflows/nodejs-package.yml'
- 'src/**'
- 'include/**'
- 'cmake/**'
- 'CMakeLists.txt'
- 'vcpkg*.json'
- 'ports/**'
- 'LICENSE'
pull_request:
paths:
- 'nodejs/**'
- '.github/workflows/nodejs-package.yml'
- 'src/**'
- 'include/**'
- 'cmake/**'
- 'CMakeLists.txt'
- 'vcpkg*.json'
- 'ports/**'
- 'LICENSE'
workflow_call:
workflow_dispatch:
permissions:
contents: read
packages: read
jobs:
build:
name: Build ${{ matrix.target }}
runs-on: ${{ matrix.os }}
container:
image: ${{ matrix.container }}
credentials:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
timeout-minutes: 45
defaults:
run:
working-directory: nodejs
strategy:
fail-fast: false
matrix:
include:
- os: ubuntu-26.04
target: linux-x64
container: ghcr.io/${{ github.repository }}/opendht-deps-noble:latest
- os: macos-15
target: darwin-arm64
- os: macos-15-intel
target: darwin-x64
- os: windows-2022
target: win32-x64
env:
VCPKG_DISABLE_METRICS: '1'
VCPKG_DEFAULT_TRIPLET: x64-windows-release
VCPKG_INSTALLED_DIR: ${{ github.workspace }}/vcpkg_installed
VCPKG_BINARY_SOURCES: clear;files,${{ github.workspace }}/vcpkg-binary-cache,readwrite
steps:
- uses: actions/checkout@v7
- uses: actions/setup-node@v7
with:
node-version: '22'
cache: npm
cache-dependency-path: nodejs/package-lock.json
- name: Install macOS dependencies
if: runner.os == 'macOS'
run: brew install cmake ninja pkg-config dylibbundler gnutls nettle argon2 msgpack-cxx fmt jsoncpp simdutf asio llhttp openssl
- name: Install restinio
if: runner.os == 'macOS'
env:
RESTINIO_VERSION: '0.7.10'
run: |
include="$RUNNER_TEMP/opendht-proxy/include"
mkdir -p "$include/nonstd"
curl -fsSL --retry 3 https://raw.githubusercontent.com/martinmoene/expected-lite/v0.10.0/include/nonstd/expected.hpp -o "$include/nonstd/expected.hpp"
curl -fsSL --retry 3 "https://github.com/Stiffstream/restinio/releases/download/v${RESTINIO_VERSION}/restinio-${RESTINIO_VERSION}.tar.bz2" -o "$RUNNER_TEMP/restinio.tar.bz2"
tar -xjf "$RUNNER_TEMP/restinio.tar.bz2" -C "$RUNNER_TEMP"
cp -R "$RUNNER_TEMP/restinio-${RESTINIO_VERSION}/dev/restinio" "$include/"
echo "CMAKE_PREFIX_PATH=$RUNNER_TEMP/opendht-proxy" >> "$GITHUB_ENV"
- name: Set up MSVC
if: runner.os == 'Windows'
uses: ilammy/msvc-dev-cmd@v1
with:
arch: x64
- name: Set up vcpkg
if: runner.os == 'Windows'
uses: lukka/run-vcpkg@v11
with:
vcpkgJsonGlob: vcpkg.json
vcpkgConfigurationJsonGlob: vcpkg-configuration.json
- name: Cache Windows native dependencies
if: runner.os == 'Windows'
uses: actions/cache@v6
with:
path: ${{ github.workspace }}/vcpkg-binary-cache
key: node-vcpkg-${{ env.VCPKG_DEFAULT_TRIPLET }}-${{ hashFiles('vcpkg.json', 'vcpkg-configuration.json', 'ports/**') }}-${{ github.run_id }}-${{ github.run_attempt }}
restore-keys: |
node-vcpkg-${{ env.VCPKG_DEFAULT_TRIPLET }}-${{ hashFiles('vcpkg.json', 'vcpkg-configuration.json', 'ports/**') }}-
- run: npm ci --no-audit --no-fund
- name: Build Unix addon
if: runner.os != 'Windows'
shell: bash
run: |
if [[ "$RUNNER_OS" == 'macOS' ]]; then
export MACOSX_DEPLOYMENT_TARGET=15.0
openssl_prefix=$(brew --prefix openssl)
export PKG_CONFIG_PATH="$openssl_prefix/lib/pkgconfig${PKG_CONFIG_PATH:+:$PKG_CONFIG_PATH}"
fi
npm run build -- --CDOPENDHT_DOWNLOAD_DEPS=OFF
- name: Build Windows addon
if: runner.os == 'Windows'
shell: pwsh
run: |
npm run build -- `
"--CDCMAKE_TOOLCHAIN_FILE=$env:VCPKG_ROOT/scripts/buildsystems/vcpkg.cmake" `
"--CDVCPKG_MANIFEST_DIR=$env:GITHUB_WORKSPACE" `
"--CDVCPKG_INSTALLED_DIR=$env:VCPKG_INSTALLED_DIR" `
"--CDVCPKG_TARGET_TRIPLET=$env:VCPKG_DEFAULT_TRIPLET" `
--CDVCPKG_MANIFEST_NO_DEFAULT_FEATURES=ON `
--CDVCPKG_MANIFEST_FEATURES=proxy `
--CDCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreadedDLL `
--CDOPENDHT_DOWNLOAD_DEPS=OFF
if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
"$env:VCPKG_INSTALLED_DIR/$env:VCPKG_DEFAULT_TRIPLET/bin" >> $env:GITHUB_PATH
"OPENDHT_LIBRARY_PATH=$env:VCPKG_INSTALLED_DIR/$env:VCPKG_DEFAULT_TRIPLET/bin" >> $env:GITHUB_ENV
- run: npm test
- run: npm run typecheck
- run: npm run bundle
- name: Include static simdutf licenses
if: runner.os == 'Linux'
run: |
mkdir -p prebuilds/linux-x64/licenses/simdutf
cp /usr/share/doc/simdutf/LICENSE-* prebuilds/linux-x64/licenses/simdutf/
- run: npm run test:package
- uses: actions/upload-artifact@v7
with:
name: opendht-node-prebuild-${{ matrix.target }}
path: nodejs/prebuilds/
if-no-files-found: error
package:
name: Assemble npm archive
needs: build
runs-on: ubuntu-26.04
steps:
- uses: actions/checkout@v7
- uses: actions/setup-node@v7
with:
node-version: '22'
- uses: actions/download-artifact@v8
with:
pattern: opendht-node-prebuild-*
path: nodejs/prebuilds
merge-multiple: true
- name: Set release version
if: github.event_name == 'push' && github.ref_type == 'tag'
working-directory: nodejs
env:
RELEASE_TAG: ${{ github.ref_name }}
run: |
release_version=$(npm version "${RELEASE_TAG#v}" --no-git-tag-version --ignore-scripts --allow-same-version)
if [[ "$release_version" != "$RELEASE_TAG" ]]; then
printf '::error::Release tag %s is not an exact npm semantic version (normalized to %s).\n' "$RELEASE_TAG" "$release_version" >&2
exit 1
fi
- name: Pack all platforms
working-directory: nodejs
run: |
npm ci --ignore-scripts
mkdir -p dist
npm pack --pack-destination dist
- uses: actions/upload-artifact@v7
with:
name: opendht-node-npm
path: nodejs/dist/*.tgz
if-no-files-found: error
smoke:
name: Install on ${{ matrix.os }} / Node ${{ matrix.node }}
needs: package
runs-on: ${{ matrix.os }}
timeout-minutes: 10
strategy:
fail-fast: false
matrix:
os: [ubuntu-24.04, ubuntu-26.04, macos-15, macos-15-intel, windows-2022]
node: ['22', '24', '26']
steps:
- uses: actions/checkout@v7
- uses: actions/setup-node@v7
with:
node-version: ${{ matrix.node }}
- uses: actions/download-artifact@v8
with:
name: opendht-node-npm
path: nodejs/dist
- name: Test installed archive without a native build toolchain
working-directory: nodejs
run: npm run test:package -- dist
+17 -41
View File
@@ -4,10 +4,10 @@ permissions:
contents: read
on:
release:
types: [published]
push:
branches: ['**']
pull_request:
workflow_call:
workflow_dispatch:
jobs:
@@ -23,7 +23,7 @@ jobs:
# Run locally with: pipx run cibuildwheel --platform linux python
- name: Build wheels
uses: pypa/cibuildwheel@v4.1.0
uses: pypa/cibuildwheel@v4.3.5
with:
package-dir: python
@@ -36,36 +36,28 @@ jobs:
name: Build wheels on Windows
runs-on: windows-latest
env:
VCPKG_VERSION: "2026.06.01"
VCPKG_INSTALLATION_ROOT: "C:\\vcpkg"
VCPKG_DISABLE_METRICS: "1"
VCPKG_DEFAULT_TRIPLET: x64-windows-release
VCPKG_INSTALLED_DIR: ${{ github.workspace }}\vcpkg_installed
VCPKG_DEFAULT_BINARY_CACHE: ${{ github.workspace }}\vcpkg_binary_cache
VCPKG_BINARY_SOURCES: clear;files,${{ github.workspace }}/vcpkg-binary-cache,readwrite
steps:
- uses: actions/checkout@v7
- name: Cache vcpkg installed packages
id: cache-vcpkg
- name: Set up vcpkg
uses: lukka/run-vcpkg@v11
with:
vcpkgJsonGlob: vcpkg.json
vcpkgConfigurationJsonGlob: vcpkg-configuration.json
- name: Cache vcpkg binary packages
uses: actions/cache@v6
with:
path: |
${{ env.VCPKG_INSTALLED_DIR }}
${{ env.VCPKG_DEFAULT_BINARY_CACHE }}
key: ${{ runner.os }}-vcpkg-${{ env.VCPKG_VERSION }}-${{ env.VCPKG_DEFAULT_TRIPLET }}-${{ hashFiles('**/vcpkg.json', '**/vcpkg-configuration.json') }}-${{ hashFiles('.github/workflows/windows-build.yml') }}
- name: Setup vcpkg and install dependencies
if: steps.cache-vcpkg.outputs.cache-hit != 'true'
run: |
New-Item -Path "${{ env.VCPKG_INSTALLED_DIR }}" -ItemType Directory -Force
New-Item -Path "${{ env.VCPKG_DEFAULT_BINARY_CACHE }}" -ItemType Directory -Force
vcpkg install `
--recurse `
--clean-after-build `
--triplet=${{ env.VCPKG_DEFAULT_TRIPLET }} `
--x-install-root=${{ env.VCPKG_INSTALLED_DIR }}
shell: pwsh
path: ${{ github.workspace }}/vcpkg-binary-cache
key: ${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-${{ hashFiles('vcpkg.json', 'vcpkg-configuration.json', 'ports/**') }}-${{ github.run_id }}-${{ github.run_attempt }}
restore-keys: |
${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-${{ hashFiles('vcpkg.json', 'vcpkg-configuration.json', 'ports/**') }}-
${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-
- name: Build wheels
run: |
@@ -73,7 +65,7 @@ jobs:
cmake -S . -B build `
-DCMAKE_BUILD_TYPE=Release `
-DVCPKG_TARGET_TRIPLET=${{ env.VCPKG_DEFAULT_TRIPLET }} `
-DCMAKE_TOOLCHAIN_FILE=${{ env.VCPKG_INSTALLATION_ROOT }}\scripts\buildsystems\vcpkg.cmake `
"-DCMAKE_TOOLCHAIN_FILE=$env:VCPKG_ROOT/scripts/buildsystems/vcpkg.cmake" `
-DVCPKG_INSTALLED_DIR=${{ env.VCPKG_INSTALLED_DIR }} `
-DBUILD_TESTING=OFF `
-DOPENDHT_C=OFF `
@@ -90,19 +82,3 @@ jobs:
with:
name: opendht-wheels-win_amd64
path: ./build/wheelhouse/*.whl
publish:
runs-on: ubuntu-latest
needs: [build_wheels, build_wheels_windows]
steps:
- uses: actions/download-artifact@v8
with:
pattern: opendht-wheels-*
path: dist
merge-multiple: true
- name: Publish package
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags')
uses: pypa/gh-action-pypi-publish@release/v1
with:
password: ${{ secrets.PYPI_API_TOKEN }}
+212 -49
View File
@@ -8,7 +8,9 @@ on:
env:
REGISTRY: ghcr.io
IMAGE_NAME_DEPS: ${{ github.repository }}/opendht-deps
IMAGE_NAME_DEPS_NOBLE: ${{ github.repository }}/opendht-deps-noble
IMAGE_NAME: ${{ github.repository }}/opendht
IMAGE_NAME_NOBLE: ${{ github.repository }}/opendht-noble
IMAGE_NAME_DEPS_LLVM: ${{ github.repository }}/opendht-deps-llvm
IMAGE_NAME_LLVM: ${{ github.repository }}/opendht-llvm
IMAGE_NAME_DHTNODE: ${{ github.repository }}/dhtnode
@@ -16,6 +18,19 @@ env:
IMAGE_NAME_ALPINE_DEPS: ${{ github.repository }}/opendht-deps-alpine
jobs:
nodejs-package:
name: Build and verify Node.js package
uses: ./.github/workflows/nodejs-package.yml
permissions:
contents: read
packages: read
python-wheels:
name: Build Python wheels
uses: ./.github/workflows/python-wheels.yml
permissions:
contents: read
build-and-push-deps-image:
name: Dependency Docker image
runs-on: ubuntu-latest
@@ -56,6 +71,46 @@ jobs:
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
build-and-push-deps-image-noble:
name: Dependency Docker image (Ubuntu 24.04)
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Checkout repository
uses: actions/checkout@v7
- name: Set up QEMU
uses: docker/setup-qemu-action@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
- name: Log in to the Container registry
uses: docker/login-action@v4
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v6
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME_DEPS_NOBLE }}
- name: Build and push Docker image
uses: docker/build-push-action@v7
with:
context: .
file: docker/DockerfileDepsNoble
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
build-and-push-image:
name: OpenDHT Docker image
runs-on: ubuntu-latest
@@ -98,6 +153,47 @@ jobs:
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
build-and-push-image-noble:
name: OpenDHT Docker image (Ubuntu 24.04)
runs-on: ubuntu-latest
needs: build-and-push-deps-image-noble
permissions:
contents: read
packages: write
steps:
- name: Checkout repository
uses: actions/checkout@v7
- name: Set up QEMU
uses: docker/setup-qemu-action@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
- name: Log in to the Container registry
uses: docker/login-action@v4
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v6
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME_NOBLE }}
- name: Build and push Docker image
uses: docker/build-push-action@v7
with:
context: .
file: docker/DockerfileNoble
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
build-and-push-deps-image-llvm:
name: Dependency Docker image (LLVM)
runs-on: ubuntu-latest
@@ -309,12 +405,10 @@ jobs:
name: Build Windows MSI Installer
runs-on: windows-latest
env:
VCPKG_VERSION: "2026.06.01"
VCPKG_INSTALLATION_ROOT: "C:\\vcpkg"
VCPKG_DISABLE_METRICS: "1"
VCPKG_DEFAULT_TRIPLET: x64-windows
VCPKG_INSTALLED_DIR: ${{ github.workspace }}\vcpkg_installed
VCPKG_DEFAULT_BINARY_CACHE: ${{ github.workspace }}\vcpkg_binary_cache
VCPKG_BINARY_SOURCES: clear;files,${{ github.workspace }}/vcpkg-binary-cache,readwrite
permissions:
contents: read
steps:
@@ -323,6 +417,12 @@ jobs:
with:
submodules: recursive
- name: Set up vcpkg
uses: lukka/run-vcpkg@v11
with:
vcpkgJsonGlob: vcpkg.json
vcpkgConfigurationJsonGlob: vcpkg-configuration.json
- name: Set up MSVC developer environment
uses: microsoft/setup-msbuild@v3
@@ -332,27 +432,14 @@ jobs:
wix extension add --global WixToolset.UI.wixext/6.0.1
shell: pwsh
- name: Cache vcpkg installed packages
id: cache-vcpkg
- name: Cache vcpkg binary packages
uses: actions/cache@v6
with:
path: |
${{ env.VCPKG_INSTALLED_DIR }}
${{ env.VCPKG_DEFAULT_BINARY_CACHE }}
key: ${{ runner.os }}-vcpkg-${{ env.VCPKG_VERSION }}-x64-shared-${{ hashFiles('**/vcpkg.json', '**/vcpkg-configuration.json') }}-${{ hashFiles('.github/workflows/windows-build.yml') }}
- name: Setup vcpkg and install dependencies
if: steps.cache-vcpkg.outputs.cache-hit != 'true'
run: |
New-Item -Path "${{ env.VCPKG_INSTALLED_DIR }}" -ItemType Directory -Force
New-Item -Path "${{ env.VCPKG_DEFAULT_BINARY_CACHE }}" -ItemType Directory -Force
vcpkg install `
--recurse `
--clean-after-build `
--triplet=x64-windows `
--x-install-root=${{ env.VCPKG_INSTALLED_DIR }} `
--x-feature=test
shell: pwsh
path: ${{ github.workspace }}/vcpkg-binary-cache
key: ${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-${{ hashFiles('vcpkg.json', 'vcpkg-configuration.json', 'ports/**') }}-${{ github.run_id }}-${{ github.run_attempt }}
restore-keys: |
${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-${{ hashFiles('vcpkg.json', 'vcpkg-configuration.json', 'ports/**') }}-
${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-
- name: Configure CMake for MSI
shell: pwsh
@@ -360,7 +447,7 @@ jobs:
mkdir build -ErrorAction SilentlyContinue
cmake -B build `
-S . `
-DCMAKE_TOOLCHAIN_FILE=${{ env.VCPKG_INSTALLATION_ROOT }}/scripts/buildsystems/vcpkg.cmake `
"-DCMAKE_TOOLCHAIN_FILE=$env:VCPKG_ROOT/scripts/buildsystems/vcpkg.cmake" `
-DVCPKG_TARGET_TRIPLET=x64-windows `
-DVCPKG_INSTALLED_DIR=${{ env.VCPKG_INSTALLED_DIR }} `
-DVCPKG_MANIFEST_FEATURES=test `
@@ -413,12 +500,10 @@ jobs:
name: Build Windows dhtnode
runs-on: windows-latest
env:
VCPKG_VERSION: "2026.06.01"
VCPKG_INSTALLATION_ROOT: "C:\\vcpkg"
VCPKG_DISABLE_METRICS: "1"
VCPKG_DEFAULT_TRIPLET: x64-windows-static-release
VCPKG_INSTALLED_DIR: ${{ github.workspace }}\vcpkg_installed
VCPKG_DEFAULT_BINARY_CACHE: ${{ github.workspace }}\vcpkg_binary_cache
VCPKG_BINARY_SOURCES: clear;files,${{ github.workspace }}/vcpkg-binary-cache,readwrite
permissions:
contents: read
steps:
@@ -427,33 +512,23 @@ jobs:
with:
submodules: recursive
- name: Set up vcpkg
uses: lukka/run-vcpkg@v11
with:
vcpkgJsonGlob: vcpkg.json
vcpkgConfigurationJsonGlob: vcpkg-configuration.json
- name: Set up MSVC developer environment
uses: microsoft/setup-msbuild@v3
- name: Cache vcpkg static installed packages
id: cache-vcpkg-static
- name: Cache vcpkg binary packages
uses: actions/cache@v6
with:
path: |
${{ env.VCPKG_INSTALLED_DIR }}
${{ env.VCPKG_DEFAULT_BINARY_CACHE }}
key: ${{ runner.os }}-vcpkg-${{ env.VCPKG_VERSION }}-x64-static-${{ hashFiles('**/vcpkg.json', '**/vcpkg-configuration.json') }}-${{ hashFiles('.github/workflows/windows-build.yml') }}
path: ${{ github.workspace }}/vcpkg-binary-cache
key: ${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-${{ hashFiles('vcpkg.json', 'vcpkg-configuration.json', 'ports/**') }}-${{ github.run_id }}-${{ github.run_attempt }}
restore-keys: |
${{ runner.os }}-vcpkg-${{ env.VCPKG_VERSION }}-x64-static-${{ hashFiles('**/vcpkg.json', '**/vcpkg-configuration.json') }}
${{ runner.os }}-vcpkg-${{ env.VCPKG_VERSION }}-x64-static-
- name: Setup vcpkg and install static dependencies
if: steps.cache-vcpkg-static.outputs.cache-hit != 'true'
shell: pwsh
run: |
New-Item -Path "${{ env.VCPKG_INSTALLED_DIR }}" -ItemType Directory -Force
New-Item -Path "${{ env.VCPKG_DEFAULT_BINARY_CACHE }}" -ItemType Directory -Force
vcpkg install `
--recurse `
--clean-after-build `
--triplet=x64-windows-static-release `
--x-install-root=${{ env.VCPKG_INSTALLED_DIR }} `
--x-feature=test
${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-${{ hashFiles('vcpkg.json', 'vcpkg-configuration.json', 'ports/**') }}-
${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-
- name: Configure CMake for Static Build
shell: pwsh
@@ -461,7 +536,7 @@ jobs:
mkdir build_static -ErrorAction SilentlyContinue
cmake -B build_static `
-S . `
-DCMAKE_TOOLCHAIN_FILE=${{ env.VCPKG_INSTALLATION_ROOT }}/scripts/buildsystems/vcpkg.cmake `
"-DCMAKE_TOOLCHAIN_FILE=$env:VCPKG_ROOT/scripts/buildsystems/vcpkg.cmake" `
-DVCPKG_TARGET_TRIPLET=x64-windows-static-release `
-DVCPKG_INSTALLED_DIR=${{ env.VCPKG_INSTALLED_DIR }} `
-DVCPKG_MANIFEST_FEATURES=test `
@@ -493,7 +568,7 @@ jobs:
create-github-release:
name: Create GitHub Release
runs-on: ubuntu-latest
needs: [build-windows-msi, build-windows-dhtnode]
needs: [build-windows-msi, build-windows-dhtnode, nodejs-package]
permissions:
contents: write
@@ -513,11 +588,99 @@ jobs:
name: opendht-windows-dhtnode
path: downloaded_artifacts/static_dhtnode
- name: Download npm archive
uses: actions/download-artifact@v8
with:
name: opendht-node-npm
path: downloaded_artifacts/nodejs
- name: Create Release
id: create_release
uses: softprops/action-gh-release@v3
with:
draft: true
prerelease: ${{ contains(github.ref_name, '-') }}
files: |
downloaded_artifacts/windows_msi/*.msi
downloaded_artifacts/static_dhtnode/**/*dhtnode.exe
downloaded_artifacts/nodejs/*.tgz
publish-npm:
name: Publish opendht to npm
needs: create-github-release
runs-on: ubuntu-26.04
environment:
name: npm
url: https://www.npmjs.com/package/opendht
permissions:
contents: read
id-token: write
steps:
- uses: actions/checkout@v7
- uses: actions/setup-node@v7
with:
node-version: '24'
registry-url: https://registry.npmjs.org/
package-manager-cache: false
- uses: actions/download-artifact@v8
with:
name: opendht-node-npm
path: nodejs/dist
- name: Publish verified archive
working-directory: nodejs
env:
RELEASE_TAG: ${{ github.ref_name }}
run: |
dist_tag=latest
if [[ "${RELEASE_TAG#v}" == *-* ]]; then
dist_tag=next
fi
npm publish ./dist/*.tgz --access public --provenance --tag "$dist_tag"
publish-pypi:
name: Publish opendht to PyPI
needs: [create-github-release, python-wheels]
runs-on: ubuntu-latest
environment:
name: pypi
url: https://pypi.org/p/opendht
permissions:
contents: read
id-token: write
steps:
- uses: actions/download-artifact@v8
with:
pattern: opendht-wheels-*
path: dist
merge-multiple: true
- name: Publish wheels
uses: pypa/gh-action-pypi-publish@release/v1
publish-crate:
name: Publish opendht to crates.io
needs: create-github-release
runs-on: ubuntu-latest
environment:
name: crates-io
url: https://crates.io/crates/opendht
permissions:
contents: read
id-token: write
steps:
- uses: actions/checkout@v7
- uses: taiki-e/install-action@v2
with:
tool: cargo-edit
- name: Set crate version from release tag
working-directory: rust
env:
RELEASE_TAG: ${{ github.ref_name }}
run: cargo set-version "${RELEASE_TAG#v}"
- name: Authenticate with crates.io
id: auth
uses: rust-lang/crates-io-auth-action@v1
- name: Publish crate
working-directory: rust
env:
CARGO_REGISTRY_TOKEN: ${{ steps.auth.outputs.token }}
run: cargo publish --allow-dirty
+64 -80
View File
@@ -9,18 +9,16 @@ on:
branches: [ main, master ]
env:
VCPKG_VERSION: "2026.06.01"
VCPKG_INSTALLATION_ROOT: "C:\\vcpkg"
VCPKG_DISABLE_METRICS: "1"
VCPKG_INSTALLED_DIR: ${{ github.workspace }}\vcpkg_installed
VCPKG_DEFAULT_BINARY_CACHE: ${{ github.workspace }}\vcpkg_binary_cache
VCPKG_BINARY_SOURCES: clear;files,${{ github.workspace }}/vcpkg-binary-cache,readwrite
jobs:
prepare-dependencies:
name: Prepare Windows Dependencies
runs-on: windows-latest
outputs:
vcpkg_cache_key: ${{ steps.generate_cache_key.outputs.key }}
env:
VCPKG_DEFAULT_TRIPLET: ${{ matrix.arch }}-windows
strategy:
matrix:
arch: [x64]
@@ -32,43 +30,31 @@ jobs:
- name: Set up MSVC
uses: microsoft/setup-msbuild@v3
- name: Generate Cache Key
id: generate_cache_key
shell: pwsh
run: |
$key_value = "${{ runner.os }}-vcpkg-${{ env.VCPKG_VERSION }}-${{ matrix.arch }}-shared-${{ hashFiles('**/vcpkg.json', '**/vcpkg-configuration.json') }}-${{ hashFiles('.github/workflows/windows-build.yml') }}"
echo "key=$key_value" >> $env:GITHUB_OUTPUT
- name: Cache vcpkg installed packages
id: cache-vcpkg
- name: Cache vcpkg binary packages
uses: actions/cache@v6
with:
path: |
${{ env.VCPKG_INSTALLED_DIR }}
${{ env.VCPKG_DEFAULT_BINARY_CACHE }}
key: ${{ steps.generate_cache_key.outputs.key }}
path: ${{ github.workspace }}/vcpkg-binary-cache
key: ${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-${{ hashFiles('vcpkg.json', 'vcpkg-configuration.json', 'ports/**') }}-${{ github.run_id }}-${{ github.run_attempt }}
restore-keys: |
${{ runner.os }}-vcpkg-${{ env.VCPKG_VERSION }}-${{ matrix.arch }}-shared-${{ hashFiles('**/vcpkg.json', '**/vcpkg-configuration.json') }}
${{ runner.os }}-vcpkg-${{ env.VCPKG_VERSION }}-${{ matrix.arch }}-shared-
${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-${{ hashFiles('vcpkg.json', 'vcpkg-configuration.json', 'ports/**') }}-
${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-
- name: Setup vcpkg and install dependencies
if: steps.cache-vcpkg.outputs.cache-hit != 'true'
run: |
New-Item -Path "${{ env.VCPKG_INSTALLED_DIR }}" -ItemType Directory -Force
New-Item -Path "${{ env.VCPKG_DEFAULT_BINARY_CACHE }}" -ItemType Directory -Force
vcpkg install `
--recurse `
--clean-after-build `
--triplet=${{ matrix.arch }}-windows `
--x-install-root=${{ env.VCPKG_INSTALLED_DIR }} `
--x-feature=test
shell: pwsh
- name: Set up vcpkg and install dependencies
uses: lukka/run-vcpkg@v11
with:
vcpkgJsonGlob: vcpkg.json
vcpkgConfigurationJsonGlob: vcpkg-configuration.json
runVcpkgInstall: true
runVcpkgFormatString: >-
[`install`, `--clean-after-build`, `--x-feature=test`,
`--triplet`, `$[env.VCPKG_DEFAULT_TRIPLET]`,
`--x-install-root`, `$[env.VCPKG_INSTALLED_DIR]`]
prepare-static-dependencies:
name: Prepare Windows Static Dependencies
runs-on: windows-latest
outputs:
vcpkg_static_cache_key: ${{ steps.generate_static_cache_key.outputs.key }}
env:
VCPKG_DEFAULT_TRIPLET: ${{ matrix.arch }}-windows-static-release
strategy:
matrix:
arch: [x64]
@@ -80,42 +66,32 @@ jobs:
- name: Set up MSVC
uses: microsoft/setup-msbuild@v3
- name: Generate Cache Key for Static Libs
id: generate_static_cache_key
shell: pwsh
run: |
$key_value = "${{ runner.os }}-vcpkg-${{ env.VCPKG_VERSION }}-${{ matrix.arch }}-static-${{ hashFiles('**/vcpkg.json', '**/vcpkg-configuration.json') }}-${{ hashFiles('.github/workflows/windows-build.yml') }}"
echo "key=$key_value" >> $env:GITHUB_OUTPUT
- name: Cache vcpkg static installed packages
id: cache-vcpkg-static
- name: Cache vcpkg binary packages
uses: actions/cache@v6
with:
path: |
${{ env.VCPKG_INSTALLED_DIR }}
${{ env.VCPKG_DEFAULT_BINARY_CACHE }}
key: ${{ steps.generate_static_cache_key.outputs.key }}
path: ${{ github.workspace }}/vcpkg-binary-cache
key: ${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-${{ hashFiles('vcpkg.json', 'vcpkg-configuration.json', 'ports/**') }}-${{ github.run_id }}-${{ github.run_attempt }}
restore-keys: |
${{ runner.os }}-vcpkg-${{ env.VCPKG_VERSION }}-${{ matrix.arch }}-static-${{ hashFiles('**/vcpkg.json', '**/vcpkg-configuration.json') }}
${{ runner.os }}-vcpkg-${{ env.VCPKG_VERSION }}-${{ matrix.arch }}-static-
${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-${{ hashFiles('vcpkg.json', 'vcpkg-configuration.json', 'ports/**') }}-
${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-
- name: Setup vcpkg and install static dependencies
if: steps.cache-vcpkg-static.outputs.cache-hit != 'true'
shell: pwsh
run: |
New-Item -Path "${{ env.VCPKG_INSTALLED_DIR }}" -ItemType Directory -Force
New-Item -Path "${{ env.VCPKG_DEFAULT_BINARY_CACHE }}" -ItemType Directory -Force
vcpkg install `
--recurse `
--clean-after-build `
--triplet=${{ matrix.arch }}-windows-static-release `
--x-install-root=${{ env.VCPKG_INSTALLED_DIR }} `
--x-feature=test
- name: Set up vcpkg and install dependencies
uses: lukka/run-vcpkg@v11
with:
vcpkgJsonGlob: vcpkg.json
vcpkgConfigurationJsonGlob: vcpkg-configuration.json
runVcpkgInstall: true
runVcpkgFormatString: >-
[`install`, `--clean-after-build`, `--x-feature=test`,
`--triplet`, `$[env.VCPKG_DEFAULT_TRIPLET]`,
`--x-install-root`, `$[env.VCPKG_INSTALLED_DIR]`]
build-windows:
name: Windows MSVC Build
runs-on: windows-latest
needs: prepare-dependencies
env:
VCPKG_DEFAULT_TRIPLET: ${{ matrix.arch }}-windows
strategy:
matrix:
@@ -127,20 +103,23 @@ jobs:
- name: Checkout repository
uses: actions/checkout@v7
- name: Set up vcpkg
uses: lukka/run-vcpkg@v11
with:
vcpkgJsonGlob: vcpkg.json
vcpkgConfigurationJsonGlob: vcpkg-configuration.json
- name: Set up MSVC
uses: microsoft/setup-msbuild@v3
- name: Restore vcpkg installed packages
id: cache-vcpkg-restore
- name: Restore vcpkg binary packages
uses: actions/cache/restore@v6
with:
path: |
${{ env.VCPKG_INSTALLED_DIR }}
${{ env.VCPKG_DEFAULT_BINARY_CACHE }}
key: ${{ needs.prepare-dependencies.outputs.vcpkg_cache_key }}
path: ${{ github.workspace }}/vcpkg-binary-cache
key: ${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-${{ hashFiles('vcpkg.json', 'vcpkg-configuration.json', 'ports/**') }}-${{ github.run_id }}-${{ github.run_attempt }}
restore-keys: |
${{ runner.os }}-vcpkg-${{ env.VCPKG_VERSION }}-${{ matrix.arch }}-shared-${{ hashFiles('**/vcpkg.json', '**/vcpkg-configuration.json') }}
${{ runner.os }}-vcpkg-${{ env.VCPKG_VERSION }}-${{ matrix.arch }}-shared-
${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-${{ hashFiles('vcpkg.json', 'vcpkg-configuration.json', 'ports/**') }}-
${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-
- name: Configure CMake
shell: pwsh
@@ -148,7 +127,7 @@ jobs:
mkdir build -ErrorAction SilentlyContinue
cmake -B build `
-S . `
-DCMAKE_TOOLCHAIN_FILE=${{ env.VCPKG_INSTALLATION_ROOT }}/scripts/buildsystems/vcpkg.cmake `
"-DCMAKE_TOOLCHAIN_FILE=$env:VCPKG_ROOT/scripts/buildsystems/vcpkg.cmake" `
-DVCPKG_TARGET_TRIPLET=${{ matrix.arch }}-windows `
-DVCPKG_INSTALLED_DIR=${{ env.VCPKG_INSTALLED_DIR }} `
-DVCPKG_MANIFEST_FEATURES=test `
@@ -194,6 +173,8 @@ jobs:
name: Windows Static MSVC Build
runs-on: windows-latest
needs: prepare-static-dependencies
env:
VCPKG_DEFAULT_TRIPLET: ${{ matrix.arch }}-windows-static-release
strategy:
matrix:
build_type: [Release]
@@ -204,20 +185,23 @@ jobs:
- name: Checkout repository
uses: actions/checkout@v7
- name: Set up vcpkg
uses: lukka/run-vcpkg@v11
with:
vcpkgJsonGlob: vcpkg.json
vcpkgConfigurationJsonGlob: vcpkg-configuration.json
- name: Set up MSVC
uses: microsoft/setup-msbuild@v3
- name: Restore vcpkg static installed packages
id: cache-vcpkg-static-restore
- name: Restore vcpkg binary packages
uses: actions/cache/restore@v6
with:
path: |
${{ env.VCPKG_INSTALLED_DIR }}
${{ env.VCPKG_DEFAULT_BINARY_CACHE }}
key: ${{ needs.prepare-static-dependencies.outputs.vcpkg_static_cache_key }}
path: ${{ github.workspace }}/vcpkg-binary-cache
key: ${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-${{ hashFiles('vcpkg.json', 'vcpkg-configuration.json', 'ports/**') }}-${{ github.run_id }}-${{ github.run_attempt }}
restore-keys: |
${{ runner.os }}-vcpkg-${{ env.VCPKG_VERSION }}-${{ matrix.arch }}-static-${{ hashFiles('**/vcpkg.json', '**/vcpkg-configuration.json') }}
${{ runner.os }}-vcpkg-${{ env.VCPKG_VERSION }}-${{ matrix.arch }}-static-
${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-${{ hashFiles('vcpkg.json', 'vcpkg-configuration.json', 'ports/**') }}-
${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-
- name: Configure CMake for Static Build
shell: pwsh
@@ -225,7 +209,7 @@ jobs:
mkdir build -ErrorAction SilentlyContinue
cmake -B build `
-S . `
-DCMAKE_TOOLCHAIN_FILE=${{ env.VCPKG_INSTALLATION_ROOT }}/scripts/buildsystems/vcpkg.cmake `
"-DCMAKE_TOOLCHAIN_FILE=$env:VCPKG_ROOT/scripts/buildsystems/vcpkg.cmake" `
-DVCPKG_TARGET_TRIPLET=${{ matrix.arch }}-windows-static-release `
-DVCPKG_INSTALLED_DIR=${{ env.VCPKG_INSTALLED_DIR }} `
-DVCPKG_MANIFEST_FEATURES=test `
@@ -304,7 +288,7 @@ jobs:
shell: msys2 {0}
env:
LLHTTP_VERSION: "v9.2.1"
RESTINIO_VERSION: "0.7.6"
RESTINIO_VERSION: "0.7.10"
EXPECTED_LITE_URL: "https://raw.githubusercontent.com/martinmoene/expected-lite/master/include/nonstd/expected.hpp"
run: |
MSYS2_SYSROOT=$(cygpath -u "$MSYSTEM_PREFIX")
+66 -2
View File
@@ -115,16 +115,64 @@ cmake -DOPENDHT_PYTHON=ON \
| `OPENDHT_PUSH_NOTIFICATIONS` | OFF | Push notification support |
| `OPENDHT_SANITIZE` | OFF | AddressSanitizer + stack protector |
## Node.js Bindings
Node-API bindings wrap the C++ API directly with `node-addon-api`, including
RSA/EC keys, certificates, identities, and signed/encrypted DHT operations.
With the native dependencies above, Node.js 22.9+, and CMake installed, run
`npm ci`, `npm run build`, `npm test`, and `npm run typecheck` from `nodejs/`.
This creates a separate native build without changing the existing build tree.
See [the Node.js binding guide](nodejs/README.md) for the API and current limits.
`npm run package` creates a platform prebuild with bundled native dependencies;
`npm run test:package` verifies installation outside the source tree. The Node.js
package CI builds Linux x64, macOS ARM64/x64, and Windows x64 prebuilds and checks
the assembled npm archive on Node 22, 24, and 26 before release attachment.
## C and Rust Bindings
Enable `OPENDHT_C=ON` to build the C ABI. With `BUILD_TESTING=ON`, CTest runs
`test_c`; Meson registers the same test as `CBindings` when `c` and `tests` are
enabled. Rust regression and callback-lifetime tests run with Cargo; see
[the Rust build guide](rust/BUILD.md) for build and library-path settings.
`dht_infohash` and `dht_pkid` have `uint32_t` alignment, matching the native hash
types, while retaining their 20-byte and 32-byte sizes. Rebuild consumers against
the updated header; foreign-language declarations must match this alignment.
C handles returned by constructors, imports, and reference functions are owned
and require their matching delete/unref function. Data views, string getters,
query fields, and callback arguments are borrowed: copy them or take a reference
before returning from the callback. `dht_sockaddr_array_delete` releases both the
public-address array and its elements.
Operation completion callbacks run once, including failure when pending work is
discarded on runner teardown. Listener release callbacks run after the native
registration is destroyed; deleting an operation token alone does not cancel its
listener. Cancel first, then delete the consumed token. Startup-context release
callbacks run after all native status/store callback captures are released, also
on startup failure. Certificate-store callbacks must invoke the supplied emitter
synchronously with borrowed certificates. Query-result views are callback-scoped.
`dht_runner_loop` drives non-threaded nodes. `dht_runner_join` waits for callbacks
and allows restart; never call it from a DHT callback. For persisted shutdown,
wait for the shutdown callback before joining. Push tokens can be polled without
blocking and must be deleted separately. `dht_push_token_poll` returns `EAGAIN`
while pending, 0 with the native outcome when ready, and an error code otherwise;
consume a successful result only once. Native push outcomes are 0 through 3 for
refresh/value events, 128 through 131 for ignored events, and 132 for errors.
## Windows/MSVC
Building on Windows with MSVC is supported with VCPKG and CMake.
### Build with VCPKG in manifest mode
1. Install [VCPKG](https://github.com/microsoft/vcpkg) and required dependencies:
1. Install [VCPKG](https://github.com/microsoft/vcpkg) 2026.10.16 and required dependencies:
```ps1
git clone https://github.com/microsoft/vcpkg.git
cd vcpkg; .\bootstrap-vcpkg.bat
cd vcpkg
git checkout 9eec002df1cd109f8f104cd966a763daf9751489
.\bootstrap-vcpkg.bat
```
2. Use CMake preset for vcpkg:
@@ -133,3 +181,19 @@ cd opendht
cmake --preset=vcpkg_manifest .
cmake --build .
```
### CI
Windows CI uses [lukka/run-vcpkg](https://github.com/lukka/run-vcpkg) to acquire
and bootstrap the revision pinned by `default-registry.baseline` in
`vcpkg-configuration.json`. CMake uses the action's `VCPKG_ROOT` and installs
manifest dependencies on every run. Dependency-preparation jobs warm the binary
cache before the Windows build matrix starts.
Only ABI-keyed binary package archives are cached, not installed trees. The
supported [files provider](https://learn.microsoft.com/en-us/vcpkg/reference/binarycaching#files-provider)
is persisted with `actions/cache`; it does not use the removed `x-gha` provider
or require package-feed credentials. Cache keys include the target triplet,
manifests, overlay ports, run ID, and attempt, with restore prefixes for reuse.
Each run can save newly built archives without overwriting an immutable cache;
vcpkg validates package ABIs before reuse.
+23 -4
View File
@@ -10,10 +10,10 @@ if(POLICY CMP0167)
endif()
set (opendht_VERSION_MAJOR 4)
set (opendht_VERSION_MINOR 1.1)
set (opendht_VERSION_MINOR 5.0)
set (opendht_VERSION ${opendht_VERSION_MAJOR}.${opendht_VERSION_MINOR})
set (PACKAGE_VERSION ${opendht_VERSION})
set (VERSION "${opendht_VERSION}")
set (PACKAGE_VERSION "${opendht_VERSION}-rc.1")
set (VERSION "${PACKAGE_VERSION}")
project (opendht
VERSION ${opendht_VERSION}
DESCRIPTION "OpenDHT: A lightweight C++17 Distributed Hash Table library."
@@ -421,7 +421,7 @@ else ()
endif ()
set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -DMSGPACK_NO_BOOST -DMSGPACK_DISABLE_LEGACY_NIL -DMSGPACK_DISABLE_LEGACY_CONVERT")
add_definitions(-DPACKAGE_VERSION="${opendht_VERSION}")
add_definitions(-DPACKAGE_VERSION="${PACKAGE_VERSION}")
if (ASIO_INCLUDE_DIR)
include_directories (SYSTEM "${ASIO_INCLUDE_DIR}")
@@ -845,6 +845,15 @@ endif()
# Unit tests
if (BUILD_TESTING)
if (OPENDHT_C)
add_executable(test_c tests/test_c.c)
target_include_directories(test_c PRIVATE c include)
target_link_libraries(test_c PRIVATE opendht-c)
if (MSVC)
target_link_libraries(test_c PRIVATE ${GNUTLS_LIBRARIES})
endif()
add_test(NAME test_c COMMAND test_c)
endif()
if (NOT MSVC)
pkg_search_module(Cppunit REQUIRED IMPORTED_TARGET cppunit)
set(cppunit_LIBRARIES PkgConfig::Cppunit)
@@ -870,7 +879,17 @@ if (BUILD_TESTING)
tests/test_parsedmessage.cpp
tests/test_networkengine.h
tests/test_networkengine.cpp
tests/test_opcache.h
tests/test_opcache.cpp
tests/test_valuecache.h
tests/test_valuecache.cpp
)
if (OPENDHT_INDEX)
list (APPEND test_FILES
tests/test_pht.h
tests/test_pht.cpp
)
endif()
if (OPENDHT_PROXY_CLIENT)
list (APPEND test_FILES
tests/test_dhtproxy_client.h
+5 -2
View File
@@ -4,7 +4,10 @@
<a id="user-content-opendht-" class="anchor" href="/savoirfairelinux/opendht/blob/master/README.md#opendht-" aria-hidden="true"></a>OpenDHT
</h1>
![PyPI - Version](https://img.shields.io/pypi/v/opendht?style=flat)
[![PyPI - Version](https://img.shields.io/pypi/v/opendht?style=flat)](https://pypi.org/project/opendht/)
[![crates.io - Version](https://img.shields.io/crates/v/opendht?style=flat)](https://crates.io/crates/opendht)
[![npm - Version](https://img.shields.io/npm/v/opendht?style=flat)](https://www.npmjs.com/package/opendht)
[![Docker Image](https://img.shields.io/badge/Docker-ghcr.io-2496ED?style=flat&logo=docker&logoColor=white)](https://github.com/savoirfairelinux/opendht/pkgs/container/opendht%2Fopendht)
A lightweight C++17 Distributed Hash Table implementation.
@@ -17,7 +20,7 @@ Values are distributed over the network, with redundancy.
* Public key cryptography layer providing optional data signature and encryption (using GnuTLS)
* IPv4 and IPv6 support
* Clean and powerful **C++17** map API
* Bindings for **C, Rust & Python 3**
* Bindings for **C, Rust, Node.js & Python 3**
* REST API with optional HTTP client+server with push notification support
## Documentation
+573 -99
View File
@@ -5,12 +5,35 @@
#include <opendht.h>
#include <opendht/log.h>
#include <atomic>
using ValueSp = std::shared_ptr<dht::Value>;
using PrivkeySp = std::shared_ptr<dht::crypto::PrivateKey>;
using PubkeySp = std::shared_ptr<const dht::crypto::PublicKey>;
using CertSp = std::shared_ptr<dht::crypto::Certificate>;
static_assert(sizeof(dht_infohash) == sizeof(dht::InfoHash));
static_assert(alignof(dht_infohash) == alignof(dht::InfoHash));
static_assert(sizeof(dht_pkid) == sizeof(dht::PkId));
static_assert(alignof(dht_pkid) == alignof(dht::PkId));
struct CompletionGuard
{
CompletionGuard(dht_done_cb callback, void* data)
: callback(callback)
, data(data)
{}
~CompletionGuard() { finish(false); }
void finish(bool ok)
{
if (!finished.exchange(true) && callback)
callback(ok, data);
}
dht_done_cb callback;
void* data;
std::atomic_bool finished {false};
};
#ifdef __cplusplus
extern "C" {
#endif
@@ -156,6 +179,47 @@ dht_value_set_user_type(dht_value* data, const char* user_type)
(*reinterpret_cast<ValueSp*>(data))->user_type = user_type;
}
uint16_t
dht_value_get_type(const dht_value* value)
{
return (*reinterpret_cast<const ValueSp*>(value))->type;
}
void
dht_value_set_type(dht_value* value, uint16_t type)
{
(*reinterpret_cast<ValueSp*>(value))->type = type;
}
uint16_t
dht_value_get_seq(const dht_value* value)
{
return (*reinterpret_cast<const ValueSp*>(value))->seq;
}
void
dht_value_set_seq(dht_value* value, uint16_t seq)
{
(*reinterpret_cast<ValueSp*>(value))->seq = seq;
}
unsigned
dht_value_get_priority(const dht_value* value)
{
return (*reinterpret_cast<const ValueSp*>(value))->priority;
}
void
dht_value_set_priority(dht_value* value, unsigned priority)
{
(*reinterpret_cast<ValueSp*>(value))->priority = priority;
}
const char*
dht_value_get_push_type(const dht_value* value)
{
return (*reinterpret_cast<const ValueSp*>(value))->pushType.c_str();
}
void
dht_value_set_push_type(dht_value* value, const char* push_type)
{
(*reinterpret_cast<ValueSp*>(value))->pushType = push_type;
}
dht_value*
dht_value_with_id_new(const uint8_t* data, size_t size, uint64_t value_id)
{
@@ -183,6 +247,29 @@ dht_value_ref(const dht_value* v)
return reinterpret_cast<dht_value*>(new ValueSp(*reinterpret_cast<const ValueSp*>(v)));
}
dht_value*
dht_value_copy(const dht_value* value)
{
try {
const auto& source = **reinterpret_cast<const ValueSp*>(value);
auto copy = std::make_shared<dht::Value>();
copy->id = source.id;
copy->owner = source.owner;
copy->recipient = source.recipient;
copy->type = source.type;
copy->data = source.data;
copy->user_type = source.user_type;
copy->seq = source.seq;
copy->signature = source.signature;
copy->cypher = source.cypher;
copy->priority = source.priority;
copy->pushType = source.pushType;
return reinterpret_cast<dht_value*>(new ValueSp(std::move(copy)));
} catch (...) {
return nullptr;
}
}
void
dht_value_unref(dht_value* v)
{
@@ -196,7 +283,7 @@ dht_publickey_import(const uint8_t* dat, size_t dat_size)
try {
return reinterpret_cast<dht_publickey*>(
new PubkeySp(std::make_shared<const dht::crypto::PublicKey>(dat, dat_size)));
} catch (const dht::crypto::CryptoException& e) {
} catch (...) {
return nullptr;
}
}
@@ -214,6 +301,29 @@ dht_publickey_export(const dht_publickey* pk, char* out, size_t* outlen)
return pkey->pack((uint8_t*) out, outlen);
}
dht_blob*
dht_publickey_export_blob(const dht_publickey* pk)
{
try {
auto data = std::make_unique<dht::Blob>();
(*reinterpret_cast<const PubkeySp*>(pk))->pack(*data);
return reinterpret_cast<dht_blob*>(data.release());
} catch (...) {
return nullptr;
}
}
int
dht_publickey_unpack(dht_publickey* pk, const uint8_t* data, size_t size)
{
try {
*reinterpret_cast<PubkeySp*>(pk) = std::make_shared<const dht::crypto::PublicKey>(data, size);
return 0;
} catch (...) {
return EINVAL;
}
}
dht_infohash
dht_publickey_get_id(const dht_publickey* pk)
{
@@ -255,8 +365,12 @@ dht_privatekey_generate(unsigned key_length_bits)
{
if (key_length_bits == 0)
key_length_bits = 4096;
return reinterpret_cast<dht_privatekey*>(
new PrivkeySp(std::make_shared<dht::crypto::PrivateKey>(dht::crypto::PrivateKey::generate(key_length_bits))));
try {
return reinterpret_cast<dht_privatekey*>(new PrivkeySp(
std::make_shared<dht::crypto::PrivateKey>(dht::crypto::PrivateKey::generate(key_length_bits))));
} catch (...) {
return nullptr;
}
}
dht_privatekey*
@@ -265,7 +379,7 @@ dht_privatekey_import(const uint8_t* dat, size_t dat_size, const char* password)
try {
return reinterpret_cast<dht_privatekey*>(
new PrivkeySp(std::make_shared<dht::crypto::PrivateKey>(dat, dat_size, password)));
} catch (const dht::crypto::CryptoException& e) {
} catch (...) {
return nullptr;
}
}
@@ -276,7 +390,43 @@ dht_privatekey_export(const dht_privatekey* k, char* out, size_t* out_size, cons
if (!out or !out_size or !*out_size)
return -1;
const auto& key = *reinterpret_cast<const PrivkeySp*>(k);
return key->serialize((uint8_t*) out, out_size, password);
try {
return key->serialize((uint8_t*) out, out_size, password ? password : "");
} catch (...) {
return -1;
}
}
dht_privatekey*
dht_privatekey_generate_ec(void)
{
try {
return reinterpret_cast<dht_privatekey*>(
new PrivkeySp(std::make_shared<dht::crypto::PrivateKey>(dht::crypto::PrivateKey::generateEC())));
} catch (...) {
return nullptr;
}
}
dht_blob*
dht_privatekey_export_blob(const dht_privatekey* key, const char* password)
{
try {
return reinterpret_cast<dht_blob*>(
new dht::Blob((*reinterpret_cast<const PrivkeySp*>(key))->serialize(password ? password : "")));
} catch (...) {
return nullptr;
}
}
dht_blob*
dht_privatekey_sign(const dht_privatekey* key, const uint8_t* data, size_t size)
{
try {
return reinterpret_cast<dht_blob*>(new dht::Blob((*reinterpret_cast<const PrivkeySp*>(key))->sign(data, size)));
} catch (...) {
return nullptr;
}
}
dht_publickey*
@@ -311,11 +461,28 @@ dht_certificate_import(const uint8_t* dat, size_t dat_size)
{
try {
return reinterpret_cast<dht_certificate*>(new CertSp(std::make_shared<dht::crypto::Certificate>(dat, dat_size)));
} catch (const dht::crypto::CryptoException& e) {
} catch (...) {
return nullptr;
}
}
dht_blob*
dht_certificate_export(const dht_certificate* certificate)
{
try {
return reinterpret_cast<dht_blob*>(new dht::Blob((*reinterpret_cast<const CertSp*>(certificate))->getPacked()));
} catch (...) {
return nullptr;
}
}
dht_certificate*
dht_certificate_ref(const dht_certificate* certificate)
{
return certificate ? reinterpret_cast<dht_certificate*>(new CertSp(*reinterpret_cast<const CertSp*>(certificate)))
: nullptr;
}
void
dht_certificate_delete(dht_certificate* c)
{
@@ -364,8 +531,10 @@ inline dht_identity
dht_identity_to_c(const dht::crypto::Identity& id)
{
dht_identity cid {};
cid.privatekey = id.first ? reinterpret_cast<dht_privatekey*>(new PrivkeySp(id.first)) : NULL;
cid.certificate = id.second ? reinterpret_cast<dht_certificate*>(new CertSp(id.second)) : NULL;
auto key = id.first ? std::make_unique<PrivkeySp>(id.first) : nullptr;
auto certificate = id.second ? std::make_unique<CertSp>(id.second) : nullptr;
cid.privatekey = reinterpret_cast<dht_privatekey*>(key.release());
cid.certificate = reinterpret_cast<dht_certificate*>(certificate.release());
return cid;
}
@@ -376,7 +545,12 @@ extern "C" {
OPENDHT_C_PUBLIC dht_identity
dht_identity_generate(const char* common_name, const dht_identity* ca)
{
return dht_identity_to_c(dht::crypto::generateIdentity(common_name, dht_identity_from_c(ca)));
try {
return dht_identity_to_c(
dht::crypto::generateIdentity(common_name ? common_name : "dhtnode", dht_identity_from_c(ca)));
} catch (...) {
return {};
}
}
OPENDHT_C_PUBLIC void
@@ -392,6 +566,17 @@ dht_identity_delete(dht_identity* id)
}
}
dht_identity
dht_identity_generate_ec(const char* common_name, const dht_identity* ca)
{
try {
return dht_identity_to_c(
dht::crypto::generateEcIdentity(common_name ? common_name : "dhtnode", dht_identity_from_c(ca)));
} catch (...) {
return {};
}
}
// config
void
dht_runner_config_default(dht_runner_config* config)
@@ -413,6 +598,28 @@ dht_runner_delete(dht_runner* runner)
delete reinterpret_cast<dht::DhtRunner*>(runner);
}
int
dht_runner_loop(dht_runner* runner)
{
try {
reinterpret_cast<dht::DhtRunner*>(runner)->loop();
return 0;
} catch (...) {
return EINVAL;
}
}
int
dht_runner_join(dht_runner* runner)
{
try {
reinterpret_cast<dht::DhtRunner*>(runner)->join();
return 0;
} catch (...) {
return EDEADLK;
}
}
int
dht_runner_run(dht_runner* r, in_port_t port)
{
@@ -424,61 +631,163 @@ dht_runner_run(dht_runner* r, in_port_t port)
}
return 0;
}
}
static dht::DhtRunner::Config
config_from_c(const dht_runner_config* conf)
{
dht::DhtRunner::Config config;
config.dht_config.node_config.is_bootstrap = conf->dht_config.node_config.is_bootstrap;
config.dht_config.node_config.maintain_storage = conf->dht_config.node_config.maintain_storage;
config.dht_config.node_config.node_id = *reinterpret_cast<const dht::InfoHash*>(
&conf->dht_config.node_config.node_id);
config.dht_config.node_config.network = conf->dht_config.node_config.network;
config.dht_config.node_config.persist_path = conf->dht_config.node_config.persist_path
? std::string(conf->dht_config.node_config.persist_path)
: std::string {};
if (conf->dht_config.id.privatekey)
config.dht_config.id.first = *reinterpret_cast<const PrivkeySp*>(conf->dht_config.id.privatekey);
if (conf->dht_config.id.certificate)
config.dht_config.id.second = *reinterpret_cast<const CertSp*>(conf->dht_config.id.certificate);
config.threaded = conf->threaded;
config.proxy_server = conf->proxy_server ? std::string(conf->proxy_server) : std::string {};
config.push_node_id = conf->push_node_id ? std::string(conf->push_node_id) : std::string {};
config.push_token = conf->push_token ? std::string(conf->push_token) : std::string {};
config.push_topic = conf->push_topic ? std::string(conf->push_topic) : std::string {};
config.push_platform = conf->push_platform ? std::string(conf->push_platform) : std::string {};
config.proxy_user_agent = conf->proxy_user_agent ? std::string(conf->proxy_user_agent) : std::string {};
config.peer_discovery = conf->peer_discovery;
config.peer_publish = conf->peer_publish;
if (conf->server_ca)
config.server_ca = *reinterpret_cast<const CertSp*>(conf->server_ca);
config.client_identity = dht_identity_from_c(&conf->client_identity);
return config;
}
extern "C" {
int
dht_runner_run_config(dht_runner* r, in_port_t port, const dht_runner_config* conf)
dht_runner_run_config(dht_runner* runner, in_port_t port, const dht_runner_config* config)
{
return dht_runner_run_bind(runner, nullptr, nullptr, port, config, nullptr);
}
int
dht_runner_run_bind(dht_runner* runner,
const char* ipv4,
const char* ipv6,
in_port_t port,
const dht_runner_config* conf,
const dht_runner_context* callbacks)
{
auto runner = reinterpret_cast<dht::DhtRunner*>(r);
try {
dht::DhtRunner::Config config;
config.dht_config.node_config.is_bootstrap = conf->dht_config.node_config.is_bootstrap;
config.dht_config.node_config.maintain_storage = conf->dht_config.node_config.maintain_storage;
config.dht_config.node_config.node_id = *reinterpret_cast<const dht::InfoHash*>(
&conf->dht_config.node_config.node_id);
config.dht_config.node_config.network = conf->dht_config.node_config.network;
config.dht_config.node_config.persist_path = conf->dht_config.node_config.persist_path
? std::string(conf->dht_config.node_config.persist_path)
: std::string {};
if (conf->dht_config.id.privatekey)
config.dht_config.id.first = *reinterpret_cast<const PrivkeySp*>(conf->dht_config.id.privatekey);
if (conf->dht_config.id.certificate)
config.dht_config.id.second = *reinterpret_cast<const CertSp*>(conf->dht_config.id.certificate);
config.threaded = conf->threaded;
config.proxy_server = conf->proxy_server ? std::string(conf->proxy_server) : std::string {};
config.push_node_id = conf->push_node_id ? std::string(conf->push_node_id) : std::string {};
config.push_token = conf->push_token ? std::string(conf->push_token) : std::string {};
config.push_topic = conf->push_topic ? std::string(conf->push_topic) : std::string {};
config.push_platform = conf->push_platform ? std::string(conf->push_platform) : std::string {};
config.proxy_user_agent = conf->proxy_user_agent ? std::string(conf->proxy_user_agent) : std::string {};
config.peer_discovery = conf->peer_discovery;
config.peer_publish = conf->peer_publish;
if (conf->server_ca)
config.server_ca = *reinterpret_cast<const CertSp*>(conf->server_ca);
config.client_identity = dht_identity_from_c(&conf->client_identity);
auto lifetime = callbacks ? std::shared_ptr<void>(callbacks->user_data,
[release = callbacks->released_cb](void* data) {
if (release)
release(data);
})
: std::shared_ptr<void> {};
auto config = config_from_c(conf);
dht::DhtRunner::Context context;
if (conf->log) {
context.logger = dht::log::getStdLogger();
}
runner->run(port, config, std::move(context));
if (callbacks) {
if (callbacks->status_cb) {
context.statusChangedCallback = [lifetime, cb = callbacks->status_cb](dht::NodeStatus ipv4,
dht::NodeStatus ipv6) {
cb(static_cast<int>(ipv4), static_cast<int>(ipv6), lifetime.get());
};
}
if (callbacks->certificate_store_cb) {
context.certificateStore = [lifetime, cb = callbacks->certificate_store_cb](const dht::PkId& id) {
std::vector<std::shared_ptr<dht::crypto::Certificate>> found;
auto emit = [](const dht_certificate* certificate, void* data) {
if (certificate)
static_cast<decltype(found)*>(data)->push_back(
*reinterpret_cast<const CertSp*>(certificate));
};
auto hash = dht_pkid_to_c(id);
cb(&hash, emit, &found, lifetime.get());
return found;
};
}
}
auto service = std::to_string(port);
reinterpret_cast<dht::DhtRunner*>(runner)
->run(ipv4 ? ipv4 : "0.0.0.0", ipv6 ? ipv6 : "::", service.c_str(), config, std::move(context));
} catch (...) {
return ENOTCONN;
}
return 0;
}
int
dht_runner_connectivity_changed(dht_runner* runner)
{
try {
reinterpret_cast<dht::DhtRunner*>(runner)->connectivityChanged();
return 0;
} catch (...) {
return EINVAL;
}
}
int
dht_runner_register_type(dht_runner* runner, uint16_t id, const char* name, uint64_t expiration_seconds, bool insecure)
{
if (!dht_runner_is_running(runner) || !name
|| expiration_seconds
> static_cast<uint64_t>(std::chrono::duration_cast<std::chrono::seconds>(dht::duration::max()).count()))
return EINVAL;
try {
dht::ValueType type(id, name, std::chrono::seconds(expiration_seconds));
if (insecure)
reinterpret_cast<dht::DhtRunner*>(runner)->registerInsecureType(type);
else
reinterpret_cast<dht::DhtRunner*>(runner)->registerType(type);
return 0;
} catch (...) {
return EINVAL;
}
}
dht_publickey*
dht_runner_get_publickey(const dht_runner* runner)
{
try {
auto key = reinterpret_cast<const dht::DhtRunner*>(runner)->getPublicKey();
return key ? reinterpret_cast<dht_publickey*>(new PubkeySp(std::move(key))) : nullptr;
} catch (...) {
return nullptr;
}
}
int
dht_runner_register_certificate(dht_runner* runner, const dht_certificate* certificate)
{
if (!dht_runner_is_running(runner))
return ENOTCONN;
try {
reinterpret_cast<dht::DhtRunner*>(runner)->registerCertificate(*reinterpret_cast<const CertSp*>(certificate));
return 0;
} catch (...) {
return EINVAL;
}
}
void
dht_runner_ping(dht_runner* r, struct sockaddr* addr, socklen_t addr_len, dht_done_cb done_cb, void* cb_user_data)
{
auto runner = reinterpret_cast<dht::DhtRunner*>(r);
if (done_cb) {
runner->bootstrap(dht::SockAddr(addr, addr_len),
[done_cb, cb_user_data](bool ok) { done_cb(ok, cb_user_data); });
} else {
runner->bootstrap(dht::SockAddr(addr, addr_len));
auto guard = std::make_shared<CompletionGuard>(done_cb, cb_user_data);
try {
runner->bootstrap(dht::SockAddr(addr, addr_len), [guard](bool ok) { guard->finish(ok); });
} catch (...) {
guard->finish(false);
}
}
@@ -494,18 +803,29 @@ dht_runner_bootstrap(dht_runner* r, const char* host, const char* service)
void
dht_runner_get(dht_runner* r, const dht_infohash* h, dht_get_cb cb, dht_done_cb done_cb, void* cb_user_data)
{
dht_runner_get_filtered(r, h, nullptr, cb, done_cb, cb_user_data);
}
void
dht_runner_get_filtered(
dht_runner* r, const dht_infohash* h, const char* where, dht_get_cb cb, dht_done_cb done_cb, void* cb_user_data)
{
auto runner = reinterpret_cast<dht::DhtRunner*>(r);
auto hash = reinterpret_cast<const dht::InfoHash*>(h);
runner->get(
*hash,
[cb, cb_user_data](std::shared_ptr<dht::Value> value) {
return cb(reinterpret_cast<const dht_value*>(&value), cb_user_data);
},
[done_cb, cb_user_data](bool ok) {
if (done_cb)
done_cb(ok, cb_user_data);
});
auto guard = std::make_shared<CompletionGuard>(done_cb, cb_user_data);
try {
runner->get(
*hash,
[cb, cb_user_data, guard](std::shared_ptr<dht::Value> value) {
return cb ? cb(reinterpret_cast<const dht_value*>(&value), cb_user_data) : true;
},
[guard](bool ok) { guard->finish(ok); },
{},
where ? dht::Where(where) : dht::Where {});
} catch (...) {
guard->finish(false);
}
}
struct ScopeGuardCb
@@ -528,23 +848,38 @@ private:
dht_op_token*
dht_runner_listen(dht_runner* r, const dht_infohash* h, dht_value_cb cb, dht_shutdown_cb done_cb, void* cb_user_data)
{
return dht_runner_listen_filtered(r, h, nullptr, cb, done_cb, cb_user_data);
}
dht_op_token*
dht_runner_listen_filtered(dht_runner* r,
const dht_infohash* h,
const char* where,
dht_value_cb cb,
dht_shutdown_cb done_cb,
void* cb_user_data)
{
auto runner = reinterpret_cast<dht::DhtRunner*>(r);
auto hash = reinterpret_cast<const dht::InfoHash*>(h);
auto fret = new std::future<size_t>;
*fret = runner->listen(*hash,
[cb,
cb_user_data,
guard = done_cb ? std::make_shared<ScopeGuardCb>(done_cb, cb_user_data)
: std::shared_ptr<ScopeGuardCb> {}](
const std::vector<std::shared_ptr<dht::Value>>& values, bool expired) {
for (const auto& value : values) {
if (not cb(reinterpret_cast<const dht_value*>(&value), expired, cb_user_data))
return false;
}
return true;
});
return (dht_op_token*) fret;
auto guard = std::make_shared<ScopeGuardCb>(done_cb, cb_user_data);
try {
auto fret = std::make_unique<std::future<size_t>>();
*fret = runner->listen(
*hash,
[cb, cb_user_data, guard](const std::vector<std::shared_ptr<dht::Value>>& values, bool expired) {
for (const auto& value : values) {
if (cb && not cb(reinterpret_cast<const dht_value*>(&value), expired, cb_user_data))
return false;
}
return true;
},
{},
where ? dht::Where(where) : dht::Where {});
return reinterpret_cast<dht_op_token*>(fret.release());
} catch (...) {
return nullptr;
}
}
void
@@ -562,6 +897,73 @@ dht_op_token_delete(dht_op_token* token)
delete reinterpret_cast<std::future<size_t>*>(token);
}
void
dht_runner_query(
dht_runner* r, const dht_infohash* h, const char* query, dht_query_cb cb, dht_done_cb done_cb, void* user_data)
{
auto guard = std::make_shared<CompletionGuard>(done_cb, user_data);
try {
reinterpret_cast<dht::DhtRunner*>(r)->query(
*reinterpret_cast<const dht::InfoHash*>(h),
[guard, cb, user_data](const std::vector<std::shared_ptr<dht::FieldValueIndex>>& results) {
for (const auto& index : results) {
dht_query_result result {};
dht::Blob user_type;
for (const auto& entry : index->index) {
switch (entry.first) {
case dht::Value::Field::Id:
result.fields |= DHT_FIELD_ID;
result.id = entry.second.getInt();
break;
case dht::Value::Field::ValueType:
result.fields |= DHT_FIELD_TYPE;
result.value_type = entry.second.getInt();
break;
case dht::Value::Field::SeqNum:
result.fields |= DHT_FIELD_SEQ;
result.seq = entry.second.getInt();
break;
case dht::Value::Field::OwnerPk:
result.fields |= DHT_FIELD_OWNER;
result.owner = dht_infohash_to_c(entry.second.getHash());
break;
case dht::Value::Field::UserType:
result.fields |= DHT_FIELD_USER_TYPE;
user_type = entry.second.getBlob();
result.user_type = {user_type.data(), user_type.size()};
break;
default:
break;
}
}
if (cb && !cb(&result, user_data))
return false;
}
return true;
},
[guard](bool ok) { guard->finish(ok); },
dht::Query(query ? query : ""));
} catch (...) {
guard->finish(false);
}
}
void
dht_runner_find_certificate(
dht_runner* runner, const dht_pkid* id, dht_certificate_cb callback, dht_shutdown_cb released_cb, void* user_data)
{
auto guard = std::make_shared<ScopeGuardCb>(released_cb, user_data);
try {
reinterpret_cast<dht::DhtRunner*>(runner)->findCertificate(
*reinterpret_cast<const dht::PkId*>(id),
[guard, callback, user_data](const std::shared_ptr<dht::crypto::Certificate>& certificate) {
if (callback)
callback(certificate ? reinterpret_cast<const dht_certificate*>(&certificate) : nullptr, user_data);
});
} catch (...) {
}
}
void
dht_runner_put(
dht_runner* r, const dht_infohash* h, const dht_value* v, dht_done_cb done_cb, void* cb_user_data, bool permanent)
@@ -569,15 +971,12 @@ dht_runner_put(
auto runner = reinterpret_cast<dht::DhtRunner*>(r);
auto hash = reinterpret_cast<const dht::InfoHash*>(h);
auto value = reinterpret_cast<const ValueSp*>(v);
runner->put(
*hash,
*value,
[done_cb, cb_user_data](bool ok) {
if (done_cb)
done_cb(ok, cb_user_data);
},
dht::time_point::max(),
permanent);
auto guard = std::make_shared<CompletionGuard>(done_cb, cb_user_data);
try {
runner->put(*hash, *value, [guard](bool ok) { guard->finish(ok); }, dht::time_point::max(), permanent);
} catch (...) {
guard->finish(false);
}
}
void
@@ -587,14 +986,12 @@ dht_runner_put_signed(
auto runner = reinterpret_cast<dht::DhtRunner*>(r);
auto hash = reinterpret_cast<const dht::InfoHash*>(h);
auto value = reinterpret_cast<const ValueSp*>(v);
runner->putSigned(
*hash,
*value,
[done_cb, cb_user_data](bool ok) {
if (done_cb)
done_cb(ok, cb_user_data);
},
permanent);
auto guard = std::make_shared<CompletionGuard>(done_cb, cb_user_data);
try {
runner->putSigned(*hash, *value, [guard](bool ok) { guard->finish(ok); }, permanent);
} catch (...) {
guard->finish(false);
}
}
void
@@ -610,15 +1007,12 @@ dht_runner_put_encrypted(dht_runner* r,
auto hash = reinterpret_cast<const dht::InfoHash*>(h);
auto toHash = reinterpret_cast<const dht::PkId*>(to);
auto value = reinterpret_cast<const ValueSp*>(v);
runner->putEncrypted(
*hash,
*toHash,
*value,
[done_cb, cb_user_data](bool ok) {
if (done_cb)
done_cb(ok, cb_user_data);
},
permanent);
auto guard = std::make_shared<CompletionGuard>(done_cb, cb_user_data);
try {
runner->putEncrypted(*hash, *toHash, *value, [guard](bool ok) { guard->finish(ok); }, permanent);
} catch (...) {
guard->finish(false);
}
}
void
@@ -633,10 +1027,8 @@ void
dht_runner_shutdown(dht_runner* r, dht_shutdown_cb done_cb, void* cb_user_data)
{
auto runner = reinterpret_cast<dht::DhtRunner*>(r);
runner->shutdown([done_cb, cb_user_data]() {
if (done_cb)
done_cb(cb_user_data);
});
auto guard = std::make_shared<ScopeGuardCb>(done_cb, cb_user_data);
runner->shutdown([guard]() {});
}
bool
@@ -681,12 +1073,94 @@ dht_runner_get_public_address(const dht_runner* r)
if (addrs.empty())
return nullptr;
auto ret = (struct sockaddr**) malloc(sizeof(struct sockaddr*) * (addrs.size() + 1));
if (!ret)
return nullptr;
for (size_t i = 0; i < addrs.size(); i++)
ret[i] = addrs[i].release();
ret[addrs.size()] = nullptr;
return ret;
}
void
dht_sockaddr_array_delete(struct sockaddr** addresses)
{
if (addresses) {
for (auto current = addresses; *current; ++current)
free(*current);
free(addresses);
}
}
int
dht_runner_set_push_notification_token(dht_runner* runner, const char* value)
{
try {
reinterpret_cast<dht::DhtRunner*>(runner)->setPushNotificationToken(value ? value : "");
return 0;
} catch (...) {
return EINVAL;
}
}
int
dht_runner_set_push_notification_topic(dht_runner* runner, const char* value)
{
try {
reinterpret_cast<dht::DhtRunner*>(runner)->setPushNotificationTopic(value ? value : "");
return 0;
} catch (...) {
return EINVAL;
}
}
int
dht_runner_set_push_notification_platform(dht_runner* runner, const char* value)
{
try {
reinterpret_cast<dht::DhtRunner*>(runner)->setPushNotificationPlatform(value ? value : "");
return 0;
} catch (...) {
return EINVAL;
}
}
dht_push_token*
dht_runner_push_notification_received(dht_runner* runner,
const char* const* keys,
const char* const* values,
size_t count)
{
try {
std::map<std::string, std::string> data;
for (size_t index = 0; index < count; ++index)
data.emplace(keys[index], values[index]);
return reinterpret_cast<dht_push_token*>(new std::future<dht::PushNotificationResult>(
reinterpret_cast<dht::DhtRunner*>(runner)->pushNotificationReceived(data)));
} catch (...) {
return nullptr;
}
}
int
dht_push_token_poll(dht_push_token* token, int* result)
{
if (!token || !result)
return EINVAL;
try {
auto future = reinterpret_cast<std::future<dht::PushNotificationResult>*>(token);
if (future->wait_for(std::chrono::seconds(0)) != std::future_status::ready)
return EAGAIN;
*result = static_cast<int>(future->get());
return 0;
} catch (...) {
return EIO;
}
}
void
dht_push_token_delete(dht_push_token* token)
{
delete reinterpret_cast<std::future<dht::PushNotificationResult>*>(token);
}
#ifdef __cplusplus
}
#endif
+104 -2
View File
@@ -39,9 +39,17 @@ OPENDHT_C_PUBLIC dht_data_view dht_blob_get_data(const dht_blob* data);
OPENDHT_C_PUBLIC void dht_blob_delete(dht_blob* data);
// dht::InfoHash
#ifdef __cplusplus
#define OPENDHT_C_ALIGN_UINT32 alignas(uint32_t)
#elif defined(_MSC_VER)
#define OPENDHT_C_ALIGN_UINT32 __declspec(align(4))
#else
#define OPENDHT_C_ALIGN_UINT32 _Alignas(uint32_t)
#endif
struct OPENDHT_C_PUBLIC dht_infohash
{
uint8_t d[HASH_LEN];
OPENDHT_C_ALIGN_UINT32 uint8_t d[HASH_LEN];
};
typedef struct dht_infohash dht_infohash;
OPENDHT_C_PUBLIC void dht_infohash_zero(dht_infohash* h);
@@ -56,9 +64,10 @@ OPENDHT_C_PUBLIC bool dht_infohash_is_zero(const dht_infohash* h);
// dht::PkId
struct OPENDHT_C_PUBLIC dht_pkid
{
uint8_t d[32];
OPENDHT_C_ALIGN_UINT32 uint8_t d[32];
};
typedef struct dht_pkid dht_pkid;
#undef OPENDHT_C_ALIGN_UINT32
OPENDHT_C_PUBLIC const char* dht_pkid_print(const dht_pkid* h);
// dht::crypto::PublicKey
@@ -67,6 +76,8 @@ typedef struct dht_publickey dht_publickey;
OPENDHT_C_PUBLIC dht_publickey* dht_publickey_import(const uint8_t* dat, size_t dat_size);
OPENDHT_C_PUBLIC void dht_publickey_delete(dht_publickey* pk);
OPENDHT_C_PUBLIC int dht_publickey_export(const dht_publickey* pk, char* out, size_t* out_size);
OPENDHT_C_PUBLIC dht_blob* dht_publickey_export_blob(const dht_publickey* pk);
OPENDHT_C_PUBLIC int dht_publickey_unpack(dht_publickey* pk, const uint8_t* data, size_t size);
OPENDHT_C_PUBLIC dht_infohash dht_publickey_get_id(const dht_publickey* pk);
OPENDHT_C_PUBLIC dht_pkid dht_publickey_get_long_id(const dht_publickey* pk);
OPENDHT_C_PUBLIC bool dht_publickey_check_signature(
@@ -77,8 +88,11 @@ OPENDHT_C_PUBLIC dht_blob* dht_publickey_encrypt(const dht_publickey* pk, const
struct OPENDHT_C_PUBLIC dht_privatekey;
typedef struct dht_privatekey dht_privatekey;
OPENDHT_C_PUBLIC dht_privatekey* dht_privatekey_generate(unsigned key_length_bits);
OPENDHT_C_PUBLIC dht_privatekey* dht_privatekey_generate_ec(void);
OPENDHT_C_PUBLIC dht_privatekey* dht_privatekey_import(const uint8_t* dat, size_t dat_size, const char* password);
OPENDHT_C_PUBLIC int dht_privatekey_export(const dht_privatekey*, char* out, size_t* out_size, const char* password);
OPENDHT_C_PUBLIC dht_blob* dht_privatekey_export_blob(const dht_privatekey*, const char* password);
OPENDHT_C_PUBLIC dht_blob* dht_privatekey_sign(const dht_privatekey*, const uint8_t* data, size_t size);
OPENDHT_C_PUBLIC dht_publickey* dht_privatekey_get_publickey(const dht_privatekey*);
OPENDHT_C_PUBLIC dht_blob* dht_privatekey_decrypt(const dht_privatekey*, const char* data, size_t data_size);
OPENDHT_C_PUBLIC void dht_privatekey_delete(dht_privatekey*);
@@ -87,6 +101,8 @@ OPENDHT_C_PUBLIC void dht_privatekey_delete(dht_privatekey*);
struct OPENDHT_C_PUBLIC dht_certificate;
typedef struct dht_certificate dht_certificate;
OPENDHT_C_PUBLIC dht_certificate* dht_certificate_import(const uint8_t* dat, size_t dat_size);
OPENDHT_C_PUBLIC dht_blob* dht_certificate_export(const dht_certificate* certificate);
OPENDHT_C_PUBLIC dht_certificate* dht_certificate_ref(const dht_certificate* certificate);
OPENDHT_C_PUBLIC dht_infohash dht_certificate_get_id(const dht_certificate*);
OPENDHT_C_PUBLIC dht_pkid dht_certificate_get_long_id(const dht_certificate*);
OPENDHT_C_PUBLIC dht_publickey* dht_certificate_get_publickey(const dht_certificate*);
@@ -99,6 +115,7 @@ struct OPENDHT_C_PUBLIC dht_identity
};
typedef struct dht_identity dht_identity;
OPENDHT_C_PUBLIC dht_identity dht_identity_generate(const char* common_name, const dht_identity* ca);
OPENDHT_C_PUBLIC dht_identity dht_identity_generate_ec(const char* common_name, const dht_identity* ca);
OPENDHT_C_PUBLIC void dht_identity_delete(dht_identity*);
// dht::Value
@@ -109,6 +126,7 @@ OPENDHT_C_PUBLIC dht_value* dht_value_new(const uint8_t* data, size_t size);
OPENDHT_C_PUBLIC dht_value* dht_value_with_id_new(const uint8_t* data, size_t size, uint64_t value_id);
OPENDHT_C_PUBLIC dht_value* dht_value_new_from_string(const char* str);
OPENDHT_C_PUBLIC dht_value* dht_value_ref(const dht_value*);
OPENDHT_C_PUBLIC dht_value* dht_value_copy(const dht_value*);
OPENDHT_C_PUBLIC void dht_value_unref(dht_value*);
OPENDHT_C_PUBLIC dht_data_view dht_value_get_data(const dht_value* data);
OPENDHT_C_PUBLIC dht_value_id dht_value_get_id(const dht_value* data);
@@ -116,12 +134,53 @@ OPENDHT_C_PUBLIC dht_publickey* dht_value_get_owner(const dht_value* data);
OPENDHT_C_PUBLIC dht_infohash dht_value_get_recipient(const dht_value* data);
OPENDHT_C_PUBLIC const char* dht_value_get_user_type(const dht_value* data);
OPENDHT_C_PUBLIC void dht_value_set_user_type(dht_value* data, const char* user_type);
OPENDHT_C_PUBLIC uint16_t dht_value_get_type(const dht_value* value);
OPENDHT_C_PUBLIC void dht_value_set_type(dht_value* value, uint16_t type);
OPENDHT_C_PUBLIC uint16_t dht_value_get_seq(const dht_value* value);
OPENDHT_C_PUBLIC void dht_value_set_seq(dht_value* value, uint16_t seq);
OPENDHT_C_PUBLIC unsigned dht_value_get_priority(const dht_value* value);
OPENDHT_C_PUBLIC void dht_value_set_priority(dht_value* value, unsigned priority);
OPENDHT_C_PUBLIC const char* dht_value_get_push_type(const dht_value* value);
OPENDHT_C_PUBLIC void dht_value_set_push_type(dht_value* value, const char* push_type);
// callbacks
typedef bool (*dht_get_cb)(const dht_value* value, void* user_data);
typedef bool (*dht_value_cb)(const dht_value* value, bool expired, void* user_data);
typedef void (*dht_done_cb)(bool ok, void* user_data);
typedef void (*dht_shutdown_cb)(void* user_data);
typedef void (*dht_status_cb)(int ipv4_status, int ipv6_status, void* user_data);
typedef void (*dht_certificate_cb)(const dht_certificate* certificate, void* user_data);
typedef void (*dht_certificate_store_cb)(const dht_pkid* id, dht_certificate_cb emit, void* emit_data, void* user_data);
typedef struct dht_runner_context
{
dht_status_cb status_cb;
dht_certificate_store_cb certificate_store_cb;
dht_shutdown_cb released_cb;
void* user_data;
} dht_runner_context;
enum dht_query_field {
DHT_FIELD_ID = 1,
DHT_FIELD_TYPE = 2,
DHT_FIELD_SEQ = 4,
DHT_FIELD_OWNER = 8,
DHT_FIELD_USER_TYPE = 16
};
typedef struct dht_query_result
{
uint32_t fields;
uint64_t id;
uint16_t value_type;
uint16_t seq;
dht_infohash owner;
dht_data_view user_type;
} dht_query_result;
typedef bool (*dht_query_cb)(const dht_query_result* result, void* user_data);
struct dht_push_token;
typedef struct dht_push_token dht_push_token;
struct OPENDHT_C_PUBLIC dht_op_token;
typedef struct dht_op_token dht_op_token;
@@ -169,17 +228,50 @@ struct OPENDHT_C_PUBLIC dht_runner;
typedef struct dht_runner dht_runner;
OPENDHT_C_PUBLIC dht_runner* dht_runner_new(void);
OPENDHT_C_PUBLIC void dht_runner_delete(dht_runner* runner);
OPENDHT_C_PUBLIC int dht_runner_loop(dht_runner* runner);
OPENDHT_C_PUBLIC int dht_runner_join(dht_runner* runner);
/* Returns 0 on success, standard error code on failure */
OPENDHT_C_PUBLIC int dht_runner_run(dht_runner* runner, in_port_t port);
/* Returns 0 on success, standard error code on failure */
OPENDHT_C_PUBLIC int dht_runner_run_config(dht_runner* runner, in_port_t port, const dht_runner_config* config);
OPENDHT_C_PUBLIC int dht_runner_run_bind(dht_runner* runner,
const char* ipv4,
const char* ipv6,
in_port_t port,
const dht_runner_config* config,
const dht_runner_context* context);
OPENDHT_C_PUBLIC int dht_runner_connectivity_changed(dht_runner* runner);
OPENDHT_C_PUBLIC int dht_runner_register_type(
dht_runner* runner, uint16_t id, const char* name, uint64_t expiration_seconds, bool insecure);
OPENDHT_C_PUBLIC dht_publickey* dht_runner_get_publickey(const dht_runner* runner);
OPENDHT_C_PUBLIC void dht_runner_find_certificate(
dht_runner* runner, const dht_pkid* id, dht_certificate_cb callback, dht_shutdown_cb released_cb, void* user_data);
OPENDHT_C_PUBLIC int dht_runner_register_certificate(dht_runner* runner, const dht_certificate* certificate);
OPENDHT_C_PUBLIC void dht_runner_ping(
dht_runner* runner, struct sockaddr* addr, socklen_t addr_len, dht_done_cb done_cb, void* cb_user_data);
OPENDHT_C_PUBLIC void dht_runner_bootstrap(dht_runner* runner, const char* host, const char* service);
OPENDHT_C_PUBLIC void dht_runner_get(
dht_runner* runner, const dht_infohash* hash, dht_get_cb cb, dht_done_cb done_cb, void* cb_user_data);
OPENDHT_C_PUBLIC void dht_runner_get_filtered(dht_runner* runner,
const dht_infohash* hash,
const char* where,
dht_get_cb cb,
dht_done_cb done_cb,
void* user_data);
OPENDHT_C_PUBLIC void dht_runner_query(dht_runner* runner,
const dht_infohash* hash,
const char* query,
dht_query_cb cb,
dht_done_cb done_cb,
void* user_data);
OPENDHT_C_PUBLIC dht_op_token* dht_runner_listen(
dht_runner* runner, const dht_infohash* hash, dht_value_cb cb, dht_shutdown_cb done_cb, void* cb_user_data);
OPENDHT_C_PUBLIC dht_op_token* dht_runner_listen_filtered(dht_runner* runner,
const dht_infohash* hash,
const char* where,
dht_value_cb cb,
dht_shutdown_cb released_cb,
void* user_data);
OPENDHT_C_PUBLIC void dht_runner_cancel_listen(dht_runner* runner, const dht_infohash* hash, dht_op_token* token);
OPENDHT_C_PUBLIC void dht_runner_put(dht_runner* runner,
const dht_infohash* hash,
@@ -208,6 +300,16 @@ OPENDHT_C_PUBLIC bool dht_runner_is_running(const dht_runner* runner);
OPENDHT_C_PUBLIC in_port_t dht_runner_get_bound_port(const dht_runner* runner, sa_family_t af);
/** Returns null-terminated array that must be freed after use as well as each element */
OPENDHT_C_PUBLIC struct sockaddr** dht_runner_get_public_address(const dht_runner* runner);
OPENDHT_C_PUBLIC void dht_sockaddr_array_delete(struct sockaddr** addresses);
OPENDHT_C_PUBLIC int dht_runner_set_push_notification_token(dht_runner* runner, const char* value);
OPENDHT_C_PUBLIC int dht_runner_set_push_notification_topic(dht_runner* runner, const char* value);
OPENDHT_C_PUBLIC int dht_runner_set_push_notification_platform(dht_runner* runner, const char* value);
OPENDHT_C_PUBLIC dht_push_token* dht_runner_push_notification_received(dht_runner* runner,
const char* const* keys,
const char* const* values,
size_t count);
OPENDHT_C_PUBLIC int dht_push_token_poll(dht_push_token* token, int* result);
OPENDHT_C_PUBLIC void dht_push_token_delete(dht_push_token* token);
#ifdef __cplusplus
}
+12 -11
View File
@@ -5,15 +5,16 @@ LABEL org.opencontainers.image.source=https://github.com/savoirfairelinux/opendh
COPY . opendht
RUN cd opendht && mkdir build && cd build \
&& cmake .. -DCMAKE_INSTALL_PREFIX=/usr \
-DBUILD_TESTING=Off \
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=On \
-DOPENDHT_C=On \
-DOPENDHT_PEER_DISCOVERY=On \
-DOPENDHT_PYTHON=On \
-DOPENDHT_TOOLS=On \
-DOPENDHT_PROXY_SERVER=On \
-DOPENDHT_PROXY_CLIENT=On \
-DOPENDHT_SYSTEMD=On \
&& make -j8 && make install \
&& cmake -GNinja .. -DCMAKE_INSTALL_PREFIX=/usr \
-DBUILD_TESTING=Off \
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=On \
-DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DOPENDHT_C=On \
-DOPENDHT_PEER_DISCOVERY=On \
-DOPENDHT_PYTHON=On \
-DOPENDHT_TOOLS=On \
-DOPENDHT_PROXY_SERVER=On \
-DOPENDHT_PROXY_CLIENT=On \
-DOPENDHT_SYSTEMD=On \
&& ninja && ninja install \
&& cd ../.. && rm -rf opendht
+2 -1
View File
@@ -10,6 +10,7 @@ ENV DESTDIR=/install
RUN cd opendht && mkdir build && cd build \
&& cmake .. -DCMAKE_INSTALL_PREFIX=/usr \
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=On \
-DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DBUILD_TESTING=Off \
-DOPENDHT_C=On \
-DOPENDHT_PEER_DISCOVERY=On \
@@ -20,7 +21,7 @@ RUN cd opendht && mkdir build && cd build \
-DOPENDHT_SYSTEMD=On \
&& make -j8 && make install
FROM alpine:3.23 AS install
FROM alpine:3.24 AS install
COPY --from=build /install /
RUN apk add --no-cache \
libstdc++ \
+8 -6
View File
@@ -1,4 +1,4 @@
FROM ubuntu:24.04
FROM ubuntu:26.04
LABEL maintainer="Adrien Béraud <adrien.beraud@savoirfairelinux.com>"
LABEL org.opencontainers.image.source=https://github.com/savoirfairelinux/opendht
@@ -8,23 +8,25 @@ RUN apt-get update && apt-get install -y \
&& echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections
RUN apt-get update && apt-get install -y \
build-essential pkg-config cmake git wget meson \
build-essential pkg-config cmake git wget meson ninja-build \
libtool \
python3-dev python3-setuptools python3-build python3-virtualenv cython3 \
libncurses5-dev libreadline-dev nettle-dev libcppunit-dev \
libgnutls28-dev libuv1-dev libjsoncpp-dev libargon2-dev \
libssl-dev libfmt-dev libasio-dev libmsgpack-dev \
libssl-dev libfmt-dev libasio-dev libmsgpack-cxx-dev \
libllhttp-dev libsimdutf-dev \
&& apt-get clean && rm -rf /var/lib/apt/lists/* /var/cache/apt/*
RUN mkdir /usr/include/nonstd \
&& wget https://raw.githubusercontent.com/martinmoene/expected-lite/master/include/nonstd/expected.hpp \
-O /usr/include/nonstd/expected.hpp
ARG RESTINIO_VERSION=0.7.10
RUN echo "*** Downloading RESTinio ***" \
&& mkdir restinio && cd restinio \
&& wget https://github.com/Stiffstream/restinio/releases/download/v0.7.8.1/restinio-0.7.8.1.tar.bz2 \
&& ls -l && tar -xjf restinio-0.7.8.1.tar.bz2 \
&& cd restinio-0.7.8.1/dev \
&& wget "https://github.com/Stiffstream/restinio/releases/download/v${RESTINIO_VERSION}/restinio-${RESTINIO_VERSION}.tar.bz2" \
&& ls -l && tar -xjf "restinio-${RESTINIO_VERSION}.tar.bz2" \
&& cd "restinio-${RESTINIO_VERSION}/dev" \
&& cmake -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=Off -DRESTINIO_SAMPLE=Off -DRESTINIO_BENCHMARK=Off \
-DRESTINIO_WITH_SOBJECTIZER=Off -DRESTINIO_DEP_STANDALONE_ASIO=system -DRESTINIO_DEP_LLHTTP=system \
-DRESTINIO_DEP_FMT=system -DRESTINIO_DEP_EXPECTED_LITE=system . \
+6 -5
View File
@@ -1,4 +1,4 @@
FROM alpine:3.23
FROM alpine:3.24
LABEL maintainer="Adrien Béraud <adrien.beraud@savoirfairelinux.com>"
LABEL org.opencontainers.image.source=https://github.com/savoirfairelinux/opendht
@@ -7,7 +7,7 @@ RUN apk add --no-cache \
cython python3-dev py3-setuptools \
ncurses-dev readline-dev nettle-dev \
cppunit-dev gnutls-dev jsoncpp-dev \
argon2-dev openssl-dev fmt-dev \
argon2-dev openssl-dev fmt-dev simdutf-dev \
llhttp-dev asio-dev msgpack-cxx-dev \
&& rm -rf /var/cache/apk/*
@@ -15,11 +15,12 @@ RUN mkdir /usr/include/nonstd \
&& wget https://raw.githubusercontent.com/martinmoene/expected-lite/master/include/nonstd/expected.hpp \
-O /usr/include/nonstd/expected.hpp
ARG RESTINIO_VERSION=0.7.10
RUN echo "*** Downloading RESTinio ***" \
&& mkdir restinio && cd restinio \
&& wget https://github.com/Stiffstream/restinio/releases/download/v0.7.8.1/restinio-0.7.8.1.tar.bz2 \
&& ls -l && tar -xjf restinio-0.7.8.1.tar.bz2 \
&& cd restinio-0.7.8.1/dev \
&& wget "https://github.com/Stiffstream/restinio/releases/download/v${RESTINIO_VERSION}/restinio-${RESTINIO_VERSION}.tar.bz2" \
&& ls -l && tar -xjf "restinio-${RESTINIO_VERSION}.tar.bz2" \
&& cd "restinio-${RESTINIO_VERSION}/dev" \
&& cmake -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=Off -DRESTINIO_SAMPLE=Off -DRESTINIO_BENCHMARK=Off \
-DRESTINIO_WITH_SOBJECTIZER=Off -DRESTINIO_DEP_STANDALONE_ASIO=system -DRESTINIO_DEP_LLHTTP=system \
-DRESTINIO_DEP_FMT=system -DRESTINIO_DEP_EXPECTED_LITE=system . \
+4 -3
View File
@@ -15,11 +15,12 @@ RUN mkdir /usr/include/nonstd \
&& wget https://raw.githubusercontent.com/martinmoene/expected-lite/master/include/nonstd/expected.hpp \
-O /usr/include/nonstd/expected.hpp
ARG RESTINIO_VERSION=0.7.10
RUN echo "*** Downloading RESTinio ***" \
&& mkdir restinio && cd restinio \
&& wget https://github.com/Stiffstream/restinio/releases/download/v0.7.8.1/restinio-0.7.8.1.tar.bz2 \
&& ls -l && tar -xjf restinio-0.7.8.1.tar.bz2 \
&& cd restinio-0.7.8.1/dev \
&& wget "https://github.com/Stiffstream/restinio/releases/download/v${RESTINIO_VERSION}/restinio-${RESTINIO_VERSION}.tar.bz2" \
&& ls -l && tar -xjf "restinio-${RESTINIO_VERSION}.tar.bz2" \
&& cd "restinio-${RESTINIO_VERSION}/dev" \
&& cmake -GNinja -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=Off -DRESTINIO_SAMPLE=Off -DRESTINIO_BENCHMARK=Off \
-DRESTINIO_WITH_SOBJECTIZER=Off -DRESTINIO_DEP_STANDALONE_ASIO=system -DRESTINIO_DEP_LLHTTP=system \
-DRESTINIO_DEP_FMT=system -DRESTINIO_DEP_EXPECTED_LITE=system . \
+17 -3
View File
@@ -18,15 +18,29 @@ RUN apt-get update && apt-get install -y \
RUN pip3 install meson Cython
ARG SIMDUTF_VERSION=8.0.0
RUN mkdir simdutf \
&& wget -q "https://github.com/simdutf/simdutf/archive/refs/tags/v${SIMDUTF_VERSION}.tar.gz" -O simdutf.tar.gz \
&& tar -xzf simdutf.tar.gz -C simdutf --strip-components=1 \
&& cmake -S simdutf -B simdutf/build -DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=/usr/local -DCMAKE_POSITION_INDEPENDENT_CODE=ON \
-DBUILD_SHARED_LIBS=OFF -DSIMDUTF_TESTS=OFF -DSIMDUTF_TOOLS=OFF \
&& cmake --build simdutf/build --parallel 2 \
&& cmake --install simdutf/build \
&& install -D simdutf/LICENSE-MIT /usr/share/doc/simdutf/LICENSE-MIT \
&& install -D simdutf/LICENSE-APACHE /usr/share/doc/simdutf/LICENSE-APACHE \
&& rm -rf simdutf simdutf.tar.gz
RUN mkdir /usr/include/nonstd \
&& wget https://raw.githubusercontent.com/martinmoene/expected-lite/master/include/nonstd/expected.hpp \
-O /usr/include/nonstd/expected.hpp
ARG RESTINIO_VERSION=0.7.10
RUN echo "*** Downloading RESTinio ***" \
&& mkdir restinio && cd restinio \
&& wget https://github.com/Stiffstream/restinio/releases/download/v.0.7.3/restinio-0.7.3.tar.bz2 \
&& ls -l && tar -xjf restinio-0.7.3.tar.bz2 \
&& cd restinio-0.7.3/dev \
&& wget "https://github.com/Stiffstream/restinio/releases/download/v${RESTINIO_VERSION}/restinio-${RESTINIO_VERSION}.tar.bz2" \
&& ls -l && tar -xjf "restinio-${RESTINIO_VERSION}.tar.bz2" \
&& cd "restinio-${RESTINIO_VERSION}/dev" \
&& cmake -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=Off -DRESTINIO_SAMPLE=Off -DRESTINIO_BENCHMARK=Off \
-DRESTINIO_WITH_SOBJECTIZER=Off -DRESTINIO_DEP_STANDALONE_ASIO=system -DRESTINIO_DEP_LLHTTP=system \
-DRESTINIO_DEP_FMT=system -DRESTINIO_DEP_EXPECTED_LITE=system . \
+7 -6
View File
@@ -1,4 +1,4 @@
FROM ubuntu:24.04
FROM ubuntu:26.04
LABEL maintainer="Adrien Béraud <adrien.beraud@savoirfairelinux.com>"
LABEL org.opencontainers.image.source=https://github.com/savoirfairelinux/opendht
@@ -10,9 +10,9 @@ RUN apt-get update && apt-get install -y \
RUN apt-get update \
&& apt-get install -y llvm llvm-dev lldb clang gdb make cmake meson pkg-config \
libtool git wget libncurses5-dev libreadline-dev \
nettle-dev libgnutls28-dev libuv1-dev libmsgpack-dev libjsoncpp-dev python3-dev \
nettle-dev libgnutls28-dev libuv1-dev libmsgpack-cxx-dev libjsoncpp-dev python3-dev \
python3-setuptools libcppunit-dev python3-pip python3-build python3-virtualenv \
libssl-dev libargon2-dev \
libssl-dev libargon2-dev libllhttp-dev libsimdutf-dev \
libfmt-dev libasio-dev \
&& apt-get remove -y gcc g++ && apt-get autoremove -y \
&& apt-get clean && rm -rf /var/lib/apt/lists/* /var/cache/apt/*
@@ -26,11 +26,12 @@ RUN mkdir /usr/include/nonstd \
&& wget https://raw.githubusercontent.com/martinmoene/expected-lite/master/include/nonstd/expected.hpp \
-O /usr/include/nonstd/expected.hpp
ARG RESTINIO_VERSION=0.7.10
RUN echo "*** Downloading RESTinio ***" \
&& mkdir restinio && cd restinio \
&& wget https://github.com/Stiffstream/restinio/releases/download/v0.7.8.1/restinio-0.7.8.1.tar.bz2 \
&& ls -l && tar -xjf restinio-0.7.8.1.tar.bz2 \
&& cd restinio-0.7.8.1/dev \
&& wget "https://github.com/Stiffstream/restinio/releases/download/v${RESTINIO_VERSION}/restinio-${RESTINIO_VERSION}.tar.bz2" \
&& ls -l && tar -xjf "restinio-${RESTINIO_VERSION}.tar.bz2" \
&& cd "restinio-${RESTINIO_VERSION}/dev" \
&& cmake -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=Off -DRESTINIO_SAMPLE=Off -DRESTINIO_BENCHMARK=Off \
-DRESTINIO_WITH_SOBJECTIZER=Off -DRESTINIO_DEP_STANDALONE_ASIO=system -DRESTINIO_DEP_LLHTTP=system \
-DRESTINIO_DEP_FMT=system -DRESTINIO_DEP_EXPECTED_LITE=system . \
+56
View File
@@ -0,0 +1,56 @@
FROM ubuntu:24.04
LABEL maintainer="Adrien Béraud <adrien.beraud@savoirfairelinux.com>"
LABEL org.opencontainers.image.source=https://github.com/savoirfairelinux/opendht
RUN apt-get update && apt-get install -y \
dialog apt-utils \
&& apt-get clean \
&& echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections
RUN apt-get update && apt-get install -y \
build-essential pkg-config cmake git wget meson ninja-build patchelf \
libtool \
python3-dev python3-setuptools python3-build python3-virtualenv cython3 \
libncurses5-dev libreadline-dev nettle-dev libcppunit-dev \
libgnutls28-dev libuv1-dev libjsoncpp-dev libargon2-dev \
libssl-dev libfmt-dev libasio-dev libmsgpack-cxx-dev \
&& apt-get clean && rm -rf /var/lib/apt/lists/* /var/cache/apt/*
ARG LLHTTP_VERSION=9.2.1
RUN mkdir llhttp \
&& wget -q "https://github.com/nodejs/llhttp/archive/refs/tags/release/v${LLHTTP_VERSION}.tar.gz" -O llhttp.tar.gz \
&& tar -xzf llhttp.tar.gz -C llhttp --strip-components=1 \
&& cmake -S llhttp -B llhttp/build -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr \
&& cmake --build llhttp/build --parallel 2 \
&& cmake --install llhttp/build \
&& rm -rf llhttp llhttp.tar.gz
ARG SIMDUTF_VERSION=8.0.0
RUN mkdir simdutf \
&& wget -q "https://github.com/simdutf/simdutf/archive/refs/tags/v${SIMDUTF_VERSION}.tar.gz" -O simdutf.tar.gz \
&& tar -xzf simdutf.tar.gz -C simdutf --strip-components=1 \
&& cmake -S simdutf -B simdutf/build -DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=/usr/local -DCMAKE_POSITION_INDEPENDENT_CODE=ON \
-DBUILD_SHARED_LIBS=OFF -DSIMDUTF_TESTS=OFF -DSIMDUTF_TOOLS=OFF \
&& cmake --build simdutf/build --parallel 2 \
&& cmake --install simdutf/build \
&& install -D simdutf/LICENSE-MIT /usr/share/doc/simdutf/LICENSE-MIT \
&& install -D simdutf/LICENSE-APACHE /usr/share/doc/simdutf/LICENSE-APACHE \
&& rm -rf simdutf simdutf.tar.gz
RUN mkdir /usr/include/nonstd \
&& wget https://raw.githubusercontent.com/martinmoene/expected-lite/master/include/nonstd/expected.hpp \
-O /usr/include/nonstd/expected.hpp
ARG RESTINIO_VERSION=0.7.10
RUN echo "*** Downloading RESTinio ***" \
&& mkdir restinio && cd restinio \
&& wget "https://github.com/Stiffstream/restinio/releases/download/v${RESTINIO_VERSION}/restinio-${RESTINIO_VERSION}.tar.bz2" \
&& ls -l && tar -xjf "restinio-${RESTINIO_VERSION}.tar.bz2" \
&& cd "restinio-${RESTINIO_VERSION}/dev" \
&& cmake -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=Off -DRESTINIO_SAMPLE=Off -DRESTINIO_BENCHMARK=Off \
-DRESTINIO_WITH_SOBJECTIZER=Off -DRESTINIO_DEP_STANDALONE_ASIO=system -DRESTINIO_DEP_LLHTTP=system \
-DRESTINIO_DEP_FMT=system -DRESTINIO_DEP_EXPECTED_LITE=system . \
&& make -j2 && make install \
&& cd ../../ && rm -rf restinio* \
&& ldconfig
+1
View File
@@ -8,6 +8,7 @@ RUN cd opendht && mkdir build && cd build \
&& cmake .. -DCMAKE_INSTALL_PREFIX=/usr \
-DBUILD_TESTING=Off \
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=On \
-DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DOPENDHT_C=On \
-DOPENDHT_PEER_DISCOVERY=On \
-DOPENDHT_PYTHON=On \
+1
View File
@@ -7,6 +7,7 @@ COPY . opendht
RUN cd opendht && mkdir build && cd build \
&& cmake .. -DCMAKE_INSTALL_PREFIX=/usr \
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=On \
-DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DOPENDHT_C=On \
-DOPENDHT_PEER_DISCOVERY=On \
-DOPENDHT_PYTHON=On \
+20
View File
@@ -0,0 +1,20 @@
FROM ghcr.io/savoirfairelinux/opendht/opendht-deps-noble:latest
LABEL maintainer="Adrien Béraud <adrien.beraud@savoirfairelinux.com>"
LABEL org.opencontainers.image.source=https://github.com/savoirfairelinux/opendht
COPY . opendht
RUN cd opendht && mkdir build && cd build \
&& cmake .. -DCMAKE_INSTALL_PREFIX=/usr \
-DBUILD_TESTING=Off \
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=On \
-DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DOPENDHT_C=On \
-DOPENDHT_PEER_DISCOVERY=On \
-DOPENDHT_PYTHON=On \
-DOPENDHT_TOOLS=On \
-DOPENDHT_PROXY_SERVER=On \
-DOPENDHT_PROXY_CLIENT=On \
-DOPENDHT_SYSTEMD=On \
&& make -j8 && make install \
&& cd ../.. && rm -rf opendht
+15 -3
View File
@@ -204,7 +204,13 @@ struct OPENDHT_PUBLIC PrivateKey
* @param algo : the public key algorithm to use, GNUTLS_PK_RSA or GNUTLS_PK_RSA_OAEP
*/
static PrivateKey generate(unsigned key_length = 4096, gnutls_pk_algorithm_t algo = GNUTLS_PK_RSA);
static PrivateKey generateEC();
/**
* Generate a new EC key pair
* @param curve : the elliptic curve to use. If GNUTLS_ECC_CURVE_INVALID is provided, the default curve will be used
* (currently GNUTLS_ECC_CURVE_SECP384R1).
*/
static PrivateKey generateEC(gnutls_ecc_curve_t curve = GNUTLS_ECC_CURVE_INVALID);
gnutls_privkey_t key {};
gnutls_x509_privkey_t x509_key {};
@@ -267,9 +273,15 @@ public:
/**
* Sign this revocation list using provided key and certificate.
* Validity_period sets the duration until next update (default to no next update).
* If `number` is non-zero, it is stored verbatim (big-endian) as the CRL Number
* extension. Otherwise the number is initialized randomly on the first signature
* and incremented on subsequent ones.
*/
void sign(const PrivateKey&, const Certificate&, duration validity_period = {});
void sign(const Identity& id) { sign(*id.first, *id.second); }
void sign(const PrivateKey&, const Certificate&, duration validity_period = {}, uint64_t number = 0);
void sign(const Identity& id, duration validity_period = {}, uint64_t number = 0)
{
sign(*id.first, *id.second, validity_period, number);
}
bool isSignedBy(const Certificate& issuer) const;
+23 -14
View File
@@ -19,7 +19,9 @@
#include <mutex>
namespace dht {
enum class PushType { None = 0, Android, iOS, UnifiedPush };
// Appended only: the values are serialized as integers by MSGPACK_ADD_ENUM,
// so reordering would break compatibility with deployed proxies.
enum class PushType { None = 0, Android, iOS, UnifiedPush, Huawei };
}
MSGPACK_ADD_ENUM(dht::PushType)
@@ -76,8 +78,14 @@ public:
asio::io_context& io_context() const;
// Monotonic clock for elapsed-time statistics (request rate, uptime).
// Must stay on steady_clock so NTP/wall-clock corrections don't skew durations.
using clock = std::chrono::steady_clock;
using time_point = clock::time_point;
// Wall clock for persisted subscribe/permanent-put lease expirations,
// so they map directly to epoch time and survive restarts.
using sys_clock = std::chrono::system_clock;
using sys_time_point = sys_clock::time_point;
struct PushStats
{
@@ -126,6 +134,7 @@ public:
PushStats androidPush;
PushStats iosPush;
PushStats unifiedPush;
PushStats huaweiPush;
/** Average requests per second */
double requestRate {0};
@@ -266,6 +275,9 @@ private:
{
std::mutex lock;
std::string sessionId;
/** Where to push: follows the latest subscription refresh of the device. */
PushType type {PushType::None};
std::string topic;
PushSessionContext(const std::string& id)
: sessionId(id)
{}
@@ -335,20 +347,16 @@ private:
*
* @param infoHash The information hash associated with the push listen request.
* @param pushToken The push token associated with the push listen request.
* @param type The type of the push listen request.
* @param clientId The client ID associated with the push listen request.
* @param sessionCtx The shared pointer to the push session context associated with the push listen request.
* @param topic The topic associated with the push listen request.
* @param sessionCtx The push session context of the listener, giving the session, push type and topic to use.
* @param values The vector of shared pointers to values associated with the push listen request.
* @param expired A boolean indicating whether the push listen request has expired.
* @return true.
*/
bool handlePushListen(const InfoHash& infoHash,
const std::string& pushToken,
PushType type,
const std::string& clientId,
const std::shared_ptr<DhtProxyServer::PushSessionContext>& sessionCtx,
const std::string& topic,
const std::vector<std::shared_ptr<Value>>& values,
bool expired);
@@ -392,6 +400,7 @@ private:
PushStats androidPush_;
PushStats iosPush_;
PushStats unifiedPush_;
PushStats huaweiPush_;
// Thread-safe access to listeners map.
std::mutex lockListener_;
@@ -401,12 +410,12 @@ private:
std::shared_ptr<ConnectionListener> connListener_;
struct PermanentPut
{
time_point expiration;
sys_time_point expiration;
std::string pushToken;
std::string clientId;
std::shared_ptr<PushSessionContext> sessionCtx;
std::unique_ptr<asio::steady_timer> expireTimer;
std::unique_ptr<asio::steady_timer> expireNotifyTimer;
std::unique_ptr<asio::system_timer> expireTimer;
std::unique_ptr<asio::system_timer> expireNotifyTimer;
Sp<Value> value;
PushType type;
std::string topic;
@@ -419,7 +428,7 @@ private:
p.pack("value");
p.pack(value);
p.pack("exp");
p.pack(to_time_t(expiration));
p.pack(sys_clock::to_time_t(expiration));
if (not clientId.empty()) {
p.pack("cid");
p.pack(clientId);
@@ -460,12 +469,12 @@ private:
#ifdef OPENDHT_PUSH_NOTIFICATIONS
struct Listener
{
time_point expiration;
sys_time_point expiration;
std::string clientId;
std::shared_ptr<PushSessionContext> sessionCtx;
std::future<size_t> internalToken;
std::unique_ptr<asio::steady_timer> expireTimer;
std::unique_ptr<asio::steady_timer> expireNotifyTimer;
std::unique_ptr<asio::system_timer> expireTimer;
std::unique_ptr<asio::system_timer> expireNotifyTimer;
PushType type;
std::string topic;
@@ -476,7 +485,7 @@ private:
p.pack("cid");
p.pack(clientId);
p.pack("exp");
p.pack(to_time_t(expiration));
p.pack(sys_clock::to_time_t(expiration));
if (sessionCtx) {
std::lock_guard l(sessionCtx->lock);
p.pack("sid");
+2
View File
@@ -532,6 +532,8 @@ public:
/**
* Insert a push notification to process for OpenDHT
* Returns an immediately ready future with IgnoredStopped if the runner is idle,
* or IgnoredDisabled if push notification support is disabled at build time.
*/
std::future<PushNotificationResult> pushNotificationReceived(const std::map<std::string, std::string>& data);
+7
View File
@@ -106,9 +106,16 @@ public:
void timeout(const std::chrono::seconds& timeout, HandlerCb cb = {});
/**
* Close the underlying socket.
* Safe to call from any thread: the actual close is dispatched to
* the thread running the io_context.
*/
void close();
private:
void doClose();
template<typename T>
T wrapCallback(T cb) const
{
+12 -20
View File
@@ -123,28 +123,19 @@ ee *
*/
static inline unsigned commonBits(const Prefix& p1, const Prefix& p2)
{
unsigned i, j;
uint8_t x;
/* size_ counts bits, so the comparison walks bits: reading it as a
byte count would run past the end of the shorter content. */
auto longest_prefix_size = std::min(p1.size_, p2.size_);
for (i = 0; i < longest_prefix_size; i++) {
if (p1.content_.data()[i] != p2.content_.data()[i] or not p1.isFlagActive(i) or not p2.isFlagActive(i)) {
unsigned common = 0;
while (common < longest_prefix_size) {
if (not p1.isFlagActive(common) or not p2.isFlagActive(common)
or p1.isContentBitActive(common) != p2.isContentBitActive(common))
break;
}
common++;
}
if (i == longest_prefix_size)
return 8 * longest_prefix_size;
x = p1.content_.data()[i] ^ p2.content_.data()[i];
j = 0;
while ((x & 0x80) == 0) {
x <<= 1;
j++;
}
return 8 * i + j;
return common;
}
/**
@@ -205,7 +196,7 @@ private:
for (auto i = copy.size(); i < size; i++)
copy.push_back(0);
swapBit(copy, size_ + 1);
swapBit(copy, size_);
return copy;
}
@@ -242,8 +233,9 @@ private:
if (bit >= b.size() * 8)
throw std::out_of_range("bit larger than prefix size.");
size_t offset_bit = (8 - bit) % 8;
b[bit / 8] ^= (1 << offset_bit);
/* Bit 0 is the most significant one of the first byte, as
isActiveBit reads it. */
b[bit / 8] ^= (1 << (7 - (bit % 8)));
}
};
+7 -2
View File
@@ -489,8 +489,12 @@ private:
static constexpr size_t MTU {1280};
static constexpr size_t MAX_PACKET_VALUE_SIZE {600};
static constexpr size_t MAX_MESSAGE_VALUE_SIZE {56 * 1024};
static constexpr size_t RX_MAX_EARLY_BYTES {MAX_MESSAGE_VALUE_SIZE + MAX_VALUE_SIZE + 32};
static constexpr size_t RX_MAX_EARLY_PARTS {RX_MAX_EARLY_BYTES / MTU + 2};
void process(std::unique_ptr<ParsedMessage>&&, const SockAddr& from);
bool isExpectedReply(const ParsedMessage&, const SockAddr& from);
bool tryCompleteMessage(PartialMessage&, const SockAddr& from);
bool rateLimit(const SockAddr& addr);
@@ -527,7 +531,8 @@ private:
{
return packValueHeader(buf, values.begin(), values.end());
}
void maintainRxBuffer(Tid tid);
using PartialMessageKey = std::pair<SockAddr, Tid>;
void maintainRxBuffer(const PartialMessageKey& key);
/*************
* Answers *
@@ -573,7 +578,7 @@ private:
// requests handling
std::map<Tid, Sp<Request>> requests {};
std::map<Tid, PartialMessage> partial_messages;
std::map<PartialMessageKey, PartialMessage> partial_messages;
MessageStats in_stats {}, out_stats {};
std::set<SockAddr> blacklist {};
+1 -1
View File
@@ -8,7 +8,7 @@ namespace dht {
namespace proxy {
constexpr const std::chrono::hours OP_TIMEOUT {24}; // one day
constexpr const std::chrono::hours OP_MARGIN {2}; // two hours
constexpr const std::chrono::hours OP_MARGIN {8}; // eight hours
using ListenToken = uint64_t;
+5
View File
@@ -284,6 +284,11 @@ public:
void forwardAllMessages(bool forward) { forward_all_ = forward; }
void setPushNotificationToken(const std::string& token = "") override { dht_->setPushNotificationToken(token); }
void setPushNotificationTopic(const std::string& topic) override { dht_->setPushNotificationTopic(topic); }
void setPushNotificationPlatform(const std::string& platform) override
{
dht_->setPushNotificationPlatform(platform);
}
/**
* Call linked callback with push_notification
+8
View File
@@ -85,6 +85,14 @@ using system_clock = std::chrono::system_clock;
using time_point = clock::time_point;
using duration = clock::duration;
OPENDHT_PUBLIC time_point from_system_time(system_clock::time_point t,
time_point steady_now,
system_clock::time_point system_now);
OPENDHT_PUBLIC time_point from_system_time(system_clock::time_point t);
OPENDHT_PUBLIC system_clock::time_point to_system_time(time_point t,
time_point steady_now,
system_clock::time_point system_now);
OPENDHT_PUBLIC system_clock::time_point to_system_time(time_point t);
OPENDHT_PUBLIC time_point from_time_t(std::time_t t);
OPENDHT_PUBLIC std::time_t to_time_t(time_point t);
+17 -1
View File
@@ -742,6 +742,7 @@ struct OPENDHT_PUBLIC FieldValue
switch (field) {
case Value::Field::Id:
case Value::Field::ValueType:
case Value::Field::SeqNum:
p.pack(intValue);
break;
case Value::Field::OwnerPk:
@@ -773,6 +774,7 @@ struct OPENDHT_PUBLIC FieldValue
switch (field) {
case Value::Field::Id:
case Value::Field::ValueType:
case Value::Field::SeqNum:
intValue = v->as<decltype(intValue)>();
break;
case Value::Field::OwnerPk:
@@ -1015,7 +1017,7 @@ struct OPENDHT_PUBLIC Query
*
* where
*
* - $field$ = *|id|value_type|owner_pk|user_type
* - $field$ = *|id|value_type|seq|owner_pk|user_type
* - $value$ = $string$|$integer$
* - $string$: a simple string WITHOUT SPACES.
* - $integer$: a simple integer.
@@ -1032,9 +1034,23 @@ struct OPENDHT_PUBLIC Query
/**
* Tell if the query is satisfied by another query.
*
* The answer to `q` contains every value this query asks for, but it may
* contain more: the caller is responsible for applying this query's own
* filters to the values it receives.
*/
bool isSatisfiedBy(const Query& q) const;
/**
* Tell if the projected answer to another query can be reported for this
* query without any further filtering.
*
* A field-projected answer only carries the fields selected by `q`, so a
* narrower filter can no longer be evaluated on it. Both queries must
* therefore apply exactly the same filters.
*/
bool isProjectionSatisfiedBy(const Query& q) const;
template<typename Packer>
void msgpack_pack(Packer& pk) const
{
+44 -4
View File
@@ -5,7 +5,7 @@ project(
'cython',
license: 'MIT',
license_files: 'LICENSE',
version: '4.1.1',
version: '4.5.0-rc.1',
meson_version: '>=1.1.0',
default_options: ['cpp_std=c++20', 'warning_level=3'],
)
@@ -55,7 +55,12 @@ if host_machine.system() == 'linux' and io_uring.found()
endif
conf_data = configuration_data()
add_project_arguments('-DMSGPACK_NO_BOOST', '-DASIO_STANDALONE', language: 'cpp')
add_project_arguments(
'-DMSGPACK_NO_BOOST',
'-DASIO_STANDALONE',
'-DPACKAGE_VERSION="' + meson.project_version() + '"',
language: 'cpp',
)
if io_uring.found()
add_project_arguments('-DASIO_HAS_IO_URING', '-DASIO_DISABLE_EPOLL', language: 'cpp')
conf_data.set('iouring_lib', ', liburing')
@@ -209,15 +214,22 @@ if get_option('python').require(
get_option('indexation').enabled(),
error_message: 'python requires indexation',
).allowed()
py = import('python').find_installation()
py = import('python').find_installation(pure: false)
dep_py = py.dependency()
py.extension_module(
'opendht',
'_core',
'python/opendht.pyx',
subdir: 'opendht',
cython_args: ['--module-name', 'opendht._core'],
override_options: ['cython_language=cpp'],
dependencies: [opendht_dep, msgpack, jsoncpp, fmt, dep_py],
install: true,
)
py.install_sources(
'python/opendht/__init__.py',
'python/opendht/aio.py',
subdir: 'opendht',
)
endif
if get_option('tools').enabled()
@@ -256,6 +268,16 @@ endif
# Unit tests
if get_option('tests').enabled()
if get_option('c').enabled()
test_c = executable(
'test_c',
'tests/test_c.c',
include_directories: ['c', opendht_interface_inc],
dependencies: [gnutls],
link_with: opendht_c,
)
test('CBindings', test_c)
endif
cppunit = dependency('cppunit')
test_infohash = executable(
@@ -292,6 +314,24 @@ if get_option('tests').enabled()
)
test('NetworkEngine', test_networkengine)
test_opcache = executable(
'test_opcache',
'tests/test_opcache.cpp',
'tests/tests_runner.cpp',
include_directories: opendht_inc,
dependencies: [opendht_dep, cppunit, jsoncpp, fmt, openssl, msgpack],
)
test('OpCache', test_opcache)
test_valuecache = executable(
'test_valuecache',
'tests/test_valuecache.cpp',
'tests/tests_runner.cpp',
include_directories: opendht_inc,
dependencies: [opendht_dep, cppunit, jsoncpp, fmt, openssl, msgpack],
)
test('ValueCache', test_valuecache)
test_crypto = executable(
'test_crypto',
'tests/test_crypto.cpp',
+5
View File
@@ -0,0 +1,5 @@
node_modules/
build*/
prebuilds/
dist/
examples/dhtnode.js
+64
View File
@@ -0,0 +1,64 @@
cmake_minimum_required(VERSION 3.21)
project(opendht_node LANGUAGES C CXX)
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
set(OPENDHT_TOOLS OFF CACHE BOOL "Build DHT tools")
set(OPENDHT_PYTHON OFF CACHE BOOL "Build Python bindings")
set(OPENDHT_PROXY_CLIENT ON CACHE BOOL "Enable DHT proxy client")
set(OPENDHT_PEER_DISCOVERY OFF CACHE BOOL "Enable multicast peer discovery")
set(OPENDHT_DOCUMENTATION OFF CACHE BOOL "Build documentation")
set(BUILD_TESTING OFF CACHE BOOL "Build native tests")
add_subdirectory(.. native EXCLUDE_FROM_ALL)
find_program(NODE_EXECUTABLE node REQUIRED)
execute_process(
COMMAND "${NODE_EXECUTABLE}" --input-type=module -e "import addon from 'node-addon-api'; process.stdout.write(addon.include_dir)"
WORKING_DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}"
OUTPUT_VARIABLE NODE_ADDON_API_DIR
OUTPUT_STRIP_TRAILING_WHITESPACE
COMMAND_ERROR_IS_FATAL ANY
)
add_library(opendht_node MODULE addon.cpp crypto.cpp ${CMAKE_JS_SRC})
target_compile_definitions(opendht_node PRIVATE NAPI_VERSION=8 NAPI_CPP_EXCEPTIONS)
target_include_directories(opendht_node PRIVATE
${CMAKE_JS_INC}
"${CMAKE_CURRENT_SOURCE_DIR}/${NODE_ADDON_API_DIR}"
"${CMAKE_CURRENT_SOURCE_DIR}/../include"
"${CMAKE_CURRENT_BINARY_DIR}/native/include"
)
target_link_libraries(opendht_node PRIVATE opendht ${CMAKE_JS_LIB})
if(OPENDHT_SANITIZE AND CMAKE_CXX_COMPILER_ID MATCHES "GNU|Clang")
target_compile_options(opendht_node PRIVATE -fsanitize=address -fstack-protector-strong)
target_link_options(opendht_node PRIVATE -fsanitize=address)
endif()
set_target_properties(opendht_node PROPERTIES
PREFIX ""
SUFFIX ".node"
OUTPUT_NAME "opendht"
LIBRARY_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}/$<CONFIG>"
RUNTIME_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}/$<CONFIG>"
)
if(APPLE)
target_link_options(opendht_node PRIVATE "-undefined" "dynamic_lookup")
endif()
if(WIN32)
add_custom_command(TARGET opendht_node POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different
$<TARGET_RUNTIME_DLLS:opendht_node> $<TARGET_FILE_DIR:opendht_node>
COMMAND_EXPAND_LISTS
)
endif()
if(MSVC)
target_compile_options(opendht_node PRIVATE /EHsc)
target_include_directories(opendht_node PRIVATE
"${CMAKE_CURRENT_SOURCE_DIR}/../src/compat/msvc"
"${CMAKE_CURRENT_SOURCE_DIR}/../src/compat/win32"
)
if(CMAKE_JS_NODELIB_DEF AND CMAKE_JS_NODELIB_TARGET)
execute_process(COMMAND ${CMAKE_AR} /def:${CMAKE_JS_NODELIB_DEF}
/out:${CMAKE_JS_NODELIB_TARGET} ${CMAKE_STATIC_LINKER_FLAGS})
endif()
endif()
+243
View File
@@ -0,0 +1,243 @@
# OpenDHT Node.js Bindings
Initial Node-API bindings using `node-addon-api` directly over the C++ library.
Node.js 22.9 or later is required; the addon targets Node-API version 8.
## Install
```sh
npm install opendht
```
## Build
Install the native dependencies described in [BUILD.md](../BUILD.md), plus Node.js,
a C++17 compiler, and CMake 3.21 or later. Ninja is optional. Proxy-client support
defaults to enabled and requires ASIO, llhttp, RESTinio, and expected-lite headers.
```sh
cd nodejs
npm ci
npm run build
npm test
npm run typecheck
```
CMake.js builds OpenDHT from the parent source directory in `nodejs/build/native`.
It does not change the repository's existing build directory. Tools, Python,
peer discovery, documentation, and native tests default to disabled in this build.
Native CMake options can be passed through, for example:
```sh
npm run build -- --CDOPENDHT_PEER_DISCOVERY=ON
```
For an existing build, change cached options directly with CMake, for example
`cmake -S . -B build -DOPENDHT_PROXY_CLIENT=ON`, then rebuild. A custom build can
disable proxy support with `-DOPENDHT_PROXY_CLIENT=OFF`; requesting a proxy then
throws an explicit unsupported-build error.
The loader checks local builds, then compatible Node-API binaries under `prebuilds/`.
Set `OPENDHT_NODE_BINDING` to an absolute addon path to select an explicit build.
Packaged binaries include OpenDHT and non-system native dependencies, so archive
installation does not require a compiler or a system OpenDHT installation.
## Usage
```js
import { DhtRunner, hash } from 'opendht';
async function main() {
const writer = new DhtRunner();
const reader = new DhtRunner();
try {
writer.run({ port: 0, ipv4: '127.0.0.1', ipv6: '::1' });
reader.run({ port: 0, ipv4: '127.0.0.1', ipv6: '::1' });
writer.bootstrap('127.0.0.1', reader.getBoundPort());
reader.bootstrap('127.0.0.1', writer.getBoundPort());
const key = hash('example');
const stored = await writer.put(key, Buffer.from('hello'), { id: 42n });
if (!stored) throw new Error('DHT put failed');
for await (const value of reader.get(key)) {
console.log(value.id, value.data.toString());
}
} finally {
await Promise.all([writer.close(), reader.close()]);
}
}
main().catch(console.error);
```
## API
- `hash(string | Uint8Array)` returns a SHA-1 hash as a 40-character hex string.
All get/put/listen keys must be hex hashes, not unhashed application keys.
- `version` is the underlying OpenDHT version.
- `run({port, ipv4, ipv6, network, persistPath, identity})` starts a threaded node. Port 0
selects a free port. The defaults bind both IP families on all interfaces.
`persistPath` is optional; use an empty string to disable persistence.
`proxyServer` and `proxyUserAgent` configure proxy mode (see below).
- `getBoundPort(4 | 6)` defaults to IPv4; `getNodeId()` returns the node's hex ID.
- `getStatus()` returns the latest JS-dispatched `{ipv4, ipv6}` connectivity
snapshot. Each family is `'disconnected'`, `'connecting'`, or `'connected'`.
`runner.on('statusChanged', status => {})` receives changes on the JS thread.
New and closing/closed runners report both families as disconnected. No status
events are delivered after close begins; close does not emit a final event.
- `bootstrap(host, port)` queues bootstrap discovery. Returning does not mean the
peer is connected; subsequent DHT operations report their own completion.
- `put(key, Uint8Array, {id, type, userType, permanent})` copies the input and
resolves to the native success boolean. IDs are unsigned 64-bit `bigint` values.
- `get(key)` returns an async iterator of values as they arrive. The iterator
ends when the lookup completes, or rejects with `ERR_DHT_GET_FAILED` on failure.
`getAll(key)` returns a single promise resolving to the complete value array.
Values contain `id`, `type`, `seq`, `userType`, copied `Buffer` data, `isSigned`,
`isEncrypted`, and an `owner` long fingerprint (or null for unsigned values).
- `listen(key, (value, expired) => {})` returns a listener with idempotent
`cancel()`. Queued deliveries are suppressed after cancellation. Callback
return values are ignored; callbacks may cancel or close the runner.
- `listen(key)` without a callback returns an async iterator of `{value, expired}`
events. It continues until cancelled or the runner closes.
- `close()` is idempotent and returns the same promise on repeated calls. It
waits off the JS thread for graceful shutdown, then joins the native threads.
After two seconds it forces the join if graceful shutdown has not completed.
Pending operations reject with `ERR_DHT_CLOSED`. A closed runner cannot restart.
- `ref()` and `unref()` control whether the runner keeps Node's event loop alive.
Running nodes are referenced by default. `Symbol.asyncDispose` calls `close()`.
Callbacks run only on their owning JS thread. There is no polling timer or worker
thread waiting on each get/put. Explicitly close runners; GC cleanup and Node
environment cleanup hooks are fallbacks, including forced worker termination.
Pending operations, active listeners, and registered status handlers keep their
runner alive until completion, cancellation, handler removal, or explicit close.
Graceful persistence cannot be guaranteed if the process is killed.
The binding does not cap the callback queue, accumulated `getAll()` results, or
iterator buffers. Slow iterator consumers can accumulate values without
backpressure. Queued callbacks retain their native values until delivery or cleanup.
## Async Iteration
`get()` and callback-free `listen()` start immediately and return iterators with
`next()`, `return()`, `throw()`, `cancel()`, and `Symbol.asyncDispose`.
Leaving a `for await` loop with `break` or an exception cancels the subscription.
Explicit cancellation discards buffered values and settles pending reads as done.
Closing the runner rejects active get iterators with `ERR_DHT_CLOSED`; listener
iterators end normally. Successful get completion preserves unread buffered values.
```js
for await (const value of runner.get(hash('example'))) {
console.log(value.id, value.data);
}
const snapshots = await runner.getAll(hash('example'));
for await (const { value, expired } of runner.listen(hash('updates'))) {
console.log(value.id, expired);
if (expired) break;
}
```
## Proxy Configuration
Pass the URL of an OpenDHT HTTP(S) proxy at startup:
```js
runner.run({
proxyServer: 'https://proxy.example.com',
proxyUserAgent: 'my-app/1.0',
});
```
This selects the native OpenDHT proxy client, not an HTTP CONNECT or SOCKS proxy.
The node does not bind UDP sockets in this mode; `port`, `ipv4`, `ipv6`, and
`bootstrap()` are for direct DHT nodes. Omit `proxyServer` or use an empty string
for direct networking. Proxy settings are startup-only; create a new runner to
change transports or servers. Packaged binaries include proxy-client support.
## Crypto
`PrivateKey.generate(options)` and `Identity.generate(name, options)` return
promises and generate keys off the JS thread. Options accept `algorithm: 'ec'`
(the default, using OpenDHT's default EC curve) or `'rsa'`. RSA `bits` must be an
integer between 2048 and 8192; the default is 2048.
- `new PrivateKey(bytes, password?)` imports PEM/DER; `serialize(password?)`
exports PKCS#8 PEM. A nonempty password encrypts the exported key.
- `privateKey.getPublicKey()` returns an owned public key. `sign(bytes)` and
`decrypt(bytes)` return copied Buffers.
- `new PublicKey(bytes)` imports a public key; `serialize()` exports DER.
`checkSignature(data, signature)` returns a boolean. `encrypt(bytes)` requires
an RSA key; EC keys support signatures, not this encryption scheme.
- `new Certificate(bytes)` imports PEM/DER or a PEM certificate chain;
`serialize()` exports a PEM chain. Import does not establish trust or validate
the chain against a trust store. `getPublicKey()` returns its public key.
- Public keys and certificates expose `getId()` (40 hex characters) and
`getLongId()` (64 hex characters).
- `new Identity(privateKey, certificate)` requires matching fingerprints.
Its read-only `privateKey` and `certificate` properties return owned wrappers.
- Pass `identity` to `run()` to enable `putSigned(key, data, options?)` and
`putEncrypted(key, data, recipientPublicKey, options?)`. Both return the same
success boolean as `put()`. Encrypted values are signed by the sender and
transparently decrypted for the recipient by OpenDHT's get/listen layer.
```js
import { Identity, DhtRunner } from 'opendht';
async function createAuthenticatedNode() {
const identity = await Identity.generate('example', { algorithm: 'rsa', bits: 2048 });
const runner = new DhtRunner();
runner.run({ port: 0, identity });
return { runner, identity };
}
```
Crypto methods take Uint8Array/Buffer inputs, not implicit text encodings. Imports
must be nonempty. Malformed handles/data,
mismatched identities, and unsupported encryption keys raise JS exceptions.
Only generation is asynchronous; import/export/sign/encrypt/decrypt are
synchronous and should be used from worker threads for large workloads.
## Packaging And CI
From a successful Release build:
```sh
npm run package
npm run test:package
```
`bundle` copies the addon into `prebuilds/<platform>-<arch>/`, gathers its runtime
dependencies, rewrites native loader paths, and copies dependency license
notices. macOS needs `dylibbundler` and Apple developer tools; Linux needs
`patchelf` and Debian/Ubuntu dependency copyright files. Windows uses the vcpkg
installed tree (`VCPKG_INSTALLED_DIR`, `VCPKG_DEFAULT_TRIPLET`) and MSVC tools.
`OPENDHT_LIBRARY_PATH` adds native library search directories when bundling.
An explicit addon path may be passed to `npm run bundle -- /absolute/addon.node`.
Bundling replaces only the current platform's generated prebuild directory.
`test:package` creates an npm archive, installs it in a temporary consumer with
install scripts disabled and no dev dependencies, forces prebuild-only loading,
and runs the full runtime suite. Pass an archive file or a directory containing
one archive to test it directly, for example `npm run test:package -- dist`.
Only runtime JS, declarations, the example's TypeScript source, documentation,
and prebuilds are included in the archive. `prepack` compiles the example, so
archive creation needs the development dependencies; archive installation does not.
The [Node.js package workflow](../.github/workflows/nodejs-package.yml) builds:
| Target | Build Baseline |
| --- | --- |
| Linux x64 | Ubuntu 24.04 / glibc 2.39 (Noble dependency image) |
| macOS ARM64 and x64 | macOS 15 |
| Windows x64 | Windows Server 2022 / MSVC |
CI combines platform prebuilds into one npm archive and verifies that archive on
Node 22, 24, and 26 on each platform, including Ubuntu 24.04 and 26.04. Branch
pushes, pull requests, and manual runs build and test without publishing.
musl/Alpine and other architectures are not currently built. Node-API stability
does not remove OS/libc or native dependency compatibility requirements. Review
the included dependency licenses and source-distribution obligations before
redistributing binaries.
Query filters, AbortSignal support, certificate trust-store validation, CA
issuance controls, and runtime proxy switching remain future work.
+734
View File
@@ -0,0 +1,734 @@
#include <napi.h>
#include <opendht.h>
#include <gnutls/gnutls.h>
#include "crypto.h"
#include <algorithm>
#include <atomic>
#include <chrono>
#include <cmath>
#include <condition_variable>
#include <functional>
#include <mutex>
#include <optional>
#include <unordered_map>
namespace {
struct GnuTlsContext
{
const int result = gnutls_global_init();
~GnuTlsContext()
{
if (result == GNUTLS_E_SUCCESS)
gnutls_global_deinit();
}
};
using opendht_node::guarded;
uint32_t
unsignedNumber(Napi::Value value, uint32_t maximum, const char* name)
{
if (!value.IsNumber())
throw Napi::TypeError::New(value.Env(), std::string(name) + " must be a number");
const auto number = value.As<Napi::Number>().DoubleValue();
if (!std::isfinite(number) || number < 0 || number > maximum || std::floor(number) != number)
throw Napi::RangeError::New(value.Env(), std::string(name) + " is out of range");
return static_cast<uint32_t>(number);
}
uint64_t
unsignedBigInt(Napi::Value value)
{
if (!value.IsBigInt())
throw Napi::TypeError::New(value.Env(), "id must be a bigint");
int sign = 0;
size_t count = 1;
uint64_t word = 0;
value.As<Napi::BigInt>().ToWords(&sign, &count, &word);
if (sign || count > 1)
throw Napi::RangeError::New(value.Env(), "id must fit in an unsigned 64-bit integer");
return word;
}
std::string
text(Napi::Value value, const char* name)
{
if (!value.IsString())
throw Napi::TypeError::New(value.Env(), std::string(name) + " must be a string");
auto result = value.As<Napi::String>().Utf8Value();
if (result.find('\0') != std::string::npos)
throw Napi::TypeError::New(value.Env(), std::string(name) + " must not contain NUL bytes");
return result;
}
dht::InfoHash
infoHash(Napi::Value value)
{
const auto result = text(value, "hash");
if (result.size() != 40 || result.find_first_not_of("0123456789abcdefABCDEF") != std::string::npos)
throw Napi::TypeError::New(value.Env(), "Expected a 40-character hexadecimal hash");
return dht::InfoHash(result);
}
Napi::Object
optionsObject(Napi::Value value)
{
if (!value.IsObject() || value.IsArray() || value.IsNull())
throw Napi::TypeError::New(value.Env(), "options must be an object");
return value.As<Napi::Object>();
}
Napi::Object
valueToJS(Napi::Env env, const dht::Value& value)
{
auto result = Napi::Object::New(env);
result.Set("id", Napi::BigInt::New(env, value.id));
result.Set("type", Napi::Number::New(env, value.type));
result.Set("seq", Napi::Number::New(env, value.seq));
result.Set("data", Napi::Buffer<uint8_t>::Copy(env, value.data.data(), value.data.size()));
result.Set("userType", Napi::String::New(env, value.user_type));
result.Set("owner",
value.owner ? Napi::Value(Napi::String::New(env, value.owner->getLongId().to_view()))
: Napi::Value(env.Null()));
result.Set("isSigned", Napi::Boolean::New(env, value.isSigned()));
result.Set("isEncrypted", Napi::Boolean::New(env, value.isEncrypted()));
return result;
}
struct PendingOperation
{
Napi::Promise::Deferred deferred;
bool get;
std::vector<std::shared_ptr<const dht::Value>> values;
Napi::FunctionReference onValue;
std::shared_ptr<std::atomic_bool> cancelled;
};
struct Subscription
{
dht::InfoHash key;
std::shared_future<size_t> token;
Napi::FunctionReference callback;
};
struct RunnerState
{
using Callback = std::function<void(RunnerState&)>;
explicit RunnerState(Napi::Env environment)
: env(environment)
{}
static std::shared_ptr<RunnerState> create(Napi::Env env)
{
auto state = std::make_shared<RunnerState>(env);
auto dispatchOwner = std::make_unique<std::shared_ptr<RunnerState>>(state);
napi_threadsafe_function function;
if (napi_create_threadsafe_function(env,
nullptr,
nullptr,
Napi::String::New(env, "OpenDHT callbacks"),
0,
1,
dispatchOwner.get(),
finalizeDispatcher,
state.get(),
dispatch,
&function)
!= napi_ok)
throw Napi::Error::New(env, "Could not create OpenDHT callback dispatcher");
dispatchOwner.release();
state->dispatcher = Napi::ThreadSafeFunction(function);
state->dispatcher.Unref(env);
auto closeOwner = std::make_unique<std::shared_ptr<RunnerState>>(state);
if (napi_create_async_work(env,
nullptr,
Napi::String::New(env, "OpenDHT close"),
executeClose,
completeClose,
closeOwner.get(),
&state->closeWork)
!= napi_ok) {
state->dispatcher.Release();
throw Napi::Error::New(env, "Could not create OpenDHT cleanup work");
}
if (napi_add_async_cleanup_hook(env, cleanup, state.get(), &state->cleanupHook) != napi_ok) {
napi_delete_async_work(env, state->closeWork);
state->dispatcher.Release();
throw Napi::Error::New(env, "Could not register OpenDHT cleanup hook");
}
state->closeOwner = closeOwner.release();
return state;
}
void requireRunning() const
{
if (closing)
throw Napi::Error::New(env, "DHT runner is closed");
if (!started)
throw Napi::Error::New(env, "DHT runner is not running");
}
void startClose()
{
if (closing.exchange(true))
return;
if (napi_queue_async_work(env, closeWork) != napi_ok) {
executeClose(env, closeOwner);
completeClose(env, napi_ok, closeOwner);
}
}
void send(Callback callback)
{
if (closing)
return;
auto task = std::make_unique<Callback>(std::move(callback));
if (dispatcher.NonBlockingCall(static_cast<void*>(task.get())) == napi_ok)
task.release();
}
void sendValue(uint64_t operation, std::shared_ptr<const dht::Value> value, bool expired = false)
{
send([operation, value = std::move(value), expired](RunnerState& state) mutable {
const auto listener = state.listeners.find(operation);
if (listener != state.listeners.end()) {
listener->second.callback.Call({valueToJS(state.env, *value), Napi::Boolean::New(state.env, expired)});
} else if (const auto pending = state.operations.find(operation); pending != state.operations.end()) {
if (!pending->second.onValue.IsEmpty())
pending->second.onValue.Call({valueToJS(state.env, *value)});
else
pending->second.values.emplace_back(std::move(value));
}
});
}
void completeOperation(uint64_t identifier, bool ok)
{
const auto pending = operations.find(identifier);
if (pending == operations.end())
return;
auto operation = std::move(pending->second);
operations.erase(pending);
if (!operation.get) {
operation.deferred.Resolve(Napi::Boolean::New(env, ok));
} else if (!ok) {
auto error = Napi::Error::New(env, "DHT get failed");
error.Set("code", "ERR_DHT_GET_FAILED");
operation.deferred.Reject(error.Value());
} else if (!operation.onValue.IsEmpty()) {
operation.deferred.Resolve(env.Undefined());
} else {
auto values = Napi::Array::New(env, operation.values.size());
for (size_t index = 0; index < operation.values.size(); ++index)
values.Set(static_cast<uint32_t>(index), valueToJS(env, *operation.values[index]));
operation.deferred.Resolve(values);
}
}
Napi::Object status() const
{
auto result = Napi::Object::New(env);
result.Set("ipv4", Napi::String::New(env, dht::statusToStr(closing ? dht::NodeStatus::Disconnected : status4)));
result.Set("ipv6", Napi::String::New(env, dht::statusToStr(closing ? dht::NodeStatus::Disconnected : status6)));
return result;
}
static void dispatch(napi_env environment, napi_value, void* context, void* data)
{
auto state = static_cast<RunnerState*>(context);
std::unique_ptr<Callback> callback(static_cast<Callback*>(data));
if (!environment || state->envClosing || state->closing)
return;
try {
(*callback)(*state);
} catch (const Napi::Error& error) {
napi_fatal_exception(environment, error.Value());
} catch (const std::exception& error) {
napi_fatal_exception(environment, Napi::Error::New(environment, error.what()).Value());
}
}
static void finalizeDispatcher(napi_env, void* data, void*)
{
delete static_cast<std::shared_ptr<RunnerState>*>(data);
}
static void cleanup(napi_async_cleanup_hook_handle, void* data)
{
auto state = static_cast<RunnerState*>(data);
state->envClosing = true;
state->startClose();
}
static void executeClose(napi_env, void* data)
{
auto state = *static_cast<std::shared_ptr<RunnerState>*>(data);
struct ShutdownSignal
{
std::mutex mutex;
std::condition_variable condition;
bool done = false;
};
try {
auto signal = std::make_shared<ShutdownSignal>();
state->runner.shutdown(
[signal] {
std::lock_guard lock(signal->mutex);
signal->done = true;
signal->condition.notify_one();
},
true);
std::unique_lock lock(signal->mutex);
signal->condition.wait_for(lock, std::chrono::seconds(2), [&] { return signal->done; });
} catch (const std::exception& error) {
state->closeError = error.what();
}
state->runner.join();
}
Napi::Error makeCloseError() const
{
auto error = Napi::Error::New(env, closeError.empty() ? "DHT runner is closed" : closeError);
error.Set("code", "ERR_DHT_CLOSED");
return error;
}
static void completeClose(napi_env environment, napi_status, void* data)
{
std::unique_ptr<std::shared_ptr<RunnerState>> owner(static_cast<std::shared_ptr<RunnerState>*>(data));
auto state = *owner;
state->finished = true;
if (!state->envClosing) {
state->dispatcher.Release();
auto error = state->makeCloseError();
for (auto& pending : state->operations)
pending.second.deferred.Reject(error.Value());
if (state->closeDeferred) {
if (state->closeError.empty())
state->closeDeferred->Resolve(Napi::Env(environment).Undefined());
else
state->closeDeferred->Reject(error.Value());
}
}
state->operations.clear();
state->listeners.clear();
state->closeDeferred.reset();
state->closePromise.Reset();
state->onStatus.Reset();
state->owner.Reset();
napi_delete_async_work(environment, state->closeWork);
napi_remove_async_cleanup_hook(state->cleanupHook);
state->closeOwner = nullptr;
}
Napi::Env env;
dht::DhtRunner runner;
Napi::ThreadSafeFunction dispatcher;
napi_async_work closeWork {};
napi_async_cleanup_hook_handle cleanupHook {};
std::shared_ptr<RunnerState>* closeOwner {};
std::atomic_bool closing {false};
bool envClosing = false;
bool started = false;
bool hasIdentity = false;
bool finished = false;
uint64_t nextOperation = 1;
std::string closeError;
std::unordered_map<uint64_t, PendingOperation> operations;
std::unordered_map<uint64_t, Subscription> listeners;
std::optional<Napi::Promise::Deferred> closeDeferred;
Napi::Reference<Napi::Promise> closePromise;
Napi::FunctionReference onStatus;
Napi::ObjectReference owner;
dht::NodeStatus status4 = dht::NodeStatus::Disconnected;
dht::NodeStatus status6 = dht::NodeStatus::Disconnected;
};
class Runner : public Napi::ObjectWrap<Runner>
{
public:
explicit Runner(const Napi::CallbackInfo& info)
: Napi::ObjectWrap<Runner>(info)
, state_(RunnerState::create(info.Env()))
{
if (info.Length() && info[0].IsFunction())
state_->onStatus = Napi::Persistent(info[0].As<Napi::Function>());
}
~Runner() override { state_->startClose(); }
static Napi::Function define(Napi::Env env)
{
return DefineClass(env,
"NativeRunner",
{
InstanceMethod("run", &Runner::run),
InstanceMethod("close", &Runner::close),
InstanceMethod("isRunning", &Runner::isRunning),
InstanceMethod("getBoundPort", &Runner::getBoundPort),
InstanceMethod("getNodeId", &Runner::getNodeId),
InstanceMethod("getStatus", &Runner::getStatus),
InstanceMethod("bootstrap", &Runner::bootstrap),
InstanceMethod("get", &Runner::get),
InstanceMethod("getAll", &Runner::getAll),
InstanceMethod("cancelGet", &Runner::cancelGet),
InstanceMethod("put", &Runner::put),
InstanceMethod("putSigned", &Runner::putSigned),
InstanceMethod("putEncrypted", &Runner::putEncrypted),
InstanceMethod("listen", &Runner::listen),
InstanceMethod("cancelListen", &Runner::cancelListen),
InstanceMethod("ref", &Runner::ref),
InstanceMethod("unref", &Runner::unref),
InstanceMethod("setOwner", &Runner::setOwner),
});
}
private:
Napi::Value run(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
if (state_->closing)
throw Napi::Error::New(info.Env(), "DHT runner is closed");
if (state_->started)
throw Napi::Error::New(info.Env(), "DHT runner is already running");
auto options = info.Length() && !info[0].IsUndefined() ? optionsObject(info[0])
: Napi::Object::New(info.Env());
const auto port = options.Has("port") ? unsignedNumber(options.Get("port"), 65535, "port") : 0;
const auto ipv4 = options.Has("ipv4") ? text(options.Get("ipv4"), "ipv4") : "0.0.0.0";
const auto ipv6 = options.Has("ipv6") ? text(options.Get("ipv6"), "ipv6") : "::";
dht::DhtRunner::Config config;
if (options.Has("network"))
config.dht_config.node_config.network = unsignedNumber(options.Get("network"), UINT32_MAX, "network");
if (options.Has("persistPath"))
config.dht_config.node_config.persist_path = text(options.Get("persistPath"), "persistPath");
if (options.Has("identity"))
config.dht_config.id = opendht_node::readIdentity(options.Get("identity"));
if (options.Has("proxyServer"))
config.proxy_server = text(options.Get("proxyServer"), "proxyServer");
if (options.Has("proxyUserAgent")) {
config.proxy_user_agent = text(options.Get("proxyUserAgent"), "proxyUserAgent");
if (config.proxy_user_agent.find_first_of("\r\n") != std::string::npos)
throw Napi::TypeError::New(info.Env(), "proxyUserAgent must not contain line breaks");
}
dht::DhtRunner::Context context;
const std::weak_ptr<RunnerState> state = state_;
context.statusChangedCallback = [state](dht::NodeStatus ipv4, dht::NodeStatus ipv6) {
if (auto owner = state.lock())
owner->send([ipv4, ipv6](RunnerState& state) {
state.status4 = ipv4;
state.status6 = ipv6;
if (!state.onStatus.IsEmpty())
state.onStatus.Call({state.status()});
});
};
const auto service = std::to_string(port);
state_->runner.run(ipv4.c_str(), ipv6.c_str(), service.c_str(), config, std::move(context));
state_->started = true;
state_->hasIdentity = bool(config.dht_config.id.first);
state_->dispatcher.Ref(info.Env());
return info.Env().Undefined();
});
}
Napi::Value close(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
if (!state_->closePromise.IsEmpty())
return state_->closePromise.Value();
auto deferred = Napi::Promise::Deferred::New(info.Env());
if (state_->finished) {
if (!state_->closeError.empty())
deferred.Reject(state_->makeCloseError().Value());
else
deferred.Resolve(info.Env().Undefined());
} else {
state_->closeDeferred = deferred;
state_->closePromise = Napi::Persistent(deferred.Promise());
state_->startClose();
}
return deferred.Promise();
});
}
Napi::Value isRunning(const Napi::CallbackInfo& info)
{
return Napi::Boolean::New(info.Env(), !state_->closing && state_->runner.isRunning());
}
Napi::Value getStatus(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value { return state_->status(); });
}
Napi::Value getBoundPort(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
state_->requireRunning();
const auto family = info.Length() ? unsignedNumber(info[0], 6, "family") : 4;
if (family != 4 && family != 6)
throw Napi::RangeError::New(info.Env(), "family must be 4 or 6");
return Napi::Number::New(info.Env(), state_->runner.getBoundPort(family == 4 ? AF_INET : AF_INET6));
});
}
Napi::Value getNodeId(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
state_->requireRunning();
return Napi::String::New(info.Env(), state_->runner.getNodeId().to_view());
});
}
Napi::Value bootstrap(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
state_->requireRunning();
const auto host = text(info[0], "host");
const auto port = unsignedNumber(info[1], 65535, "port");
if (host.empty() || !port)
throw Napi::RangeError::New(info.Env(), "bootstrap requires a host and nonzero port");
state_->runner.bootstrap(host, std::to_string(port));
return info.Env().Undefined();
});
}
Napi::Value get(const Napi::CallbackInfo& info) { return getValues(info, true); }
Napi::Value getAll(const Napi::CallbackInfo& info) { return getValues(info, false); }
Napi::Value getValues(const Napi::CallbackInfo& info, bool stream)
{
return guarded(info, [&]() -> Napi::Value {
state_->requireRunning();
const auto key = infoHash(info[0]);
if (stream && !info[1].IsFunction())
throw Napi::TypeError::New(info.Env(), "callback must be a function");
const auto operation = state_->nextOperation++;
auto deferred = Napi::Promise::Deferred::New(info.Env());
auto cancelled = std::make_shared<std::atomic_bool>(false);
PendingOperation pending {deferred, true, {}, {}, cancelled};
if (stream)
pending.onValue = Napi::Persistent(info[1].As<Napi::Function>());
state_->operations.emplace(operation, std::move(pending));
const std::weak_ptr<RunnerState> state = state_;
try {
state_->runner.get(key,
dht::GetCallback([state, operation, cancelled](
const std::vector<std::shared_ptr<dht::Value>>& values) {
const auto owner = state.lock();
if (!owner || owner->closing || *cancelled)
return false;
for (const auto& value : values)
owner->sendValue(operation, value);
return !owner->closing && !*cancelled;
}),
dht::DoneCallbackSimple([state, operation](bool ok) {
if (const auto owner = state.lock())
owner->send([operation, ok](RunnerState& state) {
state.completeOperation(operation, ok);
});
}));
} catch (...) {
state_->operations.erase(operation);
throw;
}
if (stream) {
auto result = Napi::Object::New(info.Env());
result.Set("token", Napi::BigInt::New(info.Env(), operation));
result.Set("done", deferred.Promise());
return result;
}
return deferred.Promise();
});
}
Napi::Value cancelGet(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
const auto operation = state_->operations.find(unsignedBigInt(info[0]));
if (operation != state_->operations.end() && !operation->second.onValue.IsEmpty()) {
*operation->second.cancelled = true;
operation->second.deferred.Resolve(info.Env().Undefined());
state_->operations.erase(operation);
}
return info.Env().Undefined();
});
}
Napi::Value put(const Napi::CallbackInfo& info) { return putValue(info, PutMode::Plain); }
Napi::Value putSigned(const Napi::CallbackInfo& info) { return putValue(info, PutMode::Signed); }
Napi::Value putEncrypted(const Napi::CallbackInfo& info) { return putValue(info, PutMode::Encrypted); }
enum class PutMode { Plain, Signed, Encrypted };
Napi::Value putValue(const Napi::CallbackInfo& info, PutMode mode)
{
return guarded(info, [&]() -> Napi::Value {
state_->requireRunning();
if (mode != PutMode::Plain && !state_->hasIdentity)
throw Napi::Error::New(info.Env(), "Signed and encrypted puts require an identity");
std::shared_ptr<dht::crypto::PublicKey> recipient;
if (mode == PutMode::Encrypted) {
recipient = opendht_node::readEncryptionKey(info[3]);
}
const auto key = infoHash(info[0]);
if (!info[1].IsTypedArray() || info[1].As<Napi::TypedArray>().TypedArrayType() != napi_uint8_array)
throw Napi::TypeError::New(info.Env(), "data must be a Uint8Array");
const auto data = info[1].As<Napi::Uint8Array>();
dht::Blob bytes(data.ElementLength());
if (!bytes.empty())
std::copy_n(data.Data(), bytes.size(), bytes.data());
auto value = std::make_shared<dht::Value>(std::move(bytes));
auto options = info.Length() > 2 && !info[2].IsUndefined() ? optionsObject(info[2])
: Napi::Object::New(info.Env());
if (options.Has("id"))
value->id = unsignedBigInt(options.Get("id"));
if (options.Has("type"))
value->type = unsignedNumber(options.Get("type"), 65535, "type");
if (options.Has("userType")) {
if (!options.Get("userType").IsString())
throw Napi::TypeError::New(info.Env(), "userType must be a string");
value->user_type = options.Get("userType").As<Napi::String>().Utf8Value();
}
bool permanent = false;
if (options.Has("permanent")) {
if (!options.Get("permanent").IsBoolean())
throw Napi::TypeError::New(info.Env(), "permanent must be a boolean");
permanent = options.Get("permanent").As<Napi::Boolean>().Value();
}
const auto operation = state_->nextOperation++;
auto deferred = Napi::Promise::Deferred::New(info.Env());
state_->operations.emplace(operation, PendingOperation {deferred, false, {}, {}, {}});
const std::weak_ptr<RunnerState> state = state_;
try {
dht::DoneCallbackSimple done = [state, operation](bool ok) {
if (const auto owner = state.lock())
owner->send([operation, ok](RunnerState& state) { state.completeOperation(operation, ok); });
};
if (mode == PutMode::Signed)
state_->runner.putSigned(key, value, std::move(done), permanent);
else if (mode == PutMode::Encrypted)
state_->runner.putEncrypted(key, recipient, value, std::move(done), permanent);
else
state_->runner.put(key, value, std::move(done), dht::time_point::max(), permanent);
} catch (...) {
state_->operations.erase(operation);
throw;
}
return deferred.Promise();
});
}
Napi::Value listen(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
state_->requireRunning();
const auto key = infoHash(info[0]);
if (!info[1].IsFunction())
throw Napi::TypeError::New(info.Env(), "callback must be a function");
const auto operation = state_->nextOperation++;
const std::weak_ptr<RunnerState> state = state_;
auto token = state_->runner
.listen(key,
dht::ValueCallback(
[state, operation](const std::vector<std::shared_ptr<dht::Value>>& values,
bool expired) {
const auto owner = state.lock();
if (!owner || owner->closing)
return false;
for (const auto& value : values)
owner->sendValue(operation, value, expired);
return !owner->closing;
}))
.share();
state_->listeners.emplace(operation,
Subscription {key,
std::move(token),
Napi::Persistent(info[1].As<Napi::Function>())});
return Napi::BigInt::New(info.Env(), operation);
});
}
Napi::Value cancelListen(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
if (state_->closing)
return info.Env().Undefined();
const auto listener = state_->listeners.find(unsignedBigInt(info[0]));
if (listener != state_->listeners.end()) {
state_->runner.cancelListen(listener->second.key, listener->second.token);
state_->listeners.erase(listener);
}
return info.Env().Undefined();
});
}
Napi::Value ref(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
state_->requireRunning();
state_->dispatcher.Ref(info.Env());
return info.This();
});
}
Napi::Value unref(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
state_->requireRunning();
state_->dispatcher.Unref(info.Env());
return info.This();
});
}
Napi::Value setOwner(const Napi::CallbackInfo& info)
{
if (!state_->closing) {
state_->owner.Reset();
if (info[0].IsObject())
state_->owner = Napi::Persistent(info[0].As<Napi::Object>());
}
return info.Env().Undefined();
}
std::shared_ptr<RunnerState> state_;
};
Napi::Value
hash(const Napi::CallbackInfo& info)
{
const auto env = info.Env();
dht::InfoHash result;
if (info.Length() == 1 && info[0].IsString()) {
result = dht::InfoHash::get(info[0].As<Napi::String>().Utf8Value());
} else if (info.Length() == 1 && info[0].IsTypedArray()
&& info[0].As<Napi::TypedArray>().TypedArrayType() == napi_uint8_array) {
const auto data = info[0].As<Napi::Uint8Array>();
result = dht::InfoHash::get(data.Data(), data.ElementLength());
} else {
throw Napi::TypeError::New(env, "Expected a string or Uint8Array");
}
return Napi::String::New(env, result.to_view());
}
Napi::Object
initialize(Napi::Env env, Napi::Object exports)
{
static const GnuTlsContext crypto;
if (crypto.result != GNUTLS_E_SUCCESS)
throw Napi::Error::New(env, std::string("Unable to initialize GnuTLS: ") + gnutls_strerror(crypto.result));
opendht_node::initializeCrypto(env, exports);
exports.Set("NativeRunner", Runner::define(env));
exports.Set("hash", Napi::Function::New(env, hash));
exports.Set("version", Napi::String::New(env, dht::version()));
return exports;
}
} // namespace
NODE_API_MODULE(opendht, initialize)
+304
View File
@@ -0,0 +1,304 @@
#include "crypto.h"
#include <algorithm>
#include <cmath>
#include <variant>
namespace opendht_node {
namespace {
using PublicKey = std::shared_ptr<dht::crypto::PublicKey>;
using PrivateKey = std::shared_ptr<dht::crypto::PrivateKey>;
using Certificate = std::shared_ptr<dht::crypto::Certificate>;
using Identity = dht::crypto::Identity;
using Handle = std::variant<PublicKey, PrivateKey, Certificate, Identity>;
constexpr napi_type_tag cryptoTag {0x37cdf3e4648e4cb5ULL, 0x964fb031062c245aULL};
Handle&
handle(Napi::Value value)
{
bool matches = false;
if (!value.IsExternal() || napi_check_object_type_tag(value.Env(), value, &cryptoTag, &matches) != napi_ok
|| !matches)
throw Napi::TypeError::New(value.Env(), "Invalid OpenDHT crypto handle");
return *value.As<Napi::External<Handle>>().Data();
}
template<typename Type>
Type
read(Napi::Value value)
{
if (auto result = std::get_if<Type>(&handle(value)))
return *result;
throw Napi::TypeError::New(value.Env(), "Wrong OpenDHT crypto handle type");
}
Napi::Value
wrap(Napi::Env env, Handle value)
{
auto owner = std::make_unique<Handle>(std::move(value));
auto result = Napi::External<Handle>::New(env, owner.get(), [](Napi::Env, Handle* data) { delete data; });
owner.release();
if (napi_type_tag_object(env, result, &cryptoTag) != napi_ok)
throw Napi::Error::New(env, "Could not tag OpenDHT crypto handle");
return result;
}
dht::Blob
bytes(Napi::Value value)
{
if (!value.IsTypedArray() || value.As<Napi::TypedArray>().TypedArrayType() != napi_uint8_array)
throw Napi::TypeError::New(value.Env(), "Expected a Uint8Array");
auto array = value.As<Napi::Uint8Array>();
dht::Blob result(array.ElementLength());
if (!result.empty())
std::copy_n(array.Data(), result.size(), result.data());
return result;
}
std::string
string(Napi::Value value)
{
if (!value.IsString())
throw Napi::TypeError::New(value.Env(), "Expected a string");
auto result = value.As<Napi::String>().Utf8Value();
if (result.find('\0') != std::string::npos)
throw Napi::TypeError::New(value.Env(), "Crypto strings must not contain NUL bytes");
return result;
}
Napi::Buffer<uint8_t>
buffer(Napi::Env env, const dht::Blob& value)
{
return Napi::Buffer<uint8_t>::Copy(env, value.data(), value.size());
}
PublicKey
publicKey(const Handle& value)
{
if (auto key = std::get_if<PublicKey>(&value))
return *key;
if (auto key = std::get_if<PrivateKey>(&value))
return (*key)->getSharedPublicKey();
if (auto certificate = std::get_if<Certificate>(&value))
return (*certificate)->getSharedPublicKey();
return std::get<Identity>(value).second->getSharedPublicKey();
}
class GenerateWorker : public Napi::AsyncWorker
{
public:
GenerateWorker(Napi::Env env, bool identity, std::string name, bool rsa, unsigned bits)
: Napi::AsyncWorker(env, "OpenDHT crypto generation")
, deferred_(Napi::Promise::Deferred::New(env))
, identity_(identity)
, name_(std::move(name))
, rsa_(rsa)
, bits_(bits)
{}
Napi::Promise promise() const { return deferred_.Promise(); }
void Execute() override
{
try {
if (identity_)
result_ = rsa_ ? dht::crypto::generateIdentity(name_, {}, bits_)
: dht::crypto::generateEcIdentity(name_);
else
result_ = std::make_shared<dht::crypto::PrivateKey>(rsa_ ? dht::crypto::PrivateKey::generate(bits_)
: dht::crypto::PrivateKey::generateEC());
} catch (const std::exception& error) {
SetError(error.what());
}
}
void OnOK() override { deferred_.Resolve(wrap(Env(), std::move(result_))); }
void OnError(const Napi::Error& error) override { deferred_.Reject(error.Value()); }
private:
Napi::Promise::Deferred deferred_;
bool identity_;
std::string name_;
bool rsa_;
unsigned bits_;
Handle result_;
};
Napi::Value
generate(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
const auto kind = string(info[0]);
const auto name = string(info[1]);
const auto algorithm = string(info[2]);
if (algorithm != "rsa" && algorithm != "ec")
throw Napi::TypeError::New(info.Env(), "algorithm must be rsa or ec");
if (!info[3].IsNumber())
throw Napi::TypeError::New(info.Env(), "bits must be a number");
const auto bits = info[3].As<Napi::Number>().DoubleValue();
if (!std::isfinite(bits) || std::floor(bits) != bits || bits < 2048 || bits > 8192)
throw Napi::RangeError::New(info.Env(), "RSA bits must be an integer between 2048 and 8192");
if (kind != "private" && kind != "identity")
throw Napi::TypeError::New(info.Env(), "Invalid crypto generation type");
auto worker = std::make_unique<GenerateWorker>(info.Env(), kind == "identity", name, algorithm == "rsa", bits);
auto promise = worker->promise();
worker->Queue();
worker.release();
return promise;
});
}
Napi::Value
import(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
const auto kind = string(info[0]);
auto data = bytes(info[1]);
if (data.empty())
throw Napi::TypeError::New(info.Env(), "Crypto imports must not be empty");
if (kind == "private")
return wrap(info.Env(), std::make_shared<dht::crypto::PrivateKey>(data, string(info[2])));
if (kind == "public")
return wrap(info.Env(), std::make_shared<dht::crypto::PublicKey>(data));
if (kind == "certificate")
return wrap(info.Env(), std::make_shared<dht::crypto::Certificate>(data));
throw Napi::TypeError::New(info.Env(), "Invalid crypto import type");
});
}
Napi::Value
serialize(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
auto& value = handle(info[0]);
if (auto key = std::get_if<PrivateKey>(&value))
return buffer(info.Env(), (*key)->serialize(string(info[1])));
if (auto certificate = std::get_if<Certificate>(&value))
return buffer(info.Env(), (*certificate)->getPacked());
dht::Blob result;
readPublicKey(info[0])->pack(result);
return buffer(info.Env(), result);
});
}
Napi::Value
getPublicKey(const Napi::CallbackInfo& info)
{
return guarded(info, [&] { return wrap(info.Env(), publicKey(handle(info[0]))); });
}
Napi::Value
getId(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
const auto key = publicKey(handle(info[0]));
if (!info[1].IsBoolean())
throw Napi::TypeError::New(info.Env(), "Expected a boolean");
return Napi::String::New(info.Env(),
info[1].As<Napi::Boolean>().Value() ? key->getLongId().toString()
: key->getId().toString());
});
}
Napi::Value
sign(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
return buffer(info.Env(), read<PrivateKey>(info[0])->sign(bytes(info[1])));
});
}
Napi::Value
verify(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
return Napi::Boolean::New(info.Env(), readPublicKey(info[0])->checkSignature(bytes(info[1]), bytes(info[2])));
});
}
Napi::Value
encrypt(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
const auto key = readEncryptionKey(info[0]);
return buffer(info.Env(), key->encrypt(bytes(info[1])));
});
}
Napi::Value
decrypt(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
return buffer(info.Env(), read<PrivateKey>(info[0])->decrypt(bytes(info[1])));
});
}
Napi::Value
identity(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
auto key = read<PrivateKey>(info[0]);
auto certificate = read<Certificate>(info[1]);
if (key->getPublicKey().getLongId() != certificate->getLongId())
throw Napi::TypeError::New(info.Env(), "Identity key and certificate do not match");
return wrap(info.Env(), Identity {std::move(key), std::move(certificate)});
});
}
Napi::Value
identityPart(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
const auto value = readIdentity(info[0]);
if (!info[1].IsBoolean())
throw Napi::TypeError::New(info.Env(), "Expected a boolean");
return info[1].As<Napi::Boolean>().Value() ? wrap(info.Env(), value.first) : wrap(info.Env(), value.second);
});
}
} // namespace
dht::crypto::Identity
readIdentity(Napi::Value value)
{
return read<Identity>(value);
}
std::shared_ptr<dht::crypto::PublicKey>
readPublicKey(Napi::Value value)
{
return read<PublicKey>(value);
}
std::shared_ptr<dht::crypto::PublicKey>
readEncryptionKey(Napi::Value value)
{
const auto key = readPublicKey(value);
const auto algorithm = gnutls_pubkey_get_pk_algorithm(key->pk, nullptr);
if (algorithm != GNUTLS_PK_RSA
#if GNUTLS_VERSION_NUMBER >= 0x030804
&& algorithm != GNUTLS_PK_RSA_OAEP
#endif
)
throw Napi::TypeError::New(value.Env(), "Encryption requires an RSA public key");
return key;
}
void
initializeCrypto(Napi::Env env, Napi::Object exports)
{
exports.Set("cryptoGenerate", Napi::Function::New(env, generate));
exports.Set("cryptoImport", Napi::Function::New(env, import));
exports.Set("cryptoSerialize", Napi::Function::New(env, serialize));
exports.Set("cryptoPublicKey", Napi::Function::New(env, getPublicKey));
exports.Set("cryptoId", Napi::Function::New(env, getId));
exports.Set("cryptoSign", Napi::Function::New(env, sign));
exports.Set("cryptoVerify", Napi::Function::New(env, verify));
exports.Set("cryptoEncrypt", Napi::Function::New(env, encrypt));
exports.Set("cryptoDecrypt", Napi::Function::New(env, decrypt));
exports.Set("cryptoIdentity", Napi::Function::New(env, identity));
exports.Set("cryptoIdentityPart", Napi::Function::New(env, identityPart));
}
} // namespace opendht_node
+26
View File
@@ -0,0 +1,26 @@
#pragma once
#include <napi.h>
#include <opendht/crypto.h>
namespace opendht_node {
template<typename Callback>
Napi::Value
guarded(const Napi::CallbackInfo& info, Callback&& callback)
{
try {
return callback();
} catch (const Napi::Error&) {
throw;
} catch (const std::exception& error) {
throw Napi::Error::New(info.Env(), error.what());
}
}
dht::crypto::Identity readIdentity(Napi::Value value);
std::shared_ptr<dht::crypto::PublicKey> readPublicKey(Napi::Value value);
std::shared_ptr<dht::crypto::PublicKey> readEncryptionKey(Napi::Value value);
void initializeCrypto(Napi::Env env, Napi::Object exports);
} // namespace opendht_node
+56
View File
@@ -0,0 +1,56 @@
export default function createCrypto(native) {
const handles = new WeakMap();
function handle(value, Type) {
if (!(value instanceof Type) || !handles.has(value)) throw new TypeError(`Expected ${Type.name}`);
return handles.get(value);
}
function wrap(Type, nativeHandle) {
const result = Object.create(Type.prototype);
handles.set(result, nativeHandle);
return result;
}
function generate(kind, name, options = {}) {
if (!options || typeof options !== 'object' || Array.isArray(options)) throw new TypeError('Invalid crypto options');
return native.cryptoGenerate(kind, name, options.algorithm ?? 'ec', options.bits ?? 2048);
}
class PublicKey {
constructor(data) { handles.set(this, native.cryptoImport('public', data)); }
serialize() { return native.cryptoSerialize(handle(this, PublicKey)); }
getId() { return native.cryptoId(handle(this, PublicKey), false); }
getLongId() { return native.cryptoId(handle(this, PublicKey), true); }
checkSignature(data, signature) { return native.cryptoVerify(handle(this, PublicKey), data, signature); }
encrypt(data) { return native.cryptoEncrypt(handle(this, PublicKey), data); }
}
class PrivateKey {
constructor(data, password = '') { handles.set(this, native.cryptoImport('private', data, password)); }
static generate(options) { return generate('private', '', options).then(value => wrap(PrivateKey, value)); }
serialize(password = '') { return native.cryptoSerialize(handle(this, PrivateKey), password); }
getPublicKey() { return wrap(PublicKey, native.cryptoPublicKey(handle(this, PrivateKey))); }
sign(data) { return native.cryptoSign(handle(this, PrivateKey), data); }
decrypt(data) { return native.cryptoDecrypt(handle(this, PrivateKey), data); }
}
class Certificate {
constructor(data) { handles.set(this, native.cryptoImport('certificate', data)); }
serialize() { return native.cryptoSerialize(handle(this, Certificate)); }
getPublicKey() { return wrap(PublicKey, native.cryptoPublicKey(handle(this, Certificate))); }
getId() { return native.cryptoId(handle(this, Certificate), false); }
getLongId() { return native.cryptoId(handle(this, Certificate), true); }
}
class Identity {
constructor(privateKey, certificate) {
handles.set(this, native.cryptoIdentity(handle(privateKey, PrivateKey), handle(certificate, Certificate)));
}
static generate(name = 'dhtnode', options) { return generate('identity', name, options).then(value => wrap(Identity, value)); }
get privateKey() { return wrap(PrivateKey, native.cryptoIdentityPart(handle(this, Identity), true)); }
get certificate() { return wrap(Certificate, native.cryptoIdentityPart(handle(this, Identity), false)); }
}
return { PublicKey, PrivateKey, Certificate, Identity, handle };
}
+171
View File
@@ -0,0 +1,171 @@
#!/usr/bin/env node
import { clearLine, createInterface, cursorTo, type Interface } from 'node:readline';
import { parseArgs } from 'node:util';
import type { ConnectivityStatus, Listener, Value } from '../index.js';
const commands = 'get KEY | put KEY TEXT | sign KEY TEXT | listen KEY | cancel KEY | help | quit';
function portNumber(value: string, name: string, minimum = 0): number {
const port = Number(value);
if (!Number.isInteger(port) || port < minimum || port > 65535) {
throw new Error(`${name} must be an integer between ${minimum} and 65535`);
}
return port;
}
function errorMessage(error: unknown): string {
return error instanceof Error ? error.message : String(error);
}
function report(error: unknown): void {
console.error(errorMessage(error));
process.exitCode = 1;
}
async function main(): Promise<void> {
const { values } = parseArgs({
options: {
help: { type: 'boolean', short: 'h' },
port: { type: 'string', short: 'p', default: '4222' },
ipv4: { type: 'string', default: '0.0.0.0' },
ipv6: { type: 'string', default: '::' },
bootstrap: { type: 'string', short: 'b' },
'bootstrap-port': { type: 'string', default: '4222' },
},
});
if (values.help) {
console.log(`Usage: npm run dhtnode -- [options]
-p, --port PORT Local UDP port (default: 4222; 0 selects a free port)
-b, --bootstrap HOST Optional bootstrap hostname or IP address
--bootstrap-port PORT Bootstrap UDP port (default: 4222)
--ipv4 ADDRESS IPv4 bind address (default: 0.0.0.0)
--ipv6 ADDRESS IPv6 bind address (default: ::)
-h, --help Show help
Commands: ${commands}
Keys are text labels hashed with SHA-1. Values are UTF-8 text.`);
return;
}
const port = portNumber(values.port, 'port');
const bootstrapPort = portNumber(values['bootstrap-port'], 'bootstrap-port', 1);
const { DhtRunner, Identity, hash } = await import('../index.js');
const identity = await Identity.generate('dhtnode.ts');
const runner = new DhtRunner();
const listeners = new Map<string, Listener>();
const pending = new Set<Promise<void>>();
let lines: Interface | undefined;
let closing: Promise<void> | undefined;
function close(): Promise<void> {
if (!closing) {
for (const listener of listeners.values()) listener.cancel();
listeners.clear();
lines?.close();
closing = runner.close();
}
return closing;
}
function print(message: string): void {
if (process.stdout.isTTY && lines) {
clearLine(process.stdout, 0);
cursorTo(process.stdout, 0);
}
console.log(message);
if (!closing && process.stdin.isTTY) lines?.prompt(true);
}
const stop = (): void => { close().catch(report); };
const printValue = (label: string, value: Value): void => {
print(`${label} ${value.id}: ${JSON.stringify(value.data.toString('utf8'))}${value.isSigned ? ' [signed]' : ''}`);
};
const printStatus = (status: ConnectivityStatus): void => {
print(`Connectivity: IPv4 ${status.ipv4}, IPv6 ${status.ipv6}`);
};
async function execute(line: string): Promise<void> {
const [, command, key, data] = line.trim().match(/^(\S+)(?:\s+(\S+))?(?:\s+(.+))?$/) || [];
if (!command) return;
if (command === 'help') {
print(commands);
return;
}
if (!key) throw new Error('Expected a key. Type help for commands.');
const location = hash(key);
switch (command) {
case 'get': {
const found = await runner.getAll(location);
if (closing) return;
for (const value of found) printValue(key, value);
if (!found.length) print(`No values for ${key}`);
break;
}
case 'put':
case 'sign': {
if (data === undefined) throw new Error(`Usage: ${command} KEY TEXT`);
const payload = Buffer.from(data, 'utf8');
const ok = command === 'sign' ? await runner.putSigned(location, payload)
: await runner.put(location, payload);
if (closing) return;
if (!ok) throw new Error(`Put failed for ${key}`);
print(`Stored ${key}`);
break;
}
case 'listen':
if (!listeners.has(key)) {
listeners.set(key, runner.listen(location, (value, expired) => {
printValue(`${expired ? 'Expired' : 'Value'} ${key}`, value);
}));
}
print(`Listening ${key}`);
break;
case 'cancel':
listeners.get(key)?.cancel();
listeners.delete(key);
print(`Cancelled ${key}`);
break;
default:
throw new Error('Unknown command. Type help for commands.');
}
}
runner.on('error', error => { report(error); stop(); });
runner.on('statusChanged', printStatus);
process.on('SIGINT', stop);
process.on('SIGTERM', stop);
try {
runner.run({ port, ipv4: values.ipv4, ipv6: values.ipv6, identity });
if (values.bootstrap) runner.bootstrap(values.bootstrap, bootstrapPort);
lines = createInterface({ input: process.stdin, output: process.stdout });
lines.on('SIGINT', stop);
lines.setPrompt('dht> ');
print(`Node ${runner.getNodeId()} listening on UDP ${runner.getBoundPort()}`);
printStatus(runner.getStatus());
if (process.stdin.isTTY) print(commands);
for await (const line of lines) {
if (closing) break;
if (/^(quit|exit)(?:\s|$)/.test(line.trim())) {
stop();
break;
}
const operation = execute(line).catch(error => {
if (!closing) console.error(errorMessage(error));
}).finally(() => { pending.delete(operation); });
pending.add(operation);
if (process.stdin.isTTY) lines.prompt();
}
if (!closing) await Promise.all(pending);
} finally {
try {
await close();
await Promise.all(pending);
} finally {
process.removeListener('SIGINT', stop);
process.removeListener('SIGTERM', stop);
}
}
}
main().catch(report);
+120
View File
@@ -0,0 +1,120 @@
import { Buffer } from 'node:buffer';
import { EventEmitter } from 'node:events';
export const version: string;
export function hash(data: string | Uint8Array): string;
export interface KeyOptions {
algorithm?: 'ec' | 'rsa';
bits?: number;
}
export class PublicKey {
constructor(data: Uint8Array);
serialize(): Buffer;
getId(): string;
getLongId(): string;
checkSignature(data: Uint8Array, signature: Uint8Array): boolean;
encrypt(data: Uint8Array): Buffer;
}
export class PrivateKey {
constructor(data: Uint8Array, password?: string);
static generate(options?: KeyOptions): Promise<PrivateKey>;
serialize(password?: string): Buffer;
getPublicKey(): PublicKey;
sign(data: Uint8Array): Buffer;
decrypt(data: Uint8Array): Buffer;
}
export class Certificate {
constructor(data: Uint8Array);
serialize(): Buffer;
getPublicKey(): PublicKey;
getId(): string;
getLongId(): string;
}
export class Identity {
constructor(privateKey: PrivateKey, certificate: Certificate);
static generate(name?: string, options?: KeyOptions): Promise<Identity>;
readonly privateKey: PrivateKey;
readonly certificate: Certificate;
}
export interface RunOptions {
port?: number;
ipv4?: string;
ipv6?: string;
network?: number;
persistPath?: string;
identity?: Identity;
proxyServer?: string;
proxyUserAgent?: string;
}
export interface PutOptions {
id?: bigint;
type?: number;
userType?: string;
permanent?: boolean;
}
export interface Value {
id: bigint;
type: number;
seq: number;
data: Buffer;
userType: string;
owner: string | null;
isSigned: boolean;
isEncrypted: boolean;
}
export interface Listener {
cancel(): void;
}
export interface ListenEvent {
value: Value;
expired: boolean;
}
export interface ValueIterator<T> extends AsyncIterableIterator<T, undefined>, Listener, AsyncDisposable {
return(): Promise<IteratorResult<T, undefined>>;
throw(error?: unknown): Promise<IteratorResult<T, undefined>>;
}
export interface ListenerIterator extends ValueIterator<ListenEvent> { }
export type NodeStatus = 'disconnected' | 'connecting' | 'connected';
export interface ConnectivityStatus {
ipv4: NodeStatus;
ipv6: NodeStatus;
}
export interface RunnerEvents {
statusChanged: [status: ConnectivityStatus];
error: [error: Error];
}
export class DhtRunner extends EventEmitter<RunnerEvents> implements AsyncDisposable {
run(options?: RunOptions): void;
isRunning(): boolean;
getBoundPort(family?: 4 | 6): number;
getNodeId(): string;
getStatus(): ConnectivityStatus;
bootstrap(host: string, port: number): void;
get(key: string): ValueIterator<Value>;
getAll(key: string): Promise<Value[]>;
put(key: string, data: Uint8Array, options?: PutOptions): Promise<boolean>;
putSigned(key: string, data: Uint8Array, options?: PutOptions): Promise<boolean>;
putEncrypted(key: string, data: Uint8Array, recipient: PublicKey, options?: PutOptions): Promise<boolean>;
listen(key: string): ListenerIterator;
listen(key: string, callback: (value: Value, expired: boolean) => void): Listener;
close(): Promise<void>;
ref(): this;
unref(): this;
[Symbol.asyncDispose](): Promise<void>;
}
+187
View File
@@ -0,0 +1,187 @@
import { EventEmitter } from 'node:events';
import { constants } from 'node:os';
import { dlopen } from 'node:process';
import load from 'node-gyp-build';
import createCrypto from './crypto.js';
const binding = { exports: {} };
const flags = process.platform === 'linux'
? constants.dlopen.RTLD_LAZY | constants.dlopen.RTLD_DEEPBIND
: undefined;
dlopen(binding, process.env.OPENDHT_NODE_BINDING || load.path(import.meta.dirname),
flags);
const native = binding.exports;
const { PublicKey, PrivateKey, Certificate, Identity, handle } = createCrypto(native);
function eventReporter(owner, event) {
return value => { owner.deref()?.emit(event, value); };
}
class DhtRunner extends EventEmitter {
#native;
#closing;
#listeners = 0;
#pending = 0;
#iterators = new Set();
constructor() {
super();
const owner = new WeakRef(this);
this.#native = new native.NativeRunner(eventReporter(owner, 'statusChanged'));
}
#updateOwner() {
this.#native.setOwner(this.#listeners || this.#pending || this.listenerCount('statusChanged') ? this : null);
}
#track(operation) {
this.#pending++;
this.#updateOwner();
return operation.finally(() => {
this.#pending--;
this.#updateOwner();
});
}
on(event, listener) { super.on(event, listener); this.#updateOwner(); return this; }
addListener(event, listener) { return this.on(event, listener); }
prependListener(event, listener) { super.prependListener(event, listener); this.#updateOwner(); return this; }
removeListener(event, listener) { super.removeListener(event, listener); this.#updateOwner(); return this; }
off(event, listener) { return this.removeListener(event, listener); }
removeAllListeners(...args) { super.removeAllListeners(...args); this.#updateOwner(); return this; }
run(options) {
if (typeof options?.proxyServer === 'string' && options.proxyServer) {
const proxy = new URL(options.proxyServer);
if (proxy.protocol !== 'http:' && proxy.protocol !== 'https:')
throw new TypeError('proxyServer must be an HTTP(S) URL');
}
if (options?.identity !== undefined) options = { ...options, identity: handle(options.identity, Identity) };
this.#native.run(options);
}
isRunning() { return this.#native.isRunning(); }
getBoundPort(family = 4) { return this.#native.getBoundPort(family); }
getNodeId() { return this.#native.getNodeId(); }
getStatus() { return this.#native.getStatus(); }
bootstrap(host, port) { this.#native.bootstrap(host, port); }
get(key) {
return this.#iterate((push, finish) => {
const operation = this.#native.get(key, push);
this.#track(operation.done).then(() => finish(), finish);
return () => this.#native.cancelGet(operation.token);
}, true);
}
getAll(key) { return this.#track(this.#native.getAll(key)); }
put(key, data, options) { return this.#track(this.#native.put(key, data, options)); }
putSigned(key, data, options) { return this.#track(this.#native.putSigned(key, data, options)); }
putEncrypted(key, data, recipient, options) {
return this.#track(this.#native.putEncrypted(key, data, options, handle(recipient, PublicKey)));
}
listen(key, callback) {
if (callback === undefined) {
return this.#iterate(push => {
const subscription = this.listen(key, (value, expired) => push({ value, expired }));
return () => subscription.cancel();
});
}
const token = this.#native.listen(key, callback);
this.#listeners++;
this.#updateOwner();
let cancelled = false;
return {
cancel: () => {
if (cancelled) return;
this.#native.cancelListen(token);
cancelled = true;
this.#listeners--;
this.#updateOwner();
},
};
}
#iterate(subscribe, rejectOnClose = false) {
let values = [];
let offset = 0;
const pending = [];
let done = false;
let failure;
const finish = error => {
if (done) return;
done = true;
failure = error;
unsubscribe();
this.#iterators.delete(onClose);
if (failure) {
values = [];
offset = 0;
}
for (const { resolve, reject } of pending.splice(0)) {
if (failure) reject(failure);
else resolve({ value: undefined, done: true });
}
};
const cancel = () => {
values = [];
offset = 0;
failure = undefined;
finish();
};
const onClose = () => {
if (rejectOnClose) {
const error = new Error('DHT runner is closed');
error.code = 'ERR_DHT_CLOSED';
finish(error);
} else cancel();
};
const unsubscribe = subscribe(value => {
if (done) return;
const waiter = pending.shift();
if (waiter) waiter.resolve({ value, done: false });
else values.push(value);
}, finish);
this.#iterators.add(onClose);
return {
next() {
if (failure) return Promise.reject(failure);
if (offset < values.length) {
const value = values[offset];
values[offset++] = undefined;
if (offset === values.length) {
values = [];
offset = 0;
}
return Promise.resolve({ value, done: false });
}
if (done) return Promise.resolve({ value: undefined, done: true });
return new Promise((resolve, reject) => pending.push({ resolve, reject }));
},
return() {
cancel();
return Promise.resolve({ value: undefined, done: true });
},
throw(error) {
cancel();
return Promise.reject(error);
},
cancel,
[Symbol.asyncIterator]() { return this; },
[Symbol.asyncDispose]() { cancel(); return Promise.resolve(); },
};
}
close() {
if (!this.#closing) {
this.#closing = this.#native.close();
for (const cancel of this.#iterators) cancel();
}
return this.#closing;
}
ref() { this.#native.ref(); return this; }
unref() { this.#native.unref(); return this; }
[Symbol.asyncDispose]() { return this.close(); }
}
export { DhtRunner, PublicKey, PrivateKey, Certificate, Identity };
export const hash = native.hash;
export const version = native.version;
+904
View File
@@ -0,0 +1,904 @@
{
"name": "opendht",
"version": "0.1.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "opendht",
"version": "0.1.0",
"license": "MIT",
"dependencies": {
"node-gyp-build": "^4.8.4"
},
"devDependencies": {
"@types/node": "^22.20.5",
"cmake-js": "^8.0.0",
"node-addon-api": "^8.5.0",
"typescript": "^7.0.2"
},
"engines": {
"node": ">=22.9.0"
}
},
"node_modules/@isaacs/fs-minipass": {
"version": "4.0.1",
"resolved": "https://registry.npmjs.org/@isaacs/fs-minipass/-/fs-minipass-4.0.1.tgz",
"integrity": "sha512-wgm9Ehl2jpeqP3zw/7mo3kRHFp5MEDhqAdwy1fTGkHAwnkGOVsgpvQhL8B5n1qlb01jV3n/bI0ZfZp5lWA1k4w==",
"dev": true,
"license": "ISC",
"dependencies": {
"minipass": "^7.0.4"
},
"engines": {
"node": ">=18.0.0"
}
},
"node_modules/@types/node": {
"version": "22.20.5",
"resolved": "https://registry.npmjs.org/@types/node/-/node-22.20.5.tgz",
"integrity": "sha512-U2+DNr+wSjpsTS/wZGYHq7GcwfuSmKiKvoPvK22zwTlRhU91yOniN4qRR5KhIjvif7ysw/dz/hKmfDH0Ris4aA==",
"dev": true,
"license": "MIT",
"dependencies": {
"undici-types": "~6.21.0"
}
},
"node_modules/@typescript/typescript-aix-ppc64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-aix-ppc64/-/typescript-aix-ppc64-7.0.2.tgz",
"integrity": "sha512-MTKKkWB7p/0E9xi1d1tHtZ5PiLkGEMIq88pK2CubZjOsLtYTLqhgIgi6zepFa+9GHZ6h05NMCkQxGKiPXMxXtQ==",
"cpu": [
"ppc64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"aix"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-darwin-arm64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-darwin-arm64/-/typescript-darwin-arm64-7.0.2.tgz",
"integrity": "sha512-gowzar9MwS/aRWp6f3a4KUqzRjAZjOsmGNCM6LcTgXum+dBfgsBVMN+AgvOCCbguXyick6LJhpBszxMebJ8syA==",
"cpu": [
"arm64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-darwin-x64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-darwin-x64/-/typescript-darwin-x64-7.0.2.tgz",
"integrity": "sha512-SZ9xZInqApNlNGc9s0W1VSsktYSOe9cFqNOIqmN1Gs8SmkjKZYFt017G4VwPxASInODuAdbTW7sXiFUf893RgA==",
"cpu": [
"x64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-freebsd-arm64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-freebsd-arm64/-/typescript-freebsd-arm64-7.0.2.tgz",
"integrity": "sha512-W5NH4y/J0plIIS5b2xvTEkU7JFxyqdMAOgf+Ilhl0vHQXKO5dZoxd+C/jEtq56c4F3wk71RB4BMRQ2XdI+bwYQ==",
"cpu": [
"arm64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"freebsd"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-freebsd-x64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-freebsd-x64/-/typescript-freebsd-x64-7.0.2.tgz",
"integrity": "sha512-UMGDx5sTpzNw3WiPebH7l90IWfJggEd+egHt/q6p7/Cm3zqoV7VxkGXt+3DxPIw8CcmvAB0j3sVVfbhX+M4Tpw==",
"cpu": [
"x64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"freebsd"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-linux-arm": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-linux-arm/-/typescript-linux-arm-7.0.2.tgz",
"integrity": "sha512-gffT3xPz9sR7j/YJExkyPntrI0P2EP9XbOyWzth2/Gs0RstK+90RBcO0ncXoXy/beYll1SXw846Nf2zdnEz0QQ==",
"cpu": [
"arm"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-linux-arm64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-linux-arm64/-/typescript-linux-arm64-7.0.2.tgz",
"integrity": "sha512-Qh4eU4/y3yDjnfjjyPYihMj5/ODIlmt+Bzu17OI+fiSRDW57QmU5SiN63exPRNJPKUzcc1INa1NXdrJ+MqHjUQ==",
"cpu": [
"arm64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-linux-loong64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-linux-loong64/-/typescript-linux-loong64-7.0.2.tgz",
"integrity": "sha512-uEHck9i8hoAzXPiYRib1O7miOnz23SxIeVl6F4LXox+qov1K35jHcEW6VHKvZI+pyvl7fZEP4MCU5LYvIq1GuQ==",
"cpu": [
"loong64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-linux-mips64el": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-linux-mips64el/-/typescript-linux-mips64el-7.0.2.tgz",
"integrity": "sha512-R4KvAMnE43W5Qeqb0Ly56O3mWMWIAgsMyz36DCaycd5nbg/9kzm0liw3JocfRqyJY0KPmzFjbswozXyW0DnIYA==",
"cpu": [
"mips64el"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-linux-ppc64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-linux-ppc64/-/typescript-linux-ppc64-7.0.2.tgz",
"integrity": "sha512-DORx5b3sd/4S7eayxm4FQv+A7CrkUIGRaHiwI8oiHTAI1fAPWhF4J0vAlkC8biAlHSVVwxMQ3tjZ2/DVbnQiiA==",
"cpu": [
"ppc64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-linux-riscv64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-linux-riscv64/-/typescript-linux-riscv64-7.0.2.tgz",
"integrity": "sha512-wf0jqEDOjrPRnKwYRyyJDRo11KMbvMFrU+q4zqKyChODBzvlkbhNQfKvLxQCcwTpdDaXSHZTVuh0JoCrKCUMHQ==",
"cpu": [
"riscv64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-linux-s390x": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-linux-s390x/-/typescript-linux-s390x-7.0.2.tgz",
"integrity": "sha512-IkwJc3L7yhytWd/ewjyxNDfOmswCm9GWMJT/ue/dU4aZNbwZeYAetq42VyLmsmSjvoX7z74X6ZaYCtzAr0EuGw==",
"cpu": [
"s390x"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-linux-x64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-linux-x64/-/typescript-linux-x64-7.0.2.tgz",
"integrity": "sha512-EYdf2cNg7rgCWJnxCdJ+F3V39O8ihb37eHAu1LK8oAFizgTQbPOK7zHHXbPt8rX24COqODXeI3sIf0fCXG7H/A==",
"cpu": [
"x64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-netbsd-arm64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-netbsd-arm64/-/typescript-netbsd-arm64-7.0.2.tgz",
"integrity": "sha512-+polYF4MF04aPpO5FTkHran9yUQDSXqy5GiSDKpsll5jy3l3+g9QLhpf39T+ePtefhXLOGrLl0QIjkQP6VnelA==",
"cpu": [
"arm64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"netbsd"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-netbsd-x64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-netbsd-x64/-/typescript-netbsd-x64-7.0.2.tgz",
"integrity": "sha512-8YIT0EHM/3dq10ZOVF/A7pc/YSMtbcecct4rWtexrnSCHOPcpC2KTLXfTCR6vDpnSiY12heNb1GiN/wu+T/FyA==",
"cpu": [
"x64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"netbsd"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-openbsd-arm64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-openbsd-arm64/-/typescript-openbsd-arm64-7.0.2.tgz",
"integrity": "sha512-APT8+ClYnuYm1u9+kgGXoMj2VzWzcymwh2gNSQVySHfkRDGOTVkoWLjCmOQSaO+PoqQ57B0flRp9SA+7GnnkzQ==",
"cpu": [
"arm64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"openbsd"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-openbsd-x64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-openbsd-x64/-/typescript-openbsd-x64-7.0.2.tgz",
"integrity": "sha512-yX7s+Q0Dln0Dt9tEzZsAjXXR/+ytBM7AlglaqyeMPxQszJ1JhlJdZ6jLA+IzldHtflX81em7lDao1xXu+aRRkg==",
"cpu": [
"x64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"openbsd"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-sunos-x64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-sunos-x64/-/typescript-sunos-x64-7.0.2.tgz",
"integrity": "sha512-dLJDGaLZ1D4HPQn62u1n8mBDkJREwMsAkCdkwd4Ieqw+x3TUyTsqY0YiBCtE6H6OzzgGk3iuZ3vFWRS+E8/d1g==",
"cpu": [
"x64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"sunos"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-win32-arm64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-win32-arm64/-/typescript-win32-arm64-7.0.2.tgz",
"integrity": "sha512-Gyl1Vy6OsWesLzmq+EP0Fb7b4Nid5232AvcA2SFcdYreldpNtYFFofPjnt62y9hQy7VTaZp65ICJjuAQRaVcIQ==",
"cpu": [
"arm64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-win32-x64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-win32-x64/-/typescript-win32-x64-7.0.2.tgz",
"integrity": "sha512-0BQ3HkAHHlKLSp1qRvf3SUhGpGsDuhB/jgFw75guyqbxJqEaS0Cw/VFO8i2nHglJUzQCRtMMR/IBAKE3ETMC4g==",
"cpu": [
"x64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/ansi-regex": {
"version": "5.0.1",
"resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-5.0.1.tgz",
"integrity": "sha512-quJQXlTSUGL2LH9SUXo8VwsY4soanhgo6LNSm84E1LBcE8s3O0wpdiRzyR9z/ZZJMlMWv37qOOb9pdJlMUEKFQ==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=8"
}
},
"node_modules/ansi-styles": {
"version": "4.3.0",
"resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-4.3.0.tgz",
"integrity": "sha512-zbB9rCJAT1rbjiVDb2hqKFHNYLxgtk8NURxZ3IZwD3F6NtxbXZQCnnSi1Lkx+IDohdPlFp222wVALIheZJQSEg==",
"dev": true,
"license": "MIT",
"dependencies": {
"color-convert": "^2.0.1"
},
"engines": {
"node": ">=8"
},
"funding": {
"url": "https://github.com/chalk/ansi-styles?sponsor=1"
}
},
"node_modules/chownr": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/chownr/-/chownr-3.0.0.tgz",
"integrity": "sha512-+IxzY9BZOQd/XuYPRmrvEVjF/nqj5kgT4kEq7VofrDoM1MxoRjEWkrCC3EtLi59TVawxTAn+orJwFQcrqEN1+g==",
"dev": true,
"license": "BlueOak-1.0.0",
"engines": {
"node": ">=18"
}
},
"node_modules/cliui": {
"version": "8.0.1",
"resolved": "https://registry.npmjs.org/cliui/-/cliui-8.0.1.tgz",
"integrity": "sha512-BSeNnyus75C4//NQ9gQt1/csTXyo/8Sb+afLAkzAptFuMsod9HFokGNudZpi/oQV73hnVK+sR+5PVRMd+Dr7YQ==",
"dev": true,
"license": "ISC",
"dependencies": {
"string-width": "^4.2.0",
"strip-ansi": "^6.0.1",
"wrap-ansi": "^7.0.0"
},
"engines": {
"node": ">=12"
}
},
"node_modules/cmake-js": {
"version": "8.0.0",
"resolved": "https://registry.npmjs.org/cmake-js/-/cmake-js-8.0.0.tgz",
"integrity": "sha512-YbUP88RDwCvoQkZhRtGURYm9RIpWdtvZuhT87fKNoLjk8kIFIFeARpKfuZQGdwfH99GZpUmqSfcDrK62X7lTgg==",
"dev": true,
"license": "MIT",
"dependencies": {
"debug": "^4.4.3",
"fs-extra": "^11.3.3",
"node-api-headers": "^1.8.0",
"rc": "1.2.8",
"semver": "^7.7.3",
"tar": "^7.5.6",
"url-join": "^4.0.1",
"which": "^6.0.0",
"yargs": "^17.7.2"
},
"bin": {
"cmake-js": "bin/cmake-js"
},
"engines": {
"node": "^20.17.0 || >=22.9.0"
}
},
"node_modules/color-convert": {
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/color-convert/-/color-convert-2.0.1.tgz",
"integrity": "sha512-RRECPsj7iu/xb5oKYcsFHSppFNnsj/52OVTRKb4zP5onXwVF3zVmmToNcOfGC+CRDpfK/U584fMg38ZHCaElKQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"color-name": "~1.1.4"
},
"engines": {
"node": ">=7.0.0"
}
},
"node_modules/color-name": {
"version": "1.1.4",
"resolved": "https://registry.npmjs.org/color-name/-/color-name-1.1.4.tgz",
"integrity": "sha512-dOy+3AuW3a2wNbZHIuMZpTcgjGuLU/uBL/ubcZF9OXbDo8ff4O8yVp5Bf0efS8uEoYo5q4Fx7dY9OgQGXgAsQA==",
"dev": true,
"license": "MIT"
},
"node_modules/debug": {
"version": "4.4.3",
"resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz",
"integrity": "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==",
"dev": true,
"license": "MIT",
"dependencies": {
"ms": "^2.1.3"
},
"engines": {
"node": ">=6.0"
},
"peerDependenciesMeta": {
"supports-color": {
"optional": true
}
}
},
"node_modules/deep-extend": {
"version": "0.6.0",
"resolved": "https://registry.npmjs.org/deep-extend/-/deep-extend-0.6.0.tgz",
"integrity": "sha512-LOHxIOaPYdHlJRtCQfDIVZtfw/ufM8+rVj649RIHzcm/vGwQRXFt6OPqIFWsm2XEMrNIEtWR64sY1LEKD2vAOA==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=4.0.0"
}
},
"node_modules/emoji-regex": {
"version": "8.0.0",
"resolved": "https://registry.npmjs.org/emoji-regex/-/emoji-regex-8.0.0.tgz",
"integrity": "sha512-MSjYzcWNOA0ewAHpz0MxpYFvwg6yjy1NG3xteoqz644VCo/RPgnr1/GGt+ic3iJTzQ8Eu3TdM14SawnVUmGE6A==",
"dev": true,
"license": "MIT"
},
"node_modules/escalade": {
"version": "3.2.0",
"resolved": "https://registry.npmjs.org/escalade/-/escalade-3.2.0.tgz",
"integrity": "sha512-WUj2qlxaQtO4g6Pq5c29GTcWGDyd8itL8zTlipgECz3JesAiiOKotd8JU6otB3PACgG6xkJUyVhboMS+bje/jA==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=6"
}
},
"node_modules/fs-extra": {
"version": "11.4.1",
"resolved": "https://registry.npmjs.org/fs-extra/-/fs-extra-11.4.1.tgz",
"integrity": "sha512-KYAb4c9BJQI6QqGKthV68OHe0badztdXJWKo0WtBA9IuCFPTKvE5ZdUBglP833aMjhaSPNO4A5j/EkzZtGlKjA==",
"dev": true,
"license": "MIT",
"dependencies": {
"graceful-fs": "^4.2.0",
"jsonfile": "^6.0.1",
"universalify": "^2.0.0"
},
"engines": {
"node": ">=14.14"
}
},
"node_modules/get-caller-file": {
"version": "2.0.5",
"resolved": "https://registry.npmjs.org/get-caller-file/-/get-caller-file-2.0.5.tgz",
"integrity": "sha512-DyFP3BM/3YHTQOCUL/w0OZHR0lpKeGrxotcHWcqNEdnltqFwXVfhEBQ94eIo34AfQpo0rGki4cyIiftY06h2Fg==",
"dev": true,
"license": "ISC",
"engines": {
"node": "6.* || 8.* || >= 10.*"
}
},
"node_modules/graceful-fs": {
"version": "4.2.11",
"resolved": "https://registry.npmjs.org/graceful-fs/-/graceful-fs-4.2.11.tgz",
"integrity": "sha512-RbJ5/jmFcNNCcDV5o9eTnBLJ/HszWV0P73bc+Ff4nS/rJj+YaS6IGyiOL0VoBYX+l1Wrl3k63h/KrH+nhJ0XvQ==",
"dev": true,
"license": "ISC"
},
"node_modules/ini": {
"version": "1.3.8",
"resolved": "https://registry.npmjs.org/ini/-/ini-1.3.8.tgz",
"integrity": "sha512-JV/yugV2uzW5iMRSiZAyDtQd+nxtUnjeLt0acNdw98kKLrvuRVyB80tsREOE7yvGVgalhZ6RNXCmEHkUKBKxew==",
"dev": true,
"license": "ISC"
},
"node_modules/is-fullwidth-code-point": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/is-fullwidth-code-point/-/is-fullwidth-code-point-3.0.0.tgz",
"integrity": "sha512-zymm5+u+sCsSWyD9qNaejV3DFvhCKclKdizYaJUuHA83RLjb7nSuGnddCHGv0hk+KY7BMAlsWeK4Ueg6EV6XQg==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=8"
}
},
"node_modules/isexe": {
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/isexe/-/isexe-4.0.0.tgz",
"integrity": "sha512-FFUtZMpoZ8RqHS3XeXEmHWLA4thH+ZxCv2lOiPIn1Xc7CxrqhWzNSDzD+/chS/zbYezmiwWLdQC09JdQKmthOw==",
"dev": true,
"license": "BlueOak-1.0.0",
"engines": {
"node": ">=20"
}
},
"node_modules/jsonfile": {
"version": "6.2.1",
"resolved": "https://registry.npmjs.org/jsonfile/-/jsonfile-6.2.1.tgz",
"integrity": "sha512-zwOTdL3rFQ/lRdBnntKVOX6k5cKJwEc1HdilT71BWEu7J41gXIB2MRp+vxduPSwZJPWBxEzv4yH1wYLJGUHX4Q==",
"dev": true,
"license": "MIT",
"dependencies": {
"universalify": "^2.0.0"
},
"optionalDependencies": {
"graceful-fs": "^4.1.6"
}
},
"node_modules/minimist": {
"version": "1.2.8",
"resolved": "https://registry.npmjs.org/minimist/-/minimist-1.2.8.tgz",
"integrity": "sha512-2yyAR8qBkN3YuheJanUpWC5U3bb5osDywNB8RzDVlDwDHbocAJveqqj1u8+SVD7jkWT4yvsHCpWqqWqAxb0zCA==",
"dev": true,
"license": "MIT",
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/minipass": {
"version": "7.1.3",
"resolved": "https://registry.npmjs.org/minipass/-/minipass-7.1.3.tgz",
"integrity": "sha512-tEBHqDnIoM/1rXME1zgka9g6Q2lcoCkxHLuc7ODJ5BxbP5d4c2Z5cGgtXAku59200Cx7diuHTOYfSBD8n6mm8A==",
"dev": true,
"license": "BlueOak-1.0.0",
"engines": {
"node": ">=16 || 14 >=14.17"
}
},
"node_modules/minizlib": {
"version": "3.1.0",
"resolved": "https://registry.npmjs.org/minizlib/-/minizlib-3.1.0.tgz",
"integrity": "sha512-KZxYo1BUkWD2TVFLr0MQoM8vUUigWD3LlD83a/75BqC+4qE0Hb1Vo5v1FgcfaNXvfXzr+5EhQ6ing/CaBijTlw==",
"dev": true,
"license": "MIT",
"dependencies": {
"minipass": "^7.1.2"
},
"engines": {
"node": ">= 18"
}
},
"node_modules/ms": {
"version": "2.1.3",
"resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz",
"integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==",
"dev": true,
"license": "MIT"
},
"node_modules/node-addon-api": {
"version": "8.9.2",
"resolved": "https://registry.npmjs.org/node-addon-api/-/node-addon-api-8.9.2.tgz",
"integrity": "sha512-VijLXbi3UACN69I0JVXJsX4tjACjNoQDgv2gTF6sx2wWEi8tkSg2eX8p5gSIFi8z2+DL3oHmY6OyKce38SDolg==",
"dev": true,
"license": "MIT",
"engines": {
"node": "^18 || ^20 || >= 21"
}
},
"node_modules/node-api-headers": {
"version": "1.9.0",
"resolved": "https://registry.npmjs.org/node-api-headers/-/node-api-headers-1.9.0.tgz",
"integrity": "sha512-2oNILP4jXwRB4ywnYKjVk1YyJ96n2D4EOVJO6S3oYZ5PtbJrw3Yt9TpAuX3nBLMuzn74rnfGQrv13pS9vC+YiA==",
"dev": true,
"license": "MIT"
},
"node_modules/node-gyp-build": {
"version": "4.8.4",
"resolved": "https://registry.npmjs.org/node-gyp-build/-/node-gyp-build-4.8.4.tgz",
"integrity": "sha512-LA4ZjwlnUblHVgq0oBF3Jl/6h/Nvs5fzBLwdEF4nuxnFdsfajde4WfxtJr3CaiH+F6ewcIB/q4jQ4UzPyid+CQ==",
"license": "MIT",
"bin": {
"node-gyp-build": "bin.js",
"node-gyp-build-optional": "optional.js",
"node-gyp-build-test": "build-test.js"
}
},
"node_modules/rc": {
"version": "1.2.8",
"resolved": "https://registry.npmjs.org/rc/-/rc-1.2.8.tgz",
"integrity": "sha512-y3bGgqKj3QBdxLbLkomlohkvsA8gdAiUQlSBJnBhfn+BPxg4bc62d8TcBW15wavDfgexCgccckhcZvywyQYPOw==",
"dev": true,
"license": "(BSD-2-Clause OR MIT OR Apache-2.0)",
"dependencies": {
"deep-extend": "^0.6.0",
"ini": "~1.3.0",
"minimist": "^1.2.0",
"strip-json-comments": "~2.0.1"
},
"bin": {
"rc": "cli.js"
}
},
"node_modules/require-directory": {
"version": "2.1.1",
"resolved": "https://registry.npmjs.org/require-directory/-/require-directory-2.1.1.tgz",
"integrity": "sha512-fGxEI7+wsG9xrvdjsrlmL22OMTTiHRwAMroiEeMgq8gzoLC/PQr7RsRDSTLUg/bZAZtF+TVIkHc6/4RIKrui+Q==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=0.10.0"
}
},
"node_modules/semver": {
"version": "7.8.5",
"resolved": "https://registry.npmjs.org/semver/-/semver-7.8.5.tgz",
"integrity": "sha512-Y7/KDsb8LjooZpwaqGyulO6DQlksgCncchHGk+sZIY4SBvUocMBEFH5Ur1fI4dV+Jvl0w6cjvucaIi40puRioA==",
"dev": true,
"license": "ISC",
"bin": {
"semver": "bin/semver.js"
},
"engines": {
"node": ">=10"
}
},
"node_modules/string-width": {
"version": "4.2.3",
"resolved": "https://registry.npmjs.org/string-width/-/string-width-4.2.3.tgz",
"integrity": "sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g==",
"dev": true,
"license": "MIT",
"dependencies": {
"emoji-regex": "^8.0.0",
"is-fullwidth-code-point": "^3.0.0",
"strip-ansi": "^6.0.1"
},
"engines": {
"node": ">=8"
}
},
"node_modules/strip-ansi": {
"version": "6.0.1",
"resolved": "https://registry.npmjs.org/strip-ansi/-/strip-ansi-6.0.1.tgz",
"integrity": "sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A==",
"dev": true,
"license": "MIT",
"dependencies": {
"ansi-regex": "^5.0.1"
},
"engines": {
"node": ">=8"
}
},
"node_modules/strip-json-comments": {
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/strip-json-comments/-/strip-json-comments-2.0.1.tgz",
"integrity": "sha512-4gB8na07fecVVkOI6Rs4e7T6NOTki5EmL7TUduTs6bu3EdnSycntVJ4re8kgZA+wx9IueI2Y11bfbgwtzuE0KQ==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=0.10.0"
}
},
"node_modules/tar": {
"version": "7.5.22",
"resolved": "https://registry.npmjs.org/tar/-/tar-7.5.22.tgz",
"integrity": "sha512-MFO/QzvtAOmJbkhOaCTvbGcFN9L9b+JunIsDwaKljSOdcLMea3NJ1k9Usz/rjdfSXTq4dfzfeS7W4p4YOAAHeA==",
"dev": true,
"license": "BlueOak-1.0.0",
"dependencies": {
"@isaacs/fs-minipass": "^4.0.0",
"chownr": "^3.0.0",
"minipass": "^7.1.2",
"minizlib": "^3.1.0",
"yallist": "^5.0.0"
},
"engines": {
"node": ">=18"
}
},
"node_modules/typescript": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/typescript/-/typescript-7.0.2.tgz",
"integrity": "sha512-8FYau96o3NKOhbjKi/qNvG/W5jhzxkbdm5sj9AbZ/5T5sWqn3hJgLfGx27sRKZWTvyzCP8dLRBTf5tBTSRVUNA==",
"dev": true,
"license": "Apache-2.0",
"bin": {
"tsc": "bin/tsc"
},
"engines": {
"node": ">=16.20.0"
},
"optionalDependencies": {
"@typescript/typescript-aix-ppc64": "7.0.2",
"@typescript/typescript-darwin-arm64": "7.0.2",
"@typescript/typescript-darwin-x64": "7.0.2",
"@typescript/typescript-freebsd-arm64": "7.0.2",
"@typescript/typescript-freebsd-x64": "7.0.2",
"@typescript/typescript-linux-arm": "7.0.2",
"@typescript/typescript-linux-arm64": "7.0.2",
"@typescript/typescript-linux-loong64": "7.0.2",
"@typescript/typescript-linux-mips64el": "7.0.2",
"@typescript/typescript-linux-ppc64": "7.0.2",
"@typescript/typescript-linux-riscv64": "7.0.2",
"@typescript/typescript-linux-s390x": "7.0.2",
"@typescript/typescript-linux-x64": "7.0.2",
"@typescript/typescript-netbsd-arm64": "7.0.2",
"@typescript/typescript-netbsd-x64": "7.0.2",
"@typescript/typescript-openbsd-arm64": "7.0.2",
"@typescript/typescript-openbsd-x64": "7.0.2",
"@typescript/typescript-sunos-x64": "7.0.2",
"@typescript/typescript-win32-arm64": "7.0.2",
"@typescript/typescript-win32-x64": "7.0.2"
}
},
"node_modules/undici-types": {
"version": "6.21.0",
"resolved": "https://registry.npmjs.org/undici-types/-/undici-types-6.21.0.tgz",
"integrity": "sha512-iwDZqg0QAGrg9Rav5H4n0M64c3mkR59cJ6wQp+7C4nI0gsmExaedaYLNO44eT4AtBBwjbTiGPMlt2Md0T9H9JQ==",
"dev": true,
"license": "MIT"
},
"node_modules/universalify": {
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/universalify/-/universalify-2.0.1.tgz",
"integrity": "sha512-gptHNQghINnc/vTGIk0SOFGFNXw7JVrlRUtConJRlvaw6DuX0wO5Jeko9sWrMBhh+PsYAZ7oXAiOnf/UKogyiw==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">= 10.0.0"
}
},
"node_modules/url-join": {
"version": "4.0.1",
"resolved": "https://registry.npmjs.org/url-join/-/url-join-4.0.1.tgz",
"integrity": "sha512-jk1+QP6ZJqyOiuEI9AEWQfju/nB2Pw466kbA0LEZljHwKeMgd9WrAEgEGxjPDD2+TNbbb37rTyhEfrCXfuKXnA==",
"dev": true,
"license": "MIT"
},
"node_modules/which": {
"version": "6.0.1",
"resolved": "https://registry.npmjs.org/which/-/which-6.0.1.tgz",
"integrity": "sha512-oGLe46MIrCRqX7ytPUf66EAYvdeMIZYn3WaocqqKZAxrBpkqHfL/qvTyJ/bTk5+AqHCjXmrv3CEWgy368zhRUg==",
"dev": true,
"license": "ISC",
"dependencies": {
"isexe": "^4.0.0"
},
"bin": {
"node-which": "bin/which.js"
},
"engines": {
"node": "^20.17.0 || >=22.9.0"
}
},
"node_modules/wrap-ansi": {
"version": "7.0.0",
"resolved": "https://registry.npmjs.org/wrap-ansi/-/wrap-ansi-7.0.0.tgz",
"integrity": "sha512-YVGIj2kamLSTxw6NsZjoBxfSwsn0ycdesmc4p+Q21c5zPuZ1pl+NfxVdxPtdHvmNVOQ6XSYG4AUtyt/Fi7D16Q==",
"dev": true,
"license": "MIT",
"dependencies": {
"ansi-styles": "^4.0.0",
"string-width": "^4.1.0",
"strip-ansi": "^6.0.0"
},
"engines": {
"node": ">=10"
},
"funding": {
"url": "https://github.com/chalk/wrap-ansi?sponsor=1"
}
},
"node_modules/y18n": {
"version": "5.0.8",
"resolved": "https://registry.npmjs.org/y18n/-/y18n-5.0.8.tgz",
"integrity": "sha512-0pfFzegeDWJHJIAmTLRP2DwHjdF5s7jo9tuztdQxAhINCdvS+3nGINqPd00AphqJR/0LhANUS6/+7SCb98YOfA==",
"dev": true,
"license": "ISC",
"engines": {
"node": ">=10"
}
},
"node_modules/yallist": {
"version": "5.0.0",
"resolved": "https://registry.npmjs.org/yallist/-/yallist-5.0.0.tgz",
"integrity": "sha512-YgvUTfwqyc7UXVMrB+SImsVYSmTS8X/tSrtdNZMImM+n7+QTriRXyXim0mBrTXNeqzVF0KWGgHPeiyViFFrNDw==",
"dev": true,
"license": "BlueOak-1.0.0",
"engines": {
"node": ">=18"
}
},
"node_modules/yargs": {
"version": "17.7.3",
"resolved": "https://registry.npmjs.org/yargs/-/yargs-17.7.3.tgz",
"integrity": "sha512-GZtjxm/J/4TSxuL3FNYjCmLktBTnIw/rVmKSIyKeYAZpmJB2ig9VauCC5xsa82GNKVKDAqpOn3KVzNt0zmrU0g==",
"dev": true,
"license": "MIT",
"dependencies": {
"cliui": "^8.0.1",
"escalade": "^3.1.1",
"get-caller-file": "^2.0.5",
"require-directory": "^2.1.1",
"string-width": "^4.2.3",
"y18n": "^5.0.5",
"yargs-parser": "^21.1.1"
},
"engines": {
"node": ">=12"
}
},
"node_modules/yargs-parser": {
"version": "21.1.1",
"resolved": "https://registry.npmjs.org/yargs-parser/-/yargs-parser-21.1.1.tgz",
"integrity": "sha512-tVpsJW7DdjecAiFpbIB1e3qxIQsE6NoPc5/eTdrbbIC4h0LVsWhnoa3g+m2HclBIujHzsxZ4VJVA+GUuc2/LBw==",
"dev": true,
"license": "ISC",
"engines": {
"node": ">=12"
}
}
}
}
+58
View File
@@ -0,0 +1,58 @@
{
"name": "opendht",
"version": "0.1.0",
"description": "Distributed hash table for Node.js",
"license": "MIT",
"repository": {
"type": "git",
"url": "git+https://github.com/savoirfairelinux/opendht.git",
"directory": "nodejs"
},
"publishConfig": {
"access": "public",
"registry": "https://registry.npmjs.org/"
},
"type": "module",
"main": "index.js",
"types": "index.d.ts",
"exports": {
".": {
"types": "./index.d.ts",
"import": "./index.js"
},
"./package.json": "./package.json"
},
"files": [
"index.js",
"index.d.ts",
"crypto.js",
"README.md",
"examples/dhtnode.js",
"examples/dhtnode.ts",
"prebuilds/"
],
"engines": {
"node": ">=22.9.0"
},
"scripts": {
"build": "cmake-js compile",
"postbuild": "npm run build:examples",
"build:examples": "tsc --strict --types node --target es2024 --module Node16 --moduleResolution Node16 --rootDir examples --outDir examples examples/dhtnode.ts",
"dhtnode": "node examples/dhtnode.js",
"prepack": "npm run build:examples",
"bundle": "node scripts/bundle.js",
"package": "npm run bundle && npm pack --pack-destination dist",
"test:package": "node scripts/test-package.js",
"test": "node --test test/*.test.js",
"typecheck": "tsc --noEmit --strict --types node --target es2024 --module Node16 --moduleResolution Node16 test/types.ts examples/dhtnode.ts"
},
"dependencies": {
"node-gyp-build": "^4.8.4"
},
"devDependencies": {
"@types/node": "^22.20.5",
"cmake-js": "^8.0.0",
"node-addon-api": "^8.8.0",
"typescript": "^7.0.2"
}
}
+105
View File
@@ -0,0 +1,105 @@
import { execFileSync } from 'node:child_process';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
const root = path.resolve(import.meta.dirname, '..');
const repository = path.dirname(root);
const input = path.resolve(process.argv[2] || path.join(root, 'build/Release/opendht.node'));
const target = `${process.platform}-${process.arch}`;
const output = path.join(root, 'prebuilds', target);
const temporary = fs.mkdtempSync(path.join(os.tmpdir(), 'opendht-bundle-'));
const run = (command, args) => execFileSync(command, args, { encoding: 'utf8', stdio: ['ignore', 'pipe', 'inherit'] });
function copyNotices(dependencies) {
const directory = path.join(output, 'licenses');
fs.mkdirSync(directory, { recursive: true });
fs.copyFileSync(path.join(repository, 'LICENSE'), path.join(directory, 'OpenDHT-LICENSE.txt'));
const copied = new Set();
for (const dependency of dependencies) {
const real = fs.realpathSync(dependency);
if (!path.relative(repository, real).startsWith('..')) continue;
let name;
let notices;
if (process.platform === 'darwin') {
const match = real.match(/^(.*\/Cellar\/([^/]+)\/[^/]+)\//);
if (!match) throw new Error(`Cannot locate license notices for ${dependency}`);
name = match[2];
notices = fs.readdirSync(match[1]).filter(file => /^(COPYING|LICENSE|COPYRIGHT)([._-]|$)/i.test(file))
.map(file => path.join(match[1], file));
} else if (process.platform === 'linux' && /^libllhttp\.so(?:\.|$)/.test(path.basename(real))) {
name = 'llhttp';
notices = [path.join(repository, 'resources', 'llhttp-LICENSE-MIT')];
} else if (process.platform === 'linux') {
let owner;
try { owner = run('dpkg-query', ['-S', real]); }
catch { owner = run('dpkg-query', ['-S', real.replace(/^\/usr\//, '/')]); }
name = owner.split('\n')[0].split(': ')[0].split(':')[0];
notices = [path.join('/usr/share/doc', name, 'copyright')];
} else {
const installed = process.env.VCPKG_INSTALLED_DIR;
const triplet = process.env.VCPKG_DEFAULT_TRIPLET || 'x64-windows-release';
if (!installed) throw new Error('VCPKG_INSTALLED_DIR is required for Windows dependency notices');
const share = path.join(installed, triplet, 'share');
for (const entry of fs.readdirSync(share)) {
const copyright = path.join(share, entry, 'copyright');
if (fs.existsSync(copyright)) fs.copyFileSync(copyright, path.join(directory, `${entry}-copyright.txt`));
}
break;
}
if (!notices.length) throw new Error(`Missing license notices for ${name}`);
if (copied.has(name)) continue;
for (const notice of notices) fs.copyFileSync(notice, path.join(directory, `${name}-${path.basename(notice)}.txt`));
copied.add(name);
}
}
function fixMachO(file) {
const commands = run('otool', ['-l', file]);
const rpaths = [...commands.matchAll(/cmd LC_RPATH\s+cmdsize \d+\s+path (.+?) \(offset \d+\)/g)];
for (const rpath of new Set(rpaths.map(match => match[1]))) run('install_name_tool', ['-delete_rpath', rpath, file]);
const dependencies = [...run('otool', ['-L', file]).matchAll(/^\s+(.+?) \(compatibility version/gm)];
for (const [, dependency] of dependencies) {
if (!/^(@loader_path\/|\/usr\/lib\/|\/System\/Library\/)/.test(dependency)) {
throw new Error(`Non-relocatable dependency in ${path.basename(file)}: ${dependency}`);
}
}
run('codesign', ['--force', '--sign', '-', file]);
}
try {
if (!['darwin', 'linux', 'win32'].includes(process.platform)) throw new Error(`Unsupported platform ${target}`);
if (!fs.existsSync(input)) throw new Error(`Build the Release addon first: ${input}`);
if (process.platform === 'linux' && !process.report.getReport().header.glibcVersionRuntime) {
throw new Error('Linux prebuilds currently require glibc');
}
const directories = [path.dirname(input), ...(process.env.OPENDHT_LIBRARY_PATH || '').split(path.delimiter).filter(Boolean)];
const dependencyList = path.join(temporary, 'dependencies.json');
run('cmake', [`-DADDON=${input}`, `-DOUTPUT=${dependencyList}`, `-DSEARCH_DIRECTORIES=${directories.join(';')}`,
'-P', path.join(import.meta.dirname, 'dependencies.cmake')]);
const dependencies = JSON.parse(fs.readFileSync(dependencyList, 'utf8'));
fs.rmSync(output, { recursive: true, force: true });
fs.mkdirSync(output, { recursive: true });
const binary = path.join(output, process.platform === 'linux' ? 'node.napi.glibc.node' : 'node.napi.node');
fs.copyFileSync(input, binary);
if (process.platform === 'darwin') {
run('dylibbundler', ['-b', '-of', '-ns', '-x', binary, '-d', `${output}/`, '-p', '@loader_path/', '-s', path.dirname(input)]);
for (const file of fs.readdirSync(output)) fixMachO(path.join(output, file));
} else {
for (const dependency of dependencies) fs.copyFileSync(dependency, path.join(output, path.basename(dependency)));
if (process.platform === 'linux') {
for (const file of fs.readdirSync(output)) run('patchelf', ['--set-rpath', '$ORIGIN', path.join(output, file)]);
}
}
copyNotices(dependencies);
fs.writeFileSync(path.join(output, 'manifest.json'), JSON.stringify({
platform: process.platform,
arch: process.arch,
napi: 8,
libraries: fs.readdirSync(output).filter(file => /\.(node|dll|dylib)$|\.so(\.|$)/.test(file)).sort(),
}, null, 2) + '\n');
fs.mkdirSync(path.join(root, 'dist'), { recursive: true });
console.log(`Bundled ${target}: ${output}`);
} finally {
fs.rmSync(temporary, { recursive: true, force: true });
}
+39
View File
@@ -0,0 +1,39 @@
cmake_minimum_required(VERSION 3.21)
if(NOT EXISTS "${ADDON}" OR NOT DEFINED OUTPUT)
message(FATAL_ERROR "ADDON and OUTPUT are required")
endif()
set(pre_exclude)
set(post_exclude)
if(CMAKE_HOST_SYSTEM_NAME STREQUAL "Darwin")
list(APPEND pre_exclude "^/usr/lib/" "^/System/Library/")
list(APPEND post_exclude "^/usr/lib/" "^/System/Library/")
elseif(CMAKE_HOST_SYSTEM_NAME STREQUAL "Linux")
list(APPEND pre_exclude "^ld-linux" "^lib(c|m|dl|rt|pthread|resolv|util|gcc_s|stdc\\+\\+)\\.so")
elseif(CMAKE_HOST_WIN32)
list(APPEND pre_exclude "[Nn][Oo][Dd][Ee]\\.exe" "[Aa][Pp][Ii]-[Mm][Ss]-.*" "[Ee][Xx][Tt]-[Mm][Ss]-.*")
list(APPEND post_exclude ".*[Ss][Yy][Ss][Tt][Ee][Mm]32[/\\\\].*")
endif()
file(GET_RUNTIME_DEPENDENCIES
MODULES "${ADDON}"
DIRECTORIES ${SEARCH_DIRECTORIES}
RESOLVED_DEPENDENCIES_VAR dependencies
UNRESOLVED_DEPENDENCIES_VAR unresolved
CONFLICTING_DEPENDENCIES_PREFIX conflicts
PRE_EXCLUDE_REGEXES ${pre_exclude}
POST_EXCLUDE_REGEXES ${post_exclude}
)
if(unresolved OR conflicts_FILENAMES)
message(FATAL_ERROR "Unresolved dependencies: ${unresolved}; conflicting dependencies: ${conflicts_FILENAMES}")
endif()
set(result "[]")
set(index 0)
foreach(dependency IN LISTS dependencies)
file(TO_CMAKE_PATH "${dependency}" dependency)
string(JSON result SET "${result}" ${index} "\"${dependency}\"")
math(EXPR index "${index} + 1")
endforeach()
file(WRITE "${OUTPUT}" "${result}\n")
+38
View File
@@ -0,0 +1,38 @@
import { execFileSync } from 'node:child_process';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
const root = path.resolve(import.meta.dirname, '..');
const npm = process.env.npm_execpath;
if (!npm) throw new Error('Run this check with npm run test:package');
fs.mkdirSync(path.join(root, 'dist'), { recursive: true });
let archive = process.argv[2] ? path.resolve(process.argv[2]) : path.join(root, 'dist',
JSON.parse(execFileSync(process.execPath, [npm, 'pack', '--json', '--pack-destination', 'dist'],
{ cwd: root, encoding: 'utf8' }))[0].filename);
if (fs.statSync(archive).isDirectory()) {
const archives = fs.readdirSync(archive).filter(file => file.endsWith('.tgz'));
if (archives.length !== 1) throw new Error('Expected exactly one npm archive');
archive = path.join(archive, archives[0]);
}
const consumer = fs.mkdtempSync(path.join(os.tmpdir(), 'opendht-consumer-'));
const env = { ...process.env, PREBUILDS_ONLY: '1' };
for (const variable of ['OPENDHT_NODE_BINDING', 'NODE_PATH', 'LD_LIBRARY_PATH', 'DYLD_LIBRARY_PATH', 'DYLD_FALLBACK_LIBRARY_PATH']) delete env[variable];
const excluded = [path.join(root, 'build'), ...(process.env.OPENDHT_LIBRARY_PATH || '').split(path.delimiter).filter(Boolean)];
const pathKey = Object.keys(env).find(key => key.toLowerCase() === 'path');
if (pathKey) env[pathKey] = env[pathKey].split(path.delimiter)
.filter(entry => !excluded.some(directory => path.resolve(entry).toLowerCase().startsWith(path.resolve(directory).toLowerCase())))
.join(path.delimiter);
try {
execFileSync(process.execPath, [npm, 'install', '--prefix', consumer, '--ignore-scripts', '--omit=dev', '--no-audit', '--no-fund', archive],
{ cwd: consumer, env, stdio: 'inherit' });
const { name } = JSON.parse(fs.readFileSync(path.join(root, 'package.json'), 'utf8'));
const installed = path.join(consumer, 'node_modules', ...name.split('/'));
execFileSync(process.execPath, ['--input-type=module', '-e', 'import load from "node-gyp-build"; console.log(load.path(process.cwd()))'],
{ cwd: installed, env, stdio: 'inherit' });
fs.cpSync(path.join(root, 'test'), path.join(installed, 'test'), { recursive: true });
execFileSync(process.execPath, [npm, 'test'], { cwd: installed, env, stdio: 'inherit' });
console.log(`Installed archive verified: ${archive}`);
} finally {
fs.rmSync(consumer, { recursive: true, force: true });
}
+549
View File
@@ -0,0 +1,549 @@
import assert from 'node:assert/strict';
import { execFile } from 'node:child_process';
import { EventEmitter, once } from 'node:events';
import { createServer } from 'node:http';
import { test } from 'node:test';
import { fileURLToPath } from 'node:url';
import { promisify } from 'node:util';
import { Worker } from 'node:worker_threads';
import { DhtRunner, hash, version } from '../index.js';
const moduleURL = new URL('../index.js', import.meta.url).href;
const exampleURL = new URL('../examples/dhtnode.js', import.meta.url);
const example = fileURLToPath(exampleURL);
function startNode() {
const runner = new DhtRunner();
runner.run({ port: 0, ipv4: '127.0.0.1', ipv6: '::1' });
return runner;
}
test('hashes strings and binary data without truncating NUL bytes', () => {
assert.match(version, /^\d+\./);
assert.equal(hash('hello'), 'aaf4c61ddcc5e8a2dabede0f3b482cd9aea9434d');
assert.equal(hash('hello'), hash(Buffer.from('hello')));
assert.notEqual(hash('hello\0world'), hash('hello'));
assert.throws(() => hash(42), TypeError);
});
test('runner lifecycle and argument validation', async () => {
const runner = new DhtRunner();
assert.equal(runner.isRunning(), false);
assert.throws(() => runner.get(hash('idle')), /running/i);
assert.throws(() => runner.getAll(hash('idle')), /running/i);
assert.throws(() => runner.run({ port: -1 }), RangeError);
assert.throws(() => runner.run({ port: 65536 }), RangeError);
assert.throws(() => runner.run({ port: 1.5 }), RangeError);
runner.run({ port: 0, ipv4: '127.0.0.1', ipv6: '::1' });
try {
assert.equal(runner.isRunning(), true);
assert.ok(runner.getBoundPort() > 0);
assert.match(runner.getNodeId(), /^[0-9a-f]{40}$/);
assert.throws(() => runner.run(), /already/i);
assert.throws(() => runner.get('short'), /hash/i);
assert.throws(() => runner.get('z'.repeat(40)), /hash/i);
assert.throws(() => runner.put(hash('bad'), 'text'), TypeError);
assert.throws(() => runner.put(hash('bad'), Buffer.alloc(0), { id: 1 }), TypeError);
assert.throws(() => runner.put(hash('bad'), Buffer.alloc(0), { id: -1n }), RangeError);
assert.throws(() => runner.put(hash('bad'), Buffer.alloc(0), { id: 1n << 64n }), RangeError);
assert.throws(() => runner.put(hash('bad'), Buffer.alloc(0), { type: 65536 }), RangeError);
} finally {
const closing = runner.close();
assert.equal(runner.close(), closing);
await closing;
}
assert.equal(runner.isRunning(), false);
assert.throws(() => runner.run(), /closed/i);
assert.throws(() => runner.get(hash('closed')), /closed/i);
});
test('closing an idle runner is supported', async () => {
const runner = new DhtRunner();
await runner.close();
await runner.close();
assert.equal(runner.isRunning(), false);
});
test('runner reports per-family connectivity changes and snapshots', { timeout: 15000 }, async () => {
const runner = new DhtRunner();
const peer = startNode();
const changes = [];
runner.on('statusChanged', status => changes.push(status));
try {
assert.deepEqual(runner.getStatus(), { ipv4: 'disconnected', ipv6: 'disconnected' });
runner.run({ port: 0, ipv4: '127.0.0.1', ipv6: '::1' });
const connected = new Promise((resolve, reject) => {
const signal = AbortSignal.timeout(10000);
signal.addEventListener('abort', () => reject(signal.reason), { once: true });
runner.on('statusChanged', status => {
if (status.ipv4 === 'connected') resolve(status);
});
});
runner.bootstrap('127.0.0.1', peer.getBoundPort());
peer.bootstrap('127.0.0.1', runner.getBoundPort());
const status = await connected;
assert.equal(status.ipv4, 'connected');
assert.ok(['disconnected', 'connecting', 'connected'].includes(status.ipv6));
assert.deepEqual(runner.getStatus(), status);
status.ipv4 = 'changed by consumer';
assert.equal(runner.getStatus().ipv4, 'connected');
await runner.close();
const count = changes.length;
await new Promise(resolve => setImmediate(resolve));
assert.equal(changes.length, count);
assert.deepEqual(runner.getStatus(), { ipv4: 'disconnected', ipv6: 'disconnected' });
} finally {
await Promise.all([runner.close(), peer.close()]);
}
});
test('invalid bind addresses produce JS exceptions', async () => {
const runner = new DhtRunner();
try {
assert.throws(() => runner.run({ ipv4: '192.0.2.1', ipv6: '2001:db8::1' }));
} finally {
await runner.close();
}
});
test('two loopback nodes exchange binary values and cancel listeners', { timeout: 20000 }, async () => {
const writer = startNode();
const reader = startNode();
const events = new EventEmitter();
const key = hash('node-addon-api integration');
const payload = Buffer.from([0, 1, 255, 0, 42]);
let subscription;
let retainedValue;
try {
writer.bootstrap('127.0.0.1', reader.getBoundPort());
reader.bootstrap('127.0.0.1', writer.getBoundPort());
subscription = reader.listen(key, (value, expired) => {
if (!expired) events.emit('value', value);
});
const received = once(events, 'value', { signal: AbortSignal.timeout(10000) });
assert.equal(await writer.put(key, payload, {
id: 18446744073709551614n,
type: 0,
userType: 'nodejs-test',
permanent: true,
}), true);
const [value] = await received;
assert.equal(value.id, 18446744073709551614n);
assert.equal(value.type, 0);
assert.ok(Buffer.isBuffer(value.data));
assert.deepEqual(value.data, payload);
assert.equal(value.userType, 'nodejs-test');
assert.equal(typeof value.seq, 'number');
assert.equal(value.owner, null);
assert.equal(value.isSigned, false);
assert.equal(value.isEncrypted, false);
value.data.fill(7);
const values = await reader.getAll(key);
retainedValue = values.find(candidate => candidate.id === value.id);
assert.ok(retainedValue);
assert.deepEqual(retainedValue.data, payload);
subscription.cancel();
subscription.cancel();
} finally {
subscription?.cancel();
await Promise.all([writer.close(), reader.close()]);
}
assert.deepEqual(retainedValue.data, payload);
});
test('close settles pending operations and suppresses queued listener deliveries', { timeout: 10000 }, async () => {
const runner = startNode();
let deliveries = 0;
const subscription = runner.listen(hash('closing'), () => deliveries++);
const pending = runner.getAll(hash('unreachable'));
const settled = assert.rejects(pending, { code: 'ERR_DHT_CLOSED' });
await runner.close();
await settled;
subscription.cancel();
assert.equal(deliveries, 0);
});
test('get iterators settle pending reads on cancellation and reject on runner close', { timeout: 10000 }, async () => {
const runner = startNode();
try {
const cancelled = runner.get(hash('cancel get'));
assert.equal(cancelled[Symbol.asyncIterator](), cancelled);
const pending = [cancelled.next(), cancelled.next()];
await cancelled.return();
cancelled.cancel();
assert.deepEqual(await Promise.all(pending), [
{ value: undefined, done: true },
{ value: undefined, done: true },
]);
assert.deepEqual(await cancelled.next(), { value: undefined, done: true });
const closing = runner.get(hash('close get'));
const rejected = assert.rejects(closing.next(), { code: 'ERR_DHT_CLOSED' });
await runner.close();
await rejected;
await assert.rejects(closing.next(), { code: 'ERR_DHT_CLOSED' });
} finally {
await runner.close();
}
});
test('async listeners deliver queued values and cancel on loop exit', { timeout: 15000 }, async () => {
const writer = startNode();
const reader = startNode();
const key = hash('async listener');
let iterator;
try {
writer.bootstrap('127.0.0.1', reader.getBoundPort());
reader.bootstrap('127.0.0.1', writer.getBoundPort());
iterator = reader.listen(key);
assert.equal(iterator[Symbol.asyncIterator](), iterator);
const pending = iterator.next();
assert.equal(await writer.put(key, Buffer.from('first'), { id: 1n }), true);
const first = await pending;
assert.equal(first.done, false);
assert.equal(first.value.expired, false);
assert.equal(first.value.value.id, 1n);
assert.equal(first.value.value.data.toString(), 'first');
assert.equal(await writer.put(key, Buffer.from('second'), { id: 2n }), true);
for await (const { value, expired } of iterator) {
assert.equal(expired, false);
assert.equal(value.id, 2n);
assert.equal(value.data.toString(), 'second');
break;
}
assert.deepEqual(await iterator.next(), { value: undefined, done: true });
iterator.cancel();
} finally {
iterator?.cancel();
await Promise.all([writer.close(), reader.close()]);
}
});
test('async listener cancellation settles concurrent reads and drops buffered values', { timeout: 10000 }, async () => {
const runner = startNode();
const peer = startNode();
const key = hash('async cancellation');
let iterator;
try {
runner.bootstrap('127.0.0.1', peer.getBoundPort());
peer.bootstrap('127.0.0.1', runner.getBoundPort());
iterator = runner.listen(key);
const pending = [iterator.next(), iterator.next()];
iterator.cancel();
iterator.cancel();
assert.deepEqual(await Promise.all(pending), [
{ value: undefined, done: true },
{ value: undefined, done: true },
]);
assert.equal(await runner.put(key, Buffer.from('cancelled'), { id: 3n }), true);
assert.deepEqual(await iterator.next(), { value: undefined, done: true });
const buffered = runner.listen(key);
await runner.getAll(key);
await buffered.return();
assert.deepEqual(await buffered.next(), { value: undefined, done: true });
} finally {
iterator?.cancel();
await Promise.all([runner.close(), peer.close()]);
}
});
test('async listener throw, disposal, and runner close settle pending reads', { timeout: 10000 }, async () => {
const runner = startNode();
try {
const thrown = runner.listen(hash('async throw'));
const pendingThrow = thrown.next();
const failure = new Error('consumer failed');
await assert.rejects(thrown.throw(failure), error => error === failure);
assert.deepEqual(await pendingThrow, { value: undefined, done: true });
assert.deepEqual(await thrown.next(), { value: undefined, done: true });
const disposed = runner.listen(hash('async dispose'));
const pendingDispose = disposed.next();
await disposed[Symbol.asyncDispose]();
assert.deepEqual(await pendingDispose, { value: undefined, done: true });
const closing = runner.listen(hash('async close'));
const pendingClose = [closing.next(), closing.next()];
const close = runner.close();
assert.equal(runner.close(), close);
assert.deepEqual(await Promise.all(pendingClose), [
{ value: undefined, done: true },
{ value: undefined, done: true },
]);
await close;
assert.deepEqual(await closing.next(), { value: undefined, done: true });
assert.throws(() => runner.listen(hash('closed iterator')), /closed/i);
} finally {
await runner.close();
}
});
test('proxy configuration validates option types before starting', async () => {
const runner = new DhtRunner();
try {
assert.throws(() => runner.run({ proxyServer: 42 }), TypeError);
assert.throws(() => runner.run({ proxyUserAgent: false }), TypeError);
assert.throws(() => runner.run({ proxyUserAgent: 'client\r\nInjected: header' }), TypeError);
assert.throws(() => runner.run({ proxyServer: 'http://127.0.0.1\0' }), TypeError);
assert.throws(() => runner.run({ proxyServer: 'ftp://127.0.0.1' }), /http/i);
assert.throws(() => runner.run({ proxyServer: 'not a URL' }), /url/i);
assert.equal(runner.isRunning(), false);
runner.run({ port: 0, ipv4: '127.0.0.1', ipv6: '::1', proxyServer: '' });
assert.ok(runner.getBoundPort() > 0);
} finally {
await runner.close();
}
});
test('proxy configuration routes operations through HTTP with a custom user agent', { timeout: 15000 }, async () => {
const runner = new DhtRunner();
const requests = [];
const values = new Map();
let streamNextGet = false;
let streamResponse;
const server = createServer(async (request, response) => {
response.setHeader('content-type', 'application/json');
if (request.url === '/') {
response.end(JSON.stringify({
node_id: hash('proxy fixture'),
ipv4: { good: 1 },
ipv6: {},
public_ip: '127.0.0.1',
}));
return;
}
requests.push({ method: request.method, url: request.url, userAgent: request.headers['user-agent'] });
if (request.url === `/key/${hash('proxy failure')}`) {
response.statusCode = 500;
response.end();
return;
}
if (request.method === 'POST') {
const chunks = [];
for await (const chunk of request) chunks.push(chunk);
const value = JSON.parse(Buffer.concat(chunks).toString());
values.set(request.url, value);
response.end(JSON.stringify(value));
} else {
const value = values.get(request.url);
const body = value ? `${JSON.stringify(value)}\n` : '';
if (streamNextGet) {
streamNextGet = false;
streamResponse = response;
response.write(body);
} else response.end(body);
}
});
server.listen(0, '127.0.0.1');
try {
await once(server, 'listening');
runner.run({
proxyServer: `http://127.0.0.1:${server.address().port}`,
proxyUserAgent: 'opendht-node-test/1.0',
persistPath: '',
});
const key = hash('proxy configuration');
const payload = Buffer.from([0, 255, 42]);
assert.equal(await runner.put(key, payload, { id: 42n }), true);
const received = await runner.getAll(key);
assert.equal(received.length, 1);
assert.equal(received[0].id, 42n);
assert.deepEqual(received[0].data, payload);
streamNextGet = true;
const iterator = runner.get(key);
const first = await iterator.next();
assert.equal(first.done, false);
assert.equal(first.value.id, 42n);
assert.deepEqual(first.value.data, payload);
assert.equal(streamResponse.writableEnded, false);
streamResponse.end();
assert.deepEqual(await iterator.next(), { value: undefined, done: true });
assert.deepEqual(await iterator.next(), { value: undefined, done: true });
assert.deepEqual(requests.map(request => request.method), ['POST', 'GET', 'GET']);
assert.ok(requests.every(request => request.url === `/key/${key}`));
assert.ok(requests.every(request => request.userAgent === 'opendht-node-test/1.0'));
const empty = runner.get(hash('proxy empty'));
assert.deepEqual(await empty.next(), { value: undefined, done: true });
const failed = runner.get(hash('proxy failure'));
await assert.rejects(failed.next(), { code: 'ERR_DHT_GET_FAILED' });
await assert.rejects(runner.getAll(hash('proxy failure')), { code: 'ERR_DHT_GET_FAILED' });
const all = [];
for await (const value of runner.get(key)) all.push(value);
assert.equal(all.length, 1);
assert.deepEqual(all[0].data, payload);
streamNextGet = true;
const cancelled = runner.get(key);
for await (const value of cancelled) {
assert.equal(value.id, 42n);
break;
}
assert.equal(streamResponse.writableEnded, false);
assert.deepEqual(await cancelled.next(), { value: undefined, done: true });
streamResponse.end();
} finally {
streamResponse?.end();
await runner.close();
await new Promise((resolve, reject) => server.close(error => error ? reject(error) : resolve()));
}
});
test('worker termination cleans up native runners', { timeout: 10000 }, async () => {
const source = `
import { parentPort } from 'node:worker_threads';
import { DhtRunner, hash } from ${JSON.stringify(moduleURL)};
const runner = new DhtRunner();
runner.run({ port: 0, ipv4: '127.0.0.1', ipv6: '::1' });
runner.listen(hash('worker'), () => {});
parentPort.postMessage('ready');
`;
const worker = new Worker(new URL(`data:text/javascript,${encodeURIComponent(source)}`));
try {
await once(worker, 'message');
} finally {
await worker.terminate();
}
});
test('listener callbacks may cancel themselves and close the runner', { timeout: 10000 }, async () => {
const runner = startNode();
const key = hash('reentrant listener');
let deliveries = 0;
let finish;
const received = new Promise(resolve => { finish = resolve; });
const subscription = runner.listen(key, () => {
deliveries++;
subscription.cancel();
finish(runner.close());
});
const puts = [];
for (let index = 0; index < 3000; index++) {
puts.push(runner.put(key, Buffer.from('value'), { id: BigInt(index + 1) }).catch(error => {
assert.equal(error.code, 'ERR_DHT_CLOSED');
}));
}
try {
await received;
await Promise.all(puts);
assert.equal(deliveries, 1);
} finally {
await runner.close();
}
});
test('unref and garbage collection do not strand native threads', { timeout: 20000 }, async () => {
for (const finish of ['runner.unref();', 'runner = null; setImmediate(() => global.gc());']) {
await promisify(execFile)(process.execPath, ['--input-type=module', '--expose-gc', '-e', `
import { DhtRunner } from ${JSON.stringify(moduleURL)};
let runner = new DhtRunner();
runner.run({ port: 0, ipv4: '127.0.0.1', ipv6: '::1' });
${finish}
`], { timeout: 8000 });
}
});
test('callback queue completes bursts beyond the former event limit', { timeout: 15000 }, async () => {
const runner = startNode();
const key = hash('unbounded callback queue');
const errors = [];
const count = 3000;
runner.on('error', error => errors.push(error));
const puts = [];
try {
for (let index = 0; index < count; index++)
puts.push(runner.put(key, Buffer.from('value'), { id: BigInt(index + 1) }));
const results = await Promise.all(puts);
assert.equal(results.length, count);
assert.ok(results.every(result => typeof result === 'boolean'));
assert.equal(runner.isRunning(), true);
assert.deepEqual(errors, []);
} finally {
await runner.close();
}
});
test('dhtnode example prints help and validates its port', async () => {
const { stdout } = await promisify(execFile)(process.execPath, [example, '--help']);
assert.match(stdout, /Usage:.*dhtnode/);
assert.match(stdout, /bootstrap/);
await assert.rejects(promisify(execFile)(process.execPath, [example, '--port', '65536']), error => {
assert.equal(error.code, 1);
assert.match(error.stderr, /port/i);
return true;
});
});
test('dhtnode example accepts piped commands and shuts down on EOF', { timeout: 20000 }, async () => {
const peer = startNode();
try {
const execution = promisify(execFile)(process.execPath, [example,
'--port', '0', '--ipv4', '127.0.0.1',
'--bootstrap', '127.0.0.1', '--bootstrap-port', String(peer.getBoundPort()),
], { timeout: 15000 });
let output = '';
let sentReads = false;
execution.child.stdout.on('data', data => {
output += data;
if (!sentReads && /Stored example/.test(output) && /Stored signed/.test(output)) {
sentReads = true;
execution.child.stdin.end('get example\nget signed\ncancel example\n');
}
});
execution.child.stdin.write('listen example\nput example hello world\nsign signed signed message\n');
const { stdout, stderr } = await execution;
assert.equal(stderr, '');
assert.match(stdout, /Node [0-9a-f]{40}/);
assert.match(stdout, /hello world/);
assert.match(stdout, /signed message.*\[signed\]/);
assert.match(stdout, /Cancelled example/);
assert.match(stdout, /Connectivity: IPv4 connected, IPv6 (disconnected|connecting|connected)/);
} finally {
await peer.close();
}
});
test('dhtnode example handles commands and quit while a get is pending', { timeout: 10000 }, async () => {
const execution = promisify(execFile)(process.execPath, ['--input-type=module', '-e', `
import { DhtRunner } from ${JSON.stringify(moduleURL)};
const close = DhtRunner.prototype.close;
let rejectGet;
DhtRunner.prototype.getAll = function () {
console.log('Get pending');
return new Promise((resolve, reject) => { rejectGet = reject; });
};
DhtRunner.prototype.close = function () {
rejectGet?.(new Error('DHT runner is closed'));
return close.call(this);
};
await import(${JSON.stringify(exampleURL.href)});
`, '--',
'--port', '0', '--ipv4', '127.0.0.1', '--ipv6', '::1',
], { timeout: 8000 });
let output = '';
execution.child.stdout.on('data', data => {
output += data;
if (/get KEY \| put KEY TEXT/.test(output)) execution.child.stdin.end('quit\n');
});
execution.child.stdin.write('get waiting\nhelp\n');
const { stdout, stderr } = await execution;
assert.match(stdout, /Get pending/);
assert.match(stdout, /get KEY \| put KEY TEXT/);
assert.equal(stderr, '');
});
test('dhtnode example shuts down on SIGTERM', { timeout: 10000 }, async () => {
if (process.platform === 'win32') return;
const execution = promisify(execFile)(process.execPath, [example, '--port', '0', '--ipv4', '127.0.0.1'], { timeout: 8000 });
let output = '';
execution.child.stdout.on('data', data => {
output += data;
if (/Node [0-9a-f]{40}/.test(output)) execution.child.kill('SIGTERM');
});
const { stderr } = await execution;
assert.equal(stderr, '');
});
+93
View File
@@ -0,0 +1,93 @@
import assert from 'node:assert/strict';
import { test } from 'node:test';
import { Certificate, DhtRunner, Identity, PrivateKey, PublicKey, hash } from '../index.js';
const payload = Buffer.from([0, 255, 1, 0, 42]);
test('RSA keys sign, encrypt, and survive password-protected serialization', async () => {
const key = await PrivateKey.generate({ algorithm: 'rsa', bits: 2048 });
const publicKey = key.getPublicKey();
assert.match(publicKey.getId(), /^[0-9a-f]{40}$/);
assert.match(publicKey.getLongId(), /^[0-9a-f]{64}$/);
const signature = key.sign(payload);
assert.equal(publicKey.checkSignature(payload, signature), true);
assert.equal(publicKey.checkSignature(Buffer.from('other'), signature), false);
assert.deepEqual(key.decrypt(publicKey.encrypt(payload)), payload);
const restored = new PrivateKey(key.serialize('test password'), 'test password');
const restoredPublic = new PublicKey(publicKey.serialize());
assert.equal(restoredPublic.getLongId(), publicKey.getLongId());
assert.equal(restoredPublic.checkSignature(payload, restored.sign(payload)), true);
assert.deepEqual(restored.decrypt(restoredPublic.encrypt(payload)), payload);
assert.throws(() => new PrivateKey(key.serialize('password'), 'wrong password'));
assert.throws(() => key.decrypt(Buffer.from('invalid ciphertext')));
assert.throws(() => key.serialize('bad\0password'), TypeError);
});
test('EC keys sign and invalid crypto inputs raise JS errors', async () => {
const key = await PrivateKey.generate();
const restored = new PrivateKey(key.serialize());
assert.equal(restored.getPublicKey().checkSignature(payload, key.sign(payload)), true);
const largePayload = Buffer.alloc(16 * 1024 * 1024 + 1, 42);
assert.equal(restored.getPublicKey().checkSignature(largePayload, key.sign(largePayload)), true);
assert.throws(() => key.getPublicKey().encrypt(payload));
for (const Type of [PrivateKey, PublicKey, Certificate]) {
assert.throws(() => new Type(Buffer.alloc(0)));
assert.throws(() => new Type(Buffer.from('not a key or certificate')));
assert.throws(() => new Type(null), TypeError);
}
assert.throws(() => PrivateKey.generate({ algorithm: 'unknown' }), TypeError);
assert.throws(() => PrivateKey.generate({ algorithm: 'rsa', bits: 1 }), RangeError);
assert.throws(() => PrivateKey.generate({ algorithm: 'rsa', bits: 2048.5 }), RangeError);
assert.throws(() => key.sign('not bytes'), TypeError);
assert.throws(() => PrivateKey.prototype.sign.call({}, payload), TypeError);
});
test('identities own matching keys and certificates and support import', async () => {
const identity = await Identity.generate('nodejs crypto');
const certificate = new Certificate(identity.certificate.serialize());
const key = new PrivateKey(identity.privateKey.serialize());
const restored = new Identity(key, certificate);
assert.equal(restored.certificate.getLongId(), key.getPublicKey().getLongId());
assert.equal(certificate.getPublicKey().checkSignature(payload, key.sign(payload)), true);
assert.throws(() => new Identity({}, certificate), TypeError);
const other = await PrivateKey.generate();
assert.throws(() => new Identity(other, certificate), /match/i);
});
test('signed and encrypted values round-trip between authenticated nodes', { timeout: 30000 }, async () => {
const writerIdentity = await Identity.generate('writer');
const readerIdentity = await Identity.generate('reader', { algorithm: 'rsa', bits: 2048 });
const writer = new DhtRunner();
const reader = new DhtRunner();
try {
writer.run({ port: 0, ipv4: '127.0.0.1', ipv6: '::1', identity: writerIdentity });
reader.run({ port: 0, ipv4: '127.0.0.1', ipv6: '::1', identity: readerIdentity });
writer.bootstrap('127.0.0.1', reader.getBoundPort());
reader.bootstrap('127.0.0.1', writer.getBoundPort());
const signedKey = hash('nodejs signed value');
assert.equal(await writer.putSigned(signedKey, payload, { id: 41n }), true);
const signed = await reader.getAll(signedKey);
assert.ok(signed.some(value => value.data.equals(payload) && value.isSigned));
assert.equal(signed[0].owner, writerIdentity.certificate.getLongId());
const encryptedKey = hash('nodejs encrypted value');
assert.equal(await writer.putEncrypted(encryptedKey, payload, readerIdentity.certificate.getPublicKey(), { id: 42n }), true);
const decrypted = await reader.getAll(encryptedKey);
assert.ok(decrypted.some(value => value.data.equals(payload)));
assert.throws(() => writer.putEncrypted(encryptedKey, payload, {}), TypeError);
assert.throws(() => writer.putEncrypted(encryptedKey, payload, writerIdentity.certificate.getPublicKey()), /RSA/i);
} finally {
await Promise.all([writer.close(), reader.close()]);
}
});
test('signed operations require an identity and reject stopped runners', async () => {
const runner = new DhtRunner();
try {
assert.throws(() => runner.run({ identity: {} }), TypeError);
runner.run({ port: 0, ipv4: '127.0.0.1', ipv6: '::1' });
assert.throws(() => runner.putSigned(hash('unsigned'), payload), /identity/i);
} finally {
await runner.close();
}
assert.throws(() => runner.putSigned(hash('closed'), payload), /closed/i);
});
+34
View File
@@ -0,0 +1,34 @@
import assert from 'node:assert/strict';
import { execFileSync } from 'node:child_process';
import { readFileSync } from 'node:fs';
import path from 'node:path';
import { test } from 'node:test';
const root = path.resolve(import.meta.dirname, '..');
test('package exposes ESM-only named exports', () => {
const manifest = JSON.parse(readFileSync(new URL('../package.json', import.meta.url), 'utf8'));
assert.equal(manifest.type, 'module');
assert.equal(manifest.exports['.'].import, './index.js');
assert.equal(manifest.exports['.'].require, undefined);
const stdout = execFileSync(process.execPath, ['--input-type=module', '-e', `
import { DhtRunner, hash } from ${JSON.stringify(manifest.name)};
console.log(hash('hello'));
const runner = new DhtRunner();
await runner.close();
`], { cwd: root, encoding: 'utf8' });
assert.equal(stdout.trim(), 'aaf4c61ddcc5e8a2dabede0f3b482cd9aea9434d');
});
test('npm archive contains runtime files and excludes development artifacts', () => {
const npm = process.env.npm_execpath;
assert.ok(npm, 'Run the package contract with npm test');
const [archive] = JSON.parse(execFileSync(process.execPath,
[npm, 'pack', '--dry-run', '--json', '--ignore-scripts'],
{ cwd: root, encoding: 'utf8' }));
const files = archive.files.map(file => file.path);
for (const file of ['index.js', 'index.d.ts', 'crypto.js', 'README.md', 'examples/dhtnode.js', 'examples/dhtnode.ts']) assert.ok(files.includes(file), file);
assert.equal(files.some(file => /^(build[^/]*|node_modules|test|scripts)\//.test(file)), false);
assert.equal(files.some(file => /\.(cpp|h|tsbuildinfo)$/.test(file)), false);
assert.ok(files.every(file => /^(index\.js|index\.d\.ts|crypto\.js|README\.md|package\.json|examples\/dhtnode\.(js|ts)|prebuilds\/)/.test(file)));
});
+56
View File
@@ -0,0 +1,56 @@
import { Certificate, DhtRunner, Identity, PrivateKey, PublicKey, hash, version } from '../index.js';
import type { ConnectivityStatus, ListenEvent, ListenerIterator, NodeStatus, Value, ValueIterator } from '../index.js';
async function exercise(): Promise<void> {
const privateKey: PrivateKey = await PrivateKey.generate({ algorithm: 'rsa', bits: 2048 });
const publicKey: PublicKey = new PublicKey(privateKey.getPublicKey().serialize());
const identity: Identity = await Identity.generate('typed identity');
const restored = new Identity(identity.privateKey, new Certificate(identity.certificate.serialize()));
const valid: boolean = publicKey.checkSignature(Buffer.alloc(0), privateKey.sign(Buffer.alloc(0)));
const plaintext: Buffer = privateKey.decrypt(publicKey.encrypt(Buffer.from('secret')));
const runner = new DhtRunner();
runner.on('error', (error: Error) => { throw error; });
runner.on('statusChanged', status => {
const ipv4: NodeStatus = status.ipv4;
const ipv6: NodeStatus = status.ipv6;
void [ipv4, ipv6];
});
const status: ConnectivityStatus = runner.getStatus();
runner.run({ port: 0, network: 1, persistPath: '', identity: restored });
runner.bootstrap('127.0.0.1', runner.getBoundPort(4));
const key: string = hash(new Uint8Array([0, 255]));
const accepted: boolean = await runner.put(key, Buffer.from(version), { id: 42n, permanent: true });
const signed: boolean = await runner.putSigned(key, plaintext);
const encrypted: boolean = await runner.putEncrypted(key, plaintext, publicKey);
const values: Value[] = await runner.getAll(key);
const get: ValueIterator<Value> = runner.get(key);
for await (const value of get) {
const payload: Buffer = value.data;
void payload;
}
const listener = runner.listen(key, (value, expired) => {
const payload: Buffer = value.data;
const id: bigint = value.id;
void [payload, id, expired];
});
runner.ref().unref();
listener.cancel();
const iterator: ListenerIterator = runner.listen(key);
for await (const event of iterator) {
const typed: ListenEvent = event;
const value: Value = typed.value;
const expired: boolean = typed.expired;
void [value, expired];
break;
}
iterator.cancel();
await iterator.return();
await iterator[Symbol.asyncDispose]();
await runner[Symbol.asyncDispose]();
const proxy = new DhtRunner();
proxy.run({ proxyServer: 'https://proxy.example.com', proxyUserAgent: 'typed-client/1.0' });
await proxy.close();
void [accepted, values, valid, signed, encrypted, status];
}
void exercise;
@@ -0,0 +1,118 @@
diff --git a/SMP/libgnutls.vcxproj.filters b/SMP/libgnutls.vcxproj.filters
index b182189..91ee160 100644
--- a/SMP/libgnutls.vcxproj.filters
+++ b/SMP/libgnutls.vcxproj.filters
@@ -103,9 +103,6 @@
<Filter Include="Header Files\lib\inih">
<UniqueIdentifier>{ae0c3eeb-53df-4c72-a85a-6b46de35e7ba}</UniqueIdentifier>
</Filter>
- <Filter Include="Source Files\libtasn1">
- <UniqueIdentifier>{32be60b7-8c65-486e-9df5-7e529443cf07}</UniqueIdentifier>
- </Filter>
<Filter Include="Source Files\lib\nettle\gost">
<UniqueIdentifier>{1f3549a8-3c3d-475f-8fd6-20451d336464}</UniqueIdentifier>
</Filter>
@@ -1553,30 +1550,6 @@
<ClCompile Include="..\lib\nettle\sysrng-bcrypt.c">
<Filter>Source Files\lib\nettle</Filter>
</ClCompile>
- <ClCompile Include="..\devel\libtasn1\lib\coding.c">
- <Filter>Source Files\libtasn1</Filter>
- </ClCompile>
- <ClCompile Include="..\devel\libtasn1\lib\decoding.c">
- <Filter>Source Files\libtasn1</Filter>
- </ClCompile>
- <ClCompile Include="..\devel\libtasn1\lib\element.c">
- <Filter>Source Files\libtasn1</Filter>
- </ClCompile>
- <ClCompile Include="..\devel\libtasn1\lib\errors.c">
- <Filter>Source Files\libtasn1</Filter>
- </ClCompile>
- <ClCompile Include="..\devel\libtasn1\lib\gstr.c">
- <Filter>Source Files\libtasn1</Filter>
- </ClCompile>
- <ClCompile Include="..\devel\libtasn1\lib\parser_aux.c">
- <Filter>Source Files\libtasn1</Filter>
- </ClCompile>
- <ClCompile Include="..\devel\libtasn1\lib\structure.c">
- <Filter>Source Files\libtasn1</Filter>
- </ClCompile>
- <ClCompile Include="..\devel\libtasn1\lib\version.c">
- <Filter>Source Files\libtasn1</Filter>
- </ClCompile>
<ClCompile Include="..\lib\nettle\gost\acpkm.c">
<Filter>Source Files\lib\nettle\gost</Filter>
</ClCompile>
diff --git a/SMP/libgnutls_files.props b/SMP/libgnutls_files.props
index adcb074..2317c2a 100644
--- a/SMP/libgnutls_files.props
+++ b/SMP/libgnutls_files.props
@@ -179,13 +179,6 @@
<ItemGroup>
<ClCompile Include="lib\gnutls_asn1_tab.c" />
<ClCompile Include="lib\pkix_asn1_tab.c" />
- <ClCompile Include="..\devel\libtasn1\lib\coding.c" />
- <ClCompile Include="..\devel\libtasn1\lib\decoding.c" />
- <ClCompile Include="..\devel\libtasn1\lib\element.c" />
- <ClCompile Include="..\devel\libtasn1\lib\gstr.c" />
- <ClCompile Include="..\devel\libtasn1\lib\parser_aux.c" />
- <ClCompile Include="..\devel\libtasn1\lib\structure.c" />
- <ClCompile Include="..\devel\libtasn1\lib\version.c" />
<ClCompile Include="..\gnulib\lib\c-strcasecmp.c" />
<ClCompile Include="..\gnulib\lib\c-strncasecmp.c" />
<ClCompile Include="..\gnulib\lib\cloexec.c" />
@@ -572,9 +565,6 @@
<ClCompile Include="..\gnulib\lib\readdir.c">
<ForcedIncludeFiles>dirent-private.h;%(ForcedIncludeFiles)</ForcedIncludeFiles>
</ClCompile>
- <ClCompile Include="..\devel\libtasn1\lib\errors.c">
- <ObjectFileName>$(IntDir)\tasn1_$(filename).obj</ObjectFileName>
- </ClCompile>
<ClCompile Include="..\lib\algorithms\ecc.c">
<ObjectFileName>$(IntDir)\alg_%(Filename).obj</ObjectFileName>
</ClCompile>
diff --git a/SMP/libgnutls_winrt.vcxproj.filters b/SMP/libgnutls_winrt.vcxproj.filters
index 014a408..ae5ecff 100644
--- a/SMP/libgnutls_winrt.vcxproj.filters
+++ b/SMP/libgnutls_winrt.vcxproj.filters
@@ -103,9 +103,6 @@
<Filter Include="Header Files\lib\inih">
<UniqueIdentifier>{ae0c3eeb-53df-4c72-a85a-6b46de35e7ba}</UniqueIdentifier>
</Filter>
- <Filter Include="Source Files\libtasn1">
- <UniqueIdentifier>{32be60b7-8c65-486e-9df5-7e529443cf07}</UniqueIdentifier>
- </Filter>
<Filter Include="Header Files\lib\nettle\gost">
<UniqueIdentifier>{4fada990-3138-4089-a6c7-ae722a0e7fe9}</UniqueIdentifier>
</Filter>
@@ -1553,30 +1550,6 @@
<ClCompile Include="..\lib\nettle\sysrng-bcrypt.c">
<Filter>Source Files\lib\nettle</Filter>
</ClCompile>
- <ClCompile Include="..\devel\libtasn1\lib\coding.c">
- <Filter>Source Files\libtasn1</Filter>
- </ClCompile>
- <ClCompile Include="..\devel\libtasn1\lib\decoding.c">
- <Filter>Source Files\libtasn1</Filter>
- </ClCompile>
- <ClCompile Include="..\devel\libtasn1\lib\element.c">
- <Filter>Source Files\libtasn1</Filter>
- </ClCompile>
- <ClCompile Include="..\devel\libtasn1\lib\errors.c">
- <Filter>Source Files\libtasn1</Filter>
- </ClCompile>
- <ClCompile Include="..\devel\libtasn1\lib\gstr.c">
- <Filter>Source Files\libtasn1</Filter>
- </ClCompile>
- <ClCompile Include="..\devel\libtasn1\lib\parser_aux.c">
- <Filter>Source Files\libtasn1</Filter>
- </ClCompile>
- <ClCompile Include="..\devel\libtasn1\lib\structure.c">
- <Filter>Source Files\libtasn1</Filter>
- </ClCompile>
- <ClCompile Include="..\devel\libtasn1\lib\version.c">
- <Filter>Source Files\libtasn1</Filter>
- </ClCompile>
<ClCompile Include="..\lib\accelerated\afalg.c">
<Filter>Source Files\lib\accelerated</Filter>
</ClCompile>
@@ -0,0 +1,12 @@
--- a/lib/gnutls.pc.in
+++ b/lib/gnutls.pc.in
@@ -18,7 +18,7 @@
Name: GnuTLS
Description: Transport Security Layer implementation for the GNU system
URL: https://www.gnutls.org/
Version: @VERSION@
-Libs: -L${libdir} -lgnutls
+Libs: -L${libdir} @GNUTLS_LIBS@
Libs.private: @LIBINTL@ @LIBSOCKET@ @INET_PTON_LIB@ @LIBPTHREAD@ @LIB_SELECT@ @TSS_LIBS@ @GMP_LIBS@ @LIBUNISTRING@ @LIBATOMIC_LIBS@ @GNUTLS_LIBS_PRIVATE@
@GNUTLS_REQUIRES_PRIVATE@
Cflags: -I${includedir}
+187
View File
@@ -0,0 +1,187 @@
set(GNULIB_REF "1cc0125a28cba4e8503d853bf99b854fe4dd454c")
vcpkg_from_github(
OUT_SOURCE_PATH SOURCE_PATH
REPO savoirfairelinux/gnutls
REF 300b0c721fc1809f3d191b7f9e289c0267eabd01
SHA512 8a74fbf0343e658f6321da0f28bcc05c580fb7b950f6edfb6a8fdaad6d814456ed2dbf94e8dd330c7740c836f01e04552e649d529ee29424c3641861c41344bf
HEAD_REF fix/ci-msvc
PATCHES
external-libtasn1.patch
pkgconfig.patch
)
file(REMOVE_RECURSE "${SOURCE_PATH}/devel/perlasm")
vcpkg_download_distfile(
GNULIB_SNAPSHOT
URLS "https://gitlab.com/libidn/gnulib-mirror/-/archive/${GNULIB_REF}/gnulib-mirror-${GNULIB_REF}.tar.gz"
FILENAME "gnulib-mirror-${GNULIB_REF}.tar.gz"
SHA512 340539bd7b8e30b1e0f81a15fb207960e9ae3c4ef515f39f1d1bdb43ed52ad94a29c6a6ce8bbca3484ba5392e1dae7447c2f6b58124209614e884345e13914f5
)
vcpkg_extract_source_archive(
GNULIB_SOURCE_PATH
ARCHIVE "${GNULIB_SNAPSHOT}"
SOURCE_BASE ${GNULIB_REF}
)
file(REMOVE_RECURSE "${SOURCE_PATH}/gnulib")
file(RENAME "${GNULIB_SOURCE_PATH}" "${SOURCE_PATH}/gnulib")
include("${CURRENT_HOST_INSTALLED_DIR}/share/yasm-tool-helper/yasm-tool-helper.cmake")
yasm_tool_helper(OUT_VAR YASM)
file(TO_NATIVE_PATH "${YASM}" YASM)
if(VCPKG_LIBRARY_LINKAGE STREQUAL "dynamic")
set(CONFIGURATION_RELEASE ReleaseDLL)
set(CONFIGURATION_DEBUG DebugDLL)
else()
set(CONFIGURATION_RELEASE Release)
set(CONFIGURATION_DEBUG Debug)
endif()
if(VCPKG_TARGET_IS_UWP)
string(APPEND CONFIGURATION_RELEASE WinRT)
string(APPEND CONFIGURATION_DEBUG WinRT)
endif()
set(_gnutlsproject "${SOURCE_PATH}/SMP/libgnutls.vcxproj")
file(READ "${_gnutlsproject}" _contents)
string(REPLACE [[<Import Project="$(VCTargetsPath)\BuildCustomizations\yasm.props" />]]
"<Import Project=\"${CURRENT_HOST_INSTALLED_DIR}/share/vs-yasm/yasm.props\" />"
_contents "${_contents}")
string(REPLACE [[<Import Project="$(VCTargetsPath)\BuildCustomizations\yasm.targets" />]]
"<Import Project=\"${CURRENT_HOST_INSTALLED_DIR}/share/vs-yasm/yasm.targets\" />"
_contents "${_contents}")
string(REGEX REPLACE "${VCPKG_ROOT_DIR}/installed/[^/]+/share" "${CURRENT_HOST_INSTALLED_DIR}/share" _contents "${_contents}") # Above already
file(WRITE "${_gnutlsproject}" "${_contents}")
if(VCPKG_CRT_LINKAGE STREQUAL "static")
set(RuntimeLibraryExt "")
else()
set(RuntimeLibraryExt "DLL")
endif()
# patch output library file path and name
foreach(PROPS IN ITEMS
"${SOURCE_PATH}/SMP/smp_deps.props"
"${SOURCE_PATH}/SMP/smp_winrt_deps.props")
vcpkg_replace_string(
"${PROPS}"
[=[_winrt</TargetName>]=]
[=[</TargetName>]=]
IGNORE_UNCHANGED
)
vcpkg_replace_string(
"${PROPS}"
[=[<TargetName>lib$(RootNamespace)]=]
[=[<TargetName>$(RootNamespace)]=]
)
endforeach()
# patch hogweed, gmp, nettle, zlib libraries file names; inject RuntimeLibrary property to control CRT linkage
foreach(VCXPROJ IN ITEMS
"${SOURCE_PATH}/SMP/libgnutls.vcxproj"
"${SOURCE_PATH}/SMP/libgnutls_winrt.vcxproj")
vcpkg_replace_string(
"${VCXPROJ}"
"_winrt.lib"
".lib"
IGNORE_UNCHANGED
)
vcpkg_replace_string(
"${VCXPROJ}"
"libhogweed"
"hogweed"
)
vcpkg_replace_string(
"${VCXPROJ}"
"hogweedd"
"hogweed"
)
vcpkg_replace_string(
"${VCXPROJ}"
"libgmp"
"gmp"
)
vcpkg_replace_string(
"${VCXPROJ}"
"gmpd"
"gmp"
)
vcpkg_replace_string(
"${VCXPROJ}"
"libnettle"
"nettle"
)
vcpkg_replace_string(
"${VCXPROJ}"
"nettled"
"nettle"
)
set(zlib_basename "zlib")
if(EXISTS "${CURRENT_INSTALLED_DIR}/lib/zs.lib")
set(zlib_basename "zs")
elseif(EXISTS "${CURRENT_INSTALLED_DIR}/lib/z.lib")
set(zlib_basename "z")
endif()
vcpkg_replace_string(
"${VCXPROJ}"
"libzlib"
"${zlib_basename}"
)
vcpkg_replace_string(
"${VCXPROJ}"
"zlib"
"${zlib_basename}"
)
vcpkg_replace_string(
"${VCXPROJ}"
[=[</DisableSpecificWarnings>]=]
[=[</DisableSpecificWarnings><ForcedIncludeFiles>gnutls_msvc_fixes.h;%(ForcedIncludeFiles)</ForcedIncludeFiles><RuntimeLibrary>$(RuntimeLibrary)</RuntimeLibrary>]=]
)
endforeach()
vcpkg_install_msbuild(
USE_VCPKG_INTEGRATION
SOURCE_PATH "${SOURCE_PATH}"
PROJECT_SUBPATH SMP/libgnutls.sln
PLATFORM ${TRIPLET_SYSTEM_ARCH}
LICENSE_SUBPATH LICENSE
RELEASE_CONFIGURATION ${CONFIGURATION_RELEASE}
DEBUG_CONFIGURATION ${CONFIGURATION_DEBUG}
SKIP_CLEAN
OPTIONS /p:YasmPath="${YASM}" /p:OutDir=..\\msvc
OPTIONS_DEBUG /p:RuntimeLibrary=MultiThreadedDebug${RuntimeLibraryExt}
OPTIONS_RELEASE /p:RuntimeLibrary=MultiThreaded${RuntimeLibraryExt}
)
get_filename_component(SOURCE_PATH_SUFFIX "${SOURCE_PATH}" NAME)
if(VCPKG_TARGET_IS_UWP)
set(WINRT_SUBFOLDER libgnutls_winrt)
endif()
file(INSTALL "${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-rel/${SOURCE_PATH_SUFFIX}/msvc/${WINRT_SUBFOLDER}/include" DESTINATION "${CURRENT_PACKAGES_DIR}")
set(GNUTLS_REQUIRES_PRIVATE "Requires.private: gmp, nettle, hogweed, libtasn1, zlib")
set(GNUTLS_LIBS_PRIVATE "-lcrypt32 -lws2_32 -lkernel32 -lncrypt")
set(prefix "${CURRENT_INSTALLED_DIR}")
set(exec_prefix "\${prefix}")
set(libdir "\${prefix}/lib")
set(includedir "\${prefix}/include")
set(GNUTLS_LIBS "-lgnutls")
configure_file("${SOURCE_PATH}/lib/gnutls.pc.in" "${CURRENT_PACKAGES_DIR}/lib/pkgconfig/gnutls.pc" @ONLY)
if(NOT VCPKG_BUILD_TYPE)
set(prefix "${CURRENT_INSTALLED_DIR}/debug")
set(exec_prefix "\${prefix}")
set(libdir "\${prefix}/lib")
set(includedir "\${prefix}/../include")
set(GNUTLS_LIBS "-lgnutlsd")
configure_file("${SOURCE_PATH}/lib/gnutls.pc.in" "${CURRENT_PACKAGES_DIR}/debug/lib/pkgconfig/gnutls.pc" @ONLY)
endif()
vcpkg_fixup_pkgconfig()
vcpkg_copy_pdbs()
file(COPY "${CURRENT_PORT_DIR}/vcpkg-cmake-wrapper.cmake" DESTINATION "${CURRENT_PACKAGES_DIR}/share/gnutls")
@@ -0,0 +1,2 @@
find_library(GNUTLS_LIBRARY NAMES gnutls gnutlsd NAMES_PER_DIR)
_find_package(${ARGS})
+23
View File
@@ -0,0 +1,23 @@
{
"name": "shiftmedia-libgnutls",
"version": "3.8.13",
"port-version": 0,
"description": "GnuTLS fork maintained by Savoir-faire Linux with native Visual Studio project build tools.",
"homepage": "https://github.com/savoirfairelinux/gnutls",
"license": "LGPL-2.1-only",
"supports": "windows & !arm & !mingw & !xbox",
"dependencies": [
"gmp",
"libtasn1",
"nettle",
{
"name": "vs-yasm",
"host": true
},
{
"name": "yasm-tool-helper",
"host": true
},
"zlib"
]
}
+171
View File
@@ -78,6 +78,177 @@ async def dht_async_demo(key_str: str):
asyncio.run(dht_async_demo("unique_key"))
```
## Node State Persistence
Set the state file path before starting a node:
```python
import threading
import opendht as dht
config = dht.DhtConfig()
config.setPersistPath("dht-state.msgpack")
node = dht.DhtRunner()
node.run(config=config)
stopped = threading.Event()
node.shutdown(stopped.set)
stopped.wait()
node.join()
```
The node loads its saved DHT node ID, known peers, and stored values on startup,
and saves state during `shutdown()`. Wait for the shutdown callback before
calling `join()` or reusing the state path; `join()` alone does not save state.
Port hints are
stored in a sibling file named `<path>_port.txt`; saved ports are reused when
available. The parent directory must already exist and be writable. Use a
different state path for each node.
Persistence is disabled by default. `config.setPersistPath("")` disables it
explicitly. This state does not include private keys or certificates; persist
cryptographic identities separately as described below.
## Identity Persistence
Keys can be imported from PEM or DER bytes. Private keys export PKCS#8 PEM;
public keys export DER. Certificates export their complete PEM chain.
```python
import opendht as dht
identity = dht.Identity.generate("my-node")
private_key_pem = identity.key.serialize(password)
certificate_pem = bytes(identity.certificate)
public_key_der = bytes(identity.publickey)
restored = dht.Identity(
dht.PrivateKey(private_key_pem, password),
dht.Certificate(certificate_pem),
)
public_key = dht.PublicKey(public_key_der)
node = dht.DhtRunner()
node.run(id=restored)
```
Here `password` is an application-supplied string. Save the exported bytes using
your application's storage, protect private key files, and keep the password
separate from the key. `bytes(private_key)` and `private_key.serialize()` without
a password export **unencrypted** private keys. Empty or malformed key data,
incorrect passwords, and passwords containing NUL characters raise exceptions.
Exporting an uninitialized key raises `ValueError`.
## Value Metadata
The following properties are readable and writable on `Value`, including values
returned by `get()` and `listen()`:
| Property | Type | Meaning |
|----------|------|---------|
| `type` | unsigned 16-bit integer | Native value type identifier |
| `seq` | unsigned 16-bit integer | Sequence number for signed value updates |
| `priority` | unsigned integer | Push priority: `0` high, `1` normal |
| `pushType` | string | Push notification type, such as `audioCall` |
```python
value = dht.Value(b"some binary data")
value.type = 42
value.seq = 1
value.priority = 1
value.pushType = "audioCall"
```
Set metadata before publishing the value. Numeric values outside the native
integer range raise `OverflowError`.
`value.isSigned()` reports whether a public key and signature are present;
it does not verify the signature. `value.isEncrypted()` reports whether the
value currently contains ciphertext. Values delivered by `get()` and `listen()`
are transparently decrypted when possible, so decrypted values return `False`
from `isEncrypted()`, even if they were published using `putEncrypted()`.
An uninitialized `PublicKey()` cannot be used as an encryption recipient and
raises `ValueError`.
Register custom `ValueType` instances after `run()` and before `join()`.
`registerType()` raises `RuntimeError` when the runner is stopped, `TypeError`
for `None`, and `ValueError` for an uninitialized type.
## Observability
`getNodeInfo()` returns a read-only `NodeInfo` snapshot with node identifiers,
IPv4/IPv6 routing statistics, bound ports, pending-operation count, and local
and remote storage counters. `getNodesStats(socket.AF_INET)` and
`getNodesStats(socket.AF_INET6)` return `NodeStats` snapshots. Snapshots remain
valid after the runner stops. `getPublicAddress()` returns known public
`SockAddr` objects; pass an address family to restrict the results, or use
`getPublicAddressStr()` for strings. These lists may be empty before discovery.
`getNodeMessageStats(incoming=True)` selects incoming message counters.
```python
info = node.getNodeInfo()
print(info.node_id, info.bound4, info.ipv4.good_nodes, info.storage_size)
print(node.getPublicAddressStr())
node.setOnStatusChanged(lambda ipv4, ipv6: print(ipv4.name, ipv6.name))
node.setOnStatusChanged(None)
```
The status callback receives two `NodeStatus` values: `Disconnected`,
`Connecting`, or `Connected`. Registration replaces the previous callback and
persists across restarts; `None` disables it. Core callbacks run on the DHT
thread while native locks are held: do not call snapshot getters, type
registration, logging controls, `run()`, blocking operations, or `join()` from
them. The asyncio wrapper schedules status callbacks on its event-loop thread,
where snapshot getters are safe to use.
Call `node.connectivityChanged()` after a network-interface or IP-address
change. It queues native connectivity recovery, including recontacting peers
and renewing subscriptions. It is also available on the asyncio runner as a
synchronous, nonblocking method.
## Queries And Filtered Listeners
`query()` retrieves metadata rather than complete value payloads. Each row is
a dictionary keyed by `ValueField`. Network projections support `Id`,
`ValueType`, `OwnerPk`, and `UserType`; values are integers, an `InfoHash` owner
fingerprint, or a UTF-8 string respectively. Explicit sequence-number
projections are not supported by the native wire format.
```python
selection = dht.Select().field(dht.ValueField.Id).field(dht.ValueField.UserType)
query = dht.Query()
query.buildFrom(selection, dht.Where().userType("record"))
rows = node.query(key, query=query)
query = dht.Query("SELECT id,user_type WHERE user_type=record")
node.query(key, query_cb=lambda row: print(row) or True,
done_cb=lambda ok, nodes: print(ok), query=query)
```
Without an explicit query, all four supported metadata fields are selected.
Supplying `query_cb` makes the core call nonblocking; returning `False` stops
delivery. The blocking core API returns the collected rows, as `get()` does.
The asyncio API offers `query()` as a context-managed async iterator and
`await queryAll()` as a collector; these raise `RuntimeError` on failure.
```python
rows = await node.queryAll(key, query=query)
with node.query(key, query=query) as results:
async for row in results:
print(row[dht.ValueField.Id])
```
Both core and asyncio `listen()` accept optional `where` and `filter` arguments.
`Where` is sent to the DHT to narrow the subscription; the Python predicate
runs locally. Existing two-argument core listeners are unchanged.
```python
token = node.listen(key, lambda value, expired: True,
where=dht.Where().userType("record"),
filter=lambda value: len(value.data) < 1024)
node.cancelListen(token)
```
## Documentation
For more information, examples and documentation, or to repport issues, visit: https://opendht.net
+450 -79
View File
@@ -12,15 +12,21 @@ from libcpp.map cimport map as map
from libcpp cimport bool
from libcpp.utility cimport pair
from libcpp.string cimport string
from libcpp.memory cimport shared_ptr
from libcpp.memory cimport shared_ptr, unique_ptr
from cython.operator cimport dereference as deref, preincrement as inc, predecrement as dec
from cpython cimport ref
from datetime import timedelta
from enum import IntEnum as _IntEnum
from operator import index as _index
cimport opendht_cpp as cpp
ctypedef cpp.Pht* _PhtPtr
import threading
import socket as _socket
import weakref as _weakref
cdef inline void _initialize_gnutls():
cdef int err = cpp.gnutls_global_init()
@@ -39,6 +45,58 @@ _initialize_gnutls()
def version() -> str:
return cpp.version().decode()
class NodeStatus(_IntEnum):
Disconnected = 0
Connecting = 1
Connected = 2
class ValueField(_IntEnum):
Id = 1
ValueType = 2
OwnerPk = 3
SeqNum = 4
UserType = 5
class _StatusCallbackState:
__slots__ = ("callback", "__weakref__")
def __init__(self):
self.callback = None
cdef inline void status_callback(int ipv4, int ipv6, void* user_data) noexcept with gil:
state = (<object>user_data)()
if state is None:
return
callback = state.callback
if callback is not None:
callback(NodeStatus(ipv4), NodeStatus(ipv6))
cdef inline bool query_callback(shared_ptr[cpp.FieldValueIndex] value, void* user_data) noexcept with gil:
cdef cpp.Blob blob
row = {}
for entry in value.get().index:
field = ValueField(<int>entry.first)
if field == ValueField.OwnerPk:
owner = InfoHash()
owner._infohash = entry.second.getHash()
row[field] = owner
elif field == ValueField.UserType:
blob = entry.second.getBlob()
row[field] = (<char*>blob.data())[:blob.size()].decode()
else:
row[field] = entry.second.getInt()
return (<object>user_data)['query'](row)
cdef inline void query_done_callback(bool done, cpp.vector[shared_ptr[cpp.Node]]* nodes, void* user_data) noexcept with gil:
callback = (<object>user_data)['done']
if callback is not None:
entries = []
for node in deref(nodes):
entry = NodeEntry()
entry._v.first = node.get().getId()
entry._v.second = node
entries.append(entry)
callback(done, entries)
cdef inline void lookup_callback(cpp.vector[cpp.shared_ptr[cpp.IndexValue]]* values, cpp.Prefix* p, void *user_data) noexcept with gil:
cbs = <object>user_data
if 'lookup' in cbs and cbs['lookup']:
@@ -51,9 +109,11 @@ cdef inline void lookup_callback(cpp.vector[cpp.shared_ptr[cpp.IndexValue]]* val
cdef inline void shutdown_callback(void* user_data) noexcept with gil:
cbs = <object>user_data
if 'shutdown' in cbs and cbs['shutdown']:
cbs['shutdown']()
ref.Py_DECREF(cbs)
try:
if 'shutdown' in cbs and cbs['shutdown']:
cbs['shutdown']()
finally:
ref.Py_DECREF(cbs)
cdef inline bool get_callback(shared_ptr[cpp.Value] value, void *user_data) noexcept with gil:
cbs = <object>user_data
@@ -72,22 +132,26 @@ cdef inline bool value_callback(shared_ptr[cpp.Value] value, bool expired, void
return cb(pv, expired) if not f or f(pv) else True
cdef inline void done_callback(bool done, cpp.vector[shared_ptr[cpp.Node]]* nodes, void *user_data) noexcept with gil:
node_ids = []
for n in deref(nodes):
h = NodeEntry()
h._v.first = n.get().getId()
h._v.second = n
node_ids.append(h)
cbs = <object>user_data
if 'done' in cbs and cbs['done']:
cbs['done'](done, node_ids)
ref.Py_DECREF(cbs)
try:
node_ids = []
for n in deref(nodes):
h = NodeEntry()
h._v.first = n.get().getId()
h._v.second = n
node_ids.append(h)
if 'done' in cbs and cbs['done']:
cbs['done'](done, node_ids)
finally:
ref.Py_DECREF(cbs)
cdef inline void done_callback_simple(bool done, void *user_data) noexcept with gil:
cbs = <object>user_data
if 'done' in cbs and cbs['done']:
cbs['done'](done)
ref.Py_DECREF(cbs)
try:
if 'done' in cbs and cbs['done']:
cbs['done'](done)
finally:
ref.Py_DECREF(cbs)
cdef class _WithID(object):
def __repr__(self):
@@ -101,6 +165,12 @@ cdef class _WithPk(object):
def __str__(self):
return self.getLongId().toString().decode()
cdef inline unsigned _checked_bit_index(bit, unsigned bit_count) except *:
bit = _index(bit)
if bit < 0 or bit >= bit_count:
raise IndexError("bit index out of range")
return bit
cdef class InfoHash(_WithID):
cdef cpp.InfoHash _infohash
def __cinit__(self, bytes str=b''):
@@ -116,9 +186,9 @@ cdef class InfoHash(_WithID):
return self._infohash == other._infohash
return NotImplemented
def getBit(InfoHash self, bit):
return self._infohash.getBit(bit)
return self._infohash.getBit(_checked_bit_index(bit, 160))
def setBit(InfoHash self, bit, b):
self._infohash.setBit(bit, b)
self._infohash.setBit(_checked_bit_index(bit, 160), b)
def getId(InfoHash self):
return self
def toString(InfoHash self):
@@ -154,9 +224,9 @@ cdef class PkId(_WithPk):
return self._pkid == other._pkid
return NotImplemented
def getBit(PkId self, bit):
return self._pkid.getBit(bit)
return self._pkid.getBit(_checked_bit_index(bit, 256))
def setBit(PkId self, bit, b):
self._pkid.setBit(bit, b)
self._pkid.setBit(_checked_bit_index(bit, 256), b)
def getLongId(PkId self):
return self
def toString(PkId self):
@@ -212,14 +282,92 @@ cdef class SockAddr(object):
cdef class Node(_WithID):
cdef shared_ptr[cpp.Node] _node
def getId(self):
if not self._node:
raise ValueError("Node is not initialized")
h = InfoHash()
h._infohash = self._node.get().getId()
return h
def getAddr(self):
if not self._node:
raise ValueError("Node is not initialized")
return self._node.get().getAddrStr()
def isExpired(self):
if not self._node:
raise ValueError("Node is not initialized")
return self._node.get().isExpired()
cdef class NodeStats(object):
cdef cpp.NodeStats _stats
@property
def good_nodes(self):
return self._stats.good_nodes
@property
def dubious_nodes(self):
return self._stats.dubious_nodes
@property
def cached_nodes(self):
return self._stats.cached_nodes
@property
def incoming_nodes(self):
return self._stats.incoming_nodes
@property
def table_depth(self):
return self._stats.table_depth
@property
def searches(self):
return self._stats.searches
@property
def node_cache_size(self):
return self._stats.node_cache_size
def getKnownNodes(self):
return self._stats.getKnownNodes()
def getNetworkSizeEstimation(self):
return self._stats.getNetworkSizeEstimation()
cdef class NodeInfo(object):
cdef cpp.NodeInfo _info
@property
def id(self):
result = InfoHash()
result._infohash = self._info.id
return result
@property
def node_id(self):
result = InfoHash()
result._infohash = self._info.node_id
return result
@property
def ipv4(self):
result = NodeStats()
result._stats = self._info.ipv4
return result
@property
def ipv6(self):
result = NodeStats()
result._stats = self._info.ipv6
return result
@property
def ongoing_ops(self):
return self._info.ongoing_ops
@property
def storage_values(self):
return self._info.storage_values
@property
def storage_size(self):
return self._info.storage_size
@property
def local_storage_values(self):
return self._info.local_storage_values
@property
def local_storage_size(self):
return self._info.local_storage_size
@property
def bound4(self):
return self._info.bound4
@property
def bound6(self):
return self._info.bound6
cdef class NodeEntry(_WithID):
cdef cpp.pair[cpp.InfoHash, shared_ptr[cpp.Node]] _v
def getId(self):
@@ -253,8 +401,11 @@ cdef class Select(object):
return self._select.toString().decode()
def isSatisfiedBy(self, Select os):
return self._select.isSatisfiedBy(os._select)
def field(self, int field):
self._select.field(<cpp.Field> field)
def field(self, field):
cdef int field_id = ValueField(_index(field))
if field_id == ValueField.SeqNum:
raise ValueError("The native wire format does not support sequence-number projections")
self._select.field(<cpp.Field>field_id)
return self
cdef class Where(object):
@@ -290,6 +441,10 @@ cdef class Value(object):
self._value.reset(new cpp.Value(id, val, len(val)))
def __str__(self):
return self._value.get().toString().decode()
def isSigned(self) -> bool:
return self._value.get().isSigned()
def isEncrypted(self) -> bool:
return self._value.get().isEncrypted()
@property
def owner(self) -> PublicKey:
h = PublicKey()
@@ -324,25 +479,51 @@ cdef class Value(object):
def id(self, cpp.uint64_t value):
self._value.get().id = value
@property
def type(self) -> int:
return self._value.get().type
@type.setter
def type(self, cpp.uint16_t value):
self._value.get().type = value
@property
def seq(self) -> int:
return self._value.get().seq
@seq.setter
def seq(self, cpp.uint16_t value):
self._value.get().seq = value
@property
def priority(self) -> int:
return self._value.get().priority
@priority.setter
def priority(self, unsigned value):
self._value.get().priority = value
@property
def pushType(self) -> str:
return self._value.get().pushType.decode()
@pushType.setter
def pushType(self, str value):
self._value.get().pushType = value.encode()
@property
def size(self) -> int:
return self._value.get().size()
cdef class ValueType(object):
cdef cpp.ValueType * _value
cdef unique_ptr[cpp.ValueType] _value
def __init__(self, cpp.uint16_t id, str name, expiration: timedelta):
if not isinstance(expiration, timedelta):
raise TypeError("expiration argument must be of type timedelta")
cdef cpp.seconds duration = cpp.seconds(int(expiration.total_seconds()))
self._value = new cpp.ValueType(id, name.encode(), duration)
self._value.reset(new cpp.ValueType(id, name.encode(), duration))
cdef class NodeSetIter(object):
cdef map[cpp.InfoHash, shared_ptr[cpp.Node]]* _nodes
cdef NodeSet _owner
cdef map[cpp.InfoHash, shared_ptr[cpp.Node]].iterator _curIter
def __init__(self, NodeSet s):
self._nodes = &s._nodes
self._curIter = self._nodes.begin()
def __init__(self, NodeSet s not None):
self._owner = s
self._curIter = self._owner._nodes.begin()
def __iter__(self):
return self
def __next__(self):
if self._curIter == self._nodes.end():
if self._curIter == self._owner._nodes.end():
raise StopIteration
h = NodeEntry()
h._v = deref(self._curIter)
@@ -374,6 +555,8 @@ cdef class NodeSet(object):
s = ''
cdef map[cpp.InfoHash, shared_ptr[cpp.Node]].iterator it = self._nodes.begin()
while it != self._nodes.end():
if not deref(it).second:
raise ValueError("Node is not initialized")
s += deref(it).first.toString().decode() + ' ' + deref(it).second.get().getAddrStr().decode() + '\n'
inc(it)
return s
@@ -383,6 +566,30 @@ cdef class NodeSet(object):
cdef class PrivateKey(_WithPk):
cdef shared_ptr[cpp.PrivateKey] _key
def __init__(self, bytes data=None, str password=""):
cdef cpp.Blob encoded
if '\0' in password:
raise ValueError("Password must not contain NUL characters")
if data is not None:
if not data:
raise ValueError("Private key data must not be empty")
encoded = data
self._key = cpp.make_shared[cpp.PrivateKey](encoded, <cpp.string>password.encode())
def serialize(self, str password="") -> bytes:
"""Export PKCS#8 PEM, encrypted when a nonempty password is supplied."""
if not self._key:
raise ValueError("PrivateKey is not initialized")
if '\0' in password:
raise ValueError("Password must not contain NUL characters")
cdef cpp.Blob encoded = self._key.get().serialize(password.encode())
if encoded.empty():
raise RuntimeError("Could not export private key")
return (<char*>encoded.data())[:encoded.size()]
def __bytes__(self):
return self.serialize()
def getId(self) -> InfoHash:
if not self._key:
raise ValueError("PrivateKey is not initialized")
@@ -405,6 +612,8 @@ cdef class PrivateKey(_WithPk):
return pk
def decrypt(self, bytes dat) -> bytes:
if not self._key:
raise ValueError("PrivateKey is not initialized")
cdef size_t d_len = len(dat)
cdef cpp.uint8_t* d_ptr = <cpp.uint8_t*>dat
cdef cpp.Blob indat
@@ -439,6 +648,25 @@ cdef class PrivateKey(_WithPk):
cdef class PublicKey(_WithPk):
cdef cpp.shared_ptr[cpp.PublicKey] _key
def __init__(self, bytes data=None):
cdef cpp.Blob encoded
if data is not None:
if not data:
raise ValueError("Public key data must not be empty")
encoded = data
self._key = cpp.make_shared[cpp.PublicKey](encoded)
def serialize(self) -> bytes:
"""Export the public key as DER bytes."""
if not self._key:
raise ValueError("PublicKey is not initialized")
cdef cpp.Blob encoded
self._key.get().pack(encoded)
return (<char*>encoded.data())[:encoded.size()]
def __bytes__(self):
return self.serialize()
def getId(self) -> InfoHash:
if not self._key:
raise ValueError("PublicKey is not initialized")
@@ -500,11 +728,17 @@ cdef class Certificate(_WithPk):
return ""
return self._cert.get().toString().decode()
def getName(self) -> str:
if not self._cert:
raise ValueError("Certificate is not initialized")
return self._cert.get().getName().decode()
def revoke(self, PrivateKey k, Certificate c):
if not self._cert or c is None or not c._cert:
raise ValueError("Certificate is not initialized")
if k is None or not k._key:
raise ValueError("PrivateKey is not initialized")
self._cert.get().revoke(deref(k._key.get()), deref(c._cert.get()));
def __bytes__(self):
return self._cert.get().toString().encode() if self._cert else b''
return self._cert.get().toString() if self._cert else b''
@property
def issuer(self) -> Certificate|None:
if self._cert and self._cert.get().issuer:
@@ -514,6 +748,10 @@ cdef class Certificate(_WithPk):
return None
@staticmethod
def generate(PrivateKey k, str name, Identity i = Identity(), bool is_ca = False):
if k is None or not k._key:
raise ValueError("PrivateKey is not initialized")
if i is None:
i = Identity()
c = Certificate()
c._cert = cpp.make_shared[cpp.Certificate](cpp.Certificate.generate(deref(k._key.get()), name.encode(), i._id, is_ca))
return c
@@ -528,10 +766,16 @@ cdef class VerifyResult(object):
cdef class TrustList(object):
cdef cpp.TrustList _trust
def add(self, Certificate cert):
if cert is None or not cert._cert:
raise ValueError("Certificate is not initialized")
self._trust.add(deref(cert._cert.get()))
def remove(self, Certificate cert):
if cert is None or not cert._cert:
raise ValueError("Certificate is not initialized")
self._trust.remove(deref(cert._cert.get()))
def verify(self, Certificate cert):
if cert is None or not cert._cert:
raise ValueError("Certificate is not initialized")
r = VerifyResult()
r._result = self._trust.verify(deref(cert._cert.get()))
return r
@@ -539,7 +783,7 @@ cdef class TrustList(object):
cdef class ListenToken(object):
cdef cpp.InfoHash _h
cdef cpp.shared_future[size_t] _t
_cb = dict()
cdef bool _cancelled
cdef class Identity(object):
cdef cpp.Identity _id
@@ -550,6 +794,8 @@ cdef class Identity(object):
self._id.second = c._cert
@staticmethod
def generate(str name = "pydht", Identity ca = Identity(), unsigned bits = 4096):
if ca is None:
ca = Identity()
i = Identity()
i._id = cpp.generateIdentity(name.encode(), ca._id, bits)
return i
@@ -610,6 +856,13 @@ cdef class DhtConfig(object):
self._config.dht_config.node_config.network = netid
def setMaintainStorage(self, bool maintain_storage):
self._config.dht_config.node_config.maintain_storage = maintain_storage
def setPersistPath(self, str path):
"""Set the native state file path; an empty string disables persistence."""
if path is None:
raise TypeError("path must be a string")
if '\0' in path:
raise ValueError("path must not contain NUL characters")
self._config.dht_config.node_config.persist_path = path.encode()
def setRateLimit(self, ssize_t max_req_per_sec, ssize_t max_peer_req_per_sec):
self._config.dht_config.node_config.max_req_per_sec = max_req_per_sec
self._config.dht_config.node_config.max_peer_req_per_sec = max_peer_req_per_sec
@@ -630,24 +883,65 @@ cdef class DhtConfig(object):
cdef class DhtRunner(_WithID):
cdef cpp.shared_ptr[cpp.DhtRunner] thisptr
cdef map[_PhtPtr, shared_ptr[cpp.Pht]] _indexes
cdef object _status_callback
def __cinit__(self):
self.thisptr.reset(new cpp.DhtRunner())
self._status_callback = _StatusCallbackState()
def __dealloc__(self):
with nogil:
if self.thisptr:
self.thisptr.get().join()
self._indexes.clear()
self.thisptr.reset()
def getId(self) -> InfoHash:
h = InfoHash()
if self.thisptr:
h._infohash = self.thisptr.get().getId()
with nogil:
h._infohash = self.thisptr.get().getId()
return h
def getPublicKey(self) -> PublicKey:
if not self.thisptr:
raise ValueError("DhtRunner is not initialized")
pk = PublicKey()
pk._key = self.thisptr.get().getPublicKey()
with nogil:
pk._key = self.thisptr.get().getPublicKey()
return pk
def getNodeId(self) -> InfoHash:
h = InfoHash()
if self.thisptr:
h._infohash = self.thisptr.get().getNodeId()
with nogil:
h._infohash = self.thisptr.get().getNodeId()
return h
def getNodeInfo(self) -> NodeInfo:
result = NodeInfo()
with nogil:
result._info = self.thisptr.get().getNodeInfo()
return result
def getNodesStats(self, af=_socket.AF_INET) -> NodeStats:
if af not in (_socket.AF_INET, _socket.AF_INET6):
raise ValueError("af must be AF_INET or AF_INET6")
info = self.getNodeInfo()
return info.ipv4 if af == _socket.AF_INET else info.ipv6
def getPublicAddress(self, cpp.sa_family_t af=0):
if af not in (0, _socket.AF_INET, _socket.AF_INET6):
raise ValueError("af must be AF_UNSPEC, AF_INET or AF_INET6")
cdef cpp.vector[cpp.SockAddr] addresses
with nogil:
addresses = self.thisptr.get().getPublicAddress(af)
result = []
for address in addresses:
value = SockAddr()
value._addr = address
result.append(value)
return result
def getPublicAddressStr(self, cpp.sa_family_t af=0):
return [str(address) for address in self.getPublicAddress(af)]
def setOnStatusChanged(self, callback=None):
"""Replace the DHT-thread status callback, or disable it with None."""
if callback is not None and not callable(callback):
raise TypeError("callback must be callable or None")
self._status_callback.callback = callback
def ping(self, SockAddr addr, done_cb=None):
if done_cb:
cb_obj = {'done':done_cb}
@@ -673,44 +967,80 @@ cdef class DhtRunner(_WithID):
host_bytes = host.encode()
port_bytes = port.encode() if port else b'4222'
self.thisptr.get().bootstrap(<cpp.const_char*>host_bytes, <cpp.const_char*>port_bytes)
def run(self, Identity id=None, is_bootstrap=False, cpp.in_port_t port=0, str ipv4="", str ipv6="", DhtConfig config=DhtConfig()):
if id:
config.setIdentity(id)
def connectivityChanged(self):
with nogil:
self.thisptr.get().connectivityChanged()
def run(self, Identity id=None, is_bootstrap=False, cpp.in_port_t port=0, str ipv4="", str ipv6="", DhtConfig config=None):
cdef cpp.DhtRunnerConfig run_config = config._config if config is not None else cpp.DhtRunnerConfig()
cdef cpp.string bind4, bind6, service
cdef object status_ref = _weakref.ref(self._status_callback)
if id is not None:
run_config.dht_config.id = id._id
if is_bootstrap:
run_config.dht_config.node_config.is_bootstrap = True
if ipv4 or ipv6:
bind4 = ipv4.encode() if ipv4 else b''
bind6 = ipv6.encode() if ipv6 else b''
self.thisptr.get().run(bind4, bind6, str(port).encode(), config._config)
service = str(port).encode()
with nogil:
cpp.runPythonRunnerAddress(deref(self.thisptr.get()), bind4.c_str(),
bind6.c_str(), service.c_str(), run_config,
status_callback, status_ref)
else:
self.thisptr.get().run(port, config._config)
with nogil:
cpp.runPythonRunner(deref(self.thisptr.get()), port, run_config,
status_callback, status_ref)
def join(self):
self.thisptr.get().join()
with nogil:
self.thisptr.get().join()
self._indexes.clear()
def shutdown(self, shutdown_cb=None):
cb_obj = {'shutdown':shutdown_cb}
ref.Py_INCREF(cb_obj)
self.thisptr.get().shutdown(cpp.bindShutdownCb(shutdown_callback, <void*>cb_obj))
def registerType(self, ValueType type):
self.thisptr.get().registerType(deref(type._value))
def registerType(self, ValueType type not None):
if not type._value:
raise ValueError("ValueType is not initialized")
cdef cpp.ValueType run_type = deref(type._value.get())
with nogil:
self.thisptr.get().registerType(run_type)
def enableLogging(self):
cpp.enableLogging(self.thisptr.get()[0])
with nogil:
cpp.enableLogging(self.thisptr.get()[0])
def disableLogging(self):
cpp.disableLogging(self.thisptr.get()[0])
with nogil:
cpp.disableLogging(self.thisptr.get()[0])
def enableFileLogging(self, str path):
cpp.enableFileLogging(self.thisptr.get()[0], path.encode())
cdef cpp.string log_path = path.encode()
with nogil:
cpp.enableFileLogging(self.thisptr.get()[0], log_path)
def isRunning(self):
return self.thisptr.get().isRunning()
def getBound(self, cpp.sa_family_t af = 0):
s = SockAddr()
s._addr = self.thisptr.get().getBound(af)
with nogil:
s._addr = self.thisptr.get().getBound(af)
return s
def getStorageLog(self):
return self.thisptr.get().getStorageLog().decode()
cdef cpp.string result
with nogil:
result = self.thisptr.get().getStorageLog()
return result.decode()
def getRoutingTablesLog(self, cpp.sa_family_t af):
return self.thisptr.get().getRoutingTablesLog(af).decode()
cdef cpp.string result
with nogil:
result = self.thisptr.get().getRoutingTablesLog(af)
return result.decode()
def getSearchesLog(self, cpp.sa_family_t af):
return self.thisptr.get().getSearchesLog(af).decode()
def getNodeMessageStats(self):
cdef cpp.string result
with nogil:
result = self.thisptr.get().getSearchesLog(af)
return result.decode()
def getNodeMessageStats(self, bool incoming=False):
stats = []
cdef cpp.vector[unsigned] res = self.thisptr.get().getNodeMessageStats(False)
cdef cpp.vector[unsigned] res
with nogil:
res = self.thisptr.get().getNodeMessageStats(incoming)
for n in res:
stats.append(n)
return stats
@@ -753,6 +1083,25 @@ cdef class DhtRunner(_WithID):
lock.wait()
return res
def query(self, InfoHash key not None, query_cb=None, done_cb=None, Query query=None):
"""Return projected metadata rows, or stream them through query_cb."""
if query is None:
query = Query("SELECT id,value_type,owner_pk,user_type")
if query_cb is None:
completed = threading.Event()
rows = []
def on_row(row):
rows.append(row)
return True
self.query(key, on_row, lambda ok, nodes: completed.set(), query)
completed.wait()
return rows
if not callable(query_cb) or (done_cb is not None and not callable(done_cb)):
raise TypeError("query callbacks must be callable")
callbacks = {'query': query_cb, 'done': done_cb}
cpp.queryPython(deref(self.thisptr.get()), key._infohash, query_callback,
query_done_callback, callbacks, query._query)
def put(self, InfoHash key, Value val, done_cb=None, permanent=False):
"""Publish a new value on the DHT at key.
@@ -776,7 +1125,7 @@ cdef class DhtRunner(_WithID):
lock.notify()
with lock:
pending += 1
self.put(key, val, done_cb=tmp_done)
self.put(key, val, done_cb=tmp_done, permanent=permanent)
while pending > 0:
lock.wait()
return ok
@@ -798,7 +1147,7 @@ cdef class DhtRunner(_WithID):
lock.notify()
with lock:
pending += 1
self.putSigned(key, val, done_cb=tmp_done)
self.putSigned(key, val, done_cb=tmp_done, permanent=permanent)
while pending > 0:
lock.wait()
return ok
@@ -806,6 +1155,8 @@ cdef class DhtRunner(_WithID):
def putEncrypted(self, InfoHash key, Value val, to, done_cb=None, bool permanent=False):
if to is None:
raise ValueError("'to' must be provided")
if isinstance(to, PublicKey) and not (<PublicKey>to)._key:
raise ValueError("PublicKey is not initialized")
if done_cb:
cb_obj = {'done':done_cb}
ref.Py_INCREF(cb_obj)
@@ -845,19 +1196,23 @@ cdef class DhtRunner(_WithID):
def cancelPut(self, InfoHash key, Value val):
self.thisptr.get().cancelPut(key._infohash, val._value)
def listen(self, InfoHash key, value_cb):
def listen(self, InfoHash key not None, value_cb, filter=None, Where where=None):
if not callable(value_cb) or (filter is not None and not callable(filter)):
raise TypeError("listener callbacks must be callable")
if where is None:
where = Where()
t = ListenToken()
t._h = key._infohash
cb_obj = {'valcb':value_cb}
t._cb['cb'] = cb_obj
# avoid the callback being destructed if the token is destroyed
ref.Py_INCREF(cb_obj)
t._t = self.thisptr.get().listen(t._h, cpp.bindValueCb(value_callback, <void*>cb_obj)).share()
cb_obj = {'valcb':value_cb, 'filter':filter}
t._t = self.thisptr.get().listen(t._h, cpp.bindPythonValueCb(value_callback, cb_obj),
cpp.nullptr, where._where).share()
return t
def cancelListen(self, ListenToken token):
self.thisptr.get().cancelListen(token._h, token._t)
ref.Py_DECREF(<object>token._cb['cb'])
# fixme: not thread safe
if token is None:
raise TypeError("token must be a ListenToken")
if not token._cancelled and token._t.valid():
self.thisptr.get().cancelListen(token._h, token._t)
token._cancelled = True
cdef class IndexValue(object):
cdef cpp.shared_ptr[cpp.IndexValue] _value
@@ -874,12 +1229,18 @@ cdef class IndexValue(object):
return self._value.get().second
cdef class Pht(object):
cdef cpp.Pht* thisptr
def __cinit__(self, bytes name, key_spec, DhtRunner dht):
"""PHT index; active native indexes live until the runner is joined."""
cdef shared_ptr[cpp.Pht] thisptr
cdef DhtRunner _runner
def __cinit__(self, bytes name, key_spec, DhtRunner dht not None):
cdef cpp.IndexKeySpec cpp_key_spec
for kk, size in key_spec.items():
cpp_key_spec[bytes(kk, 'utf-8')] = size
self.thisptr = new cpp.Pht(name, cpp_key_spec, dht.thisptr)
self._runner = dht
self.thisptr.reset(new cpp.Pht(name, cpp_key_spec, dht.thisptr))
def __dealloc__(self):
with nogil:
self.thisptr.reset()
@property
def MAX_NODE_ENTRY_COUNT(self):
return cpp.PHT_MAX_NODE_ENTRY_COUNT
@@ -892,16 +1253,21 @@ cdef class Pht(object):
representation of the prefix where the value was indexed in
the PHT.
"""
cb_obj = {'lookup':lookup_cb, 'done':done_cb} # TODO: donecallback is to be removed
ref.Py_INCREF(cb_obj)
cdef cpp.IndexKey cppk
for kk, v in key.items():
cppk[bytes(kk, 'utf-8')] = bytes(v)
self.thisptr.lookup(
cppk,
cpp.Pht.bindLookupCb(lookup_callback, <void*>cb_obj),
cpp.bindDoneCbSimple(done_callback_simple, <void*>cb_obj)
)
self._runner._indexes[self.thisptr.get()] = self.thisptr
cb_obj = {'lookup':lookup_cb, 'done':done_cb} # TODO: donecallback is to be removed
ref.Py_INCREF(cb_obj)
try:
self.thisptr.get().lookup(
cppk,
cpp.Pht.bindLookupCb(lookup_callback, <void*>cb_obj),
cpp.bindDoneCbSimple(done_callback_simple, <void*>cb_obj)
)
except:
ref.Py_DECREF(cb_obj)
raise
def insert(self, key, IndexValue value, done_cb=None):
"""Add an index entry to the Index.
@@ -909,16 +1275,21 @@ cdef class Pht(object):
value -- an IndexValue object describing the indexed value.
done_cb -- Called when the operation is completed.
"""
cb_obj = {'done':done_cb}
ref.Py_INCREF(cb_obj)
cdef cpp.IndexKey cppk
for kk, v in key.items():
cppk[bytes(kk, 'utf-8')] = bytes(v)
cdef cpp.IndexValue val
val.first = (<InfoHash>value.getKey())._infohash
val.second = value.getValueId()
self.thisptr.insert(
cppk,
val,
cpp.bindDoneCbSimple(done_callback_simple, <void*>cb_obj)
)
self._runner._indexes[self.thisptr.get()] = self.thisptr
cb_obj = {'done':done_cb}
ref.Py_INCREF(cb_obj)
try:
self.thisptr.get().insert(
cppk,
val,
cpp.bindDoneCbSimple(done_callback_simple, <void*>cb_obj)
)
except:
ref.Py_DECREF(cb_obj)
raise
+83 -24
View File
@@ -3,12 +3,20 @@
from __future__ import annotations
import asyncio
from typing import Any, Callable, Iterable, List, Optional, Tuple
from typing import Any, Callable, Generic, Iterable, List, Optional, Tuple, TypeVar
from . import _core as _dht
# Re-export core types for convenience
from ._core import InfoHash, Value, ValueType, Where, SockAddr, PublicKey, PkId, DhtConfig, Identity
from ._core import NodeInfo, NodeStats, NodeStatus, Query, Select, ValueField
_Result = TypeVar("_Result")
def _set_result_if_pending(future: asyncio.Future[Any], result: Any) -> None:
if not future.done():
future.set_result(result)
class _Listener:
@@ -19,7 +27,11 @@ class _Listener:
__slots__ = ("_runner", "_token", "_loop", "_queue", "_closed")
def __init__(self, runner: DhtRunner, key: InfoHash) -> None:
def __init__(
self, runner: DhtRunner, key: InfoHash,
where: Optional[Where] = None,
filter: Optional[Callable[[Value], bool]] = None,
) -> None:
self._runner = runner
self._loop = runner._loop
self._queue: asyncio.Queue[Tuple[Value, bool]] = asyncio.Queue()
@@ -31,7 +43,7 @@ class _Listener:
self._loop.call_soon_threadsafe(self._queue.put_nowait, (v, expired))
return True # continue listening
self._token = self._runner._dht.listen(key, _on_value)
self._token = self._runner._dht.listen(key, _on_value, filter, where)
def __enter__(self) -> _Listener:
return self
@@ -62,10 +74,10 @@ class _Listener:
self._queue.put_nowait(None)
class _GetStream:
"""Async iterator around DHT get callback.
class _GetStream(Generic[_Result]):
"""Async iterator around a DHT get or query callback.
Yields Value instances as they are found, then ends.
Yields results as they are found, then ends.
Raises on failure.
"""
__slots__ = ("_runner", "_loop", "_queue", "_closed", "_error")
@@ -76,14 +88,15 @@ class _GetStream:
key: InfoHash,
where: Optional[Where],
filter: Optional[Callable[[Value], bool]],
query: Optional[Query] = None,
) -> None:
self._runner = runner
self._loop = runner._loop
self._queue: asyncio.Queue[Optional[Value]] = asyncio.Queue()
self._queue: asyncio.Queue[Optional[_Result]] = asyncio.Queue()
self._closed = False
self._error: Optional[BaseException] = None
def _on_value(v: Value) -> bool:
def _on_value(v: _Result) -> bool:
# Called on DHT thread
if self._closed:
return False
@@ -95,21 +108,24 @@ class _GetStream:
if self._closed:
return
if not ok:
self._error = RuntimeError("DHT get failed")
self._error = RuntimeError("DHT query failed" if query is not None else "DHT get failed")
self._loop.call_soon_threadsafe(self._queue.put_nowait, None)
self._runner._dht.get(key, _on_value, _on_done, filter, where)
if query is None:
self._runner._dht.get(key, _on_value, _on_done, filter, where)
else:
self._runner._dht.query(key, _on_value, _on_done, query)
def __enter__(self) -> _GetStream:
def __enter__(self) -> _GetStream[_Result]:
return self
def __exit__(self, exc_type, exc, tb) -> None:
self.cancel()
def __aiter__(self) -> _GetStream:
def __aiter__(self) -> _GetStream[_Result]:
return self
async def __anext__(self) -> Value:
async def __anext__(self) -> _Result:
if self._closed:
raise StopAsyncIteration
item = await self._queue.get()
@@ -132,7 +148,7 @@ class DhtRunner:
"""Async wrapper around opendht.DhtRunner.
"""
__slots__ = ("_dht", "_loop", "_config", "_bootstrap")
__slots__ = ("_dht", "_loop", "_config", "_bootstrap", "_status_generation")
def __init__(
self,
@@ -149,6 +165,7 @@ class DhtRunner:
self._dht = core or _dht.DhtRunner()
self._config = config
self._bootstrap = list(bootstrap) if bootstrap is not None else None
self._status_generation = [0]
def run(
self,
@@ -168,9 +185,32 @@ class DhtRunner:
def bootstrap(self, host: str, port: Optional[str] = None) -> None:
self._dht.bootstrap(host, port)
def setOnStatusChanged(
self, callback: Optional[Callable[[NodeStatus, NodeStatus], None]] = None,
) -> None:
if callback is not None and not callable(callback):
raise TypeError("callback must be callable or None")
state = self._status_generation
state[0] += 1
if callback is None:
self._dht.setOnStatusChanged(None)
return
generation = state[0]
loop = self._loop
def _deliver(ipv4: NodeStatus, ipv6: NodeStatus) -> None:
if state[0] == generation:
callback(ipv4, ipv6)
def _on_status(ipv4: NodeStatus, ipv6: NodeStatus) -> None:
if not loop.is_closed():
loop.call_soon_threadsafe(_deliver, ipv4, ipv6)
self._dht.setOnStatusChanged(_on_status)
async def ping(self, addr: SockAddr) -> bool:
fut: asyncio.Future[bool] = self._loop.create_future()
self._dht.ping(addr, lambda ok: self._loop.call_soon_threadsafe(fut.set_result, bool(ok)))
self._dht.ping(addr, lambda ok: self._loop.call_soon_threadsafe(_set_result_if_pending, fut, bool(ok)))
return await fut
def get(
@@ -179,9 +219,25 @@ class DhtRunner:
*,
where: Optional[Where] = None,
filter: Optional[Callable[[Value], bool]] = None,
):
) -> _GetStream[Value]:
# Return a streaming iterator of values; it ends on completion and raises on failure.
return _GetStream(self, key, where, filter)
return _GetStream[Value](self, key, where, filter)
def query(
self, key: InfoHash, *, query: Optional[Query] = None,
) -> _GetStream[dict[ValueField, Any]]:
if query is None:
query = Query("SELECT id,value_type,owner_pk,user_type")
return _GetStream[dict[ValueField, Any]](self, key, None, None, query)
async def queryAll(
self, key: InfoHash, *, query: Optional[Query] = None,
) -> List[dict[ValueField, Any]]:
rows: List[dict[ValueField, Any]] = []
with self.query(key, query=query) as stream:
async for row in stream:
rows.append(row)
return rows
async def getAll(
self,
@@ -220,7 +276,7 @@ class DhtRunner:
fut: asyncio.Future[bool] = self._loop.create_future()
def _done(ok: bool, _nodes: Iterable[Any]) -> None:
self._loop.call_soon_threadsafe(fut.set_result, bool(ok))
self._loop.call_soon_threadsafe(_set_result_if_pending, fut, bool(ok))
self._dht.put(key, val, _done, permanent)
return await fut
@@ -229,7 +285,7 @@ class DhtRunner:
fut: asyncio.Future[bool] = self._loop.create_future()
def _done(ok: bool, _nodes: Iterable[Any]) -> None:
self._loop.call_soon_threadsafe(fut.set_result, bool(ok))
self._loop.call_soon_threadsafe(_set_result_if_pending, fut, bool(ok))
self._dht.putSigned(key, val, _done, permanent)
return await fut
@@ -245,20 +301,23 @@ class DhtRunner:
fut: asyncio.Future[bool] = self._loop.create_future()
def _done(ok: bool, _nodes: Iterable[Any]) -> None:
self._loop.call_soon_threadsafe(fut.set_result, bool(ok))
self._loop.call_soon_threadsafe(_set_result_if_pending, fut, bool(ok))
self._dht.putEncrypted(key, val, to, _done, permanent)
return await fut
def listen(self, key: InfoHash) -> _Listener:
return _Listener(self, key)
def listen(
self, key: InfoHash, *, where: Optional[Where] = None,
filter: Optional[Callable[[Value], bool]] = None,
) -> _Listener:
return _Listener(self, key, where, filter)
async def cancelListen(self, listener: _Listener) -> None:
await listener.cancel()
listener.cancel()
async def shutdown(self) -> None:
fut: asyncio.Future[None] = self._loop.create_future()
self._dht.shutdown(lambda: self._loop.call_soon_threadsafe(fut.set_result, None))
self._dht.shutdown(lambda: self._loop.call_soon_threadsafe(_set_result_if_pending, fut, None))
await fut
async def join(self) -> None:
+176 -47
View File
@@ -98,49 +98,53 @@ ctypedef vector[uint8_t] Blob
cdef extern from "opendht/crypto.h" namespace "dht::crypto":
ctypedef pair[shared_ptr[PrivateKey], shared_ptr[Certificate]] Identity
cdef Identity generateIdentity(string name, Identity ca, unsigned bits)
cdef Identity generateIdentity(string name, Identity ca, unsigned bits) except +
cdef Blob aesEncrypt(Blob data, string password) except +
cdef Blob aesDecrypt(Blob encrypted, string password) except +
cdef cppclass PrivateKey:
PrivateKey()
shared_ptr[PublicKey] getSharedPublicKey() const
Blob decrypt(Blob data) const
Blob sign(Blob data) const
PrivateKey() except +
PrivateKey(Blob data, string password) except +
shared_ptr[PublicKey] getSharedPublicKey() except +
Blob serialize(string password) except +
Blob decrypt(Blob data) except +
Blob sign(Blob data) except +
@staticmethod
PrivateKey generate()
PrivateKey generate() except +
@staticmethod
PrivateKey generateEC()
PrivateKey generateEC() except +
cdef cppclass PublicKey:
PublicKey()
InfoHash getId() const
PkId getLongId() const
Blob encrypt(Blob data) const
bool checkSignature(Blob data, Blob signature) const
PublicKey() except +
PublicKey(Blob data) except +
InfoHash getId() except +
PkId getLongId() except +
void pack(Blob& data) except +
Blob encrypt(Blob data) except +
bool checkSignature(Blob data, Blob signature) except +
bool operator==(PublicKey) const
cdef cppclass Certificate:
Certificate()
Certificate(string pem)
InfoHash getId() const
PkId getLongId() const
string toString() const
string getName() const
void revoke(PrivateKey key, Certificate cert)
Certificate() except +
Certificate(string pem) except +
InfoHash getId() except +
PkId getLongId() except +
string toString() except +
string getName() except +
void revoke(PrivateKey key, Certificate cert) except +
@staticmethod
Certificate generate(PrivateKey key, string name, Identity ca, bool is_ca)
Certificate generate(PrivateKey key, string name, Identity ca, bool is_ca) except +
shared_ptr[Certificate] issuer
cdef cppclass TrustList:
cppclass VerifyResult:
bool operator bool() const
bool isValid() const
string toString() const
TrustList()
void add(Certificate)
void remove(Certificate)
VerifyResult verify(Certificate);
string toString() except +
TrustList() except +
void add(Certificate) except +
void remove(Certificate) except +
VerifyResult verify(Certificate) except +
ctypedef TrustList.VerifyResult TrustListVerifyResult
@@ -151,6 +155,7 @@ cdef extern from "opendht/value.h" namespace "dht::Value":
cdef extern from "opendht/value.h" namespace "dht::Value::Field":
cdef Field None
cdef Field Id
cdef Field ValueTypeField "dht::Value::Field::ValueType"
cdef Field OwnerPk
cdef Field SeqNum
cdef Field UserType
@@ -163,13 +168,20 @@ cdef extern from "opendht/value.h" namespace "dht":
Value(const uint16_t t, const uint8_t* dat_ptr, size_t dat_len) except +
string toString() const
size_t size() const
bool isSigned() const
bool isEncrypted() const
uint64_t id
shared_ptr[PublicKey] owner
InfoHash recipient
vector[uint8_t] data
string user_type
uint16_t type
uint16_t seq
unsigned priority
string pushType
cdef cppclass ValueType:
ValueType() except +
ValueType(uint16_t id, string name, seconds expiration) except +
uint16_t id
string name
@@ -191,7 +203,7 @@ cdef extern from "opendht/value.h" namespace "dht":
cdef cppclass Where:
Where() except +
Where(const string& q_str)
Where(const string& q_str) except +
bool isSatisfiedBy(const Where& where) const
Where& id(uint64_t id)
Where& valueType(uint16_t type)
@@ -200,6 +212,14 @@ cdef extern from "opendht/value.h" namespace "dht":
Where& userType(string user_type)
string toString() const
cdef cppclass FieldValue:
uint64_t getInt() const
InfoHash getHash() const
Blob getBlob() const
cdef cppclass FieldValueIndex:
map[Field, FieldValue] index
cdef extern from "opendht/node.h" namespace "dht":
cdef cppclass Node:
Node() except +
@@ -208,6 +228,30 @@ cdef extern from "opendht/node.h" namespace "dht":
bool isExpired() const
cdef extern from "opendht/callbacks.h" namespace "dht":
cdef cppclass NodeStats:
unsigned good_nodes
unsigned dubious_nodes
unsigned cached_nodes
unsigned incoming_nodes
unsigned table_depth
unsigned searches
unsigned node_cache_size
unsigned getKnownNodes() const
unsigned long getNetworkSizeEstimation() const
cdef cppclass NodeInfo:
InfoHash id
InfoHash node_id
NodeStats ipv4
NodeStats ipv6
size_t ongoing_ops
size_t storage_values
size_t storage_size
size_t local_storage_values
size_t local_storage_size
in_port_t bound4
in_port_t bound6
ctypedef void (*ShutdownCallbackRaw)(void *user_data)
ctypedef bool (*GetCallbackRaw)(shared_ptr[Value] values, void *user_data)
ctypedef bool (*ValueCallbackRaw)(shared_ptr[Value] values, bool expired, void *user_data)
@@ -243,9 +287,84 @@ cdef extern from "opendht/callbacks.h" namespace "dht":
Config node_config
Identity id
cdef extern from *:
"""
#include <Python.h>
#include <opendht/dhtrunner.h>
static std::shared_ptr<PyObject> opendht_python_owner(PyObject* data)
{
auto state = PyGILState_Ensure();
Py_INCREF(data);
PyGILState_Release(state);
return std::shared_ptr<PyObject>(data, [](PyObject* object) {
auto state = PyGILState_Ensure();
Py_DECREF(object);
PyGILState_Release(state);
});
}
static dht::ValueCallback opendht_bind_python_value_cb(
dht::ValueCallbackRaw callback, PyObject* data)
{
auto owner = opendht_python_owner(data);
return [callback, owner = std::move(owner)](
const std::vector<std::shared_ptr<dht::Value>>& values, bool expired) {
for (const auto& value : values)
if (!callback(value, expired, owner.get()))
return false;
return true;
};
}
static dht::DhtRunner::Context opendht_python_context(
void (*callback)(int, int, void*), PyObject* data)
{
dht::DhtRunner::Context context;
auto owner = opendht_python_owner(data);
context.statusChangedCallback = [callback, owner = std::move(owner)](
dht::NodeStatus ipv4, dht::NodeStatus ipv6) {
callback(static_cast<int>(ipv4), static_cast<int>(ipv6), owner.get());
};
return context;
}
static void opendht_python_run(dht::DhtRunner& runner, in_port_t port,
dht::DhtRunner::Config& config, void (*callback)(int, int, void*), PyObject* data)
{
runner.run(port, config, opendht_python_context(callback, data));
}
static void opendht_python_run(dht::DhtRunner& runner,
const char* ipv4, const char* ipv6, const char* port,
dht::DhtRunner::Config& config, void (*callback)(int, int, void*), PyObject* data)
{
runner.run(ipv4, ipv6, port, config, opendht_python_context(callback, data));
}
static void opendht_python_query(dht::DhtRunner& runner, const dht::InfoHash& key,
bool (*callback)(std::shared_ptr<dht::FieldValueIndex>, void*),
dht::DoneCallbackRaw done, PyObject* data, dht::Query query)
{
auto owner = opendht_python_owner(data);
runner.query(key,
[callback, owner](const std::vector<std::shared_ptr<dht::FieldValueIndex>>& rows) {
for (const auto& row : rows)
if (!callback(row, owner.get()))
return false;
return true;
},
[done, owner](bool ok, const std::vector<std::shared_ptr<dht::Node>>& nodes) {
done(ok, const_cast<std::vector<std::shared_ptr<dht::Node>>*>(&nodes), owner.get());
}, std::move(query));
}
"""
ValueCallback bindPythonValueCb "opendht_bind_python_value_cb"(ValueCallbackRaw callback, object data) except +
ctypedef future[size_t] ListenToken
ctypedef shared_future[size_t] SharedListenToken
cdef extern from "opendht/dhtrunner.h" namespace "dht":
ctypedef future[size_t] ListenToken
ctypedef shared_future[size_t] SharedListenToken
cdef cppclass DhtRunner:
DhtRunner() except +
cppclass Config:
@@ -264,37 +383,47 @@ cdef extern from "opendht/dhtrunner.h" namespace "dht":
SockAddr bind4
SockAddr bind6
InfoHash getId() const
shared_ptr[PublicKey] getPublicKey() const
InfoHash getNodeId() const
InfoHash getId() except + nogil
shared_ptr[PublicKey] getPublicKey() except + nogil
InfoHash getNodeId() except + nogil
NodeInfo getNodeInfo() except + nogil
vector[SockAddr] getPublicAddress(sa_family_t af) except + nogil
void connectivityChanged() except + nogil
void bootstrap(const_char*, const_char*)
void bootstrap(const SockAddr&, DoneCallbackSimple done_cb)
void run(in_port_t, Config config)
void run(const_char*, const_char*, const_char*, Config config)
void join()
void run(in_port_t, Config config) except +
void run(const_char*, const_char*, const_char*, Config config) except +
void join() except + nogil
void shutdown(ShutdownCallback)
void registerType(ValueType& value);
void registerType(const ValueType& value) except + nogil
bool isRunning()
SockAddr getBound(sa_family_t af) const
string getStorageLog() const
string getRoutingTablesLog(sa_family_t af) const
string getSearchesLog(sa_family_t af) const
SockAddr getBound(sa_family_t af) except + nogil
string getStorageLog() except + nogil
string getRoutingTablesLog(sa_family_t af) except + nogil
string getSearchesLog(sa_family_t af) except + nogil
void get(InfoHash key, GetCallback get_cb, DoneCallback done_cb, nullptr_t f, Where w)
void put(InfoHash key, shared_ptr[Value] val, DoneCallback done_cb, time_point created, bool permanent)
void putSigned(InfoHash key, shared_ptr[Value] val, DoneCallback done_cb, bool permanent)
void putEncrypted(InfoHash key, PkId to, shared_ptr[Value] val, DoneCallback done_cb, bool permanent)
void putEncrypted(InfoHash key, shared_ptr[PublicKey] to, shared_ptr[Value] val, DoneCallback done_cb, bool permanent)
void cancelPut(InfoHash key, shared_ptr[Value] val)
ListenToken listen(InfoHash key, ValueCallback get_cb)
void cancelListen(InfoHash key, SharedListenToken token)
vector[unsigned] getNodeMessageStats(bool i)
ListenToken listen(InfoHash key, ValueCallback get_cb, nullptr_t f, Where w) except +
void cancelListen(InfoHash key, SharedListenToken token) except +
vector[unsigned] getNodeMessageStats(bool i) except + nogil
ctypedef DhtRunner.Config DhtRunnerConfig
cdef extern from *:
ctypedef void (*StatusCallbackRaw)(int ipv4, int ipv6, void* data)
ctypedef bool (*QueryCallbackRaw)(shared_ptr[FieldValueIndex] row, void* data)
void runPythonRunner "opendht_python_run"(DhtRunner& runner, in_port_t port, DhtRunnerConfig& config, StatusCallbackRaw callback, object data) except + nogil
void runPythonRunnerAddress "opendht_python_run"(DhtRunner& runner, const_char* ipv4, const_char* ipv6, const_char* port, DhtRunnerConfig& config, StatusCallbackRaw callback, object data) except + nogil
void queryPython "opendht_python_query"(DhtRunner& runner, InfoHash key, QueryCallbackRaw callback, DoneCallbackRaw done, object data, Query query) except +
cdef extern from "opendht/log.h" namespace "dht::log":
void enableLogging(DhtRunner& dht)
void disableLogging(DhtRunner& dht)
void enableFileLogging(DhtRunner& dht, const string& path)
void enableLogging(DhtRunner& dht) except + nogil
void disableLogging(DhtRunner& dht) except + nogil
void enableFileLogging(DhtRunner& dht, const string& path) except + nogil
cdef extern from "opendht/indexation/pht.h" namespace "dht::indexation":
size_t PHT_MAX_NODE_ENTRY_COUNT "dht::indexation::Pht::MAX_NODE_ENTRY_COUNT"
@@ -310,8 +439,8 @@ cdef extern from "opendht/indexation/pht.h" namespace "dht::indexation":
cppclass LookupCallback:
LookupCallback() except +
Pht(string, IndexKeySpec, shared_ptr[DhtRunner]) except +
void lookup(IndexKey k, LookupCallback cb, DoneCallbackSimple doneCb);
void insert(IndexKey k, IndexValue v, DoneCallbackSimple cb)
void lookup(IndexKey k, LookupCallback cb, DoneCallbackSimple doneCb) except +
void insert(IndexKey k, IndexValue v, DoneCallbackSimple cb) except +
@staticmethod
LookupCallback bindLookupCb(LookupCallbackRaw cb, void *user_data)
+5 -5
View File
@@ -4,11 +4,11 @@ build-backend = "setuptools.build_meta"
[tool.cibuildwheel]
build = [
'cp3{9,10,11,12,13,14}-manylinux_x86_64',
'cp3{10,11,12,13,14}-manylinux_aarch64',
'cp3{11,12,13,14}-macosx_x86_64',
'cp3{11,12,13,14}-macosx_arm64',
'cp3{11,12,13,14}-win_amd64',
'cp3{11,12,13,14,15}-manylinux_x86_64',
'cp3{11,12,13,14,15}-manylinux_aarch64',
'cp3{11,12,13,14,15}-macosx_x86_64',
'cp3{11,12,13,14,15}-macosx_arm64',
'cp3{11,12,13,14,15}-win_amd64',
]
[tool.cibuildwheel.linux]
+35
View File
@@ -0,0 +1,35 @@
import runpy
import unittest
from pathlib import Path
from unittest.mock import Mock, patch
import netifaces
class NodeClusterTester(unittest.TestCase):
@classmethod
def setUpClass(cls):
script = Path(__file__).resolve().parents[1] / 'tools' / 'dhtcluster.py'
with patch.dict('sys.modules', {'opendht': Mock()}):
cls.NodeCluster = runpy.run_path(str(script))['NodeCluster']
def test_find_ip_interface(self):
addresses = {
netifaces.AF_INET: [{'addr': '127.0.0.1'}],
netifaces.AF_INET6: [{'addr': '::1'}],
}
with patch('netifaces.ifaddresses', return_value=addresses) as ifaddresses:
self.assertEqual(self.NodeCluster.find_ip('lo'), ('127.0.0.1', '::1'))
self.assertTrue(ifaddresses.called)
self.assertTrue(all(call.args == ('lo',) for call in ifaddresses.call_args_list))
def test_find_ip_any(self):
with patch('netifaces.ifaddresses') as ifaddresses:
for iface in (None, '', 'any'):
with self.subTest(iface=iface):
self.assertEqual(self.NodeCluster.find_ip(iface), ('0.0.0.0', '::0'))
ifaddresses.assert_not_called()
if __name__ == '__main__':
unittest.main()
File diff suppressed because it is too large. Load diff
+1
View File
@@ -9,6 +9,7 @@ import argparse
import logging
import logging.handlers
import netifaces
import opendht as dht
logger = logging.getLogger('dhtcluster')
+22
View File
@@ -0,0 +1,22 @@
This software is licensed under the MIT License.
Copyright Fedor Indutny, 2018.
Permission is hereby granted, free of charge, to any person obtaining a
copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to permit
persons to whom the Software is furnished to do so, subject to the
following conditions:
The above copyright notice and this permission notice shall be included
in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN
NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
USE OR OTHER DEALINGS IN THE SOFTWARE.
+70
View File
@@ -0,0 +1,70 @@
# Building and Testing the Rust Crate
The `opendht` crate links to the native OpenDHT C library. Cargo does not build
or bundle that library. Install Rust and the native dependencies described in
the [OpenDHT build guide](../BUILD.md).
## Install the Native Library
From the root of an OpenDHT source checkout, enable the C bindings:
```sh
cmake -S . -B build-bindings -DOPENDHT_C=ON -DBUILD_TESTING=ON
cmake --build build-bindings
ctest --test-dir build-bindings -R test_c --output-on-failure
cmake --install build-bindings
```
Installation may require administrator privileges, depending on the install
prefix. Set `CMAKE_INSTALL_PREFIX` when configuring to choose another location.
The linker and runtime loader must be able to find the installed libraries.
Applications can then depend on `opendht` from crates.io as shown in the
[README](README.md).
## Use an Uninstalled Native Build
You can use the native build directly by setting its library search paths.
For example, from the repository root on macOS:
```sh
export LIBRARY_PATH="$PWD/build-bindings${LIBRARY_PATH:+:$LIBRARY_PATH}"
export DYLD_LIBRARY_PATH="$PWD/build-bindings${DYLD_LIBRARY_PATH:+:$DYLD_LIBRARY_PATH}"
export RUSTFLAGS="-L native=$PWD/build-bindings"
cargo test --manifest-path rust/Cargo.toml --all-targets
```
On Linux use `LD_LIBRARY_PATH` instead of `DYLD_LIBRARY_PATH`; on Windows add
the native DLL directory to `PATH`. For an application outside this checkout,
use absolute paths to the native build and run Cargo in the application directory.
When linking an AddressSanitizer native build, use the same compiler runtime.
For example, with Clang:
```sh
export RUSTFLAGS="-L native=$PWD/build-bindings -C linker=clang -C link-arg=-fsanitize=address"
```
## Build and Test the Crate
With the native library available, run from the `rust` directory:
```sh
cargo build --all-targets
cargo test --all-targets
cargo test --doc
cargo test --all-features --all-targets
cargo test --all-features --doc
```
## Run the Examples
From the `rust` directory:
```sh
cargo run --example dhtnode
cargo run --features async --example dhtnode_async
cargo run --features serde --example dhtnode_async
```
The callback example contacts a public bootstrap server. The async example uses
two loopback nodes; enabling `serde` also demonstrates typed MessagePack records.
+23 -4
View File
@@ -1,11 +1,30 @@
[package]
name = "opendht"
version = "0.1.0"
authors = ["Sébastien Blin <sebastien.blin@savoirfairelinux.com>"]
edition = "2021"
version = "0.6.0"
authors = ["Sébastien Blin <sebastien.blin@savoirfairelinux.com>", "Adrien Beraud <adrien.beraud@savoirfairelinux.com>"]
edition = "2024"
description = "Rust bindings for OpenDHT"
license = "MIT"
license-file = "../LICENSE"
repository = "https://github.com/savoirfairelinux/opendht"
[features]
async = ["dep:futures-channel", "dep:futures-core"]
serde = ["async", "dep:serde", "dep:rmp-serde"]
[[example]]
name = "dhtnode_async"
required-features = ["async"]
[dependencies]
libc="0.2"
os_socketaddr="0.2.3"
futures-channel = { version = "0.3", optional = true }
futures-core = { version = "0.3", optional = true }
serde = { version = "1", optional = true }
rmp-serde = { version = "1", optional = true }
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dev-dependencies]
futures = "0.3"
serde = { version = "1", features = ["derive"] }
serde_bytes = "0.11"
+142 -11
View File
@@ -1,21 +1,152 @@
# Rust bindings for opendht
# OpenDHT: a Distributed Hash Table
## Build
OpenDHT is a Kademlia-based distributed hash table for storing and retrieving
small values over a peer-to-peer network without a central server. Common uses
include peer discovery, service announcements, and decentralized applications.
Install rust on your system (https://rustup.rs/ is the recommanded way). Once done:
The `opendht` crate provides Rust bindings to [OpenDHT](https://opendht.net), with
callback-based and runtime-independent async APIs. Capabilities include signed
and encrypted values, filtered reads, metadata queries, and typed MessagePack
records through Serde.
```
cargo build
## Get Started
Add the crate to your dependencies:
```toml
[dependencies]
opendht = "0.6"
```
## Run tests
The native OpenDHT library with C bindings must be installed separately; it is
not bundled with the crate. See the
[build guide](https://github.com/savoirfairelinux/opendht/blob/master/rust/BUILD.md)
for native installation, library paths, and source builds.
```
cargo test
`DhtRunner` manages a node, `InfoHash` identifies a DHT key, and `Value` contains
the payload and metadata stored under that key. `put` publishes values, `get`
retrieves them, and `listen` subscribes to value updates and expiration events.
This example starts a node on an OS-assigned port, bootstraps into the public
network, and listens for a value it publishes.
```rust
use opendht::{DhtRunner, InfoHash, Value};
use std::{sync::mpsc, time::Duration};
fn main() -> Result<(), Box<dyn std::error::Error>> {
let mut runner = DhtRunner::new();
runner.run(0)?;
runner.bootstrap("bootstrap.jami.net", 4222)?;
let key = InfoHash::get("my-app/hello");
let (sender, receiver) = mpsc::channel();
let listener = runner.listen(&key, move |value, expired| {
if !expired {
let _ = sender.send(value);
}
true
})?;
runner.put(&key, Value::new("Hello from Rust!"), |_| {}, false);
let value = receiver.recv_timeout(Duration::from_secs(5))?;
println!("{}", String::from_utf8_lossy(value.as_slice()));
runner.cancel_listen(&key, listener);
runner.join()?;
Ok(())
}
```
## Run dht node
A bootstrap server supplies initial peers. Applications can use public
bootstrap servers or known peers on a private network. Values expire, and their
availability depends on participating nodes; the DHT does not provide durable
storage guarantees.
## Optional Features
| Feature | Capabilities |
|---------|--------------|
| Default | Callback-based `DhtRunner`, binary values, keys, certificates, and signed/encrypted operations. |
| `async` | `aio::AsyncDhtRunner` with publication futures, bounded read/update streams, and automatic listener cancellation on drop. |
| `serde` | Serde-backed MessagePack records with `put_typed`, `get_typed`, and `listen_typed`. Enables `async`. |
Enable `async` in the dependency configuration:
```toml
[dependencies]
opendht = { version = "0.6", features = ["async"] }
futures = "0.3"
```
cargo run --example dhtnode
```
Create the adapter with `AsyncDhtRunner::new(runner)` after starting a
`DhtRunner`. The adapter is runtime-independent and thread-bound; use a local
executor or Tokio's `LocalSet`. Dropping a listener stream cancels its
subscription. Streams use bounded buffers and report `WouldBlock` on overflow.
This example waits for a bootstrap peer, publishes a value with `.await`, and
reads matching values as a stream:
```rust,no_run
use futures::{executor::block_on, TryStreamExt};
use opendht::{aio::AsyncDhtRunner, DhtRunner, InfoHash, Value};
use std::{io, net::ToSocketAddrs};
fn main() -> io::Result<()> {
let mut runner = DhtRunner::new();
runner.run(0)?;
let runner = AsyncDhtRunner::new(runner);
let peers: Vec<_> = ("bootstrap.jami.net", 4222).to_socket_addrs()?.collect();
block_on(async {
runner.bootstrap(&peers).await?;
let key = InfoHash::get("my-app/hello-async");
runner.put(&key, Value::new("Hello from async Rust!"), false).await?;
let mut values = runner.get(&key);
while let Some(value) = values.try_next().await? {
println!("{}", String::from_utf8_lossy(value.as_slice()));
}
runner.shutdown().await?;
runner.join()
})
}
```
### Typed Records
The `serde` feature adds typed async operations and also enables `async`.
`put_typed` serializes records implementing `Serialize`; `get_typed` and
`listen_typed` decode records implementing `DeserializeOwned`. Typed reads retain
the decoded record alongside its original payload and native metadata.
Structs are serialized as compact MessagePack arrays. Field order is part of
the wire format and must remain compatible across schema changes. `serde_bytes`
encodes binary fields using MessagePack's binary representation.
## Working With Values
GET and LISTEN callbacks and async streams deliver owned `Box<Value>` instances.
Received values implement `Send` but not `Sync`; they can be retained after the
runner stops, moved to another thread, modified, and republished. `as_slice()`
borrows the payload without copying; `as_bytes()` allocates a separate `Vec<u8>`.
Values expose metadata including IDs, types, sequence numbers, and owner keys.
Filters select values by metadata, and queries return selected metadata fields.
`DhtRunner::listen` returns a token that must be passed to `cancel_listen` to
unsubscribe. Dropping the token does not cancel the registration; dropping an
async listener stream does.
`join()` stops the runner and waits for its threads to exit. It must not be
called from a callback. For graceful shutdown and configured state persistence,
wait for `shutdown` to complete before calling `join()`.
## Learn More
- [API reference](https://docs.rs/opendht)
- [Callback example](https://github.com/savoirfairelinux/opendht/blob/master/rust/examples/dhtnode.rs)
- [Async and typed two-node example](https://github.com/savoirfairelinux/opendht/blob/master/rust/examples/dhtnode_async.rs)
- [Build and test guide](https://github.com/savoirfairelinux/opendht/blob/master/rust/BUILD.md)
- [OpenDHT project and issue tracker](https://github.com/savoirfairelinux/opendht)
+19 -16
View File
@@ -2,19 +2,18 @@
// SPDX-License-Identifier: MIT
extern crate opendht;
use std::{ thread, time };
use std::{thread, time};
use opendht::{ InfoHash, DhtRunner, DhtRunnerConfig, Value };
use opendht::{DhtRunner, DhtRunnerConfig, InfoHash, Value};
// use opendht::crypto::*;
fn main() {
fn main() -> std::io::Result<()> {
println!("{}", InfoHash::random());
println!("{}", InfoHash::new());
println!("{}", InfoHash::new().is_zero());
println!("{}", InfoHash::get("alice"));
println!("{}", InfoHash::get("alice").is_zero());
let mut dht = DhtRunner::new();
let /*mut*/ config = DhtRunnerConfig::new();
//// If you want to inject a certificate, uncomment the following lines and previous mut.
@@ -23,36 +22,40 @@ fn main() {
//let cert = DhtCertificate::import("example.crt").ok().expect("Invalid cert file");
//let pk = PrivateKey::import("example.key", "");
//config.set_identity(cert, pk);
dht.run_config(1412, config);
dht.bootstrap("bootstrap.jami.net", 4222);
dht.run_config(1412, config)?;
dht.bootstrap("bootstrap.jami.net", 4222)?;
println!("Current node id: {}", dht.node_id());
let /* mut */ data = 42;
let mut get_cb = |v: Box<Value>| {
let get_cb = move |v: Box<Value>| {
//data += 1;
println!("GET: VALUE CB - data: {} - v: {}", data, v);
true
};
let mut done_cb = |ok: bool| {
let done_cb = move |ok: bool| {
println!("GET: DONE CB - data: {} - ok: {}", data, ok);
};
dht.get(&InfoHash::get("alice"), &mut get_cb, &mut done_cb);
dht.get(&InfoHash::get("alice"), get_cb, done_cb);
let mut put_done_cb = |ok: bool| {
let put_done_cb = move |ok: bool| {
println!("PUT: DONE CB - data: {} - ok: {}", data, ok);
};
dht.put(&InfoHash::get("bob"), Value::new("hi!"), &mut put_done_cb, false);
dht.put(&InfoHash::get("bob"), Value::new("hi!"), put_done_cb, false);
println!("Start listening /foo");
let mut value_cb = |v, expired| {
println!("LISTEN: DONE CB - data: {} - v: {} - expired: {}", data, v, expired);
let value_cb = move |v, expired| {
println!(
"LISTEN: DONE CB - data: {} - v: {} - expired: {}",
data, v, expired
);
true
};
let token = dht.listen(&InfoHash::get("foo"), &mut value_cb);
let token = dht.listen(&InfoHash::get("foo"), value_cb)?;
let one_min = time::Duration::from_secs(10);
thread::sleep(one_min);
dht.cancel_listen(&InfoHash::get("foo"), token);
println!("Public ips: {:#?}", dht.public_addresses());
}
dht.join()?;
Ok(())
}
+80
View File
@@ -0,0 +1,80 @@
use futures::{executor::block_on, StreamExt, TryStreamExt};
use opendht::{aio::AsyncDhtRunner, DhtRunner, DhtRunnerConfig, InfoHash, Value};
use std::error::Error;
use std::io;
use std::net::SocketAddr;
fn main() -> Result<(), Box<dyn Error>> {
let mut source = DhtRunner::new();
let mut destination = DhtRunner::new();
source.run_bind("127.0.0.1", "::1", 0, DhtRunnerConfig::new())?;
destination.run_bind("127.0.0.1", "::1", 0, DhtRunnerConfig::new())?;
let address = SocketAddr::from(([127, 0, 0, 1], destination.bound_port(libc::AF_INET as _)));
let source = AsyncDhtRunner::new(source);
let destination = AsyncDhtRunner::new(destination);
block_on(async {
source.bootstrap(&[address]).await?;
let hash = InfoHash::get("async example");
let mut updates = destination.listen(&hash)?;
source
.put(&hash, Value::with_id(b"hello from Rust async", 42), false)
.await?;
let event = updates
.next()
.await
.ok_or_else(|| io::Error::other("listener ended before receiving a value"))??;
println!(
"LISTEN {}: {} (expired: {})",
event.value.id(),
String::from_utf8_lossy(event.value.as_slice()),
event.expired,
);
drop(updates);
let values: Vec<_> = destination.get(&hash).try_collect().await?;
println!("GET: {} values", values.len());
#[cfg(feature = "serde")]
{
#[derive(Debug, serde::Serialize, serde::Deserialize)]
struct Service {
version: u32,
name: String,
port: u16,
#[serde(with = "serde_bytes")]
payload: Vec<u8>,
}
let hash = InfoHash::get("typed async example");
let mut updates = destination.listen_typed::<Service>(&hash)?;
let record = Service {
version: 1,
name: "chat".into(),
port: 4222,
payload: vec![0, 0x80, 0xff],
};
source.put_typed(&hash, &record, false).await?;
let event = updates.next().await.ok_or_else(|| {
io::Error::other("typed listener ended before receiving a value")
})??;
println!(
"TYPED LISTEN {}: {:?}",
event.value.raw.id(),
event.value.data
);
drop(updates);
let values: Vec<_> = destination
.get_typed::<Service>(&hash)
.try_collect()
.await?;
println!("TYPED GET: {} records", values.len());
}
source.shutdown().await?;
destination.shutdown().await?;
source.join()?;
destination.join()?;
Ok(())
})
}
+429
View File
@@ -0,0 +1,429 @@
//! Runtime-independent futures and bounded streams, enabled by the `async` feature.
//!
//! Runners and listeners stay on their creating thread. Use a local executor or
//! Tokio `LocalSet`; received values own their native storage and can be
//! transferred between threads without another payload copy.
use crate::{DhtRunner, InfoHash, OpToken, PkId, QueryResult, Value};
use futures_channel::{mpsc, oneshot};
use futures_core::Stream;
use std::cell::RefCell;
use std::future::Future;
use std::io;
use std::net::SocketAddr;
use std::pin::Pin;
use std::rc::Rc;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Arc, Mutex};
use std::task::{Context, Poll};
/// A listener update; expiration events retain the expired value's metadata.
pub struct ValueEvent {
pub value: Box<Value>,
pub expired: bool,
}
/// A native completion mapped to an I/O result.
///
/// Operations start immediately, before this future is polled. Dropping it
/// abandons the result but does not undo a put, ping, bootstrap, or shutdown.
#[must_use = "operations run immediately; await their result or explicitly drop it"]
pub struct Operation {
receiver: oneshot::Receiver<io::Result<()>>,
}
impl Future for Operation {
type Output = io::Result<()>;
fn poll(mut self: Pin<&mut Self>, context: &mut Context<'_>) -> Poll<Self::Output> {
Pin::new(&mut self.receiver).poll(context).map(|outcome| {
outcome.unwrap_or_else(|_| Err(stopped("DHT operation ended without completion")))
})
}
}
fn operation() -> (oneshot::Sender<io::Result<()>>, Operation) {
let (sender, receiver) = oneshot::channel();
(sender, Operation { receiver })
}
fn outcome(success: bool) -> io::Result<()> {
if success {
Ok(())
} else {
Err(io::Error::other(
"DHT operation failed or the runner stopped",
))
}
}
fn stopped(message: &'static str) -> io::Error {
io::Error::new(io::ErrorKind::Interrupted, message)
}
#[derive(Clone)]
struct StreamCompletion(Arc<Mutex<Option<oneshot::Sender<io::Result<()>>>>>);
impl StreamCompletion {
fn finish(&self, result: io::Result<()>) {
let sender = self
.0
.lock()
.unwrap_or_else(|error| error.into_inner())
.take();
if let Some(sender) = sender {
let _ = sender.send(result);
}
}
}
struct StreamSender<T> {
values: mpsc::Sender<T>,
completion: StreamCompletion,
}
impl<T> StreamSender<T> {
fn send(&mut self, value: io::Result<T>) -> bool {
let value = match value {
Ok(value) => value,
Err(error) => {
self.completion.finish(Err(error));
return false;
}
};
match self.values.try_send(value) {
Ok(()) => true,
Err(error) => {
if error.is_full() {
self.completion.finish(Err(io::Error::new(
io::ErrorKind::WouldBlock,
"DHT stream capacity exceeded; restart the operation to resynchronize",
)));
}
false
}
}
}
}
/// A bounded get or query stream.
///
/// Buffered items precede a terminal error. Native failure, runner shutdown,
/// and buffer overflow are never reported as successful EOF.
/// Dropping the stream stops delivery; native work stops at its next callback
/// or when the runner is joined.
#[must_use = "streams must be polled to receive values and completion errors"]
pub struct OperationStream<T> {
values: mpsc::Receiver<T>,
completion: oneshot::Receiver<io::Result<()>>,
completed: bool,
outcome: Option<io::Result<()>>,
}
impl<T: Unpin> Stream for OperationStream<T> {
type Item = io::Result<T>;
fn poll_next(self: Pin<&mut Self>, context: &mut Context<'_>) -> Poll<Option<Self::Item>> {
let stream = self.get_mut();
if !stream.completed
&& let Poll::Ready(result) = Pin::new(&mut stream.completion).poll(context)
{
stream.completed = true;
stream.outcome = Some(
result.unwrap_or_else(|_| Err(stopped("DHT stream ended without completion"))),
);
stream.values.close();
}
match Pin::new(&mut stream.values).poll_next(context) {
Poll::Ready(Some(value)) => Poll::Ready(Some(Ok(value))),
Poll::Ready(None) if stream.completed => {
Poll::Ready(stream.outcome.take().and_then(Result::err).map(Err))
}
_ => Poll::Pending,
}
}
}
fn stream_channel<T>(capacity: usize) -> (StreamSender<T>, OperationStream<T>) {
let (values, receiver) = mpsc::channel(capacity - 1);
let (completed, completion) = oneshot::channel();
(
StreamSender {
values,
completion: StreamCompletion(Arc::new(Mutex::new(Some(completed)))),
},
OperationStream {
values: receiver,
completion,
completed: false,
outcome: None,
},
)
}
struct RunnerState {
runner: RefCell<Box<DhtRunner>>,
stopping: Arc<AtomicBool>,
}
impl Drop for RunnerState {
fn drop(&mut self) {
self.stopping.store(true, Ordering::SeqCst);
}
}
/// A listener registration that cancels itself when dropped or terminated.
///
/// The runner is retained until this registration is dropped. Calling `join()`
/// stops all registrations and makes pending listeners report an error.
#[must_use = "dropping a listener cancels its native registration"]
pub struct Listener {
state: Rc<RunnerState>,
hash: InfoHash,
token: Option<Box<OpToken>>,
stream: OperationStream<ValueEvent>,
}
impl Listener {
fn cancel(&mut self) {
if let Some(token) = self.token.take() {
self.state
.runner
.borrow_mut()
.cancel_listen(&self.hash, token);
}
}
}
impl Stream for Listener {
type Item = io::Result<ValueEvent>;
fn poll_next(mut self: Pin<&mut Self>, context: &mut Context<'_>) -> Poll<Option<Self::Item>> {
let result = Pin::new(&mut self.stream).poll_next(context);
if matches!(result, Poll::Ready(None) | Poll::Ready(Some(Err(_)))) {
self.cancel();
}
result
}
}
impl Drop for Listener {
fn drop(&mut self) {
self.cancel();
}
}
/// A shared, thread-bound async adapter for an already configured runner.
///
/// Clones share a runner without making native handles `Send` or `Sync`.
/// Native calls are short; futures and streams never hold a mutable runner
/// borrow while waiting. The default stream capacity is 64 items.
///
/// ```compile_fail
/// use opendht::{aio::AsyncDhtRunner, DhtRunner};
/// let runner = AsyncDhtRunner::new(DhtRunner::new());
/// std::thread::spawn(move || runner.join().unwrap());
/// ```
#[derive(Clone)]
pub struct AsyncDhtRunner {
state: Rc<RunnerState>,
capacity: usize,
}
impl AsyncDhtRunner {
pub fn new(runner: Box<DhtRunner>) -> Self {
Self::with_capacity(runner, 64).expect("default stream capacity is valid")
}
/// Set the maximum number of queued values per get, query, or listener.
/// A slow consumer receives `WouldBlock` and the stream terminates.
pub fn with_capacity(runner: Box<DhtRunner>, capacity: usize) -> io::Result<Self> {
if capacity == 0 {
return Err(io::Error::new(
io::ErrorKind::InvalidInput,
"stream capacity must be positive",
));
}
Ok(Self {
state: Rc::new(RunnerState {
runner: RefCell::new(runner),
stopping: Arc::new(AtomicBool::new(false)),
}),
capacity,
})
}
pub fn put(&self, hash: &InfoHash, value: Box<Value>, permanent: bool) -> Operation {
let (sender, pending) = operation();
self.state.runner.borrow_mut().put(
hash,
value,
move |success| {
let _ = sender.send(outcome(success));
},
permanent,
);
pending
}
pub fn put_signed(&self, hash: &InfoHash, value: Box<Value>, permanent: bool) -> Operation {
let (sender, pending) = operation();
self.state.runner.borrow_mut().put_signed(
hash,
value,
move |success| {
let _ = sender.send(outcome(success));
},
permanent,
);
pending
}
pub fn put_encrypted(
&self,
hash: &InfoHash,
recipient: &PkId,
value: Box<Value>,
permanent: bool,
) -> Operation {
let (sender, pending) = operation();
self.state.runner.borrow_mut().put_encrypted(
hash,
recipient,
value,
move |success| {
let _ = sender.send(outcome(success));
},
permanent,
);
pending
}
pub fn ping(&self, address: SocketAddr) -> Operation {
let (sender, pending) = operation();
self.state
.runner
.borrow_mut()
.ping(address, move |success| {
let _ = sender.send(outcome(success));
});
pending
}
/// Bootstrap pre-resolved addresses concurrently; one successful peer suffices.
/// DNS resolution is left to the application's executor.
pub fn bootstrap(&self, addresses: &[SocketAddr]) -> Operation {
let (sender, pending) = operation();
if addresses.is_empty() {
let _ = sender.send(Err(io::Error::new(
io::ErrorKind::InvalidInput,
"at least one bootstrap address is required",
)));
return pending;
}
let completion = StreamCompletion(Arc::new(Mutex::new(Some(sender))));
let remaining = Arc::new(std::sync::atomic::AtomicUsize::new(addresses.len()));
for &address in addresses {
let completion = completion.clone();
let remaining = remaining.clone();
self.state
.runner
.borrow_mut()
.ping(address, move |success| {
if success {
completion.finish(Ok(()));
}
if remaining.fetch_sub(1, Ordering::SeqCst) == 1 {
completion.finish(outcome(false));
}
});
}
pending
}
pub fn get(&self, hash: &InfoHash) -> OperationStream<Box<Value>> {
let (mut sender, stream) = stream_channel(self.capacity);
let completion = sender.completion.clone();
self.state.runner.borrow_mut().get(
hash,
move |value| sender.send(Ok(value)),
move |success| completion.finish(outcome(success)),
);
stream
}
pub fn get_filtered(
&self,
hash: &InfoHash,
filter: &str,
) -> io::Result<OperationStream<Box<Value>>> {
let (mut sender, stream) = stream_channel(self.capacity);
let completion = sender.completion.clone();
self.state.runner.borrow_mut().get_filtered(
hash,
filter,
move |value| sender.send(Ok(value)),
move |success| completion.finish(outcome(success)),
)?;
Ok(stream)
}
pub fn query(&self, hash: &InfoHash, query: &str) -> io::Result<OperationStream<QueryResult>> {
let (mut sender, stream) = stream_channel(self.capacity);
let completion = sender.completion.clone();
self.state.runner.borrow_mut().query(
hash,
query,
move |row| sender.send(Ok(row)),
move |success| completion.finish(outcome(success)),
)?;
Ok(stream)
}
pub fn listen(&self, hash: &InfoHash) -> io::Result<Listener> {
self.listen_filtered(hash, "")
}
pub fn listen_filtered(&self, hash: &InfoHash, filter: &str) -> io::Result<Listener> {
let (mut sender, stream) = stream_channel(self.capacity);
let token = self.state.runner.borrow_mut().listen_filtered(
hash,
filter,
move |value, expired| sender.send(Ok(ValueEvent { value, expired })),
)?;
Ok(Listener {
state: self.state.clone(),
hash: InfoHash { d: hash.d },
token: Some(token),
stream,
})
}
/// Request graceful shutdown. Call `join()` after this future succeeds when
/// persistence is required. Joining first reports an interrupted shutdown.
pub fn shutdown(&self) -> Operation {
let (sender, pending) = operation();
let stopping = self.state.stopping.clone();
self.state.runner.borrow_mut().shutdown(move || {
let result = if stopping.load(Ordering::SeqCst) {
Err(stopped("runner joined before graceful shutdown completed"))
} else {
Ok(())
};
let _ = sender.send(result);
});
pending
}
/// Drive a non-threaded runner without waiting for any async operation.
pub fn loop_once(&self) -> io::Result<()> {
self.state.runner.borrow_mut().loop_once()
}
/// Stop the runner and wait for native callbacks. This is a blocking call,
/// not an async wait, and must not be invoked from a native DHT callback.
pub fn join(&self) -> io::Result<()> {
self.state.stopping.store(true, Ordering::SeqCst);
self.state.runner.borrow_mut().join()
}
}
+9 -7
View File
@@ -1,18 +1,20 @@
// Copyright (c) 2014-2026 Savoir-faire Linux Inc.
// SPDX-License-Identifier: MIT
use crate::ffi::*;
use crate::ffi::{self, DataView};
pub use crate::ffi::Blob;
native_handle!(Blob, dht_blob_delete);
impl Blob {
pub fn data(&self) -> DataView {
unsafe { dht_blob_get_data(self) }
unsafe { ffi::dht_blob_get_data(self.as_ptr()) }
}
}
impl Drop for Blob {
fn drop(&mut self) {
unsafe { dht_blob_delete(&mut *self) }
pub fn as_bytes(&self) -> Vec<u8> {
let data = self.data();
if data.size == 0 {
return Vec::new();
}
unsafe { std::slice::from_raw_parts(data.data, data.size).to_vec() }
}
}
+137 -74
View File
@@ -1,150 +1,213 @@
// Copyright (c) 2014-2026 Savoir-faire Linux Inc.
// SPDX-License-Identifier: MIT
#![allow(dead_code)]
pub use crate::ffi::*;
use crate::{ffi, Blob, InfoHash, PkId};
use std::ffi::CString;
use std::fs::File;
use std::fs;
use std::io;
use std::io::prelude::*;
use std::ptr;
native_handle!(PublicKey, dht_publickey_delete);
native_handle!(PrivateKey, dht_privatekey_delete);
native_handle!(DhtCertificate, dht_certificate_delete);
impl PublicKey {
pub fn new(data: &str) -> Box<PublicKey> {
unsafe { Box::from_raw(dht_publickey_import(data.as_ptr(), data.len())) }
pub fn new(data: impl AsRef<[u8]>) -> io::Result<Box<PublicKey>> {
let data = data.as_ref();
unsafe { Self::from_raw(ffi::dht_publickey_import(data.as_ptr(), data.len())) }
}
pub fn unpack(&mut self, data: Vec<u8>) -> i32 {
unsafe { dht_publickey_unpack(&mut *self, data.as_ptr(), data.len()) }
pub fn unpack(&mut self, data: impl AsRef<[u8]>) -> io::Result<()> {
let data = data.as_ref();
let result = unsafe { ffi::dht_publickey_unpack(self.as_ptr(), data.as_ptr(), data.len()) };
if result == 0 {
Ok(())
} else {
Err(io::Error::other("invalid public key"))
}
}
pub fn pack(&mut self, data: &str) -> i32 {
let data = CString::new(data).unwrap();
unsafe { dht_publickey_pack(&mut *self, data.as_ptr(), data.as_bytes().len()) }
pub fn pack(&self) -> io::Result<Vec<u8>> {
unsafe { Ok(Blob::from_raw(ffi::dht_publickey_export_blob(self.as_ptr()))?.as_bytes()) }
}
pub fn id(&self) -> InfoHash {
unsafe { dht_publickey_get_id(self) }
unsafe { ffi::dht_publickey_get_id(self.as_ptr()) }
}
pub fn long_id(&self) -> PkId {
unsafe { dht_publickey_get_long_id(self) }
unsafe { ffi::dht_publickey_get_long_id(self.as_ptr()) }
}
pub fn check_signature(&self, data: &str, signature: &str) -> bool {
let data = CString::new(data).unwrap();
let signature = CString::new(signature).unwrap();
pub fn check_signature(&self, data: impl AsRef<[u8]>, signature: impl AsRef<[u8]>) -> bool {
let data = data.as_ref();
let signature = signature.as_ref();
unsafe {
dht_publickey_check_signature(
self,
data.as_ptr(),
data.as_bytes().len(),
signature.as_ptr(),
signature.as_bytes().len(),
ffi::dht_publickey_check_signature(
self.as_ptr(),
data.as_ptr().cast(),
data.len(),
signature.as_ptr().cast(),
signature.len(),
)
}
}
pub fn encrypt(&self, data: &str) -> Box<Blob> {
let data = CString::new(data).unwrap();
pub fn encrypt(&self, data: impl AsRef<[u8]>) -> io::Result<Box<Blob>> {
let data = data.as_ref();
unsafe {
Box::from_raw(dht_publickey_encrypt(
self,
data.as_ptr(),
data.as_bytes().len(),
Blob::from_raw(ffi::dht_publickey_encrypt(
self.as_ptr(),
data.as_ptr().cast(),
data.len(),
))
}
}
}
impl Drop for PublicKey {
fn drop(&mut self) {
unsafe { dht_publickey_delete(&mut *self) }
}
}
impl PrivateKey {
pub fn new(key_length_bits: u32) -> Box<PrivateKey> {
unsafe { Box::from_raw(dht_privatekey_generate(key_length_bits)) }
pub fn new(key_length_bits: u32) -> io::Result<Box<PrivateKey>> {
unsafe { Self::from_raw(ffi::dht_privatekey_generate(key_length_bits)) }
}
pub fn new_ec() -> io::Result<Box<PrivateKey>> {
unsafe { Self::from_raw(ffi::dht_privatekey_generate_ec()) }
}
pub fn import(file: &str, password: &str) -> io::Result<Box<PrivateKey>> {
let mut f = File::open(file)?;
let mut buffer = Vec::new();
f.read_to_end(&mut buffer)?;
Ok(PrivateKey::from_bytes(&buffer, password))
Self::from_bytes(&fs::read(file)?, password)
}
pub fn from_bytes(buffer: &Vec<u8>, password: &str) -> Box<PrivateKey> {
pub fn from_bytes(buffer: &[u8], password: &str) -> io::Result<Box<PrivateKey>> {
let password = CString::new(password)?;
unsafe {
Box::from_raw(dht_privatekey_import(
(&*buffer).as_ptr(),
Self::from_raw(ffi::dht_privatekey_import(
buffer.as_ptr(),
buffer.len(),
password.as_ptr() as *const i8,
password.as_ptr(),
))
}
}
pub fn public_key(&self) -> Box<PublicKey> {
unsafe { Box::from_raw(dht_privatekey_get_publickey(self)) }
unsafe {
PublicKey::from_raw(ffi::dht_privatekey_get_publickey(self.as_ptr()))
.expect("private key has a public key")
}
}
}
impl Drop for PrivateKey {
fn drop(&mut self) {
unsafe { dht_privatekey_delete(&mut *self) }
pub fn serialize(&self, password: &str) -> io::Result<Vec<u8>> {
let password = CString::new(password)?;
unsafe {
Ok(Blob::from_raw(ffi::dht_privatekey_export_blob(
self.as_ptr(),
password.as_ptr(),
))?
.as_bytes())
}
}
pub fn sign(&self, data: impl AsRef<[u8]>) -> io::Result<Vec<u8>> {
let data = data.as_ref();
unsafe {
Ok(Blob::from_raw(ffi::dht_privatekey_sign(
self.as_ptr(),
data.as_ptr(),
data.len(),
))?
.as_bytes())
}
}
pub fn decrypt(&self, data: &[u8]) -> io::Result<Box<Blob>> {
unsafe {
Blob::from_raw(ffi::dht_privatekey_decrypt(
self.as_ptr(),
data.as_ptr().cast(),
data.len(),
))
}
}
}
impl DhtCertificate {
pub fn import(file: &str) -> io::Result<Box<DhtCertificate>> {
let mut f = File::open(file)?;
let mut buffer = Vec::new();
f.read_to_end(&mut buffer)?;
Ok(DhtCertificate::from_bytes(&buffer))
Self::from_bytes(&fs::read(file)?)
}
pub fn from_bytes(buffer: &Vec<u8>) -> Box<DhtCertificate> {
unsafe { Box::from_raw(dht_certificate_import((&*buffer).as_ptr(), buffer.len())) }
pub fn from_bytes(buffer: &[u8]) -> io::Result<Box<DhtCertificate>> {
unsafe { Self::from_raw(ffi::dht_certificate_import(buffer.as_ptr(), buffer.len())) }
}
pub fn from_slice(buffer: &str) -> Box<DhtCertificate> {
unsafe { Box::from_raw(dht_certificate_import((&*buffer).as_ptr(), buffer.len())) }
pub fn from_slice(buffer: &str) -> io::Result<Box<DhtCertificate>> {
Self::from_bytes(buffer.as_bytes())
}
pub fn serialize(&self) -> io::Result<Vec<u8>> {
unsafe { Ok(Blob::from_raw(ffi::dht_certificate_export(self.as_ptr()))?.as_bytes()) }
}
pub fn id(&self) -> InfoHash {
unsafe { dht_certificate_get_id(&*self) }
unsafe { ffi::dht_certificate_get_id(self.as_ptr()) }
}
pub fn long_id(&self) -> PkId {
unsafe { dht_certificate_get_long_id(&*self) }
unsafe { ffi::dht_certificate_get_long_id(self.as_ptr()) }
}
pub fn publickey(&self) -> Box<PublicKey> {
unsafe { Box::from_raw(dht_certificate_get_publickey(&*self)) }
unsafe {
PublicKey::from_raw(ffi::dht_certificate_get_publickey(self.as_ptr()))
.expect("certificate has a public key")
}
}
}
impl Drop for DhtCertificate {
fn drop(&mut self) {
unsafe { dht_certificate_delete(&mut *self) }
}
pub struct DhtIdentity {
pub privatekey: Box<PrivateKey>,
pub certificate: Box<DhtCertificate>,
}
impl DhtIdentity {
pub fn new(common_name: &str) -> DhtIdentity {
unsafe { DhtIdentity::generate(common_name, Box::from_raw(ptr::null_mut())) }
pub fn new(common_name: &str) -> io::Result<Self> {
Self::generate_inner(common_name, None, false)
}
pub fn generate(common_name: &str, ca: Box<DhtIdentity>) -> DhtIdentity {
let common_name = CString::new(common_name).unwrap();
unsafe { dht_identity_generate(common_name.as_ptr(), &*ca) }
pub fn new_ec(common_name: &str) -> io::Result<Self> {
Self::generate_inner(common_name, None, true)
}
}
impl Drop for DhtIdentity {
fn drop(&mut self) {
unsafe { dht_identity_delete(&mut *self) }
pub fn generate(common_name: &str, ca: &DhtIdentity) -> io::Result<Self> {
Self::generate_inner(common_name, Some(ca), false)
}
pub(crate) fn raw(&self) -> ffi::DhtIdentity {
ffi::DhtIdentity {
privatekey: self.privatekey.as_ptr(),
certificate: self.certificate.as_ptr(),
}
}
fn generate_inner(common_name: &str, ca: Option<&Self>, ec: bool) -> io::Result<Self> {
let name = CString::new(common_name)?;
let ca = ca.map(Self::raw);
let ca_ptr = ca
.as_ref()
.map_or(ptr::null(), |identity| identity as *const _);
unsafe {
let mut raw = if ec {
ffi::dht_identity_generate_ec(name.as_ptr(), ca_ptr)
} else {
ffi::dht_identity_generate(name.as_ptr(), ca_ptr)
};
if raw.privatekey.is_null() || raw.certificate.is_null() {
ffi::dht_identity_delete(&mut raw);
return Err(io::Error::other("identity generation failed"));
}
Ok(Self {
privatekey: PrivateKey::from_raw(raw.privatekey)?,
certificate: DhtCertificate::from_raw(raw.certificate)?,
})
}
}
}
+717 -167
View File
File diff suppressed because it is too large. Load diff
+245 -80
View File
@@ -3,71 +3,66 @@
#![allow(dead_code)]
use libc::{c_char, c_int, c_uint, c_void, in_port_t, size_t};
use os_socketaddr::OsSocketAddr;
use libc::{c_char, c_int, c_uint, c_void, in_port_t, sa_family_t, size_t, sockaddr, socklen_t};
const HASH_LEN: usize = 20;
const PKID_LEN: usize = 32;
#[repr(C)]
pub struct DataView
{
pub struct DataView {
pub data: *const u8,
pub size: size_t
pub size: size_t,
}
#[repr(C)]
pub struct Value
{
_opaque: [u8; 0]
pub struct Value {
_opaque: [u8; 0],
_thread_bound: std::marker::PhantomData<*mut ()>,
}
#[repr(C)]
pub struct Blob
{
_opaque: [u8; 0]
pub struct Blob {
_opaque: [u8; 0],
_thread_bound: std::marker::PhantomData<*mut ()>,
}
#[repr(C)]
#[repr(C, align(4))]
#[derive(PartialEq)]
pub struct InfoHash
{
pub struct InfoHash {
pub d: [u8; HASH_LEN],
}
#[repr(C)]
pub struct PkId
{
#[repr(C, align(4))]
pub struct PkId {
pub d: [u8; PKID_LEN],
}
#[repr(C)]
pub struct PublicKey
{
_opaque: [u8; 0]
pub struct PublicKey {
_opaque: [u8; 0],
_thread_bound: std::marker::PhantomData<*mut ()>,
}
#[repr(C)]
pub struct PrivateKey
{
_opaque: [u8; 0]
pub struct PrivateKey {
_opaque: [u8; 0],
_thread_bound: std::marker::PhantomData<*mut ()>,
}
#[repr(C)]
pub struct OpToken
{
_opaque: [u8; 0]
pub struct OpToken {
_opaque: [u8; 0],
_thread_bound: std::marker::PhantomData<*mut ()>,
}
#[repr(C)]
pub struct DhtRunner
{
_opaque: [u8; 0]
pub struct DhtRunner {
_opaque: [u8; 0],
_thread_bound: std::marker::PhantomData<*mut ()>,
}
#[repr(C)]
pub struct DhtNodeConfig
{
pub struct DhtNodeConfig {
pub node_id: InfoHash,
pub network: u32,
pub is_bootstrap: bool,
@@ -76,28 +71,25 @@ pub struct DhtNodeConfig
}
#[repr(C)]
pub struct DhtCertificate
{
_opaque: [u8; 0]
pub struct DhtCertificate {
_opaque: [u8; 0],
_thread_bound: std::marker::PhantomData<*mut ()>,
}
#[repr(C)]
pub struct DhtIdentity
{
pub struct DhtIdentity {
pub privatekey: *mut PrivateKey,
pub certificate: *mut DhtCertificate,
}
#[repr(C)]
pub struct DhtSecureConfig
{
pub struct DhtSecureConfig {
pub node_config: DhtNodeConfig,
pub id: DhtIdentity,
}
#[repr(C)]
pub struct DhtRunnerConfig
{
pub struct DhtRunnerConfig {
pub dht_config: DhtSecureConfig,
pub threaded: bool,
pub proxy_server: *const c_char,
@@ -113,17 +105,55 @@ pub struct DhtRunnerConfig
pub log: bool,
}
pub type CertificateCallback = extern "C" fn(*const DhtCertificate, *mut c_void);
#[repr(C)]
pub struct DhtRunnerContext {
pub status_cb: Option<extern "C" fn(c_int, c_int, *mut c_void)>,
pub certificate_store_cb:
Option<extern "C" fn(*const PkId, CertificateCallback, *mut c_void, *mut c_void)>,
pub released_cb: Option<extern "C" fn(*mut c_void)>,
pub user_data: *mut c_void,
}
#[repr(C)]
pub struct QueryResult {
pub fields: u32,
pub id: u64,
pub value_type: u16,
pub seq: u16,
pub owner: InfoHash,
pub user_type: DataView,
}
#[repr(C)]
pub struct PushToken {
_opaque: [u8; 0],
_thread_bound: std::marker::PhantomData<*mut ()>,
}
#[link(name = "opendht-c")]
extern {
unsafe extern "C" {
// dht::Value
pub fn dht_value_new(data: *const u8, size: size_t) -> *mut Value;
pub fn dht_value_with_id_new(data: *const u8, size: size_t, id: u64) -> *mut Value;
pub fn dht_value_ref(data: *const Value) -> *mut Value;
pub fn dht_value_copy(data: *const Value) -> *mut Value;
pub fn dht_value_unref(data: *mut Value);
pub fn dht_value_get_data(data: *const Value) -> DataView;
pub fn dht_value_get_id(data: *const Value) -> u64;
pub fn dht_value_get_owner(data: *const Value) -> *mut PublicKey;
pub fn dht_value_get_recipient(data: *const Value) -> InfoHash;
pub fn dht_value_get_user_type(data: *const Value) -> *const c_char;
pub fn dht_value_set_user_type(data: *mut Value, user_type: *const c_char);
pub fn dht_value_get_type(data: *const Value) -> u16;
pub fn dht_value_set_type(data: *mut Value, value_type: u16);
pub fn dht_value_get_seq(data: *const Value) -> u16;
pub fn dht_value_set_seq(data: *mut Value, seq: u16);
pub fn dht_value_get_priority(data: *const Value) -> c_uint;
pub fn dht_value_set_priority(data: *mut Value, priority: c_uint);
pub fn dht_value_get_push_type(data: *const Value) -> *const c_char;
pub fn dht_value_set_push_type(data: *mut Value, push_type: *const c_char);
// dht::Blob
pub fn dht_blob_get_data(data: *const Blob) -> DataView;
@@ -132,8 +162,9 @@ extern {
// dht::InfoHash
pub fn dht_infohash_print(h: *const InfoHash) -> *const c_char;
pub fn dht_infohash_random(h: *mut InfoHash);
pub fn dht_infohash_get(h: *mut InfoHash, dat: *mut u8, dat_size: size_t);
pub fn dht_infohash_get(h: *mut InfoHash, dat: *const u8, dat_size: size_t);
pub fn dht_infohash_from_hex(h: *mut InfoHash, dat: *const c_char);
pub fn dht_infohash_from_hex_null(h: *mut InfoHash, dat: *const c_char);
pub fn dht_infohash_is_zero(j: *const InfoHash) -> bool;
// dht::PkId
@@ -142,27 +173,56 @@ extern {
// dht::crypto::PublicKey
pub fn dht_publickey_import(dat: *const u8, dat_size: size_t) -> *mut PublicKey;
pub fn dht_publickey_delete(pk: *mut PublicKey);
pub fn dht_publickey_unpack(pk: *mut PublicKey, dat: *const u8, dat_size: size_t) -> c_int;
pub fn dht_publickey_pack(pk: *mut PublicKey, out: *const c_char, out_size: size_t) -> c_int;
pub fn dht_publickey_export_blob(pk: *const PublicKey) -> *mut Blob;
pub fn dht_publickey_unpack(pk: *mut PublicKey, data: *const u8, size: size_t) -> c_int;
pub fn dht_publickey_get_id(pk: *const PublicKey) -> InfoHash;
pub fn dht_publickey_get_long_id(pk: *const PublicKey) -> PkId;
pub fn dht_publickey_check_signature(pk: *const PublicKey, data: *const c_char, data_size: size_t, signature: *const c_char, signature_size: size_t) -> bool;
pub fn dht_publickey_encrypt(pk: *const PublicKey, data: *const c_char, data_size: size_t) -> *mut Blob;
pub fn dht_publickey_check_signature(
pk: *const PublicKey,
data: *const c_char,
data_size: size_t,
signature: *const c_char,
signature_size: size_t,
) -> bool;
pub fn dht_publickey_encrypt(
pk: *const PublicKey,
data: *const c_char,
data_size: size_t,
) -> *mut Blob;
// dht::crypto::PrivateKey
pub fn dht_privatekey_generate(key_length_bits: c_uint) -> *mut PrivateKey;
pub fn dht_privatekey_import(dat: *const u8, data_size: size_t, password: *const c_char) -> *mut PrivateKey;
pub fn dht_privatekey_generate_ec() -> *mut PrivateKey;
pub fn dht_privatekey_import(
dat: *const u8,
data_size: size_t,
password: *const c_char,
) -> *mut PrivateKey;
pub fn dht_privatekey_get_publickey(pk: *const PrivateKey) -> *mut PublicKey;
pub fn dht_privatekey_export_blob(pk: *const PrivateKey, password: *const c_char) -> *mut Blob;
pub fn dht_privatekey_sign(pk: *const PrivateKey, data: *const u8, size: size_t) -> *mut Blob;
pub fn dht_privatekey_decrypt(
pk: *const PrivateKey,
data: *const c_char,
size: size_t,
) -> *mut Blob;
pub fn dht_privatekey_delete(pk: *mut PrivateKey);
// dht::crypto::Certificate
pub fn dht_certificate_import(dat: *const u8, dat_size: size_t) -> *mut DhtCertificate;
pub fn dht_certificate_export(cert: *const DhtCertificate) -> *mut Blob;
pub fn dht_certificate_ref(cert: *const DhtCertificate) -> *mut DhtCertificate;
pub fn dht_certificate_get_id(cert: *const DhtCertificate) -> InfoHash;
pub fn dht_certificate_get_long_id(cert: *const DhtCertificate) -> PkId;
pub fn dht_certificate_get_publickey(cert: *const DhtCertificate) -> *mut PublicKey;
pub fn dht_certificate_delete(cert: *mut DhtCertificate);
pub fn dht_identity_generate(common_name: *const c_char, ca: *const DhtIdentity) -> DhtIdentity;
pub fn dht_identity_generate(common_name: *const c_char, ca: *const DhtIdentity)
-> DhtIdentity;
pub fn dht_identity_generate_ec(
common_name: *const c_char,
ca: *const DhtIdentity,
) -> DhtIdentity;
pub fn dht_identity_delete(ca: *mut DhtIdentity);
// dht::OpToken
@@ -174,37 +234,142 @@ extern {
pub fn dht_runner_get_id(dht: *const DhtRunner) -> InfoHash;
pub fn dht_runner_get_node_id(dht: *const DhtRunner) -> InfoHash;
pub fn dht_runner_delete(dht: *mut DhtRunner);
pub fn dht_runner_run(dht: *mut DhtRunner, port: in_port_t);
pub fn dht_runner_run_config(dht: *mut DhtRunner, port: in_port_t, config: *const DhtRunnerConfig);
pub fn dht_runner_run(dht: *mut DhtRunner, port: in_port_t) -> c_int;
pub fn dht_runner_run_config(
dht: *mut DhtRunner,
port: in_port_t,
config: *const DhtRunnerConfig,
) -> c_int;
pub fn dht_runner_run_bind(
dht: *mut DhtRunner,
ipv4: *const c_char,
ipv6: *const c_char,
port: in_port_t,
config: *const DhtRunnerConfig,
context: *const DhtRunnerContext,
) -> c_int;
pub fn dht_runner_connectivity_changed(dht: *mut DhtRunner) -> c_int;
pub fn dht_runner_register_type(
dht: *mut DhtRunner,
id: u16,
name: *const c_char,
expiration_seconds: u64,
insecure: bool,
) -> c_int;
pub fn dht_runner_get_publickey(dht: *const DhtRunner) -> *mut PublicKey;
pub fn dht_runner_register_certificate(
dht: *mut DhtRunner,
cert: *const DhtCertificate,
) -> c_int;
pub fn dht_runner_find_certificate(
dht: *mut DhtRunner,
id: *const PkId,
callback: CertificateCallback,
released: extern "C" fn(*mut c_void),
data: *mut c_void,
);
pub fn dht_runner_bootstrap(dht: *mut DhtRunner, host: *const c_char, service: *const c_char);
pub fn dht_runner_get(dht: *mut DhtRunner, h: *const InfoHash,
get_cb: extern fn(*mut Value, *mut c_void) -> bool,
done_cb: extern fn(bool, *mut c_void),
cb_user_data: *mut c_void);
pub fn dht_runner_put(dht: *mut DhtRunner, h: *const InfoHash, v: *const Value,
done_cb: extern fn(bool, *mut c_void),
cb_user_data: *mut c_void,
permanent: bool);
pub fn dht_runner_put_signed(dht: *mut DhtRunner, h: *const InfoHash, v: *const Value,
done_cb: extern fn(bool, *mut c_void),
cb_user_data: *mut c_void,
permanent: bool);
pub fn dht_runner_put_encrypted(dht: *mut DhtRunner, h: *const InfoHash,
to: *const PkId, v: *const Value,
done_cb: extern fn(bool, *mut c_void),
cb_user_data: *mut c_void,
permanent: bool);
pub fn dht_runner_put_permanent(dht: *mut DhtRunner, h: *const InfoHash, v: *const Value,
done_cb: extern fn(bool, *mut c_void),
cb_user_data: *mut c_void);
pub fn dht_runner_get(
dht: *mut DhtRunner,
h: *const InfoHash,
get_cb: extern "C" fn(*const Value, *mut c_void) -> bool,
done_cb: extern "C" fn(bool, *mut c_void),
cb_user_data: *mut c_void,
);
pub fn dht_runner_get_filtered(
dht: *mut DhtRunner,
h: *const InfoHash,
filter: *const c_char,
callback: extern "C" fn(*const Value, *mut c_void) -> bool,
done: extern "C" fn(bool, *mut c_void),
data: *mut c_void,
);
pub fn dht_runner_query(
dht: *mut DhtRunner,
h: *const InfoHash,
query: *const c_char,
callback: extern "C" fn(*const QueryResult, *mut c_void) -> bool,
done: extern "C" fn(bool, *mut c_void),
data: *mut c_void,
);
pub fn dht_runner_put(
dht: *mut DhtRunner,
h: *const InfoHash,
v: *const Value,
done_cb: extern "C" fn(bool, *mut c_void),
cb_user_data: *mut c_void,
permanent: bool,
);
pub fn dht_runner_put_signed(
dht: *mut DhtRunner,
h: *const InfoHash,
v: *const Value,
done_cb: extern "C" fn(bool, *mut c_void),
cb_user_data: *mut c_void,
permanent: bool,
);
pub fn dht_runner_put_encrypted(
dht: *mut DhtRunner,
h: *const InfoHash,
to: *const PkId,
v: *const Value,
done_cb: extern "C" fn(bool, *mut c_void),
cb_user_data: *mut c_void,
permanent: bool,
);
pub fn dht_runner_cancel_put(dht: *mut DhtRunner, h: *const InfoHash, vid: u64);
pub fn dht_runner_listen(dht: *mut DhtRunner, h: *const InfoHash,
cb: extern fn(*mut Value, bool, *mut c_void) -> bool,
done_cb: extern fn(*mut c_void),
cb_user_data: *mut c_void) -> *mut OpToken;
pub fn dht_runner_cancel_listen(dht: *mut DhtRunner, h: *const InfoHash,
token: *const OpToken);
pub fn dht_runner_shutdown(dht: *mut DhtRunner, done_cb: extern fn(bool, *mut c_void),
cb_user_data: *mut c_void);
pub fn dht_runner_get_public_address(dht: *const DhtRunner) -> *mut *mut OsSocketAddr;
}
pub fn dht_runner_listen(
dht: *mut DhtRunner,
h: *const InfoHash,
cb: extern "C" fn(*const Value, bool, *mut c_void) -> bool,
done_cb: extern "C" fn(*mut c_void),
cb_user_data: *mut c_void,
) -> *mut OpToken;
pub fn dht_runner_listen_filtered(
dht: *mut DhtRunner,
h: *const InfoHash,
filter: *const c_char,
callback: extern "C" fn(*const Value, bool, *mut c_void) -> bool,
released: extern "C" fn(*mut c_void),
data: *mut c_void,
) -> *mut OpToken;
pub fn dht_runner_cancel_listen(dht: *mut DhtRunner, h: *const InfoHash, token: *mut OpToken);
pub fn dht_runner_shutdown(
dht: *mut DhtRunner,
done_cb: extern "C" fn(*mut c_void),
cb_user_data: *mut c_void,
);
pub fn dht_runner_get_public_address(dht: *const DhtRunner) -> *mut *mut sockaddr;
pub fn dht_sockaddr_array_delete(addresses: *mut *mut sockaddr);
pub fn dht_runner_loop(dht: *mut DhtRunner) -> c_int;
pub fn dht_runner_join(dht: *mut DhtRunner) -> c_int;
pub fn dht_runner_is_running(dht: *const DhtRunner) -> bool;
pub fn dht_runner_get_bound_port(dht: *const DhtRunner, family: sa_family_t) -> in_port_t;
pub fn dht_runner_ping(
dht: *mut DhtRunner,
addr: *mut sockaddr,
len: socklen_t,
callback: extern "C" fn(bool, *mut c_void),
data: *mut c_void,
);
pub fn dht_runner_set_push_notification_token(
dht: *mut DhtRunner,
value: *const c_char,
) -> c_int;
pub fn dht_runner_set_push_notification_topic(
dht: *mut DhtRunner,
value: *const c_char,
) -> c_int;
pub fn dht_runner_set_push_notification_platform(
dht: *mut DhtRunner,
value: *const c_char,
) -> c_int;
pub fn dht_runner_push_notification_received(
dht: *mut DhtRunner,
keys: *const *const c_char,
values: *const *const c_char,
count: size_t,
) -> *mut PushToken;
pub fn dht_push_token_poll(token: *mut PushToken, result: *mut c_int) -> c_int;
pub fn dht_push_token_delete(token: *mut PushToken);
}
+1 -1
View File
@@ -42,7 +42,7 @@ impl InfoHash {
let mut h = InfoHash::new();
unsafe {
let c_str = CString::new(data).unwrap();
dht_infohash_from_hex(&mut h, c_str.as_ptr());
dht_infohash_from_hex_null(&mut h, c_str.as_ptr());
}
h
}
+39 -2
View File
@@ -4,6 +4,34 @@
extern crate libc;
extern crate os_socketaddr;
macro_rules! native_handle {
($name:ident, $delete:ident) => {
pub struct $name {
raw: std::ptr::NonNull<crate::ffi::$name>,
}
impl $name {
pub(crate) unsafe fn from_raw(
raw: *mut crate::ffi::$name,
) -> std::io::Result<Box<Self>> {
let raw = std::ptr::NonNull::new(raw)
.ok_or_else(|| std::io::Error::other("native operation failed"))?;
Ok(Box::new(Self { raw }))
}
pub(crate) fn as_ptr(&self) -> *mut crate::ffi::$name {
self.raw.as_ptr()
}
}
impl Drop for $name {
fn drop(&mut self) {
unsafe { crate::ffi::$delete(self.as_ptr()) }
}
}
};
}
mod blob;
pub mod crypto;
mod dhtrunner;
@@ -12,8 +40,17 @@ mod infohash;
mod pkid;
mod value;
#[cfg(feature = "async")]
pub mod aio;
#[cfg(feature = "serde")]
pub mod typed;
pub use blob::Blob;
pub use dhtrunner::{ DhtRunner, DhtRunnerConfig, OpToken };
pub use dhtrunner::{
DhtNodeConfig, DhtRunner, DhtRunnerConfig, DhtSecureConfig, NodeStatus, OpToken,
PushNotification, PushNotificationResult, QueryResult,
};
pub use infohash::InfoHash;
pub use pkid::PkId;
pub use value::{ DataView, Value };
pub use value::{DataView, Value};
+201
View File
@@ -0,0 +1,201 @@
//! MessagePack-encoded typed operations, enabled by the `serde` feature.
//!
//! This feature enables `async` and adds methods to [`AsyncDhtRunner`]. Values
//! use compact MessagePack payloads, with no binding-specific envelope or
//! implicit schema version. Structs are encoded as arrays in field order.
//! Applications can include a version field in their records and use
//! `serde_bytes` for binary fields rather than arrays of integers.
//!
//! ```no_run
//! use futures::TryStreamExt;
//! use opendht::{aio::AsyncDhtRunner, InfoHash};
//! use serde::{Deserialize, Serialize};
//!
//! #[derive(Debug, Serialize, Deserialize)]
//! struct Service {
//! version: u32,
//! name: String,
//! port: u16,
//! #[serde(with = "serde_bytes")]
//! payload: Vec<u8>,
//! }
//!
//! async fn exchange(
//! runner: &AsyncDhtRunner,
//! key: &InfoHash,
//! ) -> Result<(), Box<dyn std::error::Error>> {
//! let mut updates = runner.listen_typed::<Service>(key)?;
//! let record = Service {
//! version: 1, name: "chat".into(), port: 4222, payload: vec![0, 0x80, 0xff],
//! };
//! runner.put_typed(key, &record, false).await?;
//! if let Some(event) = updates.try_next().await? {
//! println!("{}: {:?}", event.value.raw.id(), event.value.data);
//! }
//! drop(updates);
//! let records: Vec<_> = runner.get_typed::<Service>(key).try_collect().await?;
//! Ok(())
//! }
//! ```
use crate::aio::{AsyncDhtRunner, ValueEvent};
use crate::{InfoHash, Value};
use futures_core::Stream;
use serde::{de::DeserializeOwned, Serialize};
use std::error::Error;
use std::fmt;
use std::io;
use std::pin::Pin;
use std::task::{Context, Poll};
/// A decoded record together with its original payload and native metadata.
pub struct TypedValue<T> {
pub data: T,
pub raw: Box<Value>,
}
impl<T: fmt::Debug> fmt::Debug for TypedValue<T> {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
formatter
.debug_struct("TypedValue")
.field("data", &self.data)
.field("id", &self.raw.id())
.field("seq", &self.raw.seq())
.finish_non_exhaustive()
}
}
/// A decoded listener update, including whether the record expired.
#[derive(Debug)]
pub struct TypedEvent<T> {
pub value: TypedValue<T>,
pub expired: bool,
}
/// Typed operations distinguish native failures from payload format errors.
#[derive(Debug)]
#[non_exhaustive]
pub enum TypedError {
Operation(io::Error),
Serialize(rmp_serde::encode::Error),
Deserialize {
id: u64,
source: rmp_serde::decode::Error,
},
}
impl fmt::Display for TypedError {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Self::Operation(error) => write!(formatter, "DHT operation failed: {error}"),
Self::Serialize(error) => write!(formatter, "could not serialize typed value: {error}"),
Self::Deserialize { id, source } => {
write!(formatter, "could not deserialize DHT value {id}: {source}")
}
}
}
}
impl Error for TypedError {
fn source(&self) -> Option<&(dyn Error + 'static)> {
match self {
Self::Operation(error) => Some(error),
Self::Serialize(error) => Some(error),
Self::Deserialize { source, .. } => Some(source),
}
}
}
impl<T: DeserializeOwned> TryFrom<Box<Value>> for TypedValue<T> {
type Error = TypedError;
fn try_from(raw: Box<Value>) -> Result<Self, Self::Error> {
let data =
rmp_serde::from_slice(raw.as_slice()).map_err(|source| TypedError::Deserialize {
id: raw.id(),
source,
})?;
Ok(Self { data, raw })
}
}
impl<T: DeserializeOwned> TryFrom<ValueEvent> for TypedEvent<T> {
type Error = TypedError;
fn try_from(event: ValueEvent) -> Result<Self, Self::Error> {
Ok(Self {
value: event.value.try_into()?,
expired: event.expired,
})
}
}
struct DecodeStream<Source: Stream, Output> {
source: Source,
decode: fn(Source::Item) -> Result<Output, TypedError>,
}
impl<Source: Stream + Unpin, Output> Stream for DecodeStream<Source, Output> {
type Item = Result<Output, TypedError>;
fn poll_next(self: Pin<&mut Self>, context: &mut Context<'_>) -> Poll<Option<Self::Item>> {
let stream = self.get_mut();
Pin::new(&mut stream.source)
.poll_next(context)
.map(|item| item.map(stream.decode))
}
}
fn decode<Input, Output: TryFrom<Input, Error = TypedError>>(
input: io::Result<Input>,
) -> Result<Output, TypedError> {
input.map_err(TypedError::Operation)?.try_into()
}
impl AsyncDhtRunner {
/// Serialize a record as MessagePack and publish it when this future is polled.
///
/// Unlike untyped `put`, this operation is lazy. Serialization failure does
/// not start a native put. Dropping it after publication starts does not
/// undo the put. Use the untyped API to customize value IDs and metadata.
pub async fn put_typed<T: Serialize + ?Sized>(
&self,
hash: &InfoHash,
value: &T,
permanent: bool,
) -> Result<(), TypedError> {
let bytes = rmp_serde::to_vec(value).map_err(TypedError::Serialize)?;
self.put(hash, Value::from_bytes(&bytes), permanent)
.await
.map_err(TypedError::Operation)
}
/// Start a get and decode each MessagePack payload on the consumer's thread.
///
/// A malformed or incompatible record yields a per-value error; polling can
/// continue to subsequent records. Native errors remain terminal. The stream
/// does not borrow the runner or hash, and records need not be `Unpin`.
pub fn get_typed<T: DeserializeOwned>(
&self,
hash: &InfoHash,
) -> impl Stream<Item = Result<TypedValue<T>, TypedError>> + Unpin + use<T> {
DecodeStream {
source: self.get(hash),
decode: decode::<_, TypedValue<T>>,
}
}
/// Listen for MessagePack records, preserving metadata and expiration events.
///
/// Decode errors do not cancel the registration. Dropping this stream does
/// cancel it, including when a short-circuiting consumer stops on an error.
pub fn listen_typed<T: DeserializeOwned>(
&self,
hash: &InfoHash,
) -> io::Result<impl Stream<Item = Result<TypedEvent<T>, TypedError>> + Unpin + use<T>> {
Ok(DecodeStream {
source: self.listen(hash)?,
decode: decode::<_, TypedEvent<T>>,
})
}
}
+141 -42
View File
@@ -2,91 +2,190 @@
// SPDX-License-Identifier: MIT
#![allow(dead_code)]
use crate::ffi::*;
use crate::crypto::PublicKey;
use crate::{ffi, InfoHash};
use std::ffi::{CStr, CString};
use std::fmt;
use std::ffi::{ CString, IntoStringError };
use std::str;
use std::io;
use std::slice;
use std::str;
pub use crate::ffi::{DataView, Value};
pub use crate::ffi::DataView;
native_handle!(Value, dht_value_unref);
/// Rust owns an unaliased native value: constructors allocate it, received
/// values are deep-copied once, and boxed() deep-copies explicitly. Puts consume
/// the handle before native code retains it. Shared owner keys are const and
/// their lazy fingerprints are initialized before received values cross threads.
/// Mutation requires exclusive access; native storage may be freed on any thread.
/// Value is Send, not Sync.
unsafe impl Send for Value {}
impl AsRef<[u8]> for Value {
fn as_ref(&self) -> &[u8] {
self.as_slice()
}
}
impl fmt::Debug for Value {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
formatter
.debug_struct("Value")
.field("id", &self.id())
.field("seq", &self.seq())
.finish_non_exhaustive()
}
}
impl Value {
pub fn new(data: &str) -> Box<Value> {
pub(crate) unsafe fn copy_from_received(value: *const ffi::Value) -> io::Result<Box<Self>> {
let value = unsafe { Self::from_raw(ffi::dht_value_copy(value))? };
if let Some(owner) = value.owner() {
owner.id();
owner.long_id();
}
Ok(value)
}
pub fn with_id(data: &[u8], id: u64) -> Box<Value> {
unsafe {
Box::from_raw(dht_value_new(data.as_bytes().as_ptr(),
data.as_bytes().len()))
Self::from_raw(ffi::dht_value_with_id_new(data.as_ptr(), data.len(), id))
.expect("value allocation failed")
}
}
pub fn from_bytes(data: &Vec<u8>) -> Box<Value> {
pub fn new(data: &str) -> Box<Value> {
Self::from_bytes(data.as_bytes())
}
pub fn from_bytes(data: &[u8]) -> Box<Value> {
unsafe {
Box::from_raw(dht_value_new(data.as_ptr(),
data.len()))
Self::from_raw(ffi::dht_value_new(data.as_ptr(), data.len()))
.expect("value allocation failed")
}
}
pub fn id(&self) -> u64 {
unsafe {
dht_value_get_id(self)
}
unsafe { ffi::dht_value_get_id(self.as_ptr()) }
}
fn dataview(&self) -> DataView {
unsafe {
dht_value_get_data(self)
}
unsafe { ffi::dht_value_get_data(self.as_ptr()) }
}
pub fn as_bytes(&self) -> Vec<u8> {
unsafe {
let dv = dht_value_get_data(self);
let slice = slice::from_raw_parts_mut(dv.data as *mut _, dv.size);
slice.to_vec()
self.as_slice().to_vec()
}
/// Borrow the payload without allocating or copying it.
pub fn as_slice(&self) -> &[u8] {
let view = self.dataview();
if view.size == 0 {
&[]
} else {
unsafe { slice::from_raw_parts(view.data, view.size) }
}
}
pub fn boxed(&mut self) -> Box<Value> {
unsafe {
Box::from_raw(dht_value_ref(self))
}
pub fn boxed(&self) -> Box<Value> {
unsafe { Self::from_raw(ffi::dht_value_copy(self.as_ptr())).expect("value copy failed") }
}
pub fn recipient(&self) -> InfoHash {
unsafe {
dht_value_get_recipient(self)
}
unsafe { ffi::dht_value_get_recipient(self.as_ptr()) }
}
pub fn owner(&self) -> Option<Box<PublicKey>> {
unsafe {
let ptr = dht_value_get_owner(self);
if ptr.is_null() {
return None;
}
Some(Box::from_raw(ptr))
}
unsafe { PublicKey::from_raw(ffi::dht_value_get_owner(self.as_ptr())).ok() }
}
pub fn user_type(&self) -> Result<String, IntoStringError> {
pub fn user_type(&self) -> Result<&str, str::Utf8Error> {
unsafe { CStr::from_ptr(ffi::dht_value_get_user_type(self.as_ptr())).to_str() }
}
pub fn set_user_type(&mut self, value: &str) -> io::Result<()> {
let value = CString::new(value)?;
unsafe {
CString::from_raw(dht_value_get_user_type(self) as *mut _).into_string()
ffi::dht_value_set_user_type(self.as_ptr(), value.as_ptr());
}
Ok(())
}
pub fn value_type(&self) -> u16 {
unsafe { ffi::dht_value_get_type(self.as_ptr()) }
}
pub fn set_value_type(&mut self, value: u16) {
unsafe {
ffi::dht_value_set_type(self.as_ptr(), value);
}
}
}
impl Drop for Value {
fn drop(&mut self) {
pub fn seq(&self) -> u16 {
unsafe { ffi::dht_value_get_seq(self.as_ptr()) }
}
pub fn set_seq(&mut self, value: u16) {
unsafe {
dht_value_unref(&mut *self)
ffi::dht_value_set_seq(self.as_ptr(), value);
}
}
pub fn priority(&self) -> u32 {
unsafe { ffi::dht_value_get_priority(self.as_ptr()) }
}
pub fn set_priority(&mut self, value: u32) {
unsafe {
ffi::dht_value_set_priority(self.as_ptr(), value);
}
}
pub fn push_type(&self) -> Result<&str, str::Utf8Error> {
unsafe { CStr::from_ptr(ffi::dht_value_get_push_type(self.as_ptr())).to_str() }
}
pub fn set_push_type(&mut self, value: &str) -> io::Result<()> {
let value = CString::new(value)?;
unsafe {
ffi::dht_value_set_push_type(self.as_ptr(), value.as_ptr());
}
Ok(())
}
}
impl fmt::Display for Value {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
unsafe {
let dataview = self.dataview();
if dataview.size == 0 {
return Ok(());
}
let slice = slice::from_raw_parts(dataview.data, dataview.size);
write!(f, "{}", str::from_utf8(slice).unwrap_or(""))
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn value_moves_between_threads_without_copying_payload() {
let mut value = Value::from_bytes(b"native\0payload\xff");
value.set_user_type("borrowed metadata").unwrap();
let view = unsafe { ffi::dht_value_get_data(value.as_ptr()) };
let value = std::thread::spawn(move || {
value.set_seq(7);
value
})
.join()
.unwrap();
assert_eq!(value.as_slice().as_ptr(), view.data);
assert_eq!(value.as_slice(), b"native\0payload\xff");
assert_eq!(value.seq(), 7);
let user_type: &str = value.user_type().unwrap();
assert_eq!(user_type, "borrowed metadata");
}
#[test]
fn empty_value_payload_is_borrowed_safely() {
let value = Value::from_bytes(&[]);
assert!(value.as_slice().is_empty());
assert_eq!(value.as_slice(), &[]);
}
}
File diff suppressed because it is too large. Load diff
+41 -76
View File
@@ -571,6 +571,12 @@ PublicKey::toString() const
}
if (err != GNUTLS_E_SUCCESS)
throw CryptoException(std::string("Could not print public key: ") + gnutls_strerror(err));
// GnuTLS reports the PEM size including the terminating null byte, which
// does not belong to the text and would end up embedded in whatever the
// string is copied into, such as the "owner" field of a serialized value.
ret.resize(sz);
if (not ret.empty() and ret.back() == '\0')
ret.pop_back();
return ret;
}
@@ -592,7 +598,9 @@ PublicKey::checkSignature(const uint8_t* data, size_t data_len, const uint8_t* s
return false;
const gnutls_datum_t sig {(uint8_t*) signature, (unsigned) signature_len};
const gnutls_datum_t dat {(uint8_t*) data, (unsigned) data_len};
int rc = gnutls_pubkey_verify_data2(pk, GNUTLS_SIGN_RSA_SHA512, 0, &dat, &sig);
auto algorithm = gnutls_pk_to_sign(static_cast<gnutls_pk_algorithm_t>(gnutls_pubkey_get_pk_algorithm(pk, nullptr)),
GNUTLS_DIG_SHA512);
int rc = gnutls_pubkey_verify_data2(pk, algorithm, 0, &dat, &sig);
return rc >= 0;
}
@@ -1288,14 +1296,16 @@ PrivateKey::generate(unsigned key_length, gnutls_pk_algorithm_t algo)
}
PrivateKey
PrivateKey::generateEC()
PrivateKey::generateEC(gnutls_ecc_curve_t curve)
{
gnutls_x509_privkey_t key;
if (gnutls_x509_privkey_init(&key) != GNUTLS_E_SUCCESS)
throw CryptoException("Unable to initialize private key.");
int err = gnutls_x509_privkey_generate(key,
GNUTLS_PK_EC,
gnutls_sec_param_to_pk_bits(GNUTLS_PK_EC, GNUTLS_SEC_PARAM_ULTRA),
curve == GNUTLS_ECC_CURVE_INVALID
? gnutls_sec_param_to_pk_bits(GNUTLS_PK_EC, GNUTLS_SEC_PARAM_ULTRA)
: GNUTLS_CURVE_TO_BITS(curve),
0);
if (err != GNUTLS_E_SUCCESS) {
gnutls_x509_privkey_deinit(key);
@@ -1396,8 +1406,11 @@ setRandomSerial(gnutls_x509_crt_t cert)
{
std::random_device rdev;
std::uniform_int_distribution<int64_t> dist {1};
int64_t cert_serial = dist(rdev);
gnutls_x509_crt_set_serial(cert, &cert_serial, sizeof(cert_serial));
const auto cert_serial = dist(rdev);
std::array<uint8_t, sizeof(cert_serial)> packed_serial;
for (size_t i = 0; i < packed_serial.size(); i++)
packed_serial[i] = cert_serial >> (8 * (packed_serial.size() - 1 - i));
gnutls_x509_crt_set_serial(cert, packed_serial.data(), packed_serial.size());
}
Certificate
@@ -1648,24 +1661,8 @@ OcspResponse::toString(const bool compact) const
gnutls_ocsp_cert_status_t
OcspResponse::getCertificateStatus() const
{
int ret;
unsigned int status;
// MSVC/vcpkg GnuTLS cast workaround
#if _WIN32 && (GNUTLS_VERSION_NUMBER == 0x030807)
ret = gnutls_ocsp_resp_get_single(response,
0,
NULL,
NULL,
NULL,
NULL,
reinterpret_cast<gnutls_ocsp_cert_status_t*>(&status),
NULL,
NULL,
NULL,
NULL);
#else
ret = gnutls_ocsp_resp_get_single(response, 0, NULL, NULL, NULL, NULL, &status, NULL, NULL, NULL, NULL);
#endif
int ret = gnutls_ocsp_resp_get_single(response, 0, NULL, NULL, NULL, NULL, &status, NULL, NULL, NULL, NULL);
if (ret < 0)
throw CryptoException(gnutls_strerror(ret));
return (gnutls_ocsp_cert_status_t) status;
@@ -1724,22 +1721,7 @@ OcspResponse::verifyDirect(const Certificate& crt, const Blob& nonce)
// Check certificate revocation status
unsigned int status_ocsp;
// MSVC/vcpkg GnuTLS cast workaround
#if _WIN32 && (GNUTLS_VERSION_NUMBER == 0x030807)
ret = gnutls_ocsp_resp_get_single(response,
0,
NULL,
NULL,
NULL,
NULL,
reinterpret_cast<gnutls_ocsp_cert_status_t*>(&status_ocsp),
NULL,
NULL,
NULL,
NULL);
#else
ret = gnutls_ocsp_resp_get_single(response, 0, NULL, NULL, NULL, NULL, &status_ocsp, NULL, NULL, NULL, NULL);
#endif
if (ret < 0)
throw CryptoException(gnutls_strerror(ret));
return (gnutls_ocsp_cert_status_t) status_ocsp;
@@ -1873,34 +1855,8 @@ RevocationList::getUpdateTime() const
return std::chrono::system_clock::from_time_t(t);
}
enum class Endian : uint32_t { LITTLE = 0, BIG = 1 };
template<typename T>
T
endian(T w, Endian endian = Endian::BIG)
{
// this gets optimized out into if (endian == host_endian) return w;
union {
uint64_t quad;
uint32_t islittle;
} t;
t.quad = 1;
if (t.islittle ^ (uint32_t) endian)
return w;
T r = 0;
// decent compilers will unroll this (gcc)
// or even convert straight into single bswap (clang)
for (size_t i = 0; i < sizeof(r); i++) {
r <<= 8;
r |= w & 0xff;
w >>= 8;
}
return r;
}
void
RevocationList::sign(const PrivateKey& key, const Certificate& ca, duration validity)
RevocationList::sign(const PrivateKey& key, const Certificate& ca, duration validity, uint64_t crlNumber)
{
if (auto err = gnutls_x509_crl_set_version(crl, 2))
throw CryptoException(std::string("Unable to set CRL version: ") + gnutls_strerror(err));
@@ -1910,19 +1866,28 @@ RevocationList::sign(const PrivateKey& key, const Certificate& ca, duration vali
throw CryptoException(std::string("Unable to set CRL update time: ") + gnutls_strerror(err));
if (auto err = gnutls_x509_crl_set_next_update(crl, std::chrono::system_clock::to_time_t(next_update)))
throw CryptoException(std::string("Unable to set CRL next update time: ") + gnutls_strerror(err));
uint64_t number {0};
size_t number_sz {sizeof(number)};
unsigned critical {0};
gnutls_x509_crl_get_number(crl, &number, &number_sz, &critical);
uint64_t number {crlNumber};
if (number == 0) {
// initialize to a random number
number_sz = sizeof(number);
std::random_device rdev;
std::generate_n((uint8_t*) &number, sizeof(number), std::bind(rand_byte, std::ref(rdev)));
} else
number = endian(endian(number) + 1);
if (auto err = gnutls_x509_crl_set_number(crl, &number, sizeof(number)))
throw CryptoException(std::string("Unable to set CRL update time: ") + gnutls_strerror(err));
// The CRL Number extension holds a DER integer of variable length, so it must be
// read as a big-endian byte string rather than straight into an uint64_t.
auto current = getNumber();
for (uint8_t byte : current)
number = (number << 8) | byte;
if (number == 0) {
// Initialize to a random number.
std::random_device rdev;
do {
std::generate_n((uint8_t*) &number, sizeof(number), std::bind(rand_byte, std::ref(rdev)));
} while (number == 0);
} else
number++;
}
// Store the number in network byte order.
uint8_t packedNumber[sizeof(number)];
for (size_t i = 0; i < sizeof(packedNumber); i++)
packedNumber[i] = (uint8_t) ((number >> (8 * (sizeof(packedNumber) - 1 - i))) & 0xff);
if (auto err = gnutls_x509_crl_set_number(crl, packedNumber, sizeof(packedNumber)))
throw CryptoException(std::string("Unable to set CRL number: ") + gnutls_strerror(err));
if (auto err = gnutls_x509_crl_sign2(crl, ca.cert, key.x509_key, GNUTLS_DIG_SHA512, 0))
throw CryptoException(std::string("Unable to sign certificate revocation list: ") + gnutls_strerror(err));
// to be able to actually use the CRL we need to serialize/deserialize it
+39 -13
View File
@@ -304,7 +304,7 @@ Dht::searchNodeGetDone(const net::Request& req, net::RequestAnswer&& answer, std
should not be sent anymore */
for (auto& g : sr->callbacks) {
auto& q = g.second.query;
if (q->isSatisfiedBy(*query) and q != query) {
if (q != query and g.second.isSatisfiedBy(*query)) {
auto dummy_req = std::make_shared<net::Request>();
dummy_req->cancel();
srn->getStatus[q] = std::move(dummy_req);
@@ -402,17 +402,18 @@ Dht::searchSendGetValues(Sp<Search> sr, SearchNode* pn, bool update)
std::weak_ptr<Search> ws = sr;
auto cb = sr->callbacks.begin();
static const auto ANY_QUERY = std::make_shared<Query>(Select {}, Where {}, true);
static const Get ANY_GET {time_point::min(), {}, std::make_shared<Query>(Select {}, Where {}, true), {}, {}, {}};
do { /* for all requests to send */
SearchNode* n = nullptr;
auto& query = sr->callbacks.empty() ? ANY_QUERY : cb->second.query;
const auto& get = sr->callbacks.empty() ? ANY_GET : cb->second;
const auto& query = get.query;
const time_point up = (not sr->callbacks.empty() and update) ? sr->getLastGetTime(*query) : time_point::min();
if (pn and pn->canGet(now, up, query)) {
if (pn and pn->canGet(now, up, get)) {
n = pn;
} else {
for (auto& sn : sr->nodes) {
if (sn->canGet(now, up, query)) {
if (sn->canGet(now, up, get)) {
n = sn.get();
break;
}
@@ -2270,8 +2271,16 @@ Dht::onDisconnected()
void
Dht::bootstrap()
{
if (dht4.status != NodeStatus::Disconnected || dht6.status != NodeStatus::Disconnected)
if (dht4.status != NodeStatus::Disconnected || dht6.status != NodeStatus::Disconnected) {
// The job that led here has already run, and the scheduler does not
// clear the handle to a job it has executed. onDisconnected() reads a
// set handle as "a bootstrap is already scheduled" and does nothing,
// so leaving it set here means the node never tries to rejoin the
// network once it has lost it. Clearing it is what says that no
// bootstrap is pending any more.
bootstrapJob.reset();
return;
}
if (logger_)
logger_->debug("[{}] Bootstraping", myid.to_view());
bootstrap_pending = false;
@@ -2355,6 +2364,9 @@ Dht::confirmNodes()
std::vector<ValuesExport>
Dht::exportValues() const
{
const auto& now = scheduler.time();
const auto system_now = system_clock::now();
std::vector<ValuesExport> e {};
e.reserve(store.size());
for (const auto& h : store) {
@@ -2367,7 +2379,7 @@ Dht::exportValues() const
pk.pack_array(vals.size());
for (const auto& v : vals) {
pk.pack_array(4);
pk.pack(v.created.time_since_epoch().count());
pk.pack(to_system_time(v.created, now, system_now));
v.data->msgpack_pack(pk);
if (const auto& store_addr = v.store_bucket ? v.store_bucket->getAddr() : SockAddr {}) {
pk.pack_bin(store_addr.getLength());
@@ -2375,7 +2387,7 @@ Dht::exportValues() const
} else {
pk.pack_bin(0);
}
pk.pack(v.expiration.time_since_epoch().count());
pk.pack(to_system_time(v.expiration, now, system_now));
}
ve.second = {buffer.data(), buffer.data() + buffer.size()};
e.push_back(std::move(ve));
@@ -2387,6 +2399,7 @@ void
Dht::importValues(const std::vector<ValuesExport>& import)
{
const auto& now = scheduler.time();
const auto system_now = system_clock::now();
unsigned imported = 0, ignored = 0;
size_t imported_size = 0;
@@ -2411,8 +2424,15 @@ Dht::importValues(const std::vector<ValuesExport>& import)
Value tmp_val;
SockAddr store_addr;
try {
val_time = time_point {
time_point::duration {valel.via.array.ptr[0].as<time_point::duration::rep>()}};
const bool uses_system_time = valel.via.array.ptr[0].type == msgpack::type::EXT;
if (uses_system_time) {
val_time = from_system_time(valel.via.array.ptr[0].as<system_clock::time_point>(),
now,
system_now);
} else {
val_time = time_point {
time_point::duration {valel.via.array.ptr[0].as<time_point::duration::rep>()}};
}
tmp_val.msgpack_unpack(valel.via.array.ptr[1]);
if (valel.via.array.size >= 3) {
auto addr_blob = valel.via.array.ptr[2].as<Blob>();
@@ -2422,8 +2442,14 @@ Dht::importValues(const std::vector<ValuesExport>& import)
}
}
if (valel.via.array.size >= 4) {
expiration = time_point {
time_point::duration {valel.via.array.ptr[3].as<time_point::duration::rep>()}};
if (uses_system_time) {
expiration = from_system_time(valel.via.array.ptr[3].as<system_clock::time_point>(),
now,
system_now);
} else {
expiration = time_point {
time_point::duration {valel.via.array.ptr[3].as<time_point::duration::rep>()}};
}
}
} catch (const std::exception&) {
if (logger_)
@@ -2633,7 +2659,7 @@ Dht::onGetValuesDone(const Sp<Node>& node, net::RequestAnswer& a, Sp<Search>& sr
for (auto& getp : sr->callbacks) { /* call all callbacks for this search */
auto& get = getp.second;
if (not(get.get_cb or get.query_cb)
or (orig_query and get.query and not get.query->isSatisfiedBy(*orig_query)))
or (orig_query and get.query and not get.isSatisfiedBy(*orig_query)))
continue;
if (get.query_cb) { /* in case of a request with query */
+35 -21
View File
@@ -25,8 +25,10 @@ struct DhtProxyClient::OperationState
struct DhtProxyClient::Listener
{
Listener(OpValueCache&& c)
: cache(std::move(c))
// The proxy server is a single logical source that re-delivers values
// (push refresh, listen restart): don't count duplicates.
Listener(ValueCallback&& cb)
: cache(std::move(cb), false)
{}
OpValueCache cache;
@@ -34,26 +36,26 @@ struct DhtProxyClient::Listener
Sp<OperationState> opstate;
std::shared_ptr<http::Request> request;
#ifdef OPENDHT_PUSH_NOTIFICATIONS
std::unique_ptr<asio::steady_timer> refreshSubscriberTimer;
std::unique_ptr<asio::system_timer> refreshSubscriberTimer;
#endif
};
struct PermanentPut
{
PermanentPut(const Sp<Value>& v, std::unique_ptr<asio::steady_timer>&& j, const Sp<std::atomic_bool>& o)
PermanentPut(const Sp<Value>& v, std::unique_ptr<asio::system_timer>&& j, const Sp<std::atomic_bool>& o)
: value(v)
, refreshPutTimer(std::move(j))
, ok(o)
{}
Sp<Value> value;
std::unique_ptr<asio::steady_timer> refreshPutTimer;
std::unique_ptr<asio::system_timer> refreshPutTimer;
Sp<std::atomic_bool> ok;
};
struct DhtProxyClient::ProxySearch
{
SearchCache ops {};
SearchCache ops {false};
std::unique_ptr<asio::steady_timer> opExpirationTimer;
std::map<size_t, Listener> listeners {};
std::map<Value::Id, PermanentPut> puts {};
@@ -424,7 +426,7 @@ DhtProxyClient::put(const InfoHash& key, Sp<Value> val, DoneCallback cb, time_po
auto& search = searches_[key];
if (val->id) {
auto id = val->id;
auto refreshPutTimer = std::make_unique<asio::steady_timer>(httpContext_,
auto refreshPutTimer = std::make_unique<asio::system_timer>(httpContext_,
proxy::OP_TIMEOUT - proxy::OP_MARGIN);
refreshPutTimer->async_wait(
std::bind(&DhtProxyClient::handleRefreshPut, this, std::placeholders::_1, key, id));
@@ -540,7 +542,7 @@ DhtProxyClient::doPut(const InfoHash& key, Sp<Value> val, DoneCallbackSimple cb,
auto it = search.pendingPuts.find(val);
if (it != search.pendingPuts.end()) {
auto sok = std::make_shared<std::atomic_bool>(ok);
auto refreshPutTimer = std::make_unique<asio::steady_timer>(httpContext_,
auto refreshPutTimer = std::make_unique<asio::system_timer>(httpContext_,
proxy::OP_TIMEOUT
- proxy::OP_MARGIN);
refreshPutTimer->async_wait(
@@ -891,8 +893,8 @@ DhtProxyClient::listen(const InfoHash& key, ValueCallback cb, Value::Filter filt
* (if the proxy crash for any reason)
*/
if (!l->second.refreshSubscriberTimer)
l->second.refreshSubscriberTimer = std::make_unique<asio::steady_timer>(httpContext_);
l->second.refreshSubscriberTimer->expires_at(std::chrono::steady_clock::now() + proxy::OP_TIMEOUT
l->second.refreshSubscriberTimer = std::make_unique<asio::system_timer>(httpContext_);
l->second.refreshSubscriberTimer->expires_at(std::chrono::system_clock::now() + proxy::OP_TIMEOUT
- proxy::OP_MARGIN);
l->second.refreshSubscriberTimer->async_wait(
std::bind(&DhtProxyClient::handleResubscribe, this, std::placeholders::_1, key, token, opstate));
@@ -1192,9 +1194,9 @@ DhtProxyClient::restartListeners(const asio::error_code& ec)
for (auto& put : search.second.puts) {
doPut(key, put.second.value, [ok = put.second.ok](bool result) { *ok = result; }, time_point::max(), true);
if (!put.second.refreshPutTimer) {
put.second.refreshPutTimer = std::make_unique<asio::steady_timer>(httpContext_);
put.second.refreshPutTimer = std::make_unique<asio::system_timer>(httpContext_);
}
put.second.refreshPutTimer->expires_at(std::chrono::steady_clock::now() + proxy::OP_TIMEOUT
put.second.refreshPutTimer->expires_at(std::chrono::system_clock::now() + proxy::OP_TIMEOUT
- proxy::OP_MARGIN);
put.second.refreshPutTimer->async_wait(
std::bind(&DhtProxyClient::handleRefreshPut, this, std::placeholders::_1, key, put.first));
@@ -1275,10 +1277,10 @@ DhtProxyClient::pushNotificationReceived([[maybe_unused]] const std::map<std::st
auto vid = std::stoull(vidIt->second);
auto& put = search.puts.at(vid);
if (!put.refreshPutTimer)
put.refreshPutTimer = std::make_unique<asio::steady_timer>(httpContext_,
std::chrono::steady_clock::now());
put.refreshPutTimer = std::make_unique<asio::system_timer>(httpContext_,
std::chrono::system_clock::now());
else
put.refreshPutTimer->expires_at(std::chrono::steady_clock::now());
put.refreshPutTimer->expires_at(std::chrono::system_clock::now());
put.refreshPutTimer->async_wait(
std::bind(&DhtProxyClient::handleRefreshPut, this, std::placeholders::_1, key, vid));
ret = PushNotificationResult::PutRefresh;
@@ -1290,12 +1292,12 @@ DhtProxyClient::pushNotificationReceived([[maybe_unused]] const std::map<std::st
: PushNotificationResult::ListenRefresh;
}
} else {
auto key = InfoHash(notification.at("key"));
system_clock::time_point sendTime = system_clock::time_point::min();
try {
sendTime = system_clock::time_point(std::chrono::milliseconds(std::stoull(notification.at("t"))));
} catch (...) {
}
auto key = InfoHash(notification.at("key"));
auto& search = searches_.at(key);
auto expired = notification.find("exp");
for (auto& list : search.listeners) {
@@ -1308,13 +1310,25 @@ DhtProxyClient::pushNotificationReceived([[maybe_unused]] const std::map<std::st
if (expired == notification.end()) {
auto cb = list.second.cb;
auto oldValues = list.second.cache.getValues();
auto receivedIds = std::make_shared<std::set<Value::Id>>();
get(
key,
[cb, sendTime](const std::vector<Sp<Value>>& vals) { return cb(vals, false, sendTime); },
[cb, oldValues, sendTime](bool /*ok*/) {
[cb, sendTime, receivedIds](const std::vector<Sp<Value>>& vals) {
for (const auto& v : vals)
receivedIds->insert(v->id);
return cb(vals, false, sendTime);
},
[cb, oldValues, sendTime, receivedIds](bool ok) {
if (!ok)
return;
// Decrement old values refcount to expire values not
// present in the new list
cb(oldValues, true, sendTime);
std::vector<Sp<Value>> toExpire;
for (const auto& v : oldValues)
if (receivedIds->find(v->id) == receivedIds->end())
toExpire.emplace_back(v);
if (!toExpire.empty())
cb(toExpire, true, sendTime);
});
ret = PushNotificationResult::Values;
} else {
@@ -1378,9 +1392,9 @@ DhtProxyClient::resubscribe([[maybe_unused]] const InfoHash& key,
header.method(restinio::http_method_subscribe());
header.request_target("/key/" + key.toString());
if (!listener.refreshSubscriberTimer) {
listener.refreshSubscriberTimer = std::make_unique<asio::steady_timer>(httpContext_);
listener.refreshSubscriberTimer = std::make_unique<asio::system_timer>(httpContext_);
}
listener.refreshSubscriberTimer->expires_at(std::chrono::steady_clock::now() + proxy::OP_TIMEOUT - proxy::OP_MARGIN);
listener.refreshSubscriberTimer->expires_at(std::chrono::system_clock::now() + proxy::OP_TIMEOUT - proxy::OP_MARGIN);
listener.refreshSubscriberTimer->async_wait(
std::bind(&DhtProxyClient::handleResubscribe, this, std::placeholders::_1, key, token, opstate));
auto vcb = listener.cb;
+78 -45
View File
@@ -97,7 +97,7 @@ std::string
DhtProxyServer::ServerStats::toString() const
{
auto ret = fmt::format("Listens: {}, Puts: {}, PushListeners: {}\n"
"Push requests in the last {}: [Android: {}], [iOS: {}], [Unified: {}]\n"
"Push requests in the last {}: [Android: {}], [iOS: {}], [Unified: {}], [Huawei: {}]\n"
"Requests: {} per second.",
listenCount,
putCount,
@@ -106,6 +106,7 @@ DhtProxyServer::ServerStats::toString() const
androidPush.toString(),
iosPush.toString(),
unifiedPush.toString(),
huaweiPush.toString(),
requestRate);
if (nodeInfo) {
auto& ipv4 = nodeInfo->ipv4;
@@ -131,6 +132,7 @@ DhtProxyServer::ServerStats::toJson() const
result["androidPush"] = androidPush.toJson();
result["iosPush"] = iosPush.toJson();
result["unifiedPush"] = unifiedPush.toJson();
result["huaweiPush"] = huaweiPush.toJson();
result["requestRate"] = requestRate;
if (nodeInfo)
result["nodeInfo"] = nodeInfo->toJson();
@@ -210,7 +212,7 @@ DhtProxyServer::PermanentPut::msgpack_unpack(const msgpack::object& o)
clientId = cid->as<std::string>();
}
if (auto exp = findMapValue(o, "exp"sv)) {
expiration = from_time_t(exp->as<time_t>());
expiration = sys_clock::from_time_t(exp->as<time_t>());
}
if (auto token = findMapValue(o, "token"sv)) {
pushToken = token->as<std::string>();
@@ -240,7 +242,7 @@ DhtProxyServer::Listener::msgpack_unpack(const msgpack::object& o)
clientId = cid->as<std::string>();
}
if (auto exp = findMapValue(o, "exp"sv)) {
expiration = from_time_t(exp->as<time_t>());
expiration = sys_clock::from_time_t(exp->as<time_t>());
}
if (auto sid = findMapValue(o, "sid"sv)) {
if (not sessionCtx)
@@ -414,7 +416,7 @@ DhtProxyServer::loadState(Is& is, size_t size)
logger_->debug("[proxy:server] Loading {} persistent puts", puts_.size());
for (auto& put : puts_) {
for (auto& pput : put.second.puts) {
pput.second.expireTimer = std::make_unique<asio::steady_timer>(io_context(),
pput.second.expireTimer = std::make_unique<asio::system_timer>(io_context(),
pput.second.expiration);
pput.second.expireTimer->async_wait(std::bind(&DhtProxyServer::handleCancelPermamentPut,
this,
@@ -437,7 +439,7 @@ DhtProxyServer::loadState(Is& is, size_t size)
}
return json;
};
pput.second.expireNotifyTimer = std::make_unique<asio::steady_timer>(io_context(),
pput.second.expireNotifyTimer = std::make_unique<asio::system_timer>(io_context(),
pput.second.expiration
- proxy::OP_MARGIN);
pput.second.expireNotifyTimer->async_wait(
@@ -450,7 +452,7 @@ DhtProxyServer::loadState(Is& is, size_t size)
pput.second.topic));
}
#endif
dht_->put(put.first, pput.second.value, DoneCallbackSimple {}, time_point::max(), true);
dht_->put(put.first, pput.second.value, DoneCallbackSimple {}, dht::time_point::max(), true);
}
}
} else {
@@ -466,28 +468,31 @@ DhtProxyServer::loadState(Is& is, size_t size)
for (auto& pushListener : pushListeners_) {
for (auto& listeners : pushListener.second.listeners) {
for (auto& listener : listeners.second) {
if (not listener.sessionCtx)
listener.sessionCtx = std::make_shared<PushSessionContext>("");
{
std::lock_guard l(listener.sessionCtx->lock);
listener.sessionCtx->type = listener.type;
listener.sessionCtx->topic = listener.topic;
}
// start listening
listener.internalToken
= dht_->listen(listeners.first,
[this,
infoHash = listeners.first,
pushToken = pushListener.first,
type = listener.type,
clientId = listener.clientId,
sessionCtx = listener.sessionCtx,
topic = listener.topic](const std::vector<Sp<Value>>& values,
bool expired) {
sessionCtx = listener.sessionCtx](const std::vector<Sp<Value>>& values,
bool expired) {
return this->handlePushListen(infoHash,
pushToken,
type,
clientId,
sessionCtx,
topic,
values,
expired);
});
// expire notify
listener.expireNotifyTimer = std::make_unique<asio::steady_timer>(io_context(),
listener.expireNotifyTimer = std::make_unique<asio::system_timer>(io_context(),
listener.expiration
- proxy::OP_MARGIN);
auto jsonProvider = [infoHash = listeners.first.toString(),
@@ -509,7 +514,7 @@ DhtProxyServer::loadState(Is& is, size_t size)
listener.type,
listener.topic));
// cancel push listen
listener.expireTimer = std::make_unique<asio::steady_timer>(io_context(),
listener.expireTimer = std::make_unique<asio::system_timer>(io_context(),
listener.expiration);
listener.expireTimer->async_wait(std::bind(&DhtProxyServer::handleCancelPushListen,
this,
@@ -571,6 +576,18 @@ DhtProxyServer::~DhtProxyServer()
ioContext_->stop();
if (serverThread_.joinable())
serverThread_.join();
{
// ~Request notifies DONE and its handler erases from requests_:
// detach pending push requests so nothing calls back into this object.
decltype(requests_) pending;
{
std::lock_guard l(requestLock_);
pending = std::move(requests_);
}
for (auto& r : pending)
r.second->add_on_state_change_callback({});
pending.clear();
}
if (logger_)
logger_->debug("[proxy:server] http server closed");
}
@@ -624,6 +641,7 @@ DhtProxyServer::updateStats(std::shared_ptr<NodeInfo> info) const
stats.androidPush = androidPush_;
stats.iosPush = iosPush_;
stats.unifiedPush = unifiedPush_;
stats.huaweiPush = huaweiPush_;
}
#endif
stats.serverStartTime = serverStartTime_;
@@ -849,6 +867,8 @@ DhtProxyServer::getTypeFromString(const std::string& type)
return PushType::iOS;
else if (type == "unifiedpush")
return PushType::UnifiedPush;
else if (type == "huawei")
return PushType::Huawei;
return PushType::None;
}
@@ -907,6 +927,7 @@ DhtProxyServer::subscribe(restinio::request_handle_t request, restinio::router::
Json::Value r;
auto* char_data = reinterpret_cast<const char*>(request->body().data());
auto reader = std::unique_ptr<Json::CharReader>(jsonReaderBuilder_.newCharReader());
if (!reader->parse(char_data, char_data + request->body().size(), &r, &err)) {
auto response = initHttpResponse(request->create_response(restinio::status_bad_request()));
response.set_body(RESP_MSG_JSON_INCORRECT);
@@ -944,21 +965,26 @@ DhtProxyServer::subscribe(restinio::request_handle_t request, restinio::router::
listIt = std::prev(pushListeners.end());
listIt->clientId = clientId;
listIt->sessionCtx = std::make_shared<PushSessionContext>(sessionId);
} else {
}
{
// A refresh may come with another push platform or topic (e.g. a device
// whose push configuration changed): values must then be pushed there.
std::lock_guard l(listIt->sessionCtx->lock);
listIt->sessionCtx->sessionId = sessionId;
listIt->sessionCtx->type = type;
listIt->sessionCtx->topic = topic;
}
auto& listener = *listIt;
// Expiration
auto timeout = std::chrono::steady_clock::now() + proxy::OP_TIMEOUT;
// Expiration (wall-clock: persisted as epoch time, survives restarts)
auto timeout = sys_clock::now() + proxy::OP_TIMEOUT;
listener.expiration = timeout;
listener.type = type;
listener.topic = topic;
if (listener.expireNotifyTimer)
listener.expireNotifyTimer->expires_at(timeout - proxy::OP_MARGIN);
else
listener.expireNotifyTimer = std::make_unique<asio::steady_timer>(io_context(), timeout - proxy::OP_MARGIN);
listener.expireNotifyTimer = std::make_unique<asio::system_timer>(io_context(), timeout - proxy::OP_MARGIN);
auto jsonProvider = [h = infoHash.toString(), clientId, sessionCtx = listener.sessionCtx]() {
Json::Value json;
json["timeout"] = h;
@@ -975,7 +1001,7 @@ DhtProxyServer::subscribe(restinio::request_handle_t request, restinio::router::
listener.type,
listener.topic));
if (!listener.expireTimer)
listener.expireTimer = std::make_unique<asio::steady_timer>(io_context(), timeout);
listener.expireTimer = std::make_unique<asio::system_timer>(io_context(), timeout);
else
listener.expireTimer->expires_at(timeout);
listener.expireTimer->async_wait(std::bind(&DhtProxyServer::handleCancelPushListen,
@@ -1014,23 +1040,15 @@ DhtProxyServer::subscribe(restinio::request_handle_t request, restinio::router::
// Add listen on dht
if (logger_)
logger_->debug("[proxy:server] [subscribe {}] new", infoHash);
listener.internalToken = dht_->listen(infoHash,
[this,
infoHash,
pushToken,
type,
clientId,
sessionCtx = listener.sessionCtx,
topic](const std::vector<Sp<Value>>& values, bool expired) {
return this->handlePushListen(infoHash,
pushToken,
type,
clientId,
sessionCtx,
topic,
values,
expired);
});
listener.internalToken = dht_->listen(
infoHash,
[this,
infoHash,
pushToken,
clientId,
sessionCtx = listener.sessionCtx](const std::vector<Sp<Value>>& values, bool expired) {
return this->handlePushListen(infoHash, pushToken, clientId, sessionCtx, values, expired);
});
// Send response header
auto response = initHttpResponse(request->create_response());
response.set_body("{}\n");
@@ -1137,10 +1155,8 @@ DhtProxyServer::handleCancelPushListen(const asio::error_code& ec,
bool
DhtProxyServer::handlePushListen(const InfoHash& infoHash,
const std::string& pushToken,
PushType type,
const std::string& clientId,
const std::shared_ptr<DhtProxyServer::PushSessionContext>& sessionCtx,
const std::string& topic,
const std::vector<std::shared_ptr<Value>>& values,
bool expired)
{
@@ -1150,9 +1166,13 @@ DhtProxyServer::handlePushListen(const InfoHash& infoHash,
using namespace std::chrono;
json["t"] = Json::Value::Int64(duration_cast<milliseconds>(system_clock::now().time_since_epoch()).count());
PushType type;
std::string topic;
{
std::lock_guard l(sessionCtx->lock);
json["s"] = sessionCtx->sessionId;
type = sessionCtx->type;
topic = sessionCtx->topic;
}
// Build a comma-separated list of ids from the values. This is intended to be used when
@@ -1260,15 +1280,25 @@ DhtProxyServer::sendPushNotification(
Json::Value tokens(Json::arrayValue);
tokens[0] = token;
notification["tokens"] = std::move(tokens);
notification["platform"] = type == PushType::Android ? 2 : 1;
notification["data"] = std::move(json);
// gorush platform codes: 1 = iOS (APNs), 2 = Android (FCM), 3 = Huawei (HMS)
notification["platform"] = type == PushType::Android ? 2
: type == PushType::Huawei ? 3
: 1;
auto priority = highPriority ? "high" : "normal";
if (type == PushType::Android) {
if (type == PushType::Huawei) {
// HMS carries the payload as a serialized JSON string in its own field:
// the generic "data" object is ignored for this platform by gorush.
notification["huawei_data"] = Json::writeString(jsonBuilder_, json);
notification["huawei_ttl"] = "86400s"; // time to live = 24 hours
notification["priority"] = priority;
} else if (type == PushType::Android) {
notification["data"] = std::move(json);
Json::Value androidConfig(Json::objectValue);
androidConfig["priority"] = priority;
androidConfig["ttl"] = "86400s"; // time to live = 24 hours
notification["android"] = std::move(androidConfig);
} else {
notification["data"] = std::move(json);
notification["priority"] = priority;
const auto expiration = std::chrono::system_clock::now() + std::chrono::hours(24);
uint32_t exp = std::chrono::duration_cast<std::chrono::seconds>(expiration.time_since_epoch()).count();
@@ -1321,6 +1351,9 @@ DhtProxyServer::sendPushNotification(
case PushType::UnifiedPush:
unifiedPush_.increment(highPriority);
break;
case PushType::Huawei:
huaweiPush_.increment(highPriority);
break;
default:
break;
}
@@ -1405,7 +1438,7 @@ DhtProxyServer::put(restinio::request_handle_t request, restinio::router::route_
topic = pVal["topic"].asString();
}
std::lock_guard lock(lockSearchPuts_);
auto timeout = std::chrono::steady_clock::now() + proxy::OP_TIMEOUT;
auto timeout = sys_clock::now() + proxy::OP_TIMEOUT;
auto& sPuts = puts_[infoHash];
if (value->id == Value::INVALID_ID) {
for (auto& pp : sPuts.puts) {
@@ -1439,7 +1472,7 @@ DhtProxyServer::put(restinio::request_handle_t request, restinio::router::route_
pput.expiration = timeout;
if (not pput.expireTimer) {
// cancel permanent put
pput.expireTimer = std::make_unique<asio::steady_timer>(io_context(), timeout);
pput.expireTimer = std::make_unique<asio::system_timer>(io_context(), timeout);
#ifdef OPENDHT_PUSH_NOTIFICATIONS
if (not pushToken.empty()) {
pput.pushToken = pushToken;
@@ -1478,7 +1511,7 @@ DhtProxyServer::put(restinio::request_handle_t request, restinio::router::route_
return json;
};
if (!pput.expireNotifyTimer)
pput.expireNotifyTimer = std::make_unique<asio::steady_timer>(io_context(),
pput.expireNotifyTimer = std::make_unique<asio::system_timer>(io_context(),
timeout - proxy::OP_MARGIN);
else
pput.expireNotifyTimer->expires_at(timeout - proxy::OP_MARGIN);
@@ -1507,7 +1540,7 @@ DhtProxyServer::put(restinio::request_handle_t request, restinio::router::route_
response.done();
}
},
time_point::max(),
dht::time_point::max(),
permanent);
return restinio::request_handling_status_t::accepted;
} else {
+15 -2
View File
@@ -407,6 +407,7 @@ DhtRunner::join()
{
std::lock_guard lck(dht_mtx);
resetDht();
statusCbs.clear();
status4 = NodeStatus::Disconnected;
status6 = NodeStatus::Disconnected;
}
@@ -541,6 +542,8 @@ void
DhtRunner::registerType(const ValueType& type)
{
std::lock_guard lck(dht_mtx);
if (not dht_)
throw std::runtime_error("DHT runner is not running");
dht_->registerType(type);
}
@@ -554,8 +557,14 @@ DhtRunner::registerInsecureType(const ValueType& type)
void
DhtRunner::importValues(const std::vector<ValuesExport>& values)
{
std::lock_guard lck(dht_mtx);
dht_->importValues(values);
{
std::lock_guard lck(dht_mtx);
dht_->importValues(values);
}
// Recompute the runner wakeup after import schedules value expirations.
std::lock_guard lck(storage_mtx);
pending_ops_prio.emplace([](SecureDht&) {});
cv.notify_all();
}
unsigned
@@ -1233,6 +1242,10 @@ DhtRunner::pushNotificationReceived([[maybe_unused]] const std::map<std::string,
auto ret_token = std::make_shared<std::promise<PushNotificationResult>>();
auto future = ret_token->get_future();
std::lock_guard lck(storage_mtx);
if (running == State::Idle) {
ret_token->set_value(PushNotificationResult::IgnoredStopped);
return future;
}
pending_ops_prio.emplace([ret_token, this, data](SecureDht&) {
if (dht_)
ret_token->set_value(dht_->pushNotificationReceived(data));
+38 -8
View File
@@ -233,11 +233,32 @@ Connection::Connection(asio::io_context& ctx,
Connection::~Connection()
{
close();
// No asynchronous operation can be pending here: their handlers keep
// the connection alive (see wrapCallback), so closing synchronously
// is safe regardless of the calling thread.
doClose();
}
void
Connection::close()
{
// asio sockets are not thread-safe. Closing from a thread other than the
// one running the io_context races with in-flight operations on the same
// socket (e.g. async_connect closing and reopening it to try the next
// endpoint), corrupting the reactor's descriptor state.
// Always perform the close on the io_context thread.
if (ctx_.get_executor().running_in_this_thread()) {
doClose();
return;
}
asio::post(ctx_, [w = weak_from_this()] {
if (auto sthis = w.lock())
sthis->doClose();
});
}
void
Connection::doClose()
{
std::lock_guard lock(mutex_);
asio::error_code ec;
@@ -302,13 +323,13 @@ static X509*
issuer_from_chain(STACK_OF(X509) * fullchain)
{
X509 *cert, *issuer;
X509_NAME* issuer_name;
const X509_NAME* issuer_name;
cert = cert_from_chain(fullchain);
if ((issuer_name = X509_get_issuer_name(cert)) == nullptr)
return nullptr;
issuer = X509_find_by_subject(fullchain, issuer_name);
issuer = X509_find_by_subject(fullchain, const_cast<X509_NAME*>(issuer_name));
return issuer;
}
@@ -631,12 +652,13 @@ Connection::async_handshake(HandlerCb cb)
{
std::lock_guard lock(mutex_);
if (ssl_socket_) {
std::weak_ptr<Connection> wthis = shared_from_this();
// Keep the connection alive while the handshake is in flight, like
// every other asynchronous operation (see wrapCallback).
ssl_socket_
->async_handshake(asio::ssl::stream<asio::ip::tcp::socket>::client, [wthis, cb](const asio::error_code& ec) {
->async_handshake(asio::ssl::stream<asio::ip::tcp::socket>::client, [sthis = shared_from_this(), cb](const asio::error_code& ec) {
if (ec == asio::error::operation_aborted)
return;
if (auto sthis = wthis.lock()) {
{
auto& this_ = *sthis;
auto verify_ec = SSL_get_verify_result(this_.ssl_socket_->asio_ssl_stream().native_handle());
if (this_.logger_) {
@@ -917,8 +939,16 @@ Resolver::add_callback(ResolverCb cb, sa_family_t family)
else
cb(ec, filter(endpoints, family));
});
else
cb(ec_, family == AF_UNSPEC ? endpoints_ : filter(endpoints_, family));
else {
// Always dispatch the callback on the io_context thread. Invoking it
// synchronously would run the caller's continuation (e.g. the whole
// connect chain of Request::send) on the calling thread, mutating asio
// objects concurrently with the io thread running the io_context.
asio::post(resolver_.get_executor(),
[cb = std::move(cb),
ec = ec_,
endpoints = family == AF_UNSPEC ? endpoints_ : filter(endpoints_, family)] { cb(ec, endpoints); });
}
}
void
+128 -44
View File
@@ -40,8 +40,44 @@ struct NetworkEngine::PartialMessage
time_point start;
time_point last_part;
std::unique_ptr<ParsedMessage> msg;
std::vector<std::unique_ptr<ParsedMessage>> early_parts;
size_t early_bytes {};
bool request_quota_consumed {};
};
bool
NetworkEngine::isExpectedReply(const ParsedMessage& msg, const SockAddr& from)
{
auto request = requests.find(msg.tid);
if (request != requests.end() && request->second->pending() && !request->second->node->id
&& request->second->node->getAddr() == from)
return true;
auto node = cache.getNode(msg.id, from.getFamily());
if (!node || node->getAddr() != from)
return false;
auto pending = node->getRequest(msg.tid);
return (pending && pending->pending()) || node->getSocket(msg.tid);
}
bool
NetworkEngine::tryCompleteMessage(PartialMessage& partial, const SockAddr& from)
{
try {
if (!partial.msg->complete())
return false;
process(std::move(partial.msg), from);
} catch (const std::exception& e) {
if (logger_)
logger_->warn("Error while processing partial message: {}", e.what());
} catch (...) {
if (logger_)
logger_->warn("Unknown error while processing partial message");
}
return true;
}
std::vector<Blob>
serializeValues(const std::vector<Sp<Value>>& st)
{
@@ -466,35 +502,58 @@ NetworkEngine::processMessage(const uint8_t* buf, size_t buflen, SockAddr f)
// partial value data
if (msg->type == MessageType::ValueData) {
auto pmsg_it = partial_messages.find(msg->tid);
if (pmsg_it == partial_messages.end()) {
if (logIncoming_)
if (logger_)
logger_->debug("Unable to find partial message");
if (msg->fragment_parts.empty()
|| std::any_of(msg->fragment_parts.begin(), msg->fragment_parts.end(), [](const auto& fragment) {
return fragment.second.second.empty();
})) {
rateLimit(from);
return;
}
if (!pmsg_it->second.from.equals(from)) {
if (logger_)
logger_->debug("Received partial message data from unexpected IP address");
rateLimit(from);
PartialMessageKey key {from, msg->tid};
auto pmsg_it = partial_messages.find(key);
if (pmsg_it == partial_messages.end()) {
if (!rateLimit(from))
return;
auto inserted = partial_messages.emplace(key, PartialMessage {});
pmsg_it = inserted.first;
pmsg_it->second.from = from;
pmsg_it->second.start = now;
pmsg_it->second.last_part = now;
pmsg_it->second.request_quota_consumed = true;
scheduler.add(now + RX_MAX_PACKET_TIME, std::bind(&NetworkEngine::maintainRxBuffer, this, key));
}
auto& pmsg = pmsg_it->second;
if (!pmsg.msg) {
if (pmsg.early_parts.size() >= RX_MAX_EARLY_PARTS) {
partial_messages.erase(pmsg_it);
return;
}
size_t fragment_bytes {};
for (const auto& [index, fragment] : msg->fragment_parts) {
(void) index;
if (fragment.second.size() > RX_MAX_EARLY_BYTES - fragment_bytes) {
partial_messages.erase(pmsg_it);
return;
}
fragment_bytes += fragment.second.size();
}
if (fragment_bytes > RX_MAX_EARLY_BYTES - pmsg.early_bytes) {
partial_messages.erase(pmsg_it);
return;
}
pmsg.early_bytes += fragment_bytes;
pmsg.early_parts.emplace_back(std::move(msg));
pmsg.last_part = now;
scheduler.add(now + RX_TIMEOUT, std::bind(&NetworkEngine::maintainRxBuffer, this, key));
return;
}
// append data block
if (pmsg_it->second.msg->append(*msg)) {
pmsg_it->second.last_part = now;
// check data completion
if (pmsg_it->second.msg->complete()) {
try {
// process the full message
process(std::move(pmsg_it->second.msg), from);
} catch (const std::exception& e) {
if (logger_)
logger_->warn("Error while processing partial message: {}", e.what());
}
if (pmsg.msg->append(*msg)) {
pmsg.last_part = now;
if (tryCompleteMessage(pmsg, from)) {
partial_messages.erase(pmsg_it);
} else
scheduler.add(now + RX_TIMEOUT, std::bind(&NetworkEngine::maintainRxBuffer, this, msg->tid));
scheduler.add(now + RX_TIMEOUT, std::bind(&NetworkEngine::maintainRxBuffer, this, key));
}
return;
}
@@ -505,7 +564,32 @@ NetworkEngine::processMessage(const uint8_t* buf, size_t buflen, SockAddr f)
return;
}
if (msg->type > MessageType::Reply) {
if (msg->value_parts.empty()) {
if (msg->type > MessageType::Reply) {
/* Rate limit requests. */
if (!rateLimit(from)) {
if (logger_)
logger_->warn("Dropping request due to rate limiting");
return;
}
}
try {
process(std::move(msg), from);
} catch (const std::exception& e) {
if (logger_)
logger_->warn("Error while processing message: {}", e.what());
}
return;
}
PartialMessageKey partial_key {from, msg->tid};
if (msg->type == MessageType::Reply && !isExpectedReply(*msg, from))
return;
const auto partial = partial_messages.find(partial_key);
const bool request_quota_consumed = partial != partial_messages.end() && !partial->second.msg
&& partial->second.request_quota_consumed;
if (msg->type > MessageType::Reply && !request_quota_consumed) {
/* Rate limit requests. */
if (!rateLimit(from)) {
if (logger_)
@@ -514,28 +598,28 @@ NetworkEngine::processMessage(const uint8_t* buf, size_t buflen, SockAddr f)
}
}
if (msg->value_parts.empty()) {
try {
process(std::move(msg), from);
} catch (const std::exception& e) {
if (logger_)
logger_->warn("Error while processing message: {}", e.what());
return;
}
} else {
// starting partial message session
auto k = msg->tid;
auto& pmsg = partial_messages[k];
if (not pmsg.msg) {
// starting partial message session
auto inserted = partial_messages.emplace(partial_key, PartialMessage {});
auto& pmsg = inserted.first->second;
if (not pmsg.msg) {
if (inserted.second) {
pmsg.from = from;
pmsg.msg = std::move(msg);
pmsg.start = now;
pmsg.last_part = now;
scheduler.add(now + RX_MAX_PACKET_TIME, std::bind(&NetworkEngine::maintainRxBuffer, this, k));
scheduler.add(now + RX_TIMEOUT, std::bind(&NetworkEngine::maintainRxBuffer, this, k));
} else if (logger_)
logger_->error("Partial message with given TID {} already exists", k);
}
scheduler.add(now + RX_MAX_PACKET_TIME, std::bind(&NetworkEngine::maintainRxBuffer, this, partial_key));
}
pmsg.msg = std::move(msg);
pmsg.last_part = now;
for (const auto& part : pmsg.early_parts)
pmsg.msg->append(*part);
pmsg.early_parts.clear();
pmsg.early_bytes = 0;
if (tryCompleteMessage(pmsg, from)) {
partial_messages.erase(inserted.first);
} else {
scheduler.add(now + RX_TIMEOUT, std::bind(&NetworkEngine::maintainRxBuffer, this, partial_key));
}
} else if (logger_)
logger_->error("Partial message from {} with given TID {} already exists", from.toString(), msg->tid);
}
void
@@ -1614,9 +1698,9 @@ NetworkEngine::sendError(const SockAddr& addr, Tid tid, uint16_t code, const std
}
void
NetworkEngine::maintainRxBuffer(Tid tid)
NetworkEngine::maintainRxBuffer(const PartialMessageKey& key)
{
auto msg = partial_messages.find(tid);
auto msg = partial_messages.find(key);
if (msg != partial_messages.end()) {
const auto& now = scheduler.time();
if (msg->second.start + RX_MAX_PACKET_TIME < now || msg->second.last_part + RX_TIMEOUT < now) {
+26 -18
View File
@@ -12,18 +12,20 @@ OpValueCache::onValuesAdded(const std::vector<Sp<Value>>& vals, const system_clo
std::vector<Sp<Value>> newValues;
for (const auto& v : vals) {
auto viop = values.emplace(v->id, v);
auto& stored = viop.first->second;
if (viop.second) {
newValues.emplace_back(v);
} else if (*viop.first->second.data != *v) {
// Special case for edition
if (v->seq > viop.first->second.data->seq) {
viop.first->second.data = v;
newValues.emplace_back(v);
}
} else {
viop.first->second.refCount++;
} else if (*stored.data != *v) {
if (v->seq <= stored.data->seq)
continue; // older version from a lagging source
// Edited: only the reporting source is known to hold this version
stored.data = v;
stored.refCount = 1;
newValues.emplace_back(v);
} else if (countSources) {
stored.refCount++;
}
viop.first->second.updated = t;
stored.updated = t;
}
return newValues.empty() ? true : callback(newValues, false);
}
@@ -37,6 +39,10 @@ OpValueCache::onValuesExpired(const std::vector<Sp<Value>>& vals, const system_c
if (vit != values.end()) {
if (vit->second.updated > t)
continue;
// Mirror onValuesAdded: a version that was never counted (different
// content, not a newer seq) must not decrement the refcount
if (*v != *vit->second.data and v->seq <= vit->second.data->seq)
continue;
vit->second.updated = t;
vit->second.refCount--;
@@ -174,7 +180,7 @@ SearchCache::listen(const ValueCallback& get_cb, const Sp<Query>& q, Value::Filt
auto op = getOp(q);
if (op == ops.end()) {
// New query
op = ops.emplace(q, std::make_unique<OpCache>()).first;
op = ops.emplace(q, std::make_unique<OpCache>(countSources_)).first;
auto& cache = *op->second;
cache.searchToken = onListen(
q,
@@ -240,15 +246,17 @@ SearchCache::expire(const time_point& now, const std::function<void(size_t)>& on
bool
SearchCache::get(const Value::Filter& f, const Sp<Query>& q, const GetCallback& gcb, const DoneCallback& dcb) const
{
if (not gcb)
return false;
auto op = getOp(q);
if (op != ops.end()) {
auto vals = op->second->get(f);
if ((not vals.empty() and not gcb(vals)) or op->second->isSynced()) {
dcb(true, {});
return true;
}
}
return false;
if (op == ops.end() or not op->second->isSynced())
return false;
auto vals = op->second->get(f);
if (not vals.empty())
gcb(vals); // a consumer stopping early is not a failure
if (dcb)
dcb(true, {});
return true;
}
std::vector<Sp<Value>>
+28 -12
View File
@@ -21,19 +21,29 @@ struct OpCacheValueStorage
class OpValueCache
{
public:
OpValueCache(ValueCallback&& cb) noexcept
/**
* @param countSources When true, a value is kept until every source
* that reported it has expired it (per-node caches,
* IPv4/IPv6 searches, local storage). When false,
* re-deliveries of the same value are not counted
* and a single expiration removes it (single
* logical source, e.g. a proxy server).
*/
OpValueCache(ValueCallback&& cb, bool countSources = true) noexcept
: callback(std::forward<ValueCallback>(cb))
, countSources(countSources)
{}
explicit OpValueCache(OpValueCache&& o) noexcept
: values(std::move(o.values))
, callback(std::move(o.callback))
, countSources(o.countSources)
{
o.callback = {};
}
static ValueCallback cacheCallback(ValueCallback&& cb, std::function<void()>&& onCancel)
static ValueCallback cacheCallback(ValueCallback&& cb, std::function<void()>&& onCancel, bool countSources = true)
{
return [cache = std::make_shared<OpValueCache>(std::forward<ValueCallback>(cb)),
return [cache = std::make_shared<OpValueCache>(std::forward<ValueCallback>(cb), countSources),
onCancel = std::move(onCancel)](const std::vector<Sp<Value>>& vals, bool expired) {
auto ret = cache->onValue(vals, expired);
if (not ret)
@@ -89,19 +99,22 @@ private:
size_t syncedNodes {0};
std::map<Value::Id, OpCacheValueStorage> values {};
ValueCallback callback;
bool countSources {true};
};
class OpCache
{
public:
OpCache()
: cache([this](const std::vector<Sp<Value>>& vals, bool expired) {
if (expired)
onValuesExpired(vals);
else
onValuesAdded(vals);
return true;
})
explicit OpCache(bool countSources = true)
: cache(
[this](const std::vector<Sp<Value>>& vals, bool expired) {
if (expired)
onValuesExpired(vals);
else
onValuesAdded(vals);
return true;
},
countSources)
{}
bool onValue(const std::vector<Sp<Value>>& vals, bool expired) { return cache.onValue(vals, expired); }
@@ -156,7 +169,9 @@ private:
class SearchCache
{
public:
SearchCache() {}
explicit SearchCache(bool countSources = true)
: countSources_(countSources)
{}
SearchCache(SearchCache&&) = default;
using OnListen = std::function<size_t(Sp<Query>, ValueCallback, SyncCallback)>;
@@ -192,6 +207,7 @@ private:
size_t nextToken_ {1};
time_point nextExpiration_ {time_point::max()};
bool countSources_ {true};
};
} // namespace dht
+14 -4
View File
@@ -274,14 +274,21 @@ ParsedMessage::msgpack_unpack(const msgpack::object& msg)
if (!parsed.a && !parsed.r && !parsed.e && !parsed.u)
throw msgpack::type_error();
auto& req = parsed.a ? *parsed.a : (parsed.r ? *parsed.r : (parsed.u ? *parsed.u : *parsed.e));
if (parsed.e) {
if (parsed.e->type != msgpack::type::ARRAY)
if (parsed.e->type != msgpack::type::ARRAY or parsed.e->via.array.size < 1)
throw msgpack::type_error();
error_code = parsed.e->via.array.ptr[0].as<uint16_t>();
}
/* An error-only message carries no request map. The error array must never
be reached through the map view of the msgpack object union, and a
request field that is not a map must be rejected rather than
reinterpreted. */
const msgpack::object* req = parsed.a ? parsed.a : (parsed.r ? parsed.r : parsed.u);
if (req and req->type != msgpack::type::MAP)
throw msgpack::type_error();
struct ParsedReq
{
msgpack::object* values;
@@ -290,8 +297,9 @@ ParsedMessage::msgpack_unpack(const msgpack::object& msg)
msgpack::object* want;
} parsedReq {};
for (unsigned i = 0; i < req.via.map.size; i++) {
auto& o = req.via.map.ptr[i];
const uint32_t req_size = req ? req->via.map.size : 0;
for (unsigned i = 0; i < req_size; i++) {
auto& o = req->via.map.ptr[i];
if (o.key.type != msgpack::type::STR)
continue;
auto key = o.key.as<std::string_view>();
@@ -370,6 +378,8 @@ ParsedMessage::msgpack_unpack(const msgpack::object& msg)
} else if (parsedReq.fields) {
if (auto rfields = findMapValue(*parsedReq.fields, "f"sv)) {
auto vfields = rfields->as<std::set<Value::Field>>();
if (vfields.empty())
throw msgpack::type_error();
if (auto rvalues = findMapValue(*parsedReq.fields, "v"sv)) {
if (rvalues->type != msgpack::type::ARRAY)
throw msgpack::type_error();
+19 -4
View File
@@ -21,6 +21,21 @@ struct Dht::Get
QueryCallback query_cb;
GetCallback get_cb;
DoneCallback done_cb;
/**
* Tells if the answer to `answered` can be reported to this operation as-is.
*
* A vanilla get applies its own filter to the values it receives, so a
* broader answer is usable. A projected get only receives the fields
* selected by `answered` and can no longer filter them, so it requires the
* very same filters.
*/
bool isSatisfiedBy(const Query& answered) const
{
if (not query)
return true;
return query_cb ? query->isProjectionSatisfiedBy(answered) : query->isSatisfiedBy(answered);
}
};
/**
@@ -165,11 +180,11 @@ struct Dht::SearchNode
*
* @param now The time reference to now.
* @param update The time of the last 'get' op satisfying this request.
* @param q The query defining the "get" operation we're referring to.
* @param get The "get" operation we're referring to.
*
* @return true if we can send get, else false.
*/
bool canGet(time_point now, time_point update, const Sp<Query>& q) const
bool canGet(time_point now, time_point update, const Get& get) const
{
if (node->isExpired())
return false;
@@ -178,7 +193,7 @@ struct Dht::SearchNode
for (const auto& s : getStatus) {
if (s.second and s.second->pending())
is_pending = true;
if (s.first and q and q->isSatisfiedBy(*s.first) and s.second) {
if (s.first and get.query and get.isSatisfiedBy(*s.first) and s.second) {
if (s.second->pending())
pending_sq_status = true;
else if (s.second->completed() and not(update > s.second->reply_time))
@@ -189,7 +204,7 @@ struct Dht::SearchNode
}
return (not is_pending and now > last_get_reply + Node::NODE_EXPIRE_TIME)
or not(completed_sq_status or pending_sq_status or hasStartedPagination(q));
or not(completed_sq_status or pending_sq_status or hasStartedPagination(get.query));
}
/**
+3 -1
View File
@@ -232,6 +232,8 @@ Storage::store(const InfoHash& id, const Sp<Value>& value, time_point created, t
if (it->data != value) {
size_t size_old = it->data->size();
ssize_t size_diff = size_new - (ssize_t) size_old;
// Re-storing identical content is a refresh, not an edition
size_t edited = (*it->data != *value) ? 1 : 0;
// DHT_LOG.DEBUG("Updating %s -> %s", id.toString().c_str(), value->toString().c_str());
// clear quota for previous value
if (it->store_bucket)
@@ -243,7 +245,7 @@ Storage::store(const InfoHash& id, const Sp<Value>& value, time_point created, t
sb->insert(id, *value, expiration);
it->data = value;
total_size += size_diff;
return std::make_pair(&(*it), StoreDiff {size_diff, 0, 0, 1});
return std::make_pair(&(*it), StoreDiff {size_diff, 0, 0, edited});
}
} else {
// DHT_LOG.DEBUG("Storing %s -> %s", id.toString().c_str(), value->toString().c_str());
+53 -14
View File
@@ -238,28 +238,67 @@ operator==(const SockAddr& a, const SockAddr& b)
return a.equals(b);
}
time_point
from_system_time(system_clock::time_point t, time_point steady_now, system_clock::time_point system_now)
{
if (t == system_clock::time_point::min())
return time_point::min();
if (t == system_clock::time_point::max())
return time_point::max();
// Saturate before the cast: the steady clock duration may hold a narrower range than the system one.
constexpr auto max_dt = std::chrono::duration_cast<system_clock::duration>(duration::max());
constexpr auto min_dt = std::chrono::duration_cast<system_clock::duration>(duration::min());
auto system_dt = t - system_now;
if (system_dt >= max_dt)
return time_point::max();
if (system_dt <= min_dt)
return time_point::min();
auto dt = std::chrono::duration_cast<duration>(system_dt);
if (dt > duration(0) and steady_now > time_point::max() - dt)
return time_point::max();
else if (dt < duration(0) and steady_now < time_point::min() - dt)
return time_point::min();
return steady_now + dt;
}
time_point
from_system_time(system_clock::time_point t)
{
return from_system_time(t, clock::now(), system_clock::now());
}
system_clock::time_point
to_system_time(time_point t, time_point steady_now, system_clock::time_point system_now)
{
if (t == time_point::min())
return system_clock::time_point::min();
if (t == time_point::max())
return system_clock::time_point::max();
// Cast first: mixing clock durations of different periods would overflow the bound computations below.
auto dt = std::chrono::duration_cast<system_clock::duration>(t - steady_now);
if (dt > system_clock::duration(0) and system_now >= system_clock::time_point::max() - dt)
return system_clock::time_point::max();
else if (dt < system_clock::duration(0) and system_now <= system_clock::time_point::min() - dt)
return system_clock::time_point::min();
return system_now + dt;
}
system_clock::time_point
to_system_time(time_point t)
{
return to_system_time(t, clock::now(), system_clock::now());
}
time_point
from_time_t(std::time_t t)
{
auto dt = system_clock::from_time_t(t) - system_clock::now();
auto now = clock::now();
if (dt > system_clock::duration(0) and now > time_point::max() - dt)
return time_point::max();
else if (dt < system_clock::duration(0) and now < time_point::min() - dt)
return time_point::min();
return now + dt;
return from_system_time(system_clock::from_time_t(t));
}
std::time_t
to_time_t(time_point t)
{
auto dt = t - clock::now();
auto now = system_clock::now();
if (dt > duration(0) and now >= system_clock::time_point::max() - dt)
return system_clock::to_time_t(system_clock::time_point::max());
else if (dt < duration(0) and now <= system_clock::time_point::min() - dt)
return system_clock::to_time_t(system_clock::time_point::min());
return system_clock::to_time_t(now + std::chrono::duration_cast<system_clock::duration>(dt));
return system_clock::to_time_t(to_system_time(t));
}
Blob
+9 -1
View File
@@ -397,7 +397,7 @@ FieldValueIndex::containedIn(const FieldValueIndex& other) const
return false;
for (const auto& field : index) {
auto other_field = other.index.find(field.first);
if (other_field == other.index.end())
if (other_field == other.index.end() or not (field.second == other_field->second))
return false;
}
return true;
@@ -591,6 +591,14 @@ Query::isSatisfiedBy(const Query& q) const
return none or (where.isSatisfiedBy(q.where) and select.isSatisfiedBy(q.select));
}
bool
Query::isProjectionSatisfiedBy(const Query& q) const
{
/* A projected answer can not be filtered afterwards, so requiring the
filters of both queries to be a subset of each other, that is equal. */
return none or (isSatisfiedBy(q) and q.where.isSatisfiedBy(where));
}
std::ostream&
operator<<(std::ostream& s, const dht::Select& select)
{
+27 -6
View File
@@ -71,11 +71,21 @@ public:
{
std::vector<Sp<Value>> expired_values;
for (auto it = values.begin(); it != values.end();) {
if (it->second.expiration <= now) {
auto effectiveExpiration = it->second.expiration;
if (status == ListenSyncStatus::SYNCED)
// While the listen is synced, the remote node pushes
// explicit expirations and periodic refreshes. Add a
// grace period proportional to the value's type
// expiration so that a single lost UDP refresh
// notification does not cause a premature local drop.
// A grace of one type-expiration covers exactly one
// missed re-announcement cycle.
effectiveExpiration += it->second.typeExpiration;
if (effectiveExpiration <= now) {
expired_values.emplace_back(std::move(it->second.data));
it = values.erase(it);
} else {
next = std::min(next, it->second.expiration);
next = std::min(next, effectiveExpiration);
++it;
}
}
@@ -148,12 +158,14 @@ private:
Sp<Value> data {};
time_point created {};
time_point expiration {};
duration typeExpiration {}; /* value type's expiration period, used as grace when synced */
CacheValueStorage() {}
CacheValueStorage(const Sp<Value>& v, time_point t, time_point e)
CacheValueStorage(const Sp<Value>& v, time_point t, time_point e, duration te)
: data(v)
, created(t)
, expiration(e)
, typeExpiration(te)
{}
};
@@ -166,15 +178,22 @@ private:
{
std::vector<Sp<Value>> nvals;
for (const auto& value : new_values) {
auto te = types.getType(value->type).expiration;
auto v = values.find(value->id);
if (v == values.end()) {
// new value
nvals.emplace_back(value);
values.emplace(value->id, CacheValueStorage(value, now, now + types.getType(value->type).expiration));
values.emplace(value->id, CacheValueStorage(value, now, now + te, te));
} else {
// refreshed value
v->second.created = now;
v->second.expiration = now + types.getType(v->second.data->type).expiration;
v->second.expiration = now + te;
v->second.typeExpiration = te;
if (*value != *v->second.data and value->seq > v->second.data->seq) {
// edited value (same rule as OpValueCache): emit as add
v->second.data = value;
nvals.emplace_back(value);
}
}
}
CallbackQueue ret;
@@ -201,8 +220,10 @@ private:
auto v = values.find(vid);
if (v == values.end())
return;
auto te = types.getType(v->second.data->type).expiration;
v->second.created = now;
v->second.expiration = now + types.getType(v->second.data->type).expiration;
v->second.expiration = now + te;
v->second.typeExpiration = te;
}
};
+174
View File
@@ -0,0 +1,174 @@
#ifdef NDEBUG
#undef NDEBUG
#endif
#include "opendht_c.h"
#include <assert.h>
#include <stdio.h>
#include <string.h>
#ifdef _MSC_VER
#include <gnutls/gnutls.h>
#endif
_Static_assert(_Alignof(dht_infohash) == _Alignof(uint32_t), "InfoHash alignment must match uint32_t");
_Static_assert(_Alignof(dht_pkid) == _Alignof(uint32_t), "PkId alignment must match uint32_t");
_Static_assert(sizeof(dht_infohash) == HASH_LEN, "InfoHash size must remain unchanged");
_Static_assert(sizeof(dht_pkid) == 32, "PkId size must remain unchanged");
static void
done_cb(bool ok, void* user_data)
{
int* completed = user_data;
*completed += ok ? 1 : 10;
}
static bool
get_cb(const dht_value* value, void* user_data)
{
(void) value;
(void) user_data;
return true;
}
static bool
listen_cb(const dht_value* value, bool expired, void* user_data)
{
(void) expired;
(void) user_data;
assert(dht_value_get_id(value) == 42);
return true;
}
static void
released_cb(void* user_data)
{
++*(int*) user_data;
}
int
main(void)
{
#ifdef _MSC_VER
int crypto_error = gnutls_global_init();
if (crypto_error != GNUTLS_E_SUCCESS) {
fprintf(stderr, "Failed to initialize GnuTLS: %s\n", gnutls_strerror(crypto_error));
return 1;
}
#endif
const uint8_t invalid[] = {1, 2, 3};
assert(dht_publickey_import(invalid, sizeof(invalid)) == NULL);
assert(dht_privatekey_import(invalid, sizeof(invalid), "") == NULL);
assert(dht_certificate_import(invalid, sizeof(invalid)) == NULL);
assert(dht_privatekey_generate(1) == NULL);
dht_infohash hash;
dht_infohash_get_from_string(&hash, "binding regression");
assert(!dht_infohash_is_zero(&hash));
dht_infohash restored;
dht_infohash_from_hex_null(&restored, dht_infohash_print(&hash));
assert(memcmp(&hash, &restored, sizeof(hash)) == 0);
struct {
uint8_t prefix;
dht_infohash hash;
dht_pkid pkid;
} embedded = {0};
assert((uintptr_t) &embedded.hash % _Alignof(uint32_t) == 0);
assert((uintptr_t) &embedded.pkid % _Alignof(uint32_t) == 0);
dht_infohash_get_from_string(&embedded.hash, "abc");
assert(!dht_infohash_is_zero(&embedded.hash));
assert(strcmp(dht_infohash_print(&embedded.hash), "a9993e364706816aba3e25717850c26c9cd0d89d") == 0);
dht_infohash_from_hex_null(&embedded.hash, "a9993e364706816aba3e25717850c26c9cd0d89d");
dht_infohash_from_hex(&restored, dht_infohash_print(&embedded.hash));
assert(memcmp(&embedded.hash, &restored, sizeof(restored)) == 0);
dht_infohash_zero(&embedded.hash);
assert(dht_infohash_is_zero(&embedded.hash));
dht_infohash_random(&embedded.hash);
assert(!dht_infohash_is_zero(&embedded.hash));
assert(strcmp(dht_pkid_print(&embedded.pkid),
"0000000000000000000000000000000000000000000000000000000000000000") == 0);
dht_value* value = dht_value_with_id_new(invalid, sizeof(invalid), 42);
assert(value != NULL);
assert(dht_value_get_id(value) == 42);
dht_value_set_user_type(value, "application/octet-stream");
assert(strcmp(dht_value_get_user_type(value), "application/octet-stream") == 0);
dht_value* copy = dht_value_ref(value);
dht_value_unref(value);
dht_data_view data = dht_value_get_data(copy);
assert(data.size == sizeof(invalid));
assert(memcmp(data.data, invalid, data.size) == 0);
dht_value_set_type(copy, 17);
dht_value_set_seq(copy, 23);
dht_value_set_priority(copy, 1);
dht_value_set_push_type(copy, "audioCall");
assert(dht_value_get_type(copy) == 17);
assert(dht_value_get_seq(copy) == 23);
assert(dht_value_get_priority(copy) == 1);
assert(strcmp(dht_value_get_push_type(copy), "audioCall") == 0);
dht_value_unref(copy);
dht_privatekey* private_key = dht_privatekey_generate(2048);
assert(private_key != NULL);
dht_publickey* public_key = dht_privatekey_get_publickey(private_key);
dht_blob* signature = dht_privatekey_sign(private_key, invalid, sizeof(invalid));
assert(signature != NULL);
dht_data_view signature_data = dht_blob_get_data(signature);
assert(dht_publickey_check_signature(public_key,
(const char*) invalid,
sizeof(invalid),
(const char*) signature_data.data,
signature_data.size));
dht_blob_delete(signature);
dht_blob* packed = dht_privatekey_export_blob(private_key, "binding-password");
assert(packed != NULL);
dht_data_view packed_data = dht_blob_get_data(packed);
dht_privatekey* imported = dht_privatekey_import(packed_data.data, packed_data.size, "binding-password");
assert(imported != NULL);
dht_privatekey_delete(imported);
dht_blob_delete(packed);
dht_privatekey_delete(private_key);
dht_publickey_delete(public_key);
dht_identity identity = dht_identity_generate("binding regression", NULL);
assert(identity.privatekey && identity.certificate);
packed = dht_certificate_export(identity.certificate);
assert(packed != NULL);
packed_data = dht_blob_get_data(packed);
dht_certificate* certificate = dht_certificate_import(packed_data.data, packed_data.size);
assert(certificate != NULL);
dht_certificate_delete(certificate);
dht_blob_delete(packed);
dht_identity_delete(&identity);
dht_runner_config config;
dht_runner_config_default(&config);
config.threaded = false;
dht_runner* runner = dht_runner_new();
assert(dht_runner_run_config(runner, 0, &config) == 0);
assert(dht_runner_get_bound_port(runner, AF_INET) != 0);
int released = 0;
dht_op_token* token = dht_runner_listen(runner, &hash, listen_cb, released_cb, &released);
int completed = 0;
value = dht_value_with_id_new(invalid, sizeof(invalid), 42);
dht_runner_put(runner, &hash, value, done_cb, &completed, false);
dht_value_unref(value);
assert(completed == 0);
assert(dht_runner_loop(runner) == 0);
dht_runner_get(runner, &hash, get_cb, done_cb, &completed);
assert(dht_runner_join(runner) == 0);
assert(!dht_runner_is_running(runner));
assert(released == 1);
assert(completed >= 10);
dht_op_token_delete(token);
assert(dht_runner_run(runner, 0) == 0);
assert(dht_runner_join(runner) == 0);
dht_runner_delete(runner);
#ifdef _MSC_VER
gnutls_global_deinit();
#endif
puts("C binding regressions passed");
return 0;
}
+188 -2
View File
@@ -59,6 +59,24 @@ CryptoTester::testSignatureEncryption()
}
}
void
CryptoTester::testPublicKeyPemIsPlainText()
{
auto key = dht::crypto::PrivateKey::generate();
auto pem = key.getPublicKey().toString();
// The exported key is PEM text: it must not carry the terminating null
// byte GnuTLS counts in the exported size, which would otherwise be
// embedded in anything the string is serialized into.
CPPUNIT_ASSERT(not pem.empty());
CPPUNIT_ASSERT_EQUAL(std::string::npos, pem.find('\0'));
CPPUNIT_ASSERT_EQUAL('\n', pem.back());
// The exported text still reads back as the same key.
dht::crypto::PublicKey reloaded(dht::Blob(pem.begin(), pem.end()));
CPPUNIT_ASSERT_EQUAL(key.getPublicKey().getId(), reloaded.getId());
}
void
CryptoTester::testCertificateRevocation()
{
@@ -95,10 +113,159 @@ CryptoTester::testCertificateRevocation()
CPPUNIT_ASSERT_MESSAGE(v.toString(), v);
}
namespace {
/** Read a CRL Number extension blob (big-endian, variable length) as an integer. */
uint64_t
crlNumberValue(const dht::Blob& number)
{
uint64_t value {0};
for (uint8_t byte : number)
value = (value << 8) | byte;
return value;
}
} // namespace
void
CryptoTester::testRevocationListNumber()
{
auto ca = dht::crypto::generateIdentity("ca");
auto device = dht::crypto::generateIdentity("dev", ca);
dht::crypto::RevocationList crl;
CPPUNIT_ASSERT_EQUAL(uint64_t(0), crlNumberValue(crl.getNumber()));
crl.revoke(*device.second);
crl.sign(ca);
auto first = crl.getNumber();
// An unnumbered list gets a random number on its first signature.
CPPUNIT_ASSERT(not first.empty());
CPPUNIT_ASSERT(crlNumberValue(first) != 0);
// Re-signing keeps the same lineage and moves the number forward.
crl.sign(ca);
auto second = crl.getNumber();
CPPUNIT_ASSERT_EQUAL(crlNumberValue(first) + 1, crlNumberValue(second));
crl.sign(ca);
CPPUNIT_ASSERT_EQUAL(crlNumberValue(first) + 2, crlNumberValue(crl.getNumber()));
// The number is a DER integer, so it is stored with leading zeros stripped.
// Incrementing must still work for numbers shorter than 8 bytes.
for (uint64_t start : {uint64_t(1), uint64_t(0x7f), uint64_t(0x80), uint64_t(0xffff)}) {
dht::crypto::RevocationList small;
small.revoke(*device.second);
small.sign(ca, {}, start);
CPPUNIT_ASSERT_EQUAL(start, crlNumberValue(small.getNumber()));
small.sign(ca);
CPPUNIT_ASSERT_EQUAL(start + 1, crlNumberValue(small.getNumber()));
small.sign(ca);
CPPUNIT_ASSERT_EQUAL(start + 2, crlNumberValue(small.getNumber()));
}
}
void
CryptoTester::testRevocationListExplicitNumber()
{
auto ca = dht::crypto::generateIdentity("ca");
auto device = dht::crypto::generateIdentity("dev", ca);
// Values chosen to cover leading zero bytes, a full-width number and the largest
// value that still encodes as a positive 8-byte DER integer.
const std::vector<uint64_t> numbers {1,
42,
0xff,
0x100,
0x0000'0001'0000'0000ull,
0x0000'02a1'0000'0001ull,
0x7fff'ffff'ffff'ffffull};
for (auto number : numbers) {
dht::crypto::RevocationList crl;
crl.revoke(*device.second);
crl.sign(ca, {}, number);
CPPUNIT_ASSERT_EQUAL(number, crlNumberValue(crl.getNumber()));
CPPUNIT_ASSERT(crl.isSignedBy(*ca.second));
CPPUNIT_ASSERT(crl.isRevoked(*device.second));
// The number must survive a pack/unpack round-trip.
dht::crypto::RevocationList unpacked(crl.getPacked());
CPPUNIT_ASSERT_EQUAL(number, crlNumberValue(unpacked.getNumber()));
}
// An explicit number overrides the increment, in both directions.
dht::crypto::RevocationList crl;
crl.revoke(*device.second);
crl.sign(ca, {}, 1000);
CPPUNIT_ASSERT_EQUAL(uint64_t(1000), crlNumberValue(crl.getNumber()));
crl.sign(ca, {}, 5);
CPPUNIT_ASSERT_EQUAL(uint64_t(5), crlNumberValue(crl.getNumber()));
// Passing 0 falls back to the default behavior: increment the current number.
crl.sign(ca, {}, 0);
CPPUNIT_ASSERT_EQUAL(uint64_t(6), crlNumberValue(crl.getNumber()));
}
void
CryptoTester::testRevocationListPartitionedNumber()
{
// Two devices holding the same authority issue revocation lists independently.
// RFC 5280 5.2.3 requires CRL numbers to increase monotonically for a given CRL
// scope and issuer; partitioning the low bits by device keeps each sequence
// monotonic while making collisions between devices impossible.
constexpr unsigned TAG_BITS = 24;
constexpr uint64_t TAG_A = 0x0a0b0c;
constexpr uint64_t TAG_B = 0x0a0b0d;
auto ca = dht::crypto::generateIdentity("ca");
auto account = dht::crypto::generateIdentity("acc", ca, 4096, true);
auto deviceA = dht::crypto::generateIdentity("devA", account);
auto deviceB = dht::crypto::generateIdentity("devB", account);
uint64_t previous {0};
dht::crypto::RevocationList crlA;
for (unsigned i = 1; i <= 4; i++) {
crlA.revoke(*deviceA.second);
crlA.sign(account, {}, (uint64_t(i) << TAG_BITS) | TAG_A);
auto number = crlNumberValue(crlA.getNumber());
CPPUNIT_ASSERT_EQUAL(TAG_A, number & ((uint64_t(1) << TAG_BITS) - 1));
CPPUNIT_ASSERT(number > previous);
CPPUNIT_ASSERT_EQUAL(uint64_t(0), number >> 63);
previous = number;
}
// Both devices branch from the same counter but never mint the same number.
dht::crypto::RevocationList branchA;
branchA.revoke(*deviceA.second);
branchA.sign(account, {}, (uint64_t(9) << TAG_BITS) | TAG_A);
dht::crypto::RevocationList branchB;
branchB.revoke(*deviceB.second);
branchB.sign(account, {}, (uint64_t(9) << TAG_BITS) | TAG_B);
CPPUNIT_ASSERT(crlNumberValue(branchA.getNumber()) != crlNumberValue(branchB.getNumber()));
// Both are valid and revoke only their own target.
CPPUNIT_ASSERT(branchA.isSignedBy(*account.second));
CPPUNIT_ASSERT(branchB.isSignedBy(*account.second));
CPPUNIT_ASSERT(branchA.isRevoked(*deviceA.second));
CPPUNIT_ASSERT(not branchA.isRevoked(*deviceB.second));
CPPUNIT_ASSERT(branchB.isRevoked(*deviceB.second));
CPPUNIT_ASSERT(not branchB.isRevoked(*deviceA.second));
// A number minted by an older client is fully random, so bit 63 may be set. Masking
// it out and taking the counter part still yields a strictly greater number.
constexpr uint64_t legacy = 0xdead'beef'0123'4567ull;
dht::crypto::RevocationList legacyCrl;
legacyCrl.revoke(*deviceA.second);
legacyCrl.sign(account, {}, legacy);
auto counter = ((crlNumberValue(legacyCrl.getNumber()) & ~(uint64_t(1) << 63)) >> TAG_BITS) + 1;
legacyCrl.sign(account, {}, (counter << TAG_BITS) | TAG_A);
CPPUNIT_ASSERT(crlNumberValue(legacyCrl.getNumber()) > (legacy & ~(uint64_t(1) << 63)));
CPPUNIT_ASSERT_EQUAL(uint64_t(0), crlNumberValue(legacyCrl.getNumber()) >> 63);
}
void
CryptoTester::testCertificateRequest()
{
// Generate CA
{ // Generate CA
auto ca = dht::crypto::generateIdentity("Test CA");
// Generate signed request
@@ -149,6 +316,25 @@ CryptoTester::testCertificateSerialNumber()
};
auto serial = dht::crypto::Certificate(cert_pem).getSerialNumber();
CPPUNIT_ASSERT(std::equal(SERIAL.begin(), SERIAL.end(), serial.begin(), serial.end()));
auto key = dht::crypto::PrivateKey::generateEC();
for (unsigned i = 0; i < 32; i++) {
auto generated = dht::crypto::Certificate::generate(key, "generated", {});
auto generatedSerial = generated.getSerialNumber();
CPPUNIT_ASSERT(not generatedSerial.empty());
CPPUNIT_ASSERT_EQUAL(uint8_t {0}, uint8_t(generatedSerial.front() & 0x80));
CPPUNIT_ASSERT(std::any_of(generatedSerial.begin(), generatedSerial.end(), [](uint8_t byte) { return byte != 0; }));
}
}
void
CryptoTester::testEcKeyCurve()
{
auto key = dht::crypto::PrivateKey::generateEC(GNUTLS_ECC_CURVE_SECP256R1);
unsigned int bits = 0;
CPPUNIT_ASSERT_EQUAL(static_cast<int>(GNUTLS_PK_EC),
gnutls_x509_privkey_get_pk_algorithm2(key.x509_key, &bits));
CPPUNIT_ASSERT_EQUAL(256u, bits);
}
void
+25
View File
@@ -12,9 +12,14 @@ class CryptoTester : public CppUnit::TestFixture
{
CPPUNIT_TEST_SUITE(CryptoTester);
CPPUNIT_TEST(testSignatureEncryption);
CPPUNIT_TEST(testPublicKeyPemIsPlainText);
CPPUNIT_TEST(testCertificateRevocation);
CPPUNIT_TEST(testRevocationListNumber);
CPPUNIT_TEST(testRevocationListExplicitNumber);
CPPUNIT_TEST(testRevocationListPartitionedNumber);
CPPUNIT_TEST(testCertificateRequest);
CPPUNIT_TEST(testCertificateSerialNumber);
CPPUNIT_TEST(testEcKeyCurve);
CPPUNIT_TEST(testOcsp);
CPPUNIT_TEST(testAesEncryption);
CPPUNIT_TEST(testAesEncryptionWithMultipleKeySizes);
@@ -36,10 +41,26 @@ public:
* Test data signature, encryption and decryption
*/
void testSignatureEncryption();
/**
* Test that an exported public key is text only, with no embedded null byte
*/
void testPublicKeyPemIsPlainText();
/**
* Test certificate generation, validation and revocation
*/
void testCertificateRevocation();
/**
* Test the default CRL Number behavior: random on first signature, incremented after
*/
void testRevocationListNumber();
/**
* Test signing a revocation list with a caller-provided CRL Number
*/
void testRevocationListExplicitNumber();
/**
* Test that issuers sharing an authority can partition the CRL Number space
*/
void testRevocationListPartitionedNumber();
/**
* Test certificate requests
*/
@@ -48,6 +69,10 @@ public:
* Test certificate serial number extraction
*/
void testCertificateSerialNumber();
/**
* Test EC private key generation with an explicit curve
*/
void testEcKeyCurve();
/**
* Test OCSP
*/
Loaded 100 of 122 files, more files were not shown because too many files have changed in this diff. Show more