Compare commits

...
Author SHA1 Message Date
Adrien Beraud ccb0b4340e transport: make private network grants permanent 2026-11-01 22:18:02 -05:00
Adrien Beraud fa78704062 transport: test permanent grants and authority transitions 2026-11-01 22:12:28 -05:00
Adrien Beraud 625625a186 tests: isolate projection scheduling from transport handshakes 2026-10-29 23:35:22 -04:00
Adrien Beraud f0d039b4a0 tests: keep persisted transport paths as filesystem paths 2026-10-29 23:28:48 -04:00
Adrien Beraud d464214d73 tests: use portable paths for persisted transport keys 2026-10-29 23:28:48 -04:00
Adrien Beraud b69c66feed network: retain session authorization and budget credential checks 2026-10-29 23:28:48 -04:00
Adrien Beraud 3d9ee6a51a tests: verify cached private authorization expires and renews 2026-10-29 23:28:48 -04:00
Adrien Beraud f71697c81b tests: cover private client authorization and cached credential limits 2026-10-29 23:28:48 -04:00
Adrien Beraud b071022283 tests: fix transport test portability on GCC and Windows 2026-10-29 23:28:48 -04:00
Adrien Beraud 487bc2fce2 meson: add optional Rust bindings build 2026-10-29 23:28:48 -04:00
Adrien Beraud 758f280651 network: validate sessions before cache admission and isolate fragments 2026-10-29 23:28:48 -04:00
Adrien Beraud 20464e6f16 tests: cover rejected sessions and cross-sender fragments 2026-10-29 23:28:48 -04:00
Adrien Beraud d211de73e0 transport: erase temporary session keys and HMAC state 2026-10-29 23:28:48 -04:00
Adrien Beraud bde661a2d4 tests: verify session key destruction erases secrets 2026-10-29 23:28:48 -04:00
Adrien Beraud 49ef8ee84d docs: describe separate rate limits 2026-10-29 23:28:48 -04:00
Adrien Beraud 390528a1af tools: disable public-key rate limit with other limits 2026-10-29 23:28:48 -04:00
Adrien Beraud 97c9d58a5c network: separate rate limits for sessions, unauthenticated requests and public-key work 2026-10-29 23:28:48 -04:00
Adrien Beraud e5903d5c94 tests: check separate request, session and public-key rate limits 2026-10-29 23:28:48 -04:00
Adrien Beraud 65bbbead34 docs: describe control-only transport establishment 2026-10-29 23:28:48 -04:00
Adrien Beraud ffaf1e2ca3 dht: establish cold searches through target-aware routing probes 2026-10-29 23:28:48 -04:00
Adrien Beraud af7d5c2770 network: finish recovery only after accepted replies 2026-10-29 23:28:48 -04:00
Adrien Beraud c14858a82a tests: exercise private routing and unreachable search peers 2026-10-29 23:28:48 -04:00
Adrien Beraud 1010a4c90f tests: cover establishment failure and legacy fallback 2026-10-29 23:28:48 -04:00
Adrien Beraud ea6bacf0a1 network: establish sessions without spending withheld request attempts 2026-10-29 23:28:48 -04:00
Adrien Beraud 36f67747d7 tests: preserve routing progress during cold session establishment 2026-10-29 23:28:48 -04:00
Adrien Beraud b093dc0907 tests: cover cold establishment and delayed recovery replies 2026-10-29 23:28:48 -04:00
Adrien Beraud a06695734d network: require a session for keyed value requests 2026-10-29 23:28:48 -04:00
Adrien Beraud 35aa47029e tests: reject unvalidated transport value requests 2026-10-29 23:28:48 -04:00
Adrien Beraud 4af12a4c2c docs: describe per-peer session recovery 2026-10-29 23:28:48 -04:00
Adrien Beraud 3f77b6b820 network: recover lost sessions per peer 2026-10-29 23:28:48 -04:00
Adrien Beraud 858867dc56 tests: check that session recovery withholds every request to the peer 2026-10-29 23:28:48 -04:00
Adrien Beraud fcb0632a41 docs: describe session recovery without payloads in Initial envelopes 2026-10-29 23:28:48 -04:00
Adrien Beraud 3ba61e7e4e network: recover lost sessions with a ping before resending requests 2026-10-29 23:28:48 -04:00
Adrien Beraud 4ef2c21ce0 docs: describe fragment sizes per envelope 2026-10-29 23:28:48 -04:00
Adrien Beraud d98c66b2c0 network: size value fragments by the envelope in use 2026-10-29 23:28:48 -04:00
Adrien Beraud dc09983b6b tests: require session recovery before resending payloads, and envelope-sized fragments 2026-10-29 23:28:48 -04:00
Adrien Beraud da00246fe8 docs: describe the fragment size calculation 2026-10-29 23:28:48 -04:00
Adrien Beraud e092e4d89f network: size value fragments from the minimum MTU
Fragments carried 1280 bytes of value data, plus the msgpack framing and,
for encrypted peers, the envelope overhead: packets exceeded the 1280-byte
IPv6 minimum MTU. Derive both fragment sizes from the 1232-byte UDP payload
that fits in it, less an upper bound of the fragment framing.
2026-10-29 23:28:48 -04:00
Adrien Beraud ed8131a3c6 tests: require value fragments to fit the IPv6 minimum MTU 2026-10-29 23:28:48 -04:00
Adrien Beraud f29c77cadf docs: state the scope of each transport limitation 2026-10-29 23:28:48 -04:00
Adrien Beraud d90988cb6a transport: save key files through temporary replacements 2026-10-29 23:28:48 -04:00
Adrien Beraud 3cf24c89d4 tests: require atomic transport-key state persistence 2026-10-29 23:28:48 -04:00
Adrien Beraud 5e7bcd1189 node: use compact fixed-width transport badges 2026-10-29 23:28:48 -04:00
Adrien Beraud 8ac6baee22 tests: require fixed-width node transport badges 2026-10-29 23:28:48 -04:00
Adrien Beraud f9bb4bcb40 node: show transport authentication state in logs 2026-10-29 23:28:48 -04:00
Adrien Beraud 35f486e035 tests: cover node transport state formatting 2026-10-29 23:28:48 -04:00
Adrien Beraud 0d5e4c761a network: enforce transport trust and address validation boundaries 2026-10-29 23:28:48 -04:00
Adrien Beraud efc372dc3f tests: cover transport trust and address validation boundaries 2026-10-29 23:28:48 -04:00
Adrien Beraud ddfbca3106 docs: describe fragment binding to the envelope state 2026-10-29 23:28:48 -04:00
Adrien Beraud 353c3d4e11 network: bind fragmented message parts to their envelope state
Value data fragments carry no node ID and were merged into a partial
message by source address and transaction ID alone, first come first
served. Transaction IDs increment from a random start and show in any
plaintext packet of a pair, such as the first-contact ping. An attacker
spoofing the sender's address could thus pre-seed forged plaintext
fragments, or a plaintext header, for an encrypted reply: the real parts
were then dropped as overlapping, forging unsigned values or losing the
message.

Parts of a message must now all be plaintext, or all come in envelopes
from the same sender key as its first part. An encrypted part replaces
plaintext state, which anyone can forge from a spoofed address; plaintext
or foreign-key parts never join an encrypted message.
2026-10-29 23:28:48 -04:00
Adrien Beraud 51324683e8 tests: cover forged parts in encrypted fragmented messages
Value data fragments carry no node ID. Parts of a message received in
envelopes must all come under the sender key of the envelopes, so that
plaintext or foreign-key parts spoofed from the sender's address, which
would otherwise be merged first, are ignored.
2026-10-29 23:28:48 -04:00
Adrien Beraud 4633483e06 docs: describe address-validated session caching 2026-10-29 23:28:48 -04:00
Adrien Beraud 6aee7ceb5c network: cache sessions only for address-validated peers
A session was cached for an address whenever we answered a peer there,
including replies to Initial and Hello envelopes, whose source address is
not verified. A spoofed Initial or Hello therefore replaced the session of
the real owner of the address, whose Data envelopes were then dropped until
a retry, and spoofed Initials from many addresses filled the cache and
evicted legitimate sessions, which were picked by address order.

Only cache a session for a peer whose ephemeral key arrived in a Session
or Data envelope, i.e. opened with our key for its address. Replies to
Initial and Hello envelopes derive their keys without caching them, so a
node that sent an Initial only confirms the session, and switches to Data
envelopes, once the peer answered a Session envelope. Evict the least
recently used session when the cache is full.
2026-10-29 23:28:48 -04:00
Adrien Beraud 32709230c6 tests: cover session cache hijacking by unvalidated senders
Replies to Initial and Hello envelopes must not cache a session at the
source address, which can be spoofed, and the cache must evict the least
recently used session rather than the lowest address. A node that started
with an Initial envelope only switches to Data envelopes once the peer
answered a Session envelope, which is when the peer caches the session.
2026-10-29 23:28:48 -04:00
Adrien Beraud 7b526410dd docs: describe Data envelopes and session confirmation 2026-10-29 23:28:48 -04:00
Adrien Beraud fbff1eced4 network: forget the peer ephemeral key on plaintext key advertisement
A peer only advertises its key in plaintext when it doesn't know ours, e.g. after a restart: it lost our session, so answer with an Initial envelope.
2026-10-29 23:28:48 -04:00
Adrien Beraud d23cb42b27 tests: cover a peer restarting with the same key
A node restarting with the same key and contacting a peer by address gets answers encrypted with the ephemeral key of its previous run.
2026-10-29 23:28:48 -04:00
Adrien Beraud e78e54b8c5 transport: send keyless Data envelopes once the session is confirmed
Sessions are keyed by peer address, so the receiver finds the keys from the source address: Data envelopes drop both keys from the header (30 bytes of overhead instead of 94) and need no key derivation to reject. A session is confirmed when the peer answers our envelope request with the session keys, or sends Data itself. Credentials only travel in key-carrying envelopes, which replaces Peer::acked. A received Initial envelope unconfirms the session, and full session caches evict instead of refusing, so a peer that derived keys keeps them.
2026-10-29 23:28:48 -04:00
Adrien Beraud 40560eec4e tests: cover keyless Data envelopes and session confirmation
Steady-state traffic uses Data envelopes, without sender keys nor credential, once the peer answered a request. Data envelopes only open from the session address, also authorize writes, and survive the receiver's epoch rotation.
2026-10-29 23:28:48 -04:00
Adrien Beraud 853bb1eeed docs: describe address-bound ephemeral keys and write authorization 2026-10-29 23:28:48 -04:00
Adrien Beraud 22d87a0f0e transport: replace write tokens with address-bound sessions
Ephemeral keys derive from a per-epoch secret and the peer address, so only that address learns them. A Session envelope then proves return routability: encrypted put, refresh and listen are accepted without token, and messages to encrypted peers no longer carry one (40 bytes saved). Initial envelopes cannot write. Tokens for plaintext peers move from Dht to NetworkEngine and derive from the same epoch secret. Session caches are keyed by address, and cached keys of both epochs are tried before any derivation.
2026-10-29 23:28:48 -04:00
Adrien Beraud dfb66c5a23 tests: cover session-based write authorization and address-bound keys
Encrypted writes must come in a Session envelope, whose ephemeral key is bound to the sender address. Plaintext writes still need a token, now issued by the network engine.
2026-10-29 23:28:48 -04:00
Adrien Beraud 56fcc5b918 docs: describe forward-secret transport keys
Ephemeral keys, Initial and Session key schedules, the unauthenticated
hello and the retry rule, with their security properties.
2026-10-29 23:28:48 -04:00
Adrien Beraud 26b30d4162 transport: add forward secrecy with rotated ephemeral keys
Each node rotates an ephemeral X25519 key every 30 minutes and keeps the
previous one for another period. Envelopes carry the sender static and
ephemeral keys. Session envelopes use keys derived from all four DH
combinations of static and ephemeral keys, erased with the epoch.
Initial envelopes, sent before the peer's ephemeral key is known, mix
the sender ephemeral and static keys with the receiver static key.

A hello is unauthenticated: it is only answered, without adding or
moving its claimed node. A peer is acknowledged only by a reply to a
request sent in an envelope, and a request retry restarts with an
Initial envelope and our credential.
2026-10-29 23:28:48 -04:00
Adrien Beraud 791a667fe2 tests: cover ephemeral key rotation, peer restart and hello
The previous ephemeral key is accepted for one more period, then erased.
A request retry restarts with an Initial envelope and our credential.
A hello is only answered and doesn't change what is known of its sender.
2026-10-29 23:28:48 -04:00
Adrien Beraud b693e3028b tests: cover forward-secret transport keys and envelope header
Session keys are directional and mix both static and ephemeral keys,
Initial keys are used before the peer's ephemeral key is known.
2026-10-29 23:28:48 -04:00
Adrien Beraud 3e975d530a docs: describe the transport encryption design 2026-10-29 23:28:48 -04:00
Adrien Beraud a32acbfa08 dht: persist the generated transport key in the node state
With persist_path, a transport key generated by the node is saved in the
state (owner-only file permissions) and restored on start, keeping the
node ID across restarts. Configured keys are never written to the state.
2026-10-29 23:28:48 -04:00
Adrien Beraud 43147b8791 tests: check generated transport key persistence 2026-10-29 23:28:48 -04:00
Adrien Beraud c1762af7e9 tools: add transport encryption and private network options
--no-encryption, --transport-key, --network-authority and
--network-credential configure the node; --gen-network-authority and
--issue-credential manage a private network.
2026-10-29 23:28:48 -04:00
Adrien Beraud 68b5996ba9 tests: cover encrypted transport and private networks 2026-10-29 23:28:48 -04:00
Adrien Beraud 7bc287fe2f network_engine: encrypt the DHT protocol
Nodes using transport encryption derive their node ID from their X25519
transport key, advertise it in plaintext messages ("k") and in node
lists ("k4"/"k6"), and encrypt all traffic with nodes whose key is
known. Keys are bound to node IDs, so they can't be forged, and
plaintext messages never move a node known to support encryption.
Legacy nodes keep talking in plaintext.

Private networks (TransportConfig::authorities) only accept nodes with
a credential issued by one of the network authorities, never send or
accept plaintext, and introduce themselves to addresses with a Hello.
2026-10-29 23:28:48 -04:00
Adrien Beraud 523dda8b3d tests: add transport primitives tests 2026-10-29 23:28:48 -04:00
Adrien Beraud 1c32dd0efb transport: add protocol encryption and credential primitives
X25519 transport keys, node ID derivation from the public key,
HKDF-SHA256 key agreement, ChaCha20-Poly1305 envelopes that legacy
msgpack parsers reject, and Ed25519 network authority credentials.
2026-10-29 23:28:48 -04:00
Adrien Beraud 1c914d8977 BUILD.md: cleanup 2026-10-29 23:28:29 -04:00
Adrien Beraud 3c87991e92 docs: make Rust README precise and factual 2026-10-29 22:46:57 -04:00
Adrien Beraud 52287bf608 ci: use npm environment for publishing 2026-10-29 22:27:34 -04:00
Adrien Beraud 061ce2deb4 docker: build with RelWithDebInfo 2026-10-29 22:27:04 -04:00
Adrien Beraud 64b536fb53 ci: publish Python wheels from the tag release workflow 2026-10-29 21:48:33 -04:00
Adrien Beraud 67c76f17de README: add Node.js bindings 2026-10-29 21:20:25 -04:00
Adrien Beraud 25fde756e8 docs: add linked package and Docker badges 2026-10-29 21:12:35 -04:00
Adrien Beraud f6b3eda280 rust: rewrite README 2026-10-29 20:55:07 -04:00
Adrien Beraud 9aee407a42 rust: avoid dual-stack conflict in startup error test 2026-10-29 10:35:51 -04:00
Adrien Beraud 722ef44c7d rust: move owned Value directly through callbacks and streams 2026-10-28 23:51:46 -04:00
Adrien Beraud 6ed2720bd7 rust: test single-copy async value ownership 2026-10-28 23:51:46 -04:00
Adrien Beraud 9005f73141 rust: add optional Serde typed operations 2026-10-28 23:51:46 -04:00
Adrien Beraud 9785c0a127 rust: add typed operation regression tests 2026-10-28 23:51:41 -04:00
Adrien Beraud 5662cc36d1 rust: publish crate on release with trusted publishing 2026-10-28 22:04:57 -04:00
Adrien Beraud e116cfe548 rust: add optional runtime-independent async API 2026-10-28 21:11:43 -04:00
Adrien Beraud 1c5e8378b8 rust: add async adapter regression tests 2026-10-28 21:11:42 -04:00
Adrien Beraud c630e80345 ci: use trusted publishing for Python wheels 2026-10-29 00:26:07 +00:00
cyrilleandCopilot 17f83edce4 proxy: add Huawei (HMS) push type
Route push notifications to Huawei devices, which have no Google Play
Services and therefore cannot receive FCM. gorush, already used as the
push relay, supports HMS natively as platform 3.

HMS takes its payload as a serialized JSON string in "huawei_data" and
carries its own ttl/priority fields, so it needs its own branch rather
than the generic "data" object used for FCM and APNs.

The enum value is appended last: MSGPACK_ADD_ENUM serializes it as an
integer, so reordering would break compatibility with deployed proxies.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-10-29 00:02:02 +00:00
Adrien Beraud afab251376 nodejs: isolate native Linux dependency symbols
Use RTLD_DEEPBIND when loading the Linux addon so its llhttp calls do not resolve to Node's incompatible bundled parser. The CI image uses llhttp 9.2.1 while Node 22 exports 9.4.3, corrupting the heap when the proxy client starts. Preserve default loading on other platforms.
2026-10-27 23:04:32 -04:00
Adrien Beraud 86d452fbb6 nodejs: bundle source-built llhttp license notices
The Noble dependency image builds llhttp from source, so dpkg-query cannot identify its license package. Include the pinned upstream MIT notice when bundling Linux llhttp libraries while retaining package-owner checks for other dependencies.
2026-10-27 23:04:32 -04:00
Adrien Beraud 08ff775232 ci: rely on npm trusted publishing 2026-10-27 22:48:58 -04:00
Adrien Beraud 738e0e22bf nodejs: add streaming iterators and startup proxy configuration 2026-10-27 22:46:34 -04:00
Adrien Beraud 8ec5b47c86 nodejs: test streaming iterators and proxy configuration 2026-10-27 22:09:21 -04:00
Adrien Beraud e1c31568f9 readme: cleanup 2026-10-27 22:01:05 -04:00
Adrien Beraud 45bad769b4 nodejs: fix npm archive paths and validate release versions 2026-10-27 21:22:15 -04:00
Adrien Beraud 09780fd923 nodejs: publish verified npm archives from tag releases 2026-10-27 20:36:12 -04:00
Adrien Beraud 747e70d1ae nodejs: simplify design 2026-10-27 20:12:17 -04:00
Adrien Beraud 05abfa89bf nodejs: initialize GnuTLS before using the addon 2026-10-27 11:33:17 -04:00
Adrien Beraud b714714128 build: bump version to 4.4.99 2026-10-27 10:21:04 -04:00
Adrien Beraud 2e8b36477d ci: publish Noble OpenDHT release image 2026-10-27 09:59:17 -04:00
Adrien Beraud 14543ae477 docker: switch to ninja and add Noble Dockerfile
Use Ninja instead of Make in the main Dockerfile for faster builds
and install ninja-build in DockerfileDeps. Also, add a new
Dockerfile for Ubuntu Noble.
2026-10-27 09:57:33 -04:00
Adrien Beraud 044135732f docker: add simdutf to remaining dependency images 2026-10-27 00:31:34 -04:00
Adrien Beraud 21b7f0d757 docker: restore Noble dependency image for Node builds 2026-10-27 00:31:34 -04:00
Adrien Beraud a5bfd3d933 ci: test Node packages on Ubuntu 24.04 and 26.04 2026-10-27 00:26:08 -04:00
Adrien Beraud af38323728 nodejs: release dispatcher before environment teardown 2026-10-27 00:14:56 -04:00
Adrien Beraud 62e0ded803 ci: use Ubuntu 26.04 for Node.js packages 2026-10-27 00:14:56 -04:00
Adrien Beraud a09cfefe5e tests: require native alignment for C hashes 2026-10-27 00:03:07 -04:00
Adrien Beraud 13f340694e tools: rebuild proxy web client with concurrent live sessions 2026-10-27 00:03:07 -04:00
Adrien Beraud 3723c4acf1 nodejs: migrate runtime tooling and examples to ESM 2026-10-27 00:02:09 -04:00
Adrien Beraud f19bca0df1 tests: enforce ESM imports for Node bindings 2026-10-27 00:02:09 -04:00
Adrien Beraud fbc7f90897 tests: cover Node connectivity and concurrent CLI operations 2026-10-27 00:02:09 -04:00
Adrien Beraud 3a3c421ef9 nodejs: add an interactive dhtnode example 2026-10-27 00:02:09 -04:00
Adrien Beraud 9ec7a58ee3 tests: cover the Node dhtnode example CLI 2026-10-27 00:02:00 -04:00
Adrien Beraud ce73eb052e ci: build and verify Node-API npm packages 2026-10-27 00:02:00 -04:00
Adrien Beraud d716fae4bc nodejs: expose per-family connectivity status and events 2026-10-27 00:02:00 -04:00
Adrien Beraud a040bfeebb nodejs: add crypto APIs and relocatable npm prebuilds 2026-10-26 23:57:50 -04:00
Adrien Beraud 3223742d70 tests: cover Node crypto types and npm archive contents 2026-10-26 23:57:50 -04:00
Adrien Beraud d2d56fcc20 tests: cover Node crypto persistence and authenticated values 2026-10-26 23:57:50 -04:00
Adrien Beraud af814f027f nodejs: add Node-API runner bindings over the C++ library 2026-10-26 23:57:50 -04:00
Adrien Beraud 2016a9e2fa tests: verify Node TypeScript API and deferred garbage collection 2026-10-26 23:57:50 -04:00
Adrien Beraud 1db8abb7fc c: align hash structs with native types 2026-10-26 23:57:50 -04:00
Adrien Beraud 3f7b1bc3f5 tests: cover unaligned C hash storage 2026-10-26 23:56:59 -04:00
Adrien Beraud 019c25633e tests: initialize GnuTLS for MSVC C bindings 2026-10-26 23:56:43 -04:00
Adrien Beraud 8193f43593 tests: cover Node runner collection cancellation and queue overflow 2026-10-26 23:54:00 -04:00
Adrien Beraud 39564f14c1 tests: define Node-API binding lifecycle and loopback contracts 2026-10-26 23:54:00 -04:00
Adrien Beraud 83778d84fe rust: update edition to 2024
Update the Rust edition to 2024 and mark the FFI extern block as unsafe.
2026-10-26 09:57:02 -04:00
Adrien Beraud b61ee620e4 dht: schedule projected gets with matching reuse rules 2026-10-26 09:21:10 -04:00
Adrien Beraud 0d120eb5d4 tests: cover projected query request scheduling 2026-10-26 09:21:10 -04:00
Cyrille BéraudandCopilot 9b9056c0c8 dht: do not report unfiltered answers to filtered queries
Query::isSatisfiedBy() considers an unfiltered answer sufficient for a
filtered query, which is true for a vanilla get: onGetValuesDone() applies
the operation filter to the values before calling back.

A projected query is different. It receives the fields another query selected
and can no longer evaluate its own filters on them, all the more so as those
filters may target fields that were not selected at all. Two concurrent
queries such as "SELECT id" and "SELECT id WHERE value_type=1" therefore made
the filtered callback report identifiers whose type does not match, and
searchNodeGetDone() additionally cancelled its own request so it never got a
correctly filtered answer.

Add Query::isProjectionSatisfiedBy(), which additionally requires both queries
to apply the same filters, and use it for operations with a query callback.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-10-26 09:21:10 -04:00
Cyrille BéraudandCopilot bf8e3ea251 value: compare field values in FieldValueIndex::containedIn
containedIn() only checked that the other index declared the same field
keys, never that they held the same values, contradicting its documented
contract of containment "with the same value".

Dht::query() uses containedIn() to deduplicate projected results. Because
every result of a given query shares the same projection schema, any two
results compared equal: the first one suppressed all the later ones, and the
"lesser" branch even erased the already reported result. A query selecting a
field therefore reported a single value regardless of how many matched.

Compare the FieldValue as well, and add a regression covering distinct
values, equal indexes and a narrower projection contained in a wider one.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-10-26 09:21:10 -04:00
Adrien Beraud 34eb86a119 value: support sequence numbers in wire filters 2026-10-26 09:19:57 -04:00
Adrien Beraud 75c789ceb2 rust: cover sequence filters across loopback nodes 2026-10-26 09:15:36 -04:00
Adrien Beraud da438d1bd2 value: reproduce sequence filter serialization failure 2026-10-26 09:12:12 -04:00
Adrien Beraud 44e67e1389 rust: parse hexadecimal hashes with string length checks 2026-10-26 09:11:42 -04:00
Adrien Beraud 7f9126b3b0 rust: cover empty and short hexadecimal hashes 2026-10-26 09:10:39 -04:00
Adrien Beraud 574674d3a2 rust: own native pointers in movable handle wrappers 2026-10-26 09:10:16 -04:00
Adrien Beraud c60703bfc3 rust: cover moves across native handle types 2026-10-26 09:10:16 -04:00
Adrien Beraud a0220827cd rust: reproduce moving native values out of boxes 2026-10-26 09:08:03 -04:00
Adrien Beraud fb90211157 python: expose value state and harden runner operations 2026-10-26 01:10:34 -04:00
Adrien Beraud 9501465293 python: cover value state and binding safety regressions 2026-10-26 01:06:48 -04:00
Adrien Beraud 95bd26c3be rust: bump version to 0.2.0 and add author 2026-10-26 00:51:27 -04:00
Adrien Beraud 87463c2d6c rust: own callbacks and expose safe native binding APIs 2026-10-26 00:51:27 -04:00
Adrien Beraud baf413dd0d c: expose lifecycle queries crypto and callback ownership 2026-10-26 00:51:27 -04:00
Adrien Beraud 6bf01f3b6d runner: release status callback captures on join 2026-10-26 00:51:27 -04:00
Adrien Beraud 852403fbeb crypto: verify signatures using the public key algorithm 2026-10-26 00:51:27 -04:00
Adrien Beraud 5f6d5c769e rust: reject certificate registration on stopped runners 2026-10-26 00:51:27 -04:00
Adrien Beraud 42e4419b75 bindings: verify retained values and queued push delivery 2026-10-26 00:51:27 -04:00
Adrien Beraud 4e94d123b2 rust: cover threaded crypto and native callback teardown 2026-10-26 00:51:27 -04:00
Adrien Beraud 7f3f1b4ec4 rust: cover filtered queries and extension hooks 2026-10-26 00:51:27 -04:00
Adrien Beraud 6d462c0dbf rust: cover callback ownership and binding error paths 2026-10-26 00:51:27 -04:00
Adrien Beraud 0285acc766 c: cover lifecycle progress and crypto persistence APIs 2026-10-26 00:51:27 -04:00
Adrien Beraud 0626e82f4a c: contain crypto generation failures and run binding regressions 2026-10-26 00:51:27 -04:00
Adrien Beraud aa01f68cb5 c: cover binding ownership and invalid crypto input 2026-10-26 00:51:27 -04:00
Adrien Beraud af3fd618bf python: expose observability and projected query APIs 2026-10-26 00:51:27 -04:00
Adrien Beraud b96bdfbf04 python: cover observability and querying APIs 2026-10-26 00:51:27 -04:00
Adrien Beraud 60ca85f782 python: release native resources and guard callback cleanup 2026-10-26 00:51:27 -04:00
Adrien Beraud e4966aa457 python: cover native resource and callback cleanup regressions 2026-10-26 00:51:27 -04:00
Adrien Beraud aa9d23870b python: guard native errors, iterator lifetimes, and hash bounds 2026-10-26 00:51:27 -04:00
Adrien Beraud ac68db84eb python: cover binding exception and memory safety regressions 2026-10-26 00:51:27 -04:00
Adrien Beraud bbbd9122af python: use loopback addresses in bootstrap test 2026-10-26 00:51:27 -04:00
Adrien Beraud cf58d66282 python: expose node state persistence configuration 2026-10-26 00:51:27 -04:00
Adrien Beraud 2ddf1e5ec5 python: await shutdown before checking persisted state 2026-10-26 00:51:27 -04:00
Adrien Beraud e48bfd578a python: cover node state persistence configuration 2026-10-26 00:51:27 -04:00
Adrien Beraud 1c3fc22c31 meson: install Python bindings as opendht package 2026-10-26 00:51:27 -04:00
Adrien Beraud b6cf2fd246 python: expose key persistence and value metadata 2026-10-26 00:51:27 -04:00
Adrien Beraud 91c3f62c0c python: validate restored keys through certificate chains 2026-10-26 00:51:27 -04:00
Adrien Beraud 2703cf129d python: cover key persistence and value metadata 2026-10-26 00:51:27 -04:00
Adrien Beraud 51c2aadd4e python: handle cancelled and repeated async completions 2026-10-26 00:51:27 -04:00
Adrien Beraud 8b20e912d6 docker: use alpine 3.24 runtime 2026-10-26 00:51:27 -04:00
Adrien Beraud b66ff83fe2 python: preserve permanence in blocking puts 2026-10-26 00:51:27 -04:00
Adrien Beraud 552813e1b9 python: cover permanence forwarding and async cancellation 2026-10-26 00:51:27 -04:00
Adrien Beraud 601a8fbf80 python: release GIL while joining and destroying runners 2026-10-26 00:51:27 -04:00
Adrien Beraud 26488d18a6 python: translate crypto errors and guard empty objects 2026-10-26 00:51:27 -04:00
Adrien Beraud 53c509c54f python: cover crypto errors and GIL-safe teardown 2026-10-26 00:51:27 -04:00
Adrien Beraud def6a28d96 python: fix listener ownership and isolate run config 2026-10-26 00:51:27 -04:00
Adrien Beraud dbd5953b17 python: cover listener lifetime and config isolation 2026-10-26 00:51:27 -04:00
Cyrille BéraudandCopilot 2893823a77 securedht: forward push platform and topic
SecureDht only forwarded the push token to the DHT it wraps, so the
push platform and topic given to DhtRunner after the proxy client was
created never reached it. A device started without push platform
(e.g. a Jami account created by a build without push support) then
subscribed with an empty platform once given a token, and the proxy
server pushed it as an iOS device.

Co-authored-by: Copilot (Claude Opus 5.5) <223556219+Copilot@users.noreply.github.com>
2026-10-26 02:55:59 +00:00
Cyrille BéraudandCopilot 3ec03ccc0b proxy_server: keep push platform up to date
The DHT listen callback of a push subscription kept the push type and
topic of the request that created it. Refreshing the subscription
with another platform updated the listener, but values kept being
pushed to the first platform: an Android device that once subscribed
without platform was pushed as an iOS device forever (BadDeviceToken)
and never received values such as connection requests.

Keep the push type and topic in the session context shared with the
callback, and update them on each subscription refresh and when the
state is restored.

Co-authored-by: Copilot (Claude Opus 5.5) <223556219+Copilot@users.noreply.github.com>
2026-10-26 02:55:59 +00:00
Cyrille BéraudandCopilot 20fa751da0 tests: check push platform of proxy notifications
Move the fake push gateway of the full chain test into a reusable
helper, and add two tests showing that the proxy server pushes to the
wrong platform:
- when a device refreshes its subscription with another platform,
  new values keep being pushed with the platform of the first
  subscription;
- a platform set after the proxy client started is not used by its
  subscriptions.

Co-authored-by: Copilot (Claude Opus 5.5) <223556219+Copilot@users.noreply.github.com>
2026-10-26 02:55:59 +00:00
Adrien Beraud 0d572f0dab build: update supported Python versions
Drop Python 3.9 and 3.10 and add support for Python 3.15.
2026-10-25 18:16:26 -04:00
Adrien Beraud d9c401e1e2 docker: migrate to 26.04 for main LLVM image 2026-10-25 12:42:20 -04:00
Adrien Beraud dbdbe65a5d docker: fix local build context and missing dependencies 2026-10-25 12:39:30 -04:00
Adrien Beraud 6f5bec3af6 update version to 4.4.2 2026-10-25 12:11:46 -04:00
Adrien Beraud d07a9e6583 ci: update cibuildwheel to v4.3.4 2026-10-25 12:06:54 -04:00
Adrien Beraud fd43a329e7 deps: update RESTinio to 0.7.10 across Docker and CI 2026-10-25 11:58:53 -04:00
Adrien Beraud 93b2e8d5b9 ci: migrate vcpkg setup and binary caching 2026-10-25 13:46:35 +00:00
Adrien Beraud d66713d97e build: update vcpkg to 2026.10.16 2026-10-25 13:46:35 +00:00
Adrien Beraud 7af9868c47 docker: update to Alpine 3.24, Ubuntu 26.04 2026-10-24 23:25:08 -04:00
Adrien Beraud 05a524b038 docker: update RESTinio to 0.7.10 2026-10-24 23:25:08 -04:00
Adrien Beraud 57ed00579e pht: preserve prefix padding encoding 2026-10-25 03:03:35 +00:00
Adrien Beraud 9ffccb3956 pht: test prefix padding compatibility 2026-10-25 03:03:35 +00:00
Cyrille BéraudandCopilot d99324ebd4 pht: fix the bit arithmetic of Prefix
Prefix::size_ counts bits, but commonBits walked content_ a byte at a
time up to it. Two prefixes that agree on their whole common length are
therefore read well past their end, which AddressSanitizer reports as a
heap-buffer-overflow, and the count that comes back is eight times too
large. A lookup with a common prefix length hits this as soon as an
index entry matches the key it is looking for, and entry.prefix comes
straight off the network. The flags were indexed the same way, so a
partially known prefix was read at the wrong position too.

swapBit had the other half of the confusion: it flipped bit
(8 - bit) % 8 of the byte, while isActiveBit reads bit 7 - (bit % 8).
getSibling, which flips the last bit of a prefix to name the sibling
node in the trie, therefore returned a prefix differing in some other
bit and sharing the last one, so lookup and insertion looked up the
wrong node. Splitting a leaf and finding a canary both rely on it.

Add tests covering both, which fail before the fix: commonBits returns
193 for two identical 160 bit prefixes, and a prefix and its sibling
share 7 bits rather than all but the last.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-10-25 03:03:35 +00:00
copilot-swe-agent[bot]andaberaud 933f6cbca1 dhtcluster: import netifaces for interface selection
Co-authored-by: aberaud <101827+aberaud@users.noreply.github.com>
2026-10-25 02:45:22 +00:00
copilot-swe-agent[bot]andaberaud d5d7d6483f test: cover dhtcluster interface address selection
Co-authored-by: aberaud <101827+aberaud@users.noreply.github.com>
2026-10-25 02:45:22 +00:00
dependabot[bot] cc09a39ade build(deps): bump pypa/cibuildwheel from 4.1.1 to 4.3.1
Bumps [pypa/cibuildwheel](https://github.com/pypa/cibuildwheel) from 4.1.1 to 4.3.1.
- [Release notes](https://github.com/pypa/cibuildwheel/releases)
- [Changelog](https://github.com/pypa/cibuildwheel/blob/main/docs/changelog.md)
- [Commits](https://github.com/pypa/cibuildwheel/compare/v4.1.1...v4.3.1)

---
updated-dependencies:
- dependency-name: pypa/cibuildwheel
  dependency-version: 4.3.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-10-25 02:44:52 +00:00
Adrien Beraud 621b44c8e8 Revert "disable windows wheel for now"
This reverts commit fc6edb4d20.
2026-09-23 14:42:57 -04:00
Adrien Beraud e1c088f2e4 build: bump version to 4.4.1 2026-09-23 14:38:36 -04:00
Adrien Beraud a6e0e441ac crypto: use standard unsigned int* for gnutls_ocsp_resp_get_single 2026-09-22 11:31:45 -04:00
Adrien Beraud dcc2ba3492 crypto: use gnutls_ocsp_cert_status_t* in gnutls_ocsp_resp_get_single 2026-09-22 11:31:45 -04:00
Adrien Beraud ccf00d6b09 ports: update shiftmedia-libgnutls to f76d780a6 2026-09-22 11:31:45 -04:00
Adrien Beraud d8250574fc ports: use ForcedIncludeFiles for gnutls_msvc_fixes.h in shiftmedia-libgnutls 2026-09-22 11:31:45 -04:00
Adrien Beraud 0d698c76e8 ports: update shiftmedia-libgnutls to 9ebeb826d 2026-09-22 11:31:45 -04:00
Adrien Beraud e4e3b4e73b ports: set LanguageStandard_C to stdc11 in shiftmedia-libgnutls 2026-09-22 11:31:45 -04:00
Adrien Beraud b26d622f5d ports: update shiftmedia-libgnutls to 37840cfa5 2026-09-22 11:31:45 -04:00
Adrien Beraud 78694751bd ports: update shiftmedia-libgnutls to 4ab78d57e 2026-09-22 11:31:45 -04:00
Adrien Beraud f9b90c5c1c ports: update shiftmedia-libgnutls to 5a5a9dd95 2026-09-22 11:31:45 -04:00
Adrien Beraud 56b848bf47 ports: fix savoirfairelinux/gnutls commit hash and SHA512 in shiftmedia-libgnutls 2026-09-22 11:31:45 -04:00
Adrien Beraud dd875230c3 ports: update shiftmedia-libgnutls to 90eb686e8 2026-09-22 11:31:45 -04:00
Adrien Beraud de1d428e63 ports: use gitlab gnulib mirror archive in shiftmedia-libgnutls 2026-09-22 11:31:45 -04:00
Adrien Beraud 55e3a6514f ports: fix gnulib snapshot SHA512 hash in shiftmedia-libgnutls 2026-09-22 11:31:45 -04:00
Adrien Beraud f4bbef210c ports: fix savoirfairelinux/gnutls archive SHA512 hash in shiftmedia-libgnutls 2026-09-22 11:31:45 -04:00
Adrien Beraud 2c1470cef6 vcpkg: add overlay port for savoirfairelinux/gnutls on Windows MSVC 2026-09-22 11:31:45 -04:00
Adrien Beraud adb6aa57e6 update vcpkg 2026-09-22 11:31:45 -04:00
Adrien Beraud acd0abdb70 ci: fix vcpkg cache keys across workflows and remove manual version string 2026-09-22 11:31:45 -04:00
Cyrille BéraudandCopilot b9c8f5e0fd runner: complete pushes received while stopped
Resolve idle-runner push futures immediately under the operation queue lock. A stopped runner has no loop left to execute the queued promise, otherwise consumers can wait indefinitely.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-09-21 16:02:47 -04:00
Cyrille BéraudandCopilot 18fedf431f tests: cover pushes received by stopped runners
Require immediate completion before run and after join, including a burst of notifications. Keep active-runner coverage.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-09-21 16:02:47 -04:00
Adrien Beraud fc6edb4d20 disable windows wheel for now 2026-09-19 22:09:44 -04:00
Adrien Beraud d1093c77a3 ci: use libmsgpack-cxx-dev on Debian 13 2026-09-19 21:56:06 -04:00
Adrien Beraud b543f1e3cb update vcpkg to 2026.06.01 2026-09-19 21:10:44 -04:00
Adrien Beraud be210e27da update version to 4.4.0 2026-09-04 16:46:41 -04:00
Adrien Beraud f7243fca37 value_cache: only emit an edition when the content actually changed
ValueCache re-emitted a known value as an edition on a higher seq alone,
while OpValueCache requires the content to differ as well. Since
Value::operator== ignores seq, identical content re-announced with a
bumped seq was re-emitted as an add: a source-counting OpValueCache
then counted the same node twice and kept the value after the node
expired or left the search. Apply the same rule at both levels.
2026-09-04 15:44:35 -04:00
Adrien Beraud 1f6efd5736 tests: cover same-content higher-seq re-announce in ValueCache
Value::operator== ignores seq, so a node re-announcing identical content
with a bumped seq is the same value refreshed, not an edition. ValueCache
must not re-emit it: a source-counting OpValueCache above would count the
node twice and keep the value after the node is gone.
2026-09-04 15:44:35 -04:00
Adrien Beraud 54fbad3a7e op_cache: ignore expiry of versions that were never counted
onValuesAdded ignores a different payload that does not carry a higher
seq (e.g. the same signed content re-signed with a randomized
signature), but onValuesExpired only skipped strictly older versions.
The uncounted source could therefore decrement the refcount and expire
a value still held by others. Apply the same rule on both sides.
2026-09-04 15:18:35 -04:00
Adrien Beraud 1315489a7b tests: cover expiry of an uncounted same-seq version in OpValueCache
A source reporting a different payload with the same seq is ignored on
add and not counted; its later expiration must not release the value
still held by the counted sources.
2026-09-04 15:18:35 -04:00
Adrien Beraud c189fa0e82 proxy_server: detach pending push requests before destruction
~DhtProxyServer destroyed requests_ whose ~http::Request notifies DONE;
the handler registered by sendPushNotification then erased from the map
being destroyed, corrupting the heap (double free under ASan) whenever a
push request was still pending, e.g. an unreachable push server. Detach
the state-change callbacks and release the requests before the members
are destroyed.
2026-09-04 15:18:24 -04:00
Adrien Beraud 3eae03289e storage: re-storing identical content is a refresh, not an edition
Storage::store reported edited_values=1 whenever the stored Sp<Value>
differed from the incoming one, even with equal content. A local put()
of an already-stored value was therefore announced to local listeners
as a new value a second time, unbalancing listen caches that count how
many sources hold a value. Only report an edition when the content
actually changed; the expiration is still refreshed.
2026-09-04 15:18:24 -04:00
Adrien Béraud e5e5a45b91 op_cache, proxy_client: guard SearchCache::get and push refresh expiry
SearchCache::get called the GetCallback unconditionally; Dht::query
passes none, which threw bad_function_call as soon as a listen cache
existed for the key. Return false without a callback so the network
path runs. A callback returning false means the consumer has enough
values, not a failure: report done(true) as the network path does.

DhtProxyClient push refresh expired every previously cached value once
the refresh get completed, including values that were just re-delivered
or edited, producing phantom expire callbacks. Track the ids received by
the get and only expire cached values missing from it, and skip the
reconciliation when the get failed so a transient error does not expire
values still present on the DHT.
2026-09-04 15:18:24 -04:00
Adrien Beraud 7a71beb892 value_cache: detect editions by increasing seq, like OpValueCache
ValueCache emitted any content change of a known value as an edition
while OpValueCache only accepts a higher seq. A node reporting an older
version made ValueCache replace its copy and re-emit versions that were
then dropped or, with source counting, double counted downstream. Use
the same rule at both levels.
2026-09-04 15:18:24 -04:00
Adrien Béraud 74a894229f value_cache, op_cache: fix value accounting after edition and re-delivery
OpValueCache counted every re-delivery of an identical value as a new
source, but the proxy client is a single logical source that re-sends
values on push refresh or listen restart: the refcount grew unbounded
and expirations were never propagated. Add a countSources flag to
OpValueCache, OpCache and SearchCache; the proxy client disables it.

When a value is edited (same id, higher seq), only the reporting source
is known to hold the new version: replace it in place, reset the
refcount to 1 and emit a single add. Ignore older versions from lagging
sources on add, and ignore their expiration so they cannot release a
newer version still held by others.

ValueCache emits an edited value as an add instead of silently
refreshing its old copy, and applies a grace period of one type
expiration while the listen is synced so that a single lost refresh
notification does not drop a value the remote node still holds.

SearchCache::get only answers from the cache once the listen is synced,
so a get never returns a partial view that the network would complete.

Add the single-source OpCacheTester cases enabled by countSources.
2026-09-04 15:18:13 -04:00
Adrien Beraud c81c1f70c8 tests: add listen cache tests for OpValueCache, ValueCache, DhtRunner and proxy
OpCacheTester and ValueCacheTester exercise the listen caches in
isolation: add/edit/expire flows, per-source refcounting, stale
versions, sync status, SearchCache::get and node drop.

DhtRunnerTester and DhtProxyTester gain end-to-end listen tests for
value edition, duplicate puts, multi-source and partial expiration,
get after a synced listen, and the push refresh/expire paths.

The proxy fixture joins the proxy DhtRunner before destroying the
DhtProxyServer so no DHT callback can reach a destroyed server, and
push notifications are never awaited while holding the mutex taken by
listen callbacks: the result is produced on the DHT thread.

Register the new executables in CMake and Meson.
2026-09-04 15:17:34 -04:00
Adrien BeraudandCopilot 3a383ac873 http: close connections on the io_context thread
asio sockets are not thread-safe. http::Connection::close() used to close
the socket synchronously on the calling thread. Request::cancel() and
Request::~Request() (via terminate()) are called from arbitrary threads
(e.g. DhtProxyClient::stop(), cancelAllListeners(), cancelListen()),
so the close raced with in-flight operations on the io_context thread.

In particular, asio::async_connect's range_connect_op closes and
re-opens the socket when trying the next endpoint (common on mobile
where the IPv6 endpoint fails first). A concurrent close() from another
thread corrupts the epoll reactor's per-descriptor state, leading to
crashes in epoll_reactor::deregister_descriptor / register_descriptor
called from range_connect_op::process.

Dispatch the close to the io_context thread when not already on it.
Also keep the connection alive during the TLS handshake, like every
other asynchronous operation, so that the destructor can never run
while an operation is pending.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-09-04 14:13:20 -04:00
Adrien Beraud 3149fe60c4 proxy: increase OP_MARGIN to 8 hours 2026-09-04 11:04:42 -04:00
Adrien Beraud d5956e0778 proxy: use system_clock for subscriber expiration timeouts
Track subscribe/permanent-put expiration with std::chrono::system_clock
(wall clock) instead of steady_clock in both the proxy client and server,
so persisted expirations survive restarts and map directly to epoch time.

- DhtProxyServer::clock is now system_clock; expireTimer/expireNotifyTimer
  use asio::system_timer.
- DhtProxyClient::refreshSubscriberTimer uses asio::system_timer and
  system_clock::now() for subscriber refresh.
2026-09-04 11:04:42 -04:00
Adrien Beraud 8baf53d078 utils: fix clock conversion overflow with mismatched durations
to_system_time computed its saturation bounds by subtracting a
steady_clock duration from system_clock::time_point::max(). When the
two clocks have different periods (microseconds for system_clock and
nanoseconds for steady_clock on libc++), the common type promotes the
bound to nanoseconds and overflows, so every future time point was
converted to system_clock::time_point::max().

Cast the delta to the target duration before comparing against the
bounds, and saturate from_system_time before the reverse cast for
symmetry.
2026-09-01 22:56:16 -04:00
Adrien Beraud 82ab21dc3f crypto: support explicit EC curve generation 2026-09-01 21:34:36 -04:00
Adrien Beraud d676aef027 http: support legacy X509 lookup signature 2026-08-06 14:05:48 -04:00
Adrien Beraud a2d9fdf334 Recompute runner wakeup after import schedules value expirations. 2026-08-06 13:59:48 -04:00
Adrien Beraud 0cec4135c0 update to 4.3.0 2026-08-05 16:35:01 -04:00
Adrien Beraud 6f76686a3e ci: update to Debian 13 2026-08-05 16:19:46 -04:00
Adrien Beraud c211b3c376 dht: persist values with system time 2026-08-05 16:14:15 -04:00
Adrien Beraud 0179b6ecc7 test: cover system time state persistence 2026-08-05 16:14:15 -04:00
Adrien Beraud 44e1ad839e net: harden fragmented reply completion 2026-08-05 14:29:03 -04:00
Adrien Beraud dddd97daff tests: cover partial reply hardening 2026-08-05 14:29:03 -04:00
Adrien Beraud 632bec459e net: avoid partial key allocation for regular messages 2026-08-05 14:29:03 -04:00
Cyrille BéraudandCopilot 910ca10a5f net: reject malformed error and request fields
An error array was read at index 0 without checking its size, so a packet
carrying "e":[] dereferenced a null element pointer and crashed the process
on unauthenticated UDP input.

When a message had an error array but no "a", "r" or "u" map, the same object
was then walked through the map view of the msgpack union. A two-element array
appeared to hold two key/value pairs, reading past its allocation. A request
field that was not a map was never type-checked either.

Validate the error array size, restrict request parsing to "a", "r" and "u",
and require those to be maps. Add parser regressions for the empty error
array, the error-only message and the non-map request; the first one
segfaults without this change.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-08-05 14:29:03 -04:00
Cyrille BéraudandCopilot 0826822a4d net: harden fragmented message reassembly
Key partial messages by source and transaction ID, buffer bounded fragments received before their header, and avoid charging reordered requests twice against rate limits. Cover source collisions, early fragments, empty data and large values.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-08-05 14:29:03 -04:00
Cyrille BéraudandCopilot 8aa6b009c3 net: reject empty field selections
Reject field-compressed responses with no selected fields before computing the value count, avoiding a division by zero on malformed network input. Add a parser regression test.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-08-05 14:29:03 -04:00
Cyrille BéraudandCopilot 2caa21d65f dht: let a node that lost the network try to rejoin it
Dht::onDisconnected() only starts a bootstrap when bootstrapJob is
unset, reading a set handle as "a bootstrap is already scheduled".
That reading is wrong: Scheduler::run() erases the job from its timers
and calls it, but never clears the shared handle the caller holds, and
Scheduler::cancel() leaves the handle alone too when the job is no
longer in the timers. The handle therefore stays set for as long as the
Dht lives, once a bootstrap job has run.

That is harmless while bootstrap() reschedules itself, which it does as
long as the node is disconnected. It is not harmless on the early
return: a job that fires while a ping is still in flight, or while a
dubious contact keeps the status at Connecting, returns without
rescheduling. From then on the chain is dead and onDisconnected() will
not restart it, so a node that goes on to lose every contact stays
disconnected for good. Only an explicit addBootstrap() or a
connectivityChanged() can revive it, and a node whose network simply
went away gets neither.

Clearing the handle on the early return restores the meaning
onDisconnected() relies on.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-08-03 11:34:25 -04:00
Adrien Beraud cf64fa454c crypto: encode certificate serials in network order 2026-08-02 17:24:55 -04:00
Adrien Beraud ef9ef12195 tests: cover positive certificate serials 2026-08-02 17:24:55 -04:00
Cyrille BéraudandCopilot 74a1918a4d crypto: strip the terminating null byte from an exported public key
gnutls_pubkey_export reports the size of the exported PEM including its
terminating null byte, and PublicKey::toString hands that size straight
back to the caller. The returned string therefore ends with a null byte
that is not part of the PEM text.

Certificate::toString does not have the problem because
gnutls_x509_crt_export excludes the terminator, so the two exports
disagree on what a PEM string is.

The stray byte travels: Value::toJson stores owner->toString() in the
"owner" field, so a value published through the proxy carries a null
byte inside a JSON string, which is valid JSON but a surprise for any
consumer that treats the field as text.

Resize to the reported size and drop the terminator when it is there.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-08-02 17:20:04 -04:00
dependabot[bot] 56a60fb557 build(deps): bump pypa/cibuildwheel from 4.1.0 to 4.1.1
Bumps [pypa/cibuildwheel](https://github.com/pypa/cibuildwheel) from 4.1.0 to 4.1.1.
- [Release notes](https://github.com/pypa/cibuildwheel/releases)
- [Changelog](https://github.com/pypa/cibuildwheel/blob/main/docs/changelog.md)
- [Commits](https://github.com/pypa/cibuildwheel/compare/v4.1.0...v4.1.1)

---
updated-dependencies:
- dependency-name: pypa/cibuildwheel
  dependency-version: 4.1.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-07-30 10:17:04 -04:00
Adrien Beraud f98df1d615 crypto: allow signing a revocation list with an explicit CRL number
RevocationList::sign() always overwrote the CRL Number extension: random
on the first signature, incremented afterwards. This leaves no way for a
certificate authority replicated over several devices to keep numbers
unique, since every device branching from the same list mints the same
number for a different content.

Add an optional `number` parameter, used verbatim when non-zero, so
callers can partition the number space (RFC 5280 5.2.3 requires
monotonicity for a given CRL scope and issuer, not globally).

Also fix the default increment path: the extension holds a DER integer
with leading zeros stripped, so reading it straight into an uint64_t only
worked for numbers wide enough to occupy all 8 bytes. Signing a list
numbered 5 produced 0x0500000000000001 instead of 6. Read and write the
number as a big-endian byte string instead, which makes the `endian()`
helper unnecessary.

Add tests covering the default numbering, explicit numbers (including
values with leading zero bytes and the largest positive 8-byte integer),
pack/unpack round-trips, and a device-partitioned numbering scheme.
2026-07-29 13:26:23 -04:00
Adrien Beraud 8f779f2efb http: fix build on OpenSSL 4.0 2026-07-20 11:25:02 -04:00
Adrien Beraud 699073acd8 update version to 4.2.0 2026-07-17 14:09:54 -04:00
Cyrille 02d49b9f6a http: run completed resolver callbacks on io executor 2026-07-15 14:07:14 -04:00
Adrien Beraud 85b0bc52c8 test: reproduce completed resolver callback race 2026-07-15 14:07:14 -04:00
Adrien Beraud af02c3a117 ci: install brew packages for macOS 15 2026-07-15 12:16:28 -04:00
François-Simon Fauteux-Chapleau 74b99453ce op_cache: fix boundary condition
In SearchCache::expire, if there is an op whose expiration is equal to
`now`, then it's not processed and the function returns `now` as the
next expiration timepoint. This can lead to the opExpirationJob in
Search::cancelListen getting rescheduled at the same time in an infinite
loop.
2026-07-13 14:38:27 -04:00
Adrien Beraud cba3d0e18f update version to 4.1.1 2026-07-06 15:50:00 -04:00
Adrien Beraud f9332985f5 thread_pool: only include ciso646 on C++<20 2026-07-06 15:45:01 -04:00
Cyrille BéraudandCopilot 513d0eb803 dhtrunner: fall back to random port when saved port is taken
Since 15eb221c the node's bound port is persisted alongside the DHT
state and reused on the next run for a stable identity. If that port
is still held by another process - typically a fast application
restart racing the previous instance's shutdown - UdpSocket
constructor throws (both the IPv4 and IPv6 binds fail) and
DhtRunner::run() propagates the exception: the node never starts.

In Jami this left every account stuck in the connecting state after a
quick restart, with 'DhtException occurred: Can't bind socket' logged
per account, and no retry.

The persisted port is only a best-effort hint: when binding it fails,
retry with OS-assigned ports instead of throwing. The persisted file
is then refreshed with the newly bound ports as before.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-07-06 15:44:47 -04:00
Cyrille BéraudandCopilot 139852a531 tests: dhtrunner: cover saved-port fallback
Cover starting a node whose persisted port (persist_path + "_port.txt")
is already bound by another node, as happens when an application is
restarted while the previous instance still holds the socket. The node
is expected to start on a fallback port instead of failing.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-07-06 15:44:47 -04:00
Cyrille BéraudandCopilot 3a73ad3aca value: serialize pushType in JSON
The pushType field (593e59e0) lets the DHT proxy server tag push
notifications with the connection type ("pt" in the FCM payload), so
mobile clients can tell incoming calls apart from background noise.

It was only serialized in msgpack. Clients using DhtProxyClient
send their puts as JSON (Value::toJson()), where the field was
silently dropped, and the proxy server parsed the incoming JSON with
Value(const Json::Value&), which never read it back. As a result any
value put through a proxy (e.g. a call request from a mobile device)
reached the recipient's device with an empty "pt", was classified as
noise, and did not wake up the callee: calls from proxied devices
never rang, while the same call from a full DHT node (desktop) did.

Serialize pushType as "pt" in Value::toJson() when set, and parse it
back in the JSON constructor, mirroring the msgpack behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-07-04 11:57:49 -04:00
Cyrille BéraudandCopilot 9031cf06c4 tests: value: add pushType JSON round-trip test
Cover Value::toJson() / Value(const Json::Value&) preservation of the
pushType field, and its absence from the JSON when empty. This is the
serialization path used by DhtProxyClient::put(), so mobile clients
behind a DHT proxy depend on it to propagate the push notification
type.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-07-04 11:57:49 -04:00
dependabot[bot] 7b1369931e build(deps): bump actions/cache from 5 to 6
Bumps [actions/cache](https://github.com/actions/cache) from 5 to 6.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/v5...v6)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-29 19:15:42 -04:00
dependabot[bot] 64364602d7 build(deps): bump pypa/cibuildwheel from 3.4.1 to 4.1.0
Bumps [pypa/cibuildwheel](https://github.com/pypa/cibuildwheel) from 3.4.1 to 4.1.0.
- [Release notes](https://github.com/pypa/cibuildwheel/releases)
- [Changelog](https://github.com/pypa/cibuildwheel/blob/main/docs/changelog.md)
- [Commits](https://github.com/pypa/cibuildwheel/compare/v3.4.1...v4.1.0)

---
updated-dependencies:
- dependency-name: pypa/cibuildwheel
  dependency-version: 4.1.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-22 10:56:34 -04:00
dependabot[bot] 2dbec3df97 build(deps): bump actions/checkout from 6 to 7
Bumps [actions/checkout](https://github.com/actions/checkout) from 6 to 7.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v6...v7)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-22 10:53:41 -04:00
Adrien Beraud 507e2de3c4 update version to 4.1.0 2026-06-19 00:27:25 -04:00
dependabot[bot] c37abed715 build(deps): bump https://github.com/microsoft/vcpkg
Bumps [https://github.com/microsoft/vcpkg](https://github.com/microsoft/vcpkg) from HEAD to 2026.06.01. This release includes the previously tagged commit.
- [Release notes](https://github.com/microsoft/vcpkg/releases)
- [Commits](https://github.com/microsoft/vcpkg/compare/66c0373dc7fca549e5803087b9487edfe3aca0a1...f3e10653cc27d62a37a3763cd84b38bca07c6075)

---
updated-dependencies:
- dependency-name: https://github.com/microsoft/vcpkg
  dependency-version: 2026.06.01
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-19 00:12:50 -04:00
Adrien Beraud 0eb1c33ba5 ci: update vcpkg 2026-06-19 00:10:28 -04:00
Adrien Beraud 45918186a2 ci: update vcpkg 2026-06-19 00:08:20 -04:00
Adrien Beraud 29cb909228 build/meson: base64 requires simdutf 2026-06-18 23:52:24 -04:00
Adrien Beraud 9f6a03bf3d build/meson: require appropriate dependencies for proxy server 2026-06-18 20:34:45 -04:00
Adrien Beraud 732da0e478 default types: add ValueId to TrustRequest 2026-06-11 16:52:03 -04:00
Adrien Béraud 9a60329f66 proxy server: use token as request url 2026-06-03 20:00:34 -04:00
Adrien Béraud 0bb393ec36 http: allow to build Request by std::string url 2026-06-03 19:30:53 -04:00
vx32 947610003b http: allow to build Url from std::string, cleanup 2026-06-03 19:14:32 -04:00
Adrien Béraud dcbe158554 proxy server: allow url-encoded base64 for UnifiedPush 2026-06-03 19:12:40 -04:00
vx32 38e9a7fd61 base64: add support for url-encoded base64 2026-06-03 17:01:19 -04:00
148 changed files with 24777 additions and 2073 deletions

No files matched your search

+12
View File
@@ -0,0 +1,12 @@
.git
.vscode
.DS_Store
build
build-*
build_*
install
vcpkg_installed
rust/target
**/__pycache__
python/dist
python/*.egg-info
+22 -10
View File
@@ -2,12 +2,12 @@ name: C/C++ CI
env:
LLHTTP_VERSION: "v9.2.1"
RESTINIO_VERSION: "0.7.8.1"
RESTINIO_VERSION: "0.7.10"
EXPECTED_LITE_URL: "https://raw.githubusercontent.com/martinmoene/expected-lite/master/include/nonstd/expected.hpp"
UBUNTU_FULL_DEPS: "libncurses5-dev libreadline-dev nettle-dev libasio-dev libgnutls28-dev libuv1-dev python3-pip python3-dev python3-setuptools libcppunit-dev libjsoncpp-dev libfmt-dev libmsgpack-dev libargon2-dev"
UBUNTU_MESON_DEPS: "ninja-build libncurses5-dev libreadline-dev nettle-dev libasio-dev libgnutls28-dev libuv1-dev python3-dev python3-setuptools python3-pip libcppunit-dev libjsoncpp-dev libfmt-dev libmsgpack-dev libargon2-dev"
UBUNTU_MINIMAL_DEPS: "libncurses5-dev libreadline-dev nettle-dev libfmt-dev libgnutls28-dev libcppunit-dev libmsgpack-dev libargon2-dev"
DEBIAN12_DEPS: "git libtool pkg-config wget cmake libncurses5-dev libreadline-dev nettle-dev libasio-dev libgnutls28-dev libuv1-dev python3-pip python3-dev python3-setuptools libcppunit-dev libjsoncpp-dev libfmt-dev libmsgpack-dev libargon2-dev"
DEBIAN13_DEPS: "git libtool pkg-config wget cmake libncurses5-dev libreadline-dev nettle-dev libasio-dev libgnutls28-dev libuv1-dev python3-pip python3-dev python3-setuptools libcppunit-dev libjsoncpp-dev libfmt-dev libmsgpack-cxx-dev libargon2-dev"
on: [push, pull_request]
@@ -16,7 +16,7 @@ jobs:
name: Ubuntu/GCC Meson build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
- name: Install Dependencies
run: |
@@ -33,6 +33,18 @@ jobs:
- name: Build with Ninja
run: cd build && ninja
- uses: dtolnay/rust-toolchain@stable
- name: Test Rust bindings
env:
LIBRARY_PATH: ${{ github.workspace }}/build
LD_LIBRARY_PATH: ${{ github.workspace }}/build
RUSTFLAGS: -L native=${{ github.workspace }}/build
run: |
cargo test --manifest-path rust/Cargo.toml --all-targets
cargo test --manifest-path rust/Cargo.toml --all-features --all-targets
cargo test --manifest-path rust/Cargo.toml --all-features --doc
build-ubuntu-minimal:
name: Ubuntu/GCC minimal build
runs-on: ubuntu-latest
@@ -40,7 +52,7 @@ jobs:
matrix:
cxx_standard: [17, 20, 23]
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
- name: Install Minimal Dependencies
run: |
@@ -65,12 +77,12 @@ jobs:
- name: Run Unit tests
run: ctest --output-on-failure --test-dir build
build-debian12:
name: Debian 12/GCC build
build-debian13:
name: Debian 13/GCC build
runs-on: ubuntu-latest
container:
image: debian:12
image: debian:13
volumes:
- .:/workspace
options: --workdir /workspace
@@ -78,12 +90,12 @@ jobs:
DEBIAN_FRONTEND: noninteractive
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
- name: Install Dependencies
run: |
apt-get update
apt-get install -y $DEBIAN12_DEPS
apt-get install -y $DEBIAN13_DEPS
- name: Build llhttp dependency
run: |
@@ -131,7 +143,7 @@ jobs:
matrix:
cxx_standard: [17, 20, 23]
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
- name: Install Homebrew Dependencies
run: brew install msgpack-cxx asio gnutls nettle readline fmt jsoncpp argon2 openssl cppunit
+6 -4
View File
@@ -6,7 +6,7 @@ jobs:
name: Clang Static Analysis
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
- name: deps
run: |
sudo apt update && \
@@ -23,11 +23,13 @@ jobs:
-O /usr/include/nonstd/expected.hpp
- name: restinio
env:
RESTINIO_VERSION: "0.7.10"
run: |
mkdir restinio && cd restinio
wget https://github.com/Stiffstream/restinio/releases/download/v0.7.8.1/restinio-0.7.8.1.tar.bz2
ls -l && tar -xjf restinio-0.7.8.1.tar.bz2
cd restinio-0.7.8.1/dev
wget "https://github.com/Stiffstream/restinio/releases/download/v${RESTINIO_VERSION}/restinio-${RESTINIO_VERSION}.tar.bz2"
ls -l && tar -xjf "restinio-${RESTINIO_VERSION}.tar.bz2"
cd "restinio-${RESTINIO_VERSION}/dev"
cmake -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=Off -DRESTINIO_SAMPLE=Off -DRESTINIO_BENCHMARK=Off \
-DRESTINIO_WITH_SOBJECTIZER=Off -DRESTINIO_DEP_STANDALONE_ASIO=system -DRESTINIO_DEP_LLHTTP=system \
-DRESTINIO_DEP_FMT=system -DRESTINIO_DEP_EXPECTED_LITE=system .
+1 -1
View File
@@ -27,7 +27,7 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v7
- name: Initialize CodeQL
uses: github/codeql-action/init@v4
+206
View File
@@ -0,0 +1,206 @@
name: Node.js Package
on:
push:
branches: ['**']
paths:
- 'nodejs/**'
- '.github/workflows/nodejs-package.yml'
- 'src/**'
- 'include/**'
- 'cmake/**'
- 'CMakeLists.txt'
- 'vcpkg*.json'
- 'ports/**'
- 'LICENSE'
pull_request:
paths:
- 'nodejs/**'
- '.github/workflows/nodejs-package.yml'
- 'src/**'
- 'include/**'
- 'cmake/**'
- 'CMakeLists.txt'
- 'vcpkg*.json'
- 'ports/**'
- 'LICENSE'
workflow_call:
workflow_dispatch:
permissions:
contents: read
packages: read
jobs:
build:
name: Build ${{ matrix.target }}
runs-on: ${{ matrix.os }}
container:
image: ${{ matrix.container }}
credentials:
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
timeout-minutes: 45
defaults:
run:
working-directory: nodejs
strategy:
fail-fast: false
matrix:
include:
- os: ubuntu-26.04
target: linux-x64
container: ghcr.io/${{ github.repository }}/opendht-deps-noble:latest
- os: macos-15
target: darwin-arm64
- os: macos-15-intel
target: darwin-x64
- os: windows-2022
target: win32-x64
env:
VCPKG_DISABLE_METRICS: '1'
VCPKG_DEFAULT_TRIPLET: x64-windows-release
VCPKG_INSTALLED_DIR: ${{ github.workspace }}/vcpkg_installed
VCPKG_BINARY_SOURCES: clear;files,${{ github.workspace }}/vcpkg-binary-cache,readwrite
steps:
- uses: actions/checkout@v7
- uses: actions/setup-node@v6
with:
node-version: '22'
cache: npm
cache-dependency-path: nodejs/package-lock.json
- name: Install macOS dependencies
if: runner.os == 'macOS'
run: brew install cmake ninja pkg-config dylibbundler gnutls nettle argon2 msgpack-cxx fmt jsoncpp simdutf asio llhttp openssl
- name: Install restinio
if: runner.os == 'macOS'
env:
RESTINIO_VERSION: '0.7.10'
run: |
include="$RUNNER_TEMP/opendht-proxy/include"
mkdir -p "$include/nonstd"
curl -fsSL --retry 3 https://raw.githubusercontent.com/martinmoene/expected-lite/v0.10.0/include/nonstd/expected.hpp -o "$include/nonstd/expected.hpp"
curl -fsSL --retry 3 "https://github.com/Stiffstream/restinio/releases/download/v${RESTINIO_VERSION}/restinio-${RESTINIO_VERSION}.tar.bz2" -o "$RUNNER_TEMP/restinio.tar.bz2"
tar -xjf "$RUNNER_TEMP/restinio.tar.bz2" -C "$RUNNER_TEMP"
cp -R "$RUNNER_TEMP/restinio-${RESTINIO_VERSION}/dev/restinio" "$include/"
echo "CMAKE_PREFIX_PATH=$RUNNER_TEMP/opendht-proxy" >> "$GITHUB_ENV"
- name: Set up MSVC
if: runner.os == 'Windows'
uses: ilammy/msvc-dev-cmd@v1
with:
arch: x64
- name: Set up vcpkg
if: runner.os == 'Windows'
uses: lukka/run-vcpkg@v11
with:
vcpkgJsonGlob: vcpkg.json
vcpkgConfigurationJsonGlob: vcpkg-configuration.json
- name: Cache Windows native dependencies
if: runner.os == 'Windows'
uses: actions/cache@v6
with:
path: ${{ github.workspace }}/vcpkg-binary-cache
key: node-vcpkg-${{ env.VCPKG_DEFAULT_TRIPLET }}-${{ hashFiles('vcpkg.json', 'vcpkg-configuration.json', 'ports/**') }}-${{ github.run_id }}-${{ github.run_attempt }}
restore-keys: |
node-vcpkg-${{ env.VCPKG_DEFAULT_TRIPLET }}-${{ hashFiles('vcpkg.json', 'vcpkg-configuration.json', 'ports/**') }}-
- run: npm ci --no-audit --no-fund
- name: Build Unix addon
if: runner.os != 'Windows'
shell: bash
run: |
if [[ "$RUNNER_OS" == 'macOS' ]]; then
export MACOSX_DEPLOYMENT_TARGET=15.0
openssl_prefix=$(brew --prefix openssl)
export PKG_CONFIG_PATH="$openssl_prefix/lib/pkgconfig${PKG_CONFIG_PATH:+:$PKG_CONFIG_PATH}"
fi
npm run build -- --CDOPENDHT_DOWNLOAD_DEPS=OFF
- name: Build Windows addon
if: runner.os == 'Windows'
shell: pwsh
run: |
npm run build -- `
"--CDCMAKE_TOOLCHAIN_FILE=$env:VCPKG_ROOT/scripts/buildsystems/vcpkg.cmake" `
"--CDVCPKG_MANIFEST_DIR=$env:GITHUB_WORKSPACE" `
"--CDVCPKG_INSTALLED_DIR=$env:VCPKG_INSTALLED_DIR" `
"--CDVCPKG_TARGET_TRIPLET=$env:VCPKG_DEFAULT_TRIPLET" `
--CDVCPKG_MANIFEST_NO_DEFAULT_FEATURES=ON `
--CDVCPKG_MANIFEST_FEATURES=proxy `
--CDCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreadedDLL `
--CDOPENDHT_DOWNLOAD_DEPS=OFF
if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
"$env:VCPKG_INSTALLED_DIR/$env:VCPKG_DEFAULT_TRIPLET/bin" >> $env:GITHUB_PATH
"OPENDHT_LIBRARY_PATH=$env:VCPKG_INSTALLED_DIR/$env:VCPKG_DEFAULT_TRIPLET/bin" >> $env:GITHUB_ENV
- run: npm test
- run: npm run typecheck
- run: npm run bundle
- name: Include static simdutf licenses
if: runner.os == 'Linux'
run: |
mkdir -p prebuilds/linux-x64/licenses/simdutf
cp /usr/share/doc/simdutf/LICENSE-* prebuilds/linux-x64/licenses/simdutf/
- run: npm run test:package
- uses: actions/upload-artifact@v7
with:
name: opendht-node-prebuild-${{ matrix.target }}
path: nodejs/prebuilds/
if-no-files-found: error
package:
name: Assemble npm archive
needs: build
runs-on: ubuntu-26.04
steps:
- uses: actions/checkout@v7
- uses: actions/setup-node@v6
with:
node-version: '22'
- uses: actions/download-artifact@v8
with:
pattern: opendht-node-prebuild-*
path: nodejs/prebuilds
merge-multiple: true
- name: Set release version
if: github.event_name == 'push' && github.ref_type == 'tag'
working-directory: nodejs
env:
RELEASE_TAG: ${{ github.ref_name }}
run: |
release_version=$(npm version "${RELEASE_TAG#v}" --no-git-tag-version --ignore-scripts --allow-same-version)
if [[ "$release_version" != "$RELEASE_TAG" ]]; then
printf '::error::Release tag %s is not an exact npm semantic version (normalized to %s).\n' "$RELEASE_TAG" "$release_version" >&2
exit 1
fi
- name: Pack all platforms
working-directory: nodejs
run: |
npm ci --ignore-scripts
mkdir -p dist
npm pack --pack-destination dist
- uses: actions/upload-artifact@v7
with:
name: opendht-node-npm
path: nodejs/dist/*.tgz
if-no-files-found: error
smoke:
name: Install on ${{ matrix.os }} / Node ${{ matrix.node }}
needs: package
runs-on: ${{ matrix.os }}
timeout-minutes: 10
strategy:
fail-fast: false
matrix:
os: [ubuntu-24.04, ubuntu-26.04, macos-15, macos-15-intel, windows-2022]
node: ['22', '24', '26']
steps:
- uses: actions/checkout@v7
- uses: actions/setup-node@v6
with:
node-version: ${{ matrix.node }}
- uses: actions/download-artifact@v8
with:
name: opendht-node-npm
path: nodejs/dist
- name: Test installed archive without a native build toolchain
working-directory: nodejs
run: npm run test:package -- dist
+20 -44
View File
@@ -4,10 +4,10 @@ permissions:
contents: read
on:
release:
types: [published]
push:
branches: ['**']
pull_request:
workflow_call:
workflow_dispatch:
jobs:
@@ -16,14 +16,14 @@ jobs:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, ubuntu-24.04-arm, macos-latest]
os: [ubuntu-latest, ubuntu-24.04-arm, macos-15]
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
# Run locally with: pipx run cibuildwheel --platform linux python
- name: Build wheels
uses: pypa/cibuildwheel@v3.4.1
uses: pypa/cibuildwheel@v4.3.4
with:
package-dir: python
@@ -36,36 +36,28 @@ jobs:
name: Build wheels on Windows
runs-on: windows-latest
env:
VCPKG_VERSION: "2025.07.25"
VCPKG_INSTALLATION_ROOT: "C:\\vcpkg"
VCPKG_DISABLE_METRICS: "1"
VCPKG_DEFAULT_TRIPLET: x64-windows-release
VCPKG_INSTALLED_DIR: ${{ github.workspace }}\vcpkg_installed
VCPKG_DEFAULT_BINARY_CACHE: ${{ github.workspace }}\vcpkg_binary_cache
VCPKG_BINARY_SOURCES: clear;files,${{ github.workspace }}/vcpkg-binary-cache,readwrite
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7
- name: Cache vcpkg installed packages
id: cache-vcpkg
uses: actions/cache@v5
- name: Set up vcpkg
uses: lukka/run-vcpkg@v11
with:
path: |
${{ env.VCPKG_INSTALLED_DIR }}
${{ env.VCPKG_DEFAULT_BINARY_CACHE }}
key: ${{ runner.os }}-vcpkg-${{ env.VCPKG_VERSION }}-${{ env.VCPKG_DEFAULT_TRIPLET }}-${{ hashFiles('**/vcpkg.json', '**/vcpkg-configuration.json') }}-${{ hashFiles('.github/workflows/windows-build.yml') }}
vcpkgJsonGlob: vcpkg.json
vcpkgConfigurationJsonGlob: vcpkg-configuration.json
- name: Setup vcpkg and install dependencies
if: steps.cache-vcpkg.outputs.cache-hit != 'true'
run: |
New-Item -Path "${{ env.VCPKG_INSTALLED_DIR }}" -ItemType Directory -Force
New-Item -Path "${{ env.VCPKG_DEFAULT_BINARY_CACHE }}" -ItemType Directory -Force
vcpkg install `
--recurse `
--clean-after-build `
--triplet=${{ env.VCPKG_DEFAULT_TRIPLET }} `
--x-install-root=${{ env.VCPKG_INSTALLED_DIR }}
shell: pwsh
- name: Cache vcpkg binary packages
uses: actions/cache@v6
with:
path: ${{ github.workspace }}/vcpkg-binary-cache
key: ${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-${{ hashFiles('vcpkg.json', 'vcpkg-configuration.json', 'ports/**') }}-${{ github.run_id }}-${{ github.run_attempt }}
restore-keys: |
${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-${{ hashFiles('vcpkg.json', 'vcpkg-configuration.json', 'ports/**') }}-
${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-
- name: Build wheels
run: |
@@ -73,7 +65,7 @@ jobs:
cmake -S . -B build `
-DCMAKE_BUILD_TYPE=Release `
-DVCPKG_TARGET_TRIPLET=${{ env.VCPKG_DEFAULT_TRIPLET }} `
-DCMAKE_TOOLCHAIN_FILE=${{ env.VCPKG_INSTALLATION_ROOT }}\scripts\buildsystems\vcpkg.cmake `
"-DCMAKE_TOOLCHAIN_FILE=$env:VCPKG_ROOT/scripts/buildsystems/vcpkg.cmake" `
-DVCPKG_INSTALLED_DIR=${{ env.VCPKG_INSTALLED_DIR }} `
-DBUILD_TESTING=OFF `
-DOPENDHT_C=OFF `
@@ -90,19 +82,3 @@ jobs:
with:
name: opendht-wheels-win_amd64
path: ./build/wheelhouse/*.whl
publish:
runs-on: ubuntu-latest
needs: [build_wheels, build_wheels_windows]
steps:
- uses: actions/download-artifact@v8
with:
pattern: opendht-wheels-*
path: dist
merge-multiple: true
- name: Publish package
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags')
uses: pypa/gh-action-pypi-publish@release/v1
with:
password: ${{ secrets.PYPI_API_TOKEN }}
+223 -61
View File
@@ -8,7 +8,9 @@ on:
env:
REGISTRY: ghcr.io
IMAGE_NAME_DEPS: ${{ github.repository }}/opendht-deps
IMAGE_NAME_DEPS_NOBLE: ${{ github.repository }}/opendht-deps-noble
IMAGE_NAME: ${{ github.repository }}/opendht
IMAGE_NAME_NOBLE: ${{ github.repository }}/opendht-noble
IMAGE_NAME_DEPS_LLVM: ${{ github.repository }}/opendht-deps-llvm
IMAGE_NAME_LLVM: ${{ github.repository }}/opendht-llvm
IMAGE_NAME_DHTNODE: ${{ github.repository }}/dhtnode
@@ -16,6 +18,19 @@ env:
IMAGE_NAME_ALPINE_DEPS: ${{ github.repository }}/opendht-deps-alpine
jobs:
nodejs-package:
name: Build and verify Node.js package
uses: ./.github/workflows/nodejs-package.yml
permissions:
contents: read
packages: read
python-wheels:
name: Build Python wheels
uses: ./.github/workflows/python-wheels.yml
permissions:
contents: read
build-and-push-deps-image:
name: Dependency Docker image
runs-on: ubuntu-latest
@@ -25,7 +40,7 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v7
- name: Set up QEMU
uses: docker/setup-qemu-action@v4
@@ -56,6 +71,46 @@ jobs:
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
build-and-push-deps-image-noble:
name: Dependency Docker image (Ubuntu 24.04)
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Checkout repository
uses: actions/checkout@v7
- name: Set up QEMU
uses: docker/setup-qemu-action@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
- name: Log in to the Container registry
uses: docker/login-action@v4
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v6
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME_DEPS_NOBLE }}
- name: Build and push Docker image
uses: docker/build-push-action@v7
with:
context: .
file: docker/DockerfileDepsNoble
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
build-and-push-image:
name: OpenDHT Docker image
runs-on: ubuntu-latest
@@ -66,7 +121,7 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v7
- name: Set up QEMU
uses: docker/setup-qemu-action@v4
@@ -98,6 +153,47 @@ jobs:
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
build-and-push-image-noble:
name: OpenDHT Docker image (Ubuntu 24.04)
runs-on: ubuntu-latest
needs: build-and-push-deps-image-noble
permissions:
contents: read
packages: write
steps:
- name: Checkout repository
uses: actions/checkout@v7
- name: Set up QEMU
uses: docker/setup-qemu-action@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
- name: Log in to the Container registry
uses: docker/login-action@v4
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v6
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME_NOBLE }}
- name: Build and push Docker image
uses: docker/build-push-action@v7
with:
context: .
file: docker/DockerfileNoble
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
build-and-push-deps-image-llvm:
name: Dependency Docker image (LLVM)
runs-on: ubuntu-latest
@@ -107,7 +203,7 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v7
- name: Set up QEMU
uses: docker/setup-qemu-action@v4
@@ -148,7 +244,7 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v7
- name: Set up QEMU
uses: docker/setup-qemu-action@v4
@@ -190,7 +286,7 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v7
- name: Set up QEMU
uses: docker/setup-qemu-action@v4
@@ -231,7 +327,7 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v7
- name: Set up QEMU
uses: docker/setup-qemu-action@v4
@@ -273,7 +369,7 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v7
- name: Set up QEMU
uses: docker/setup-qemu-action@v4
@@ -309,20 +405,24 @@ jobs:
name: Build Windows MSI Installer
runs-on: windows-latest
env:
VCPKG_VERSION: "2025.07.25"
VCPKG_INSTALLATION_ROOT: "C:\\vcpkg"
VCPKG_DISABLE_METRICS: "1"
VCPKG_DEFAULT_TRIPLET: x64-windows
VCPKG_INSTALLED_DIR: ${{ github.workspace }}\vcpkg_installed
VCPKG_DEFAULT_BINARY_CACHE: ${{ github.workspace }}\vcpkg_binary_cache
VCPKG_BINARY_SOURCES: clear;files,${{ github.workspace }}/vcpkg-binary-cache,readwrite
permissions:
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v7
with:
submodules: recursive
- name: Set up vcpkg
uses: lukka/run-vcpkg@v11
with:
vcpkgJsonGlob: vcpkg.json
vcpkgConfigurationJsonGlob: vcpkg-configuration.json
- name: Set up MSVC developer environment
uses: microsoft/setup-msbuild@v3
@@ -332,27 +432,14 @@ jobs:
wix extension add --global WixToolset.UI.wixext/6.0.1
shell: pwsh
- name: Cache vcpkg installed packages
id: cache-vcpkg
uses: actions/cache@v5
- name: Cache vcpkg binary packages
uses: actions/cache@v6
with:
path: |
${{ env.VCPKG_INSTALLED_DIR }}
${{ env.VCPKG_DEFAULT_BINARY_CACHE }}
key: ${{ runner.os }}-vcpkg-${{ env.VCPKG_VERSION }}-x64-shared-${{ hashFiles('**/vcpkg.json', '**/vcpkg-configuration.json') }}-${{ hashFiles('.github/workflows/windows-build.yml') }}
- name: Setup vcpkg and install dependencies
if: steps.cache-vcpkg.outputs.cache-hit != 'true'
run: |
New-Item -Path "${{ env.VCPKG_INSTALLED_DIR }}" -ItemType Directory -Force
New-Item -Path "${{ env.VCPKG_DEFAULT_BINARY_CACHE }}" -ItemType Directory -Force
vcpkg install `
--recurse `
--clean-after-build `
--triplet=x64-windows `
--x-install-root=${{ env.VCPKG_INSTALLED_DIR }} `
--x-feature=test
shell: pwsh
path: ${{ github.workspace }}/vcpkg-binary-cache
key: ${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-${{ hashFiles('vcpkg.json', 'vcpkg-configuration.json', 'ports/**') }}-${{ github.run_id }}-${{ github.run_attempt }}
restore-keys: |
${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-${{ hashFiles('vcpkg.json', 'vcpkg-configuration.json', 'ports/**') }}-
${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-
- name: Configure CMake for MSI
shell: pwsh
@@ -360,7 +447,7 @@ jobs:
mkdir build -ErrorAction SilentlyContinue
cmake -B build `
-S . `
-DCMAKE_TOOLCHAIN_FILE=${{ env.VCPKG_INSTALLATION_ROOT }}/scripts/buildsystems/vcpkg.cmake `
"-DCMAKE_TOOLCHAIN_FILE=$env:VCPKG_ROOT/scripts/buildsystems/vcpkg.cmake" `
-DVCPKG_TARGET_TRIPLET=x64-windows `
-DVCPKG_INSTALLED_DIR=${{ env.VCPKG_INSTALLED_DIR }} `
-DVCPKG_MANIFEST_FEATURES=test `
@@ -413,47 +500,35 @@ jobs:
name: Build Windows dhtnode
runs-on: windows-latest
env:
VCPKG_VERSION: "2025.07.25"
VCPKG_INSTALLATION_ROOT: "C:\\vcpkg"
VCPKG_DISABLE_METRICS: "1"
VCPKG_DEFAULT_TRIPLET: x64-windows-static-release
VCPKG_INSTALLED_DIR: ${{ github.workspace }}\vcpkg_installed
VCPKG_DEFAULT_BINARY_CACHE: ${{ github.workspace }}\vcpkg_binary_cache
VCPKG_BINARY_SOURCES: clear;files,${{ github.workspace }}/vcpkg-binary-cache,readwrite
permissions:
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v7
with:
submodules: recursive
- name: Set up vcpkg
uses: lukka/run-vcpkg@v11
with:
vcpkgJsonGlob: vcpkg.json
vcpkgConfigurationJsonGlob: vcpkg-configuration.json
- name: Set up MSVC developer environment
uses: microsoft/setup-msbuild@v3
- name: Cache vcpkg static installed packages
id: cache-vcpkg-static
uses: actions/cache@v5
- name: Cache vcpkg binary packages
uses: actions/cache@v6
with:
path: |
${{ env.VCPKG_INSTALLED_DIR }}
${{ env.VCPKG_DEFAULT_BINARY_CACHE }}
key: ${{ runner.os }}-vcpkg-${{ env.VCPKG_VERSION }}-x64-static-${{ hashFiles('**/vcpkg.json', '**/vcpkg-configuration.json') }}-${{ hashFiles('.github/workflows/windows-build.yml') }}
path: ${{ github.workspace }}/vcpkg-binary-cache
key: ${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-${{ hashFiles('vcpkg.json', 'vcpkg-configuration.json', 'ports/**') }}-${{ github.run_id }}-${{ github.run_attempt }}
restore-keys: |
${{ runner.os }}-vcpkg-${{ env.VCPKG_VERSION }}-x64-static-${{ hashFiles('**/vcpkg.json', '**/vcpkg-configuration.json') }}
${{ runner.os }}-vcpkg-${{ env.VCPKG_VERSION }}-x64-static-
- name: Setup vcpkg and install static dependencies
if: steps.cache-vcpkg-static.outputs.cache-hit != 'true'
shell: pwsh
run: |
New-Item -Path "${{ env.VCPKG_INSTALLED_DIR }}" -ItemType Directory -Force
New-Item -Path "${{ env.VCPKG_DEFAULT_BINARY_CACHE }}" -ItemType Directory -Force
vcpkg install `
--recurse `
--clean-after-build `
--triplet=x64-windows-static-release `
--x-install-root=${{ env.VCPKG_INSTALLED_DIR }} `
--x-feature=test
${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-${{ hashFiles('vcpkg.json', 'vcpkg-configuration.json', 'ports/**') }}-
${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-
- name: Configure CMake for Static Build
shell: pwsh
@@ -461,7 +536,7 @@ jobs:
mkdir build_static -ErrorAction SilentlyContinue
cmake -B build_static `
-S . `
-DCMAKE_TOOLCHAIN_FILE=${{ env.VCPKG_INSTALLATION_ROOT }}/scripts/buildsystems/vcpkg.cmake `
"-DCMAKE_TOOLCHAIN_FILE=$env:VCPKG_ROOT/scripts/buildsystems/vcpkg.cmake" `
-DVCPKG_TARGET_TRIPLET=x64-windows-static-release `
-DVCPKG_INSTALLED_DIR=${{ env.VCPKG_INSTALLED_DIR }} `
-DVCPKG_MANIFEST_FEATURES=test `
@@ -493,13 +568,13 @@ jobs:
create-github-release:
name: Create GitHub Release
runs-on: ubuntu-latest
needs: [build-windows-msi, build-windows-dhtnode]
needs: [build-windows-msi, build-windows-dhtnode, nodejs-package]
permissions:
contents: write
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v7
- name: Download Windows MSI artifact
uses: actions/download-artifact@v8
@@ -513,6 +588,12 @@ jobs:
name: opendht-windows-dhtnode
path: downloaded_artifacts/static_dhtnode
- name: Download npm archive
uses: actions/download-artifact@v8
with:
name: opendht-node-npm
path: downloaded_artifacts/nodejs
- name: Create Release
id: create_release
uses: softprops/action-gh-release@v3
@@ -521,3 +602,84 @@ jobs:
files: |
downloaded_artifacts/windows_msi/*.msi
downloaded_artifacts/static_dhtnode/**/*dhtnode.exe
downloaded_artifacts/nodejs/*.tgz
publish-npm:
name: Publish opendht to npm
needs: create-github-release
runs-on: ubuntu-26.04
environment:
name: npm
url: https://www.npmjs.com/package/opendht
permissions:
contents: read
id-token: write
steps:
- uses: actions/checkout@v7
- uses: actions/setup-node@v6
with:
node-version: '24'
registry-url: https://registry.npmjs.org/
package-manager-cache: false
- uses: actions/download-artifact@v8
with:
name: opendht-node-npm
path: nodejs/dist
- name: Publish verified archive
working-directory: nodejs
env:
RELEASE_TAG: ${{ github.ref_name }}
run: |
dist_tag=latest
if [[ "${RELEASE_TAG#v}" == *-* ]]; then
dist_tag=next
fi
npm publish ./dist/*.tgz --access public --provenance --tag "$dist_tag"
publish-pypi:
name: Publish opendht to PyPI
needs: [create-github-release, python-wheels]
runs-on: ubuntu-latest
environment:
name: pypi
url: https://pypi.org/p/opendht
permissions:
contents: read
id-token: write
steps:
- uses: actions/download-artifact@v8
with:
pattern: opendht-wheels-*
path: dist
merge-multiple: true
- name: Publish wheels
uses: pypa/gh-action-pypi-publish@release/v1
publish-crate:
name: Publish opendht to crates.io
needs: create-github-release
runs-on: ubuntu-latest
environment:
name: crates-io
url: https://crates.io/crates/opendht
permissions:
contents: read
id-token: write
steps:
- uses: actions/checkout@v7
- uses: taiki-e/install-action@v2
with:
tool: cargo-edit
- name: Set crate version from release tag
working-directory: rust
env:
RELEASE_TAG: ${{ github.ref_name }}
run: cargo set-version "${RELEASE_TAG#v}"
- name: Authenticate with crates.io
id: auth
uses: rust-lang/crates-io-auth-action@v1
- name: Publish crate
working-directory: rust
env:
CARGO_REGISTRY_TOKEN: ${{ steps.auth.outputs.token }}
run: cargo publish --allow-dirty
+73 -89
View File
@@ -9,113 +9,89 @@ on:
branches: [ main, master ]
env:
VCPKG_VERSION: "2026.01.16"
VCPKG_INSTALLATION_ROOT: "C:\\vcpkg"
VCPKG_DISABLE_METRICS: "1"
VCPKG_INSTALLED_DIR: ${{ github.workspace }}\vcpkg_installed
VCPKG_DEFAULT_BINARY_CACHE: ${{ github.workspace }}\vcpkg_binary_cache
VCPKG_BINARY_SOURCES: clear;files,${{ github.workspace }}/vcpkg-binary-cache,readwrite
jobs:
prepare-dependencies:
name: Prepare Windows Dependencies
runs-on: windows-latest
outputs:
vcpkg_cache_key: ${{ steps.generate_cache_key.outputs.key }}
env:
VCPKG_DEFAULT_TRIPLET: ${{ matrix.arch }}-windows
strategy:
matrix:
arch: [x64]
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v7
- name: Set up MSVC
uses: microsoft/setup-msbuild@v3
- name: Generate Cache Key
id: generate_cache_key
shell: pwsh
run: |
$key_value = "${{ runner.os }}-vcpkg-${{ env.VCPKG_VERSION }}-${{ matrix.arch }}-shared-${{ hashFiles('**/vcpkg.json', '**/vcpkg-configuration.json') }}-${{ hashFiles('.github/workflows/windows-build.yml') }}"
echo "key=$key_value" >> $env:GITHUB_OUTPUT
- name: Cache vcpkg installed packages
id: cache-vcpkg
uses: actions/cache@v5
- name: Cache vcpkg binary packages
uses: actions/cache@v6
with:
path: |
${{ env.VCPKG_INSTALLED_DIR }}
${{ env.VCPKG_DEFAULT_BINARY_CACHE }}
key: ${{ steps.generate_cache_key.outputs.key }}
path: ${{ github.workspace }}/vcpkg-binary-cache
key: ${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-${{ hashFiles('vcpkg.json', 'vcpkg-configuration.json', 'ports/**') }}-${{ github.run_id }}-${{ github.run_attempt }}
restore-keys: |
${{ runner.os }}-vcpkg-${{ env.VCPKG_VERSION }}-${{ matrix.arch }}-shared-${{ hashFiles('**/vcpkg.json', '**/vcpkg-configuration.json') }}
${{ runner.os }}-vcpkg-${{ env.VCPKG_VERSION }}-${{ matrix.arch }}-shared-
${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-${{ hashFiles('vcpkg.json', 'vcpkg-configuration.json', 'ports/**') }}-
${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-
- name: Setup vcpkg and install dependencies
if: steps.cache-vcpkg.outputs.cache-hit != 'true'
run: |
New-Item -Path "${{ env.VCPKG_INSTALLED_DIR }}" -ItemType Directory -Force
New-Item -Path "${{ env.VCPKG_DEFAULT_BINARY_CACHE }}" -ItemType Directory -Force
vcpkg install `
--recurse `
--clean-after-build `
--triplet=${{ matrix.arch }}-windows `
--x-install-root=${{ env.VCPKG_INSTALLED_DIR }} `
--x-feature=test
shell: pwsh
- name: Set up vcpkg and install dependencies
uses: lukka/run-vcpkg@v11
with:
vcpkgJsonGlob: vcpkg.json
vcpkgConfigurationJsonGlob: vcpkg-configuration.json
runVcpkgInstall: true
runVcpkgFormatString: >-
[`install`, `--clean-after-build`, `--x-feature=test`,
`--triplet`, `$[env.VCPKG_DEFAULT_TRIPLET]`,
`--x-install-root`, `$[env.VCPKG_INSTALLED_DIR]`]
prepare-static-dependencies:
name: Prepare Windows Static Dependencies
runs-on: windows-latest
outputs:
vcpkg_static_cache_key: ${{ steps.generate_static_cache_key.outputs.key }}
env:
VCPKG_DEFAULT_TRIPLET: ${{ matrix.arch }}-windows-static-release
strategy:
matrix:
arch: [x64]
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v7
- name: Set up MSVC
uses: microsoft/setup-msbuild@v3
- name: Generate Cache Key for Static Libs
id: generate_static_cache_key
shell: pwsh
run: |
$key_value = "${{ runner.os }}-vcpkg-${{ env.VCPKG_VERSION }}-${{ matrix.arch }}-static-${{ hashFiles('**/vcpkg.json', '**/vcpkg-configuration.json') }}-${{ hashFiles('.github/workflows/windows-build.yml') }}"
echo "key=$key_value" >> $env:GITHUB_OUTPUT
- name: Cache vcpkg static installed packages
id: cache-vcpkg-static
uses: actions/cache@v5
- name: Cache vcpkg binary packages
uses: actions/cache@v6
with:
path: |
${{ env.VCPKG_INSTALLED_DIR }}
${{ env.VCPKG_DEFAULT_BINARY_CACHE }}
key: ${{ steps.generate_static_cache_key.outputs.key }}
path: ${{ github.workspace }}/vcpkg-binary-cache
key: ${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-${{ hashFiles('vcpkg.json', 'vcpkg-configuration.json', 'ports/**') }}-${{ github.run_id }}-${{ github.run_attempt }}
restore-keys: |
${{ runner.os }}-vcpkg-${{ env.VCPKG_VERSION }}-${{ matrix.arch }}-static-${{ hashFiles('**/vcpkg.json', '**/vcpkg-configuration.json') }}
${{ runner.os }}-vcpkg-${{ env.VCPKG_VERSION }}-${{ matrix.arch }}-static-
${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-${{ hashFiles('vcpkg.json', 'vcpkg-configuration.json', 'ports/**') }}-
${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-
- name: Setup vcpkg and install static dependencies
if: steps.cache-vcpkg-static.outputs.cache-hit != 'true'
shell: pwsh
run: |
New-Item -Path "${{ env.VCPKG_INSTALLED_DIR }}" -ItemType Directory -Force
New-Item -Path "${{ env.VCPKG_DEFAULT_BINARY_CACHE }}" -ItemType Directory -Force
vcpkg install `
--recurse `
--clean-after-build `
--triplet=${{ matrix.arch }}-windows-static-release `
--x-install-root=${{ env.VCPKG_INSTALLED_DIR }} `
--x-feature=test
- name: Set up vcpkg and install dependencies
uses: lukka/run-vcpkg@v11
with:
vcpkgJsonGlob: vcpkg.json
vcpkgConfigurationJsonGlob: vcpkg-configuration.json
runVcpkgInstall: true
runVcpkgFormatString: >-
[`install`, `--clean-after-build`, `--x-feature=test`,
`--triplet`, `$[env.VCPKG_DEFAULT_TRIPLET]`,
`--x-install-root`, `$[env.VCPKG_INSTALLED_DIR]`]
build-windows:
name: Windows MSVC Build
runs-on: windows-latest
needs: prepare-dependencies
env:
VCPKG_DEFAULT_TRIPLET: ${{ matrix.arch }}-windows
strategy:
matrix:
@@ -125,22 +101,25 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v7
- name: Set up vcpkg
uses: lukka/run-vcpkg@v11
with:
vcpkgJsonGlob: vcpkg.json
vcpkgConfigurationJsonGlob: vcpkg-configuration.json
- name: Set up MSVC
uses: microsoft/setup-msbuild@v3
- name: Restore vcpkg installed packages
id: cache-vcpkg-restore
uses: actions/cache/restore@v5
- name: Restore vcpkg binary packages
uses: actions/cache/restore@v6
with:
path: |
${{ env.VCPKG_INSTALLED_DIR }}
${{ env.VCPKG_DEFAULT_BINARY_CACHE }}
key: ${{ needs.prepare-dependencies.outputs.vcpkg_cache_key }}
path: ${{ github.workspace }}/vcpkg-binary-cache
key: ${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-${{ hashFiles('vcpkg.json', 'vcpkg-configuration.json', 'ports/**') }}-${{ github.run_id }}-${{ github.run_attempt }}
restore-keys: |
${{ runner.os }}-vcpkg-${{ env.VCPKG_VERSION }}-${{ matrix.arch }}-shared-${{ hashFiles('**/vcpkg.json', '**/vcpkg-configuration.json') }}
${{ runner.os }}-vcpkg-${{ env.VCPKG_VERSION }}-${{ matrix.arch }}-shared-
${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-${{ hashFiles('vcpkg.json', 'vcpkg-configuration.json', 'ports/**') }}-
${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-
- name: Configure CMake
shell: pwsh
@@ -148,7 +127,7 @@ jobs:
mkdir build -ErrorAction SilentlyContinue
cmake -B build `
-S . `
-DCMAKE_TOOLCHAIN_FILE=${{ env.VCPKG_INSTALLATION_ROOT }}/scripts/buildsystems/vcpkg.cmake `
"-DCMAKE_TOOLCHAIN_FILE=$env:VCPKG_ROOT/scripts/buildsystems/vcpkg.cmake" `
-DVCPKG_TARGET_TRIPLET=${{ matrix.arch }}-windows `
-DVCPKG_INSTALLED_DIR=${{ env.VCPKG_INSTALLED_DIR }} `
-DVCPKG_MANIFEST_FEATURES=test `
@@ -194,6 +173,8 @@ jobs:
name: Windows Static MSVC Build
runs-on: windows-latest
needs: prepare-static-dependencies
env:
VCPKG_DEFAULT_TRIPLET: ${{ matrix.arch }}-windows-static-release
strategy:
matrix:
build_type: [Release]
@@ -202,22 +183,25 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v7
- name: Set up vcpkg
uses: lukka/run-vcpkg@v11
with:
vcpkgJsonGlob: vcpkg.json
vcpkgConfigurationJsonGlob: vcpkg-configuration.json
- name: Set up MSVC
uses: microsoft/setup-msbuild@v3
- name: Restore vcpkg static installed packages
id: cache-vcpkg-static-restore
uses: actions/cache/restore@v5
- name: Restore vcpkg binary packages
uses: actions/cache/restore@v6
with:
path: |
${{ env.VCPKG_INSTALLED_DIR }}
${{ env.VCPKG_DEFAULT_BINARY_CACHE }}
key: ${{ needs.prepare-static-dependencies.outputs.vcpkg_static_cache_key }}
path: ${{ github.workspace }}/vcpkg-binary-cache
key: ${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-${{ hashFiles('vcpkg.json', 'vcpkg-configuration.json', 'ports/**') }}-${{ github.run_id }}-${{ github.run_attempt }}
restore-keys: |
${{ runner.os }}-vcpkg-${{ env.VCPKG_VERSION }}-${{ matrix.arch }}-static-${{ hashFiles('**/vcpkg.json', '**/vcpkg-configuration.json') }}
${{ runner.os }}-vcpkg-${{ env.VCPKG_VERSION }}-${{ matrix.arch }}-static-
${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-${{ hashFiles('vcpkg.json', 'vcpkg-configuration.json', 'ports/**') }}-
${{ runner.os }}-vcpkg-binary-${{ env.VCPKG_DEFAULT_TRIPLET }}-
- name: Configure CMake for Static Build
shell: pwsh
@@ -225,7 +209,7 @@ jobs:
mkdir build -ErrorAction SilentlyContinue
cmake -B build `
-S . `
-DCMAKE_TOOLCHAIN_FILE=${{ env.VCPKG_INSTALLATION_ROOT }}/scripts/buildsystems/vcpkg.cmake `
"-DCMAKE_TOOLCHAIN_FILE=$env:VCPKG_ROOT/scripts/buildsystems/vcpkg.cmake" `
-DVCPKG_TARGET_TRIPLET=${{ matrix.arch }}-windows-static-release `
-DVCPKG_INSTALLED_DIR=${{ env.VCPKG_INSTALLED_DIR }} `
-DVCPKG_MANIFEST_FEATURES=test `
@@ -275,7 +259,7 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v7
- name: Setup MinGW
uses: msys2/setup-msys2@v2
@@ -304,7 +288,7 @@ jobs:
shell: msys2 {0}
env:
LLHTTP_VERSION: "v9.2.1"
RESTINIO_VERSION: "0.7.6"
RESTINIO_VERSION: "0.7.10"
EXPECTED_LITE_URL: "https://raw.githubusercontent.com/martinmoene/expected-lite/master/include/nonstd/expected.hpp"
run: |
MSYS2_SYSROOT=$(cygpath -u "$MSYSTEM_PREFIX")
+22 -2
View File
@@ -115,16 +115,36 @@ cmake -DOPENDHT_PYTHON=ON \
| `OPENDHT_PUSH_NOTIFICATIONS` | OFF | Push notification support |
| `OPENDHT_SANITIZE` | OFF | AddressSanitizer + stack protector |
## Node.js Bindings
Node-API bindings wrap the C++ API directly with `node-addon-api`, including
RSA/EC keys, certificates, identities, and signed/encrypted DHT operations.
With the native dependencies above, Node.js 22.9+, and CMake installed, run
`npm ci`, `npm run build`, `npm test`, and `npm run typecheck` from `nodejs/`.
This creates a separate native build without changing the existing build tree.
See [the Node.js binding guide](nodejs/README.md) for the API and current limits.
`npm run package` creates a platform prebuild with bundled native dependencies;
`npm run test:package` verifies installation outside the source tree. The Node.js
package CI builds Linux x64, macOS ARM64/x64, and Windows x64 prebuilds and checks
the assembled npm archive on Node 22, 24, and 26 before release attachment.
## C and Rust Bindings
Enable `OPENDHT_C=ON` to build the C ABI.
See [the Rust build guide](rust/BUILD.md) for build instructions.
## Windows/MSVC
Building on Windows with MSVC is supported with VCPKG and CMake.
### Build with VCPKG in manifest mode
1. Install [VCPKG](https://github.com/microsoft/vcpkg) and required dependencies:
1. Install [VCPKG](https://github.com/microsoft/vcpkg) 2026.10.16 and required dependencies:
```ps1
git clone https://github.com/microsoft/vcpkg.git
cd vcpkg; .\bootstrap-vcpkg.bat
cd vcpkg
git checkout 9eec002df1cd109f8f104cd966a763daf9751489
.\bootstrap-vcpkg.bat
```
2. Use CMake preset for vcpkg:
+28 -2
View File
@@ -1,4 +1,4 @@
cmake_minimum_required (VERSION 3.21...3.31)
cmake_minimum_required (VERSION 3.21...4.4)
if(POLICY CMP0091)
cmake_policy(SET CMP0091 NEW)
endif()
@@ -10,7 +10,7 @@ if(POLICY CMP0167)
endif()
set (opendht_VERSION_MAJOR 4)
set (opendht_VERSION_MINOR 0.0)
set (opendht_VERSION_MINOR 4.99)
set (opendht_VERSION ${opendht_VERSION_MAJOR}.${opendht_VERSION_MINOR})
set (PACKAGE_VERSION ${opendht_VERSION})
set (VERSION "${opendht_VERSION}")
@@ -117,6 +117,7 @@ if (NOT MSVC)
find_package (PkgConfig REQUIRED)
pkg_search_module (GnuTLS REQUIRED IMPORTED_TARGET gnutls)
pkg_search_module (Nettle REQUIRED IMPORTED_TARGET nettle)
pkg_search_module (Hogweed REQUIRED IMPORTED_TARGET hogweed>=3.1)
pkg_search_module (argon2 REQUIRED IMPORTED_TARGET libargon2)
set(argon2_lib ", libargon2")
pkg_search_module (Jsoncpp IMPORTED_TARGET jsoncpp)
@@ -467,6 +468,7 @@ list (APPEND opendht_SOURCES
src/routing_table.cpp
src/node_cache.cpp
src/network_engine.cpp
src/transport.cpp
src/securedht.cpp
src/dhtrunner.cpp
src/log.cpp
@@ -491,6 +493,7 @@ list (APPEND opendht_HEADERS
include/opendht/routing_table.h
include/opendht/node_cache.h
include/opendht/network_engine.h
include/opendht/transport.h
include/opendht/scheduler.h
include/opendht/rate_limiter.h
include/opendht/securedht.h
@@ -613,6 +616,7 @@ else()
PRIVATE
PkgConfig::argon2
PkgConfig::Nettle
PkgConfig::Hogweed
${SIMDUTF_LIBRARIES}
PUBLIC
${CMAKE_THREAD_LIBS_INIT}
@@ -637,6 +641,7 @@ else()
PRIVATE
argon2
nettle
hogweed
PUBLIC
${CMAKE_THREAD_LIBS_INIT}
${GNUTLS_LIBRARIES}
@@ -845,6 +850,15 @@ endif()
# Unit tests
if (BUILD_TESTING)
if (OPENDHT_C)
add_executable(test_c tests/test_c.c)
target_include_directories(test_c PRIVATE c include)
target_link_libraries(test_c PRIVATE opendht-c)
if (MSVC)
target_link_libraries(test_c PRIVATE ${GNUTLS_LIBRARIES})
endif()
add_test(NAME test_c COMMAND test_c)
endif()
if (NOT MSVC)
pkg_search_module(Cppunit REQUIRED IMPORTED_TARGET cppunit)
set(cppunit_LIBRARIES PkgConfig::Cppunit)
@@ -860,6 +874,8 @@ if (BUILD_TESTING)
tests/test_value.cpp
tests/test_crypto.h
tests/test_crypto.cpp
tests/test_transport.h
tests/test_transport.cpp
tests/test_dhtrunner.h
tests/test_dhtrunner.cpp
tests/test_storage.h
@@ -870,7 +886,17 @@ if (BUILD_TESTING)
tests/test_parsedmessage.cpp
tests/test_networkengine.h
tests/test_networkengine.cpp
tests/test_opcache.h
tests/test_opcache.cpp
tests/test_valuecache.h
tests/test_valuecache.cpp
)
if (OPENDHT_INDEX)
list (APPEND test_FILES
tests/test_pht.h
tests/test_pht.cpp
)
endif()
if (OPENDHT_PROXY_CLIENT)
list (APPEND test_FILES
tests/test_dhtproxy_client.h
+5 -2
View File
@@ -4,7 +4,10 @@
<a id="user-content-opendht-" class="anchor" href="/savoirfairelinux/opendht/blob/master/README.md#opendht-" aria-hidden="true"></a>OpenDHT
</h1>
![PyPI - Version](https://img.shields.io/pypi/v/opendht?style=flat)
[![PyPI - Version](https://img.shields.io/pypi/v/opendht?style=flat)](https://pypi.org/project/opendht/)
[![crates.io - Version](https://img.shields.io/crates/v/opendht?style=flat)](https://crates.io/crates/opendht)
[![npm - Version](https://img.shields.io/npm/v/opendht?style=flat)](https://www.npmjs.com/package/opendht)
[![Docker Image](https://img.shields.io/badge/Docker-ghcr.io-2496ED?style=flat&logo=docker&logoColor=white)](https://github.com/savoirfairelinux/opendht/pkgs/container/opendht%2Fopendht)
A lightweight C++17 Distributed Hash Table implementation.
@@ -17,7 +20,7 @@ Values are distributed over the network, with redundancy.
* Public key cryptography layer providing optional data signature and encryption (using GnuTLS)
* IPv4 and IPv6 support
* Clean and powerful **C++17** map API
* Bindings for **C, Rust & Python 3**
* Bindings for **C, Rust, Node.js & Python 3**
* REST API with optional HTTP client+server with push notification support
## Documentation
+573 -99
View File
@@ -5,12 +5,35 @@
#include <opendht.h>
#include <opendht/log.h>
#include <atomic>
using ValueSp = std::shared_ptr<dht::Value>;
using PrivkeySp = std::shared_ptr<dht::crypto::PrivateKey>;
using PubkeySp = std::shared_ptr<const dht::crypto::PublicKey>;
using CertSp = std::shared_ptr<dht::crypto::Certificate>;
static_assert(sizeof(dht_infohash) == sizeof(dht::InfoHash));
static_assert(alignof(dht_infohash) == alignof(dht::InfoHash));
static_assert(sizeof(dht_pkid) == sizeof(dht::PkId));
static_assert(alignof(dht_pkid) == alignof(dht::PkId));
struct CompletionGuard
{
CompletionGuard(dht_done_cb callback, void* data)
: callback(callback)
, data(data)
{}
~CompletionGuard() { finish(false); }
void finish(bool ok)
{
if (!finished.exchange(true) && callback)
callback(ok, data);
}
dht_done_cb callback;
void* data;
std::atomic_bool finished {false};
};
#ifdef __cplusplus
extern "C" {
#endif
@@ -156,6 +179,47 @@ dht_value_set_user_type(dht_value* data, const char* user_type)
(*reinterpret_cast<ValueSp*>(data))->user_type = user_type;
}
uint16_t
dht_value_get_type(const dht_value* value)
{
return (*reinterpret_cast<const ValueSp*>(value))->type;
}
void
dht_value_set_type(dht_value* value, uint16_t type)
{
(*reinterpret_cast<ValueSp*>(value))->type = type;
}
uint16_t
dht_value_get_seq(const dht_value* value)
{
return (*reinterpret_cast<const ValueSp*>(value))->seq;
}
void
dht_value_set_seq(dht_value* value, uint16_t seq)
{
(*reinterpret_cast<ValueSp*>(value))->seq = seq;
}
unsigned
dht_value_get_priority(const dht_value* value)
{
return (*reinterpret_cast<const ValueSp*>(value))->priority;
}
void
dht_value_set_priority(dht_value* value, unsigned priority)
{
(*reinterpret_cast<ValueSp*>(value))->priority = priority;
}
const char*
dht_value_get_push_type(const dht_value* value)
{
return (*reinterpret_cast<const ValueSp*>(value))->pushType.c_str();
}
void
dht_value_set_push_type(dht_value* value, const char* push_type)
{
(*reinterpret_cast<ValueSp*>(value))->pushType = push_type;
}
dht_value*
dht_value_with_id_new(const uint8_t* data, size_t size, uint64_t value_id)
{
@@ -183,6 +247,29 @@ dht_value_ref(const dht_value* v)
return reinterpret_cast<dht_value*>(new ValueSp(*reinterpret_cast<const ValueSp*>(v)));
}
dht_value*
dht_value_copy(const dht_value* value)
{
try {
const auto& source = **reinterpret_cast<const ValueSp*>(value);
auto copy = std::make_shared<dht::Value>();
copy->id = source.id;
copy->owner = source.owner;
copy->recipient = source.recipient;
copy->type = source.type;
copy->data = source.data;
copy->user_type = source.user_type;
copy->seq = source.seq;
copy->signature = source.signature;
copy->cypher = source.cypher;
copy->priority = source.priority;
copy->pushType = source.pushType;
return reinterpret_cast<dht_value*>(new ValueSp(std::move(copy)));
} catch (...) {
return nullptr;
}
}
void
dht_value_unref(dht_value* v)
{
@@ -196,7 +283,7 @@ dht_publickey_import(const uint8_t* dat, size_t dat_size)
try {
return reinterpret_cast<dht_publickey*>(
new PubkeySp(std::make_shared<const dht::crypto::PublicKey>(dat, dat_size)));
} catch (const dht::crypto::CryptoException& e) {
} catch (...) {
return nullptr;
}
}
@@ -214,6 +301,29 @@ dht_publickey_export(const dht_publickey* pk, char* out, size_t* outlen)
return pkey->pack((uint8_t*) out, outlen);
}
dht_blob*
dht_publickey_export_blob(const dht_publickey* pk)
{
try {
auto data = std::make_unique<dht::Blob>();
(*reinterpret_cast<const PubkeySp*>(pk))->pack(*data);
return reinterpret_cast<dht_blob*>(data.release());
} catch (...) {
return nullptr;
}
}
int
dht_publickey_unpack(dht_publickey* pk, const uint8_t* data, size_t size)
{
try {
*reinterpret_cast<PubkeySp*>(pk) = std::make_shared<const dht::crypto::PublicKey>(data, size);
return 0;
} catch (...) {
return EINVAL;
}
}
dht_infohash
dht_publickey_get_id(const dht_publickey* pk)
{
@@ -255,8 +365,12 @@ dht_privatekey_generate(unsigned key_length_bits)
{
if (key_length_bits == 0)
key_length_bits = 4096;
return reinterpret_cast<dht_privatekey*>(
new PrivkeySp(std::make_shared<dht::crypto::PrivateKey>(dht::crypto::PrivateKey::generate(key_length_bits))));
try {
return reinterpret_cast<dht_privatekey*>(new PrivkeySp(
std::make_shared<dht::crypto::PrivateKey>(dht::crypto::PrivateKey::generate(key_length_bits))));
} catch (...) {
return nullptr;
}
}
dht_privatekey*
@@ -265,7 +379,7 @@ dht_privatekey_import(const uint8_t* dat, size_t dat_size, const char* password)
try {
return reinterpret_cast<dht_privatekey*>(
new PrivkeySp(std::make_shared<dht::crypto::PrivateKey>(dat, dat_size, password)));
} catch (const dht::crypto::CryptoException& e) {
} catch (...) {
return nullptr;
}
}
@@ -276,7 +390,43 @@ dht_privatekey_export(const dht_privatekey* k, char* out, size_t* out_size, cons
if (!out or !out_size or !*out_size)
return -1;
const auto& key = *reinterpret_cast<const PrivkeySp*>(k);
return key->serialize((uint8_t*) out, out_size, password);
try {
return key->serialize((uint8_t*) out, out_size, password ? password : "");
} catch (...) {
return -1;
}
}
dht_privatekey*
dht_privatekey_generate_ec(void)
{
try {
return reinterpret_cast<dht_privatekey*>(
new PrivkeySp(std::make_shared<dht::crypto::PrivateKey>(dht::crypto::PrivateKey::generateEC())));
} catch (...) {
return nullptr;
}
}
dht_blob*
dht_privatekey_export_blob(const dht_privatekey* key, const char* password)
{
try {
return reinterpret_cast<dht_blob*>(
new dht::Blob((*reinterpret_cast<const PrivkeySp*>(key))->serialize(password ? password : "")));
} catch (...) {
return nullptr;
}
}
dht_blob*
dht_privatekey_sign(const dht_privatekey* key, const uint8_t* data, size_t size)
{
try {
return reinterpret_cast<dht_blob*>(new dht::Blob((*reinterpret_cast<const PrivkeySp*>(key))->sign(data, size)));
} catch (...) {
return nullptr;
}
}
dht_publickey*
@@ -311,11 +461,28 @@ dht_certificate_import(const uint8_t* dat, size_t dat_size)
{
try {
return reinterpret_cast<dht_certificate*>(new CertSp(std::make_shared<dht::crypto::Certificate>(dat, dat_size)));
} catch (const dht::crypto::CryptoException& e) {
} catch (...) {
return nullptr;
}
}
dht_blob*
dht_certificate_export(const dht_certificate* certificate)
{
try {
return reinterpret_cast<dht_blob*>(new dht::Blob((*reinterpret_cast<const CertSp*>(certificate))->getPacked()));
} catch (...) {
return nullptr;
}
}
dht_certificate*
dht_certificate_ref(const dht_certificate* certificate)
{
return certificate ? reinterpret_cast<dht_certificate*>(new CertSp(*reinterpret_cast<const CertSp*>(certificate)))
: nullptr;
}
void
dht_certificate_delete(dht_certificate* c)
{
@@ -364,8 +531,10 @@ inline dht_identity
dht_identity_to_c(const dht::crypto::Identity& id)
{
dht_identity cid {};
cid.privatekey = id.first ? reinterpret_cast<dht_privatekey*>(new PrivkeySp(id.first)) : NULL;
cid.certificate = id.second ? reinterpret_cast<dht_certificate*>(new CertSp(id.second)) : NULL;
auto key = id.first ? std::make_unique<PrivkeySp>(id.first) : nullptr;
auto certificate = id.second ? std::make_unique<CertSp>(id.second) : nullptr;
cid.privatekey = reinterpret_cast<dht_privatekey*>(key.release());
cid.certificate = reinterpret_cast<dht_certificate*>(certificate.release());
return cid;
}
@@ -376,7 +545,12 @@ extern "C" {
OPENDHT_C_PUBLIC dht_identity
dht_identity_generate(const char* common_name, const dht_identity* ca)
{
return dht_identity_to_c(dht::crypto::generateIdentity(common_name, dht_identity_from_c(ca)));
try {
return dht_identity_to_c(
dht::crypto::generateIdentity(common_name ? common_name : "dhtnode", dht_identity_from_c(ca)));
} catch (...) {
return {};
}
}
OPENDHT_C_PUBLIC void
@@ -392,6 +566,17 @@ dht_identity_delete(dht_identity* id)
}
}
dht_identity
dht_identity_generate_ec(const char* common_name, const dht_identity* ca)
{
try {
return dht_identity_to_c(
dht::crypto::generateEcIdentity(common_name ? common_name : "dhtnode", dht_identity_from_c(ca)));
} catch (...) {
return {};
}
}
// config
void
dht_runner_config_default(dht_runner_config* config)
@@ -413,6 +598,28 @@ dht_runner_delete(dht_runner* runner)
delete reinterpret_cast<dht::DhtRunner*>(runner);
}
int
dht_runner_loop(dht_runner* runner)
{
try {
reinterpret_cast<dht::DhtRunner*>(runner)->loop();
return 0;
} catch (...) {
return EINVAL;
}
}
int
dht_runner_join(dht_runner* runner)
{
try {
reinterpret_cast<dht::DhtRunner*>(runner)->join();
return 0;
} catch (...) {
return EDEADLK;
}
}
int
dht_runner_run(dht_runner* r, in_port_t port)
{
@@ -424,61 +631,163 @@ dht_runner_run(dht_runner* r, in_port_t port)
}
return 0;
}
}
static dht::DhtRunner::Config
config_from_c(const dht_runner_config* conf)
{
dht::DhtRunner::Config config;
config.dht_config.node_config.is_bootstrap = conf->dht_config.node_config.is_bootstrap;
config.dht_config.node_config.maintain_storage = conf->dht_config.node_config.maintain_storage;
config.dht_config.node_config.node_id = *reinterpret_cast<const dht::InfoHash*>(
&conf->dht_config.node_config.node_id);
config.dht_config.node_config.network = conf->dht_config.node_config.network;
config.dht_config.node_config.persist_path = conf->dht_config.node_config.persist_path
? std::string(conf->dht_config.node_config.persist_path)
: std::string {};
if (conf->dht_config.id.privatekey)
config.dht_config.id.first = *reinterpret_cast<const PrivkeySp*>(conf->dht_config.id.privatekey);
if (conf->dht_config.id.certificate)
config.dht_config.id.second = *reinterpret_cast<const CertSp*>(conf->dht_config.id.certificate);
config.threaded = conf->threaded;
config.proxy_server = conf->proxy_server ? std::string(conf->proxy_server) : std::string {};
config.push_node_id = conf->push_node_id ? std::string(conf->push_node_id) : std::string {};
config.push_token = conf->push_token ? std::string(conf->push_token) : std::string {};
config.push_topic = conf->push_topic ? std::string(conf->push_topic) : std::string {};
config.push_platform = conf->push_platform ? std::string(conf->push_platform) : std::string {};
config.proxy_user_agent = conf->proxy_user_agent ? std::string(conf->proxy_user_agent) : std::string {};
config.peer_discovery = conf->peer_discovery;
config.peer_publish = conf->peer_publish;
if (conf->server_ca)
config.server_ca = *reinterpret_cast<const CertSp*>(conf->server_ca);
config.client_identity = dht_identity_from_c(&conf->client_identity);
return config;
}
extern "C" {
int
dht_runner_run_config(dht_runner* r, in_port_t port, const dht_runner_config* conf)
dht_runner_run_config(dht_runner* runner, in_port_t port, const dht_runner_config* config)
{
return dht_runner_run_bind(runner, nullptr, nullptr, port, config, nullptr);
}
int
dht_runner_run_bind(dht_runner* runner,
const char* ipv4,
const char* ipv6,
in_port_t port,
const dht_runner_config* conf,
const dht_runner_context* callbacks)
{
auto runner = reinterpret_cast<dht::DhtRunner*>(r);
try {
dht::DhtRunner::Config config;
config.dht_config.node_config.is_bootstrap = conf->dht_config.node_config.is_bootstrap;
config.dht_config.node_config.maintain_storage = conf->dht_config.node_config.maintain_storage;
config.dht_config.node_config.node_id = *reinterpret_cast<const dht::InfoHash*>(
&conf->dht_config.node_config.node_id);
config.dht_config.node_config.network = conf->dht_config.node_config.network;
config.dht_config.node_config.persist_path = conf->dht_config.node_config.persist_path
? std::string(conf->dht_config.node_config.persist_path)
: std::string {};
if (conf->dht_config.id.privatekey)
config.dht_config.id.first = *reinterpret_cast<const PrivkeySp*>(conf->dht_config.id.privatekey);
if (conf->dht_config.id.certificate)
config.dht_config.id.second = *reinterpret_cast<const CertSp*>(conf->dht_config.id.certificate);
config.threaded = conf->threaded;
config.proxy_server = conf->proxy_server ? std::string(conf->proxy_server) : std::string {};
config.push_node_id = conf->push_node_id ? std::string(conf->push_node_id) : std::string {};
config.push_token = conf->push_token ? std::string(conf->push_token) : std::string {};
config.push_topic = conf->push_topic ? std::string(conf->push_topic) : std::string {};
config.push_platform = conf->push_platform ? std::string(conf->push_platform) : std::string {};
config.proxy_user_agent = conf->proxy_user_agent ? std::string(conf->proxy_user_agent) : std::string {};
config.peer_discovery = conf->peer_discovery;
config.peer_publish = conf->peer_publish;
if (conf->server_ca)
config.server_ca = *reinterpret_cast<const CertSp*>(conf->server_ca);
config.client_identity = dht_identity_from_c(&conf->client_identity);
auto lifetime = callbacks ? std::shared_ptr<void>(callbacks->user_data,
[release = callbacks->released_cb](void* data) {
if (release)
release(data);
})
: std::shared_ptr<void> {};
auto config = config_from_c(conf);
dht::DhtRunner::Context context;
if (conf->log) {
context.logger = dht::log::getStdLogger();
}
runner->run(port, config, std::move(context));
if (callbacks) {
if (callbacks->status_cb) {
context.statusChangedCallback = [lifetime, cb = callbacks->status_cb](dht::NodeStatus ipv4,
dht::NodeStatus ipv6) {
cb(static_cast<int>(ipv4), static_cast<int>(ipv6), lifetime.get());
};
}
if (callbacks->certificate_store_cb) {
context.certificateStore = [lifetime, cb = callbacks->certificate_store_cb](const dht::PkId& id) {
std::vector<std::shared_ptr<dht::crypto::Certificate>> found;
auto emit = [](const dht_certificate* certificate, void* data) {
if (certificate)
static_cast<decltype(found)*>(data)->push_back(
*reinterpret_cast<const CertSp*>(certificate));
};
auto hash = dht_pkid_to_c(id);
cb(&hash, emit, &found, lifetime.get());
return found;
};
}
}
auto service = std::to_string(port);
reinterpret_cast<dht::DhtRunner*>(runner)
->run(ipv4 ? ipv4 : "0.0.0.0", ipv6 ? ipv6 : "::", service.c_str(), config, std::move(context));
} catch (...) {
return ENOTCONN;
}
return 0;
}
int
dht_runner_connectivity_changed(dht_runner* runner)
{
try {
reinterpret_cast<dht::DhtRunner*>(runner)->connectivityChanged();
return 0;
} catch (...) {
return EINVAL;
}
}
int
dht_runner_register_type(dht_runner* runner, uint16_t id, const char* name, uint64_t expiration_seconds, bool insecure)
{
if (!dht_runner_is_running(runner) || !name
|| expiration_seconds
> static_cast<uint64_t>(std::chrono::duration_cast<std::chrono::seconds>(dht::duration::max()).count()))
return EINVAL;
try {
dht::ValueType type(id, name, std::chrono::seconds(expiration_seconds));
if (insecure)
reinterpret_cast<dht::DhtRunner*>(runner)->registerInsecureType(type);
else
reinterpret_cast<dht::DhtRunner*>(runner)->registerType(type);
return 0;
} catch (...) {
return EINVAL;
}
}
dht_publickey*
dht_runner_get_publickey(const dht_runner* runner)
{
try {
auto key = reinterpret_cast<const dht::DhtRunner*>(runner)->getPublicKey();
return key ? reinterpret_cast<dht_publickey*>(new PubkeySp(std::move(key))) : nullptr;
} catch (...) {
return nullptr;
}
}
int
dht_runner_register_certificate(dht_runner* runner, const dht_certificate* certificate)
{
if (!dht_runner_is_running(runner))
return ENOTCONN;
try {
reinterpret_cast<dht::DhtRunner*>(runner)->registerCertificate(*reinterpret_cast<const CertSp*>(certificate));
return 0;
} catch (...) {
return EINVAL;
}
}
void
dht_runner_ping(dht_runner* r, struct sockaddr* addr, socklen_t addr_len, dht_done_cb done_cb, void* cb_user_data)
{
auto runner = reinterpret_cast<dht::DhtRunner*>(r);
if (done_cb) {
runner->bootstrap(dht::SockAddr(addr, addr_len),
[done_cb, cb_user_data](bool ok) { done_cb(ok, cb_user_data); });
} else {
runner->bootstrap(dht::SockAddr(addr, addr_len));
auto guard = std::make_shared<CompletionGuard>(done_cb, cb_user_data);
try {
runner->bootstrap(dht::SockAddr(addr, addr_len), [guard](bool ok) { guard->finish(ok); });
} catch (...) {
guard->finish(false);
}
}
@@ -494,18 +803,29 @@ dht_runner_bootstrap(dht_runner* r, const char* host, const char* service)
void
dht_runner_get(dht_runner* r, const dht_infohash* h, dht_get_cb cb, dht_done_cb done_cb, void* cb_user_data)
{
dht_runner_get_filtered(r, h, nullptr, cb, done_cb, cb_user_data);
}
void
dht_runner_get_filtered(
dht_runner* r, const dht_infohash* h, const char* where, dht_get_cb cb, dht_done_cb done_cb, void* cb_user_data)
{
auto runner = reinterpret_cast<dht::DhtRunner*>(r);
auto hash = reinterpret_cast<const dht::InfoHash*>(h);
runner->get(
*hash,
[cb, cb_user_data](std::shared_ptr<dht::Value> value) {
return cb(reinterpret_cast<const dht_value*>(&value), cb_user_data);
},
[done_cb, cb_user_data](bool ok) {
if (done_cb)
done_cb(ok, cb_user_data);
});
auto guard = std::make_shared<CompletionGuard>(done_cb, cb_user_data);
try {
runner->get(
*hash,
[cb, cb_user_data, guard](std::shared_ptr<dht::Value> value) {
return cb ? cb(reinterpret_cast<const dht_value*>(&value), cb_user_data) : true;
},
[guard](bool ok) { guard->finish(ok); },
{},
where ? dht::Where(where) : dht::Where {});
} catch (...) {
guard->finish(false);
}
}
struct ScopeGuardCb
@@ -528,23 +848,38 @@ private:
dht_op_token*
dht_runner_listen(dht_runner* r, const dht_infohash* h, dht_value_cb cb, dht_shutdown_cb done_cb, void* cb_user_data)
{
return dht_runner_listen_filtered(r, h, nullptr, cb, done_cb, cb_user_data);
}
dht_op_token*
dht_runner_listen_filtered(dht_runner* r,
const dht_infohash* h,
const char* where,
dht_value_cb cb,
dht_shutdown_cb done_cb,
void* cb_user_data)
{
auto runner = reinterpret_cast<dht::DhtRunner*>(r);
auto hash = reinterpret_cast<const dht::InfoHash*>(h);
auto fret = new std::future<size_t>;
*fret = runner->listen(*hash,
[cb,
cb_user_data,
guard = done_cb ? std::make_shared<ScopeGuardCb>(done_cb, cb_user_data)
: std::shared_ptr<ScopeGuardCb> {}](
const std::vector<std::shared_ptr<dht::Value>>& values, bool expired) {
for (const auto& value : values) {
if (not cb(reinterpret_cast<const dht_value*>(&value), expired, cb_user_data))
return false;
}
return true;
});
return (dht_op_token*) fret;
auto guard = std::make_shared<ScopeGuardCb>(done_cb, cb_user_data);
try {
auto fret = std::make_unique<std::future<size_t>>();
*fret = runner->listen(
*hash,
[cb, cb_user_data, guard](const std::vector<std::shared_ptr<dht::Value>>& values, bool expired) {
for (const auto& value : values) {
if (cb && not cb(reinterpret_cast<const dht_value*>(&value), expired, cb_user_data))
return false;
}
return true;
},
{},
where ? dht::Where(where) : dht::Where {});
return reinterpret_cast<dht_op_token*>(fret.release());
} catch (...) {
return nullptr;
}
}
void
@@ -562,6 +897,73 @@ dht_op_token_delete(dht_op_token* token)
delete reinterpret_cast<std::future<size_t>*>(token);
}
void
dht_runner_query(
dht_runner* r, const dht_infohash* h, const char* query, dht_query_cb cb, dht_done_cb done_cb, void* user_data)
{
auto guard = std::make_shared<CompletionGuard>(done_cb, user_data);
try {
reinterpret_cast<dht::DhtRunner*>(r)->query(
*reinterpret_cast<const dht::InfoHash*>(h),
[guard, cb, user_data](const std::vector<std::shared_ptr<dht::FieldValueIndex>>& results) {
for (const auto& index : results) {
dht_query_result result {};
dht::Blob user_type;
for (const auto& entry : index->index) {
switch (entry.first) {
case dht::Value::Field::Id:
result.fields |= DHT_FIELD_ID;
result.id = entry.second.getInt();
break;
case dht::Value::Field::ValueType:
result.fields |= DHT_FIELD_TYPE;
result.value_type = entry.second.getInt();
break;
case dht::Value::Field::SeqNum:
result.fields |= DHT_FIELD_SEQ;
result.seq = entry.second.getInt();
break;
case dht::Value::Field::OwnerPk:
result.fields |= DHT_FIELD_OWNER;
result.owner = dht_infohash_to_c(entry.second.getHash());
break;
case dht::Value::Field::UserType:
result.fields |= DHT_FIELD_USER_TYPE;
user_type = entry.second.getBlob();
result.user_type = {user_type.data(), user_type.size()};
break;
default:
break;
}
}
if (cb && !cb(&result, user_data))
return false;
}
return true;
},
[guard](bool ok) { guard->finish(ok); },
dht::Query(query ? query : ""));
} catch (...) {
guard->finish(false);
}
}
void
dht_runner_find_certificate(
dht_runner* runner, const dht_pkid* id, dht_certificate_cb callback, dht_shutdown_cb released_cb, void* user_data)
{
auto guard = std::make_shared<ScopeGuardCb>(released_cb, user_data);
try {
reinterpret_cast<dht::DhtRunner*>(runner)->findCertificate(
*reinterpret_cast<const dht::PkId*>(id),
[guard, callback, user_data](const std::shared_ptr<dht::crypto::Certificate>& certificate) {
if (callback)
callback(certificate ? reinterpret_cast<const dht_certificate*>(&certificate) : nullptr, user_data);
});
} catch (...) {
}
}
void
dht_runner_put(
dht_runner* r, const dht_infohash* h, const dht_value* v, dht_done_cb done_cb, void* cb_user_data, bool permanent)
@@ -569,15 +971,12 @@ dht_runner_put(
auto runner = reinterpret_cast<dht::DhtRunner*>(r);
auto hash = reinterpret_cast<const dht::InfoHash*>(h);
auto value = reinterpret_cast<const ValueSp*>(v);
runner->put(
*hash,
*value,
[done_cb, cb_user_data](bool ok) {
if (done_cb)
done_cb(ok, cb_user_data);
},
dht::time_point::max(),
permanent);
auto guard = std::make_shared<CompletionGuard>(done_cb, cb_user_data);
try {
runner->put(*hash, *value, [guard](bool ok) { guard->finish(ok); }, dht::time_point::max(), permanent);
} catch (...) {
guard->finish(false);
}
}
void
@@ -587,14 +986,12 @@ dht_runner_put_signed(
auto runner = reinterpret_cast<dht::DhtRunner*>(r);
auto hash = reinterpret_cast<const dht::InfoHash*>(h);
auto value = reinterpret_cast<const ValueSp*>(v);
runner->putSigned(
*hash,
*value,
[done_cb, cb_user_data](bool ok) {
if (done_cb)
done_cb(ok, cb_user_data);
},
permanent);
auto guard = std::make_shared<CompletionGuard>(done_cb, cb_user_data);
try {
runner->putSigned(*hash, *value, [guard](bool ok) { guard->finish(ok); }, permanent);
} catch (...) {
guard->finish(false);
}
}
void
@@ -610,15 +1007,12 @@ dht_runner_put_encrypted(dht_runner* r,
auto hash = reinterpret_cast<const dht::InfoHash*>(h);
auto toHash = reinterpret_cast<const dht::PkId*>(to);
auto value = reinterpret_cast<const ValueSp*>(v);
runner->putEncrypted(
*hash,
*toHash,
*value,
[done_cb, cb_user_data](bool ok) {
if (done_cb)
done_cb(ok, cb_user_data);
},
permanent);
auto guard = std::make_shared<CompletionGuard>(done_cb, cb_user_data);
try {
runner->putEncrypted(*hash, *toHash, *value, [guard](bool ok) { guard->finish(ok); }, permanent);
} catch (...) {
guard->finish(false);
}
}
void
@@ -633,10 +1027,8 @@ void
dht_runner_shutdown(dht_runner* r, dht_shutdown_cb done_cb, void* cb_user_data)
{
auto runner = reinterpret_cast<dht::DhtRunner*>(r);
runner->shutdown([done_cb, cb_user_data]() {
if (done_cb)
done_cb(cb_user_data);
});
auto guard = std::make_shared<ScopeGuardCb>(done_cb, cb_user_data);
runner->shutdown([guard]() {});
}
bool
@@ -681,12 +1073,94 @@ dht_runner_get_public_address(const dht_runner* r)
if (addrs.empty())
return nullptr;
auto ret = (struct sockaddr**) malloc(sizeof(struct sockaddr*) * (addrs.size() + 1));
if (!ret)
return nullptr;
for (size_t i = 0; i < addrs.size(); i++)
ret[i] = addrs[i].release();
ret[addrs.size()] = nullptr;
return ret;
}
void
dht_sockaddr_array_delete(struct sockaddr** addresses)
{
if (addresses) {
for (auto current = addresses; *current; ++current)
free(*current);
free(addresses);
}
}
int
dht_runner_set_push_notification_token(dht_runner* runner, const char* value)
{
try {
reinterpret_cast<dht::DhtRunner*>(runner)->setPushNotificationToken(value ? value : "");
return 0;
} catch (...) {
return EINVAL;
}
}
int
dht_runner_set_push_notification_topic(dht_runner* runner, const char* value)
{
try {
reinterpret_cast<dht::DhtRunner*>(runner)->setPushNotificationTopic(value ? value : "");
return 0;
} catch (...) {
return EINVAL;
}
}
int
dht_runner_set_push_notification_platform(dht_runner* runner, const char* value)
{
try {
reinterpret_cast<dht::DhtRunner*>(runner)->setPushNotificationPlatform(value ? value : "");
return 0;
} catch (...) {
return EINVAL;
}
}
dht_push_token*
dht_runner_push_notification_received(dht_runner* runner,
const char* const* keys,
const char* const* values,
size_t count)
{
try {
std::map<std::string, std::string> data;
for (size_t index = 0; index < count; ++index)
data.emplace(keys[index], values[index]);
return reinterpret_cast<dht_push_token*>(new std::future<dht::PushNotificationResult>(
reinterpret_cast<dht::DhtRunner*>(runner)->pushNotificationReceived(data)));
} catch (...) {
return nullptr;
}
}
int
dht_push_token_poll(dht_push_token* token, int* result)
{
if (!token || !result)
return EINVAL;
try {
auto future = reinterpret_cast<std::future<dht::PushNotificationResult>*>(token);
if (future->wait_for(std::chrono::seconds(0)) != std::future_status::ready)
return EAGAIN;
*result = static_cast<int>(future->get());
return 0;
} catch (...) {
return EIO;
}
}
void
dht_push_token_delete(dht_push_token* token)
{
delete reinterpret_cast<std::future<dht::PushNotificationResult>*>(token);
}
#ifdef __cplusplus
}
#endif
+104 -2
View File
@@ -39,9 +39,17 @@ OPENDHT_C_PUBLIC dht_data_view dht_blob_get_data(const dht_blob* data);
OPENDHT_C_PUBLIC void dht_blob_delete(dht_blob* data);
// dht::InfoHash
#ifdef __cplusplus
#define OPENDHT_C_ALIGN_UINT32 alignas(uint32_t)
#elif defined(_MSC_VER)
#define OPENDHT_C_ALIGN_UINT32 __declspec(align(4))
#else
#define OPENDHT_C_ALIGN_UINT32 _Alignas(uint32_t)
#endif
struct OPENDHT_C_PUBLIC dht_infohash
{
uint8_t d[HASH_LEN];
OPENDHT_C_ALIGN_UINT32 uint8_t d[HASH_LEN];
};
typedef struct dht_infohash dht_infohash;
OPENDHT_C_PUBLIC void dht_infohash_zero(dht_infohash* h);
@@ -56,9 +64,10 @@ OPENDHT_C_PUBLIC bool dht_infohash_is_zero(const dht_infohash* h);
// dht::PkId
struct OPENDHT_C_PUBLIC dht_pkid
{
uint8_t d[32];
OPENDHT_C_ALIGN_UINT32 uint8_t d[32];
};
typedef struct dht_pkid dht_pkid;
#undef OPENDHT_C_ALIGN_UINT32
OPENDHT_C_PUBLIC const char* dht_pkid_print(const dht_pkid* h);
// dht::crypto::PublicKey
@@ -67,6 +76,8 @@ typedef struct dht_publickey dht_publickey;
OPENDHT_C_PUBLIC dht_publickey* dht_publickey_import(const uint8_t* dat, size_t dat_size);
OPENDHT_C_PUBLIC void dht_publickey_delete(dht_publickey* pk);
OPENDHT_C_PUBLIC int dht_publickey_export(const dht_publickey* pk, char* out, size_t* out_size);
OPENDHT_C_PUBLIC dht_blob* dht_publickey_export_blob(const dht_publickey* pk);
OPENDHT_C_PUBLIC int dht_publickey_unpack(dht_publickey* pk, const uint8_t* data, size_t size);
OPENDHT_C_PUBLIC dht_infohash dht_publickey_get_id(const dht_publickey* pk);
OPENDHT_C_PUBLIC dht_pkid dht_publickey_get_long_id(const dht_publickey* pk);
OPENDHT_C_PUBLIC bool dht_publickey_check_signature(
@@ -77,8 +88,11 @@ OPENDHT_C_PUBLIC dht_blob* dht_publickey_encrypt(const dht_publickey* pk, const
struct OPENDHT_C_PUBLIC dht_privatekey;
typedef struct dht_privatekey dht_privatekey;
OPENDHT_C_PUBLIC dht_privatekey* dht_privatekey_generate(unsigned key_length_bits);
OPENDHT_C_PUBLIC dht_privatekey* dht_privatekey_generate_ec(void);
OPENDHT_C_PUBLIC dht_privatekey* dht_privatekey_import(const uint8_t* dat, size_t dat_size, const char* password);
OPENDHT_C_PUBLIC int dht_privatekey_export(const dht_privatekey*, char* out, size_t* out_size, const char* password);
OPENDHT_C_PUBLIC dht_blob* dht_privatekey_export_blob(const dht_privatekey*, const char* password);
OPENDHT_C_PUBLIC dht_blob* dht_privatekey_sign(const dht_privatekey*, const uint8_t* data, size_t size);
OPENDHT_C_PUBLIC dht_publickey* dht_privatekey_get_publickey(const dht_privatekey*);
OPENDHT_C_PUBLIC dht_blob* dht_privatekey_decrypt(const dht_privatekey*, const char* data, size_t data_size);
OPENDHT_C_PUBLIC void dht_privatekey_delete(dht_privatekey*);
@@ -87,6 +101,8 @@ OPENDHT_C_PUBLIC void dht_privatekey_delete(dht_privatekey*);
struct OPENDHT_C_PUBLIC dht_certificate;
typedef struct dht_certificate dht_certificate;
OPENDHT_C_PUBLIC dht_certificate* dht_certificate_import(const uint8_t* dat, size_t dat_size);
OPENDHT_C_PUBLIC dht_blob* dht_certificate_export(const dht_certificate* certificate);
OPENDHT_C_PUBLIC dht_certificate* dht_certificate_ref(const dht_certificate* certificate);
OPENDHT_C_PUBLIC dht_infohash dht_certificate_get_id(const dht_certificate*);
OPENDHT_C_PUBLIC dht_pkid dht_certificate_get_long_id(const dht_certificate*);
OPENDHT_C_PUBLIC dht_publickey* dht_certificate_get_publickey(const dht_certificate*);
@@ -99,6 +115,7 @@ struct OPENDHT_C_PUBLIC dht_identity
};
typedef struct dht_identity dht_identity;
OPENDHT_C_PUBLIC dht_identity dht_identity_generate(const char* common_name, const dht_identity* ca);
OPENDHT_C_PUBLIC dht_identity dht_identity_generate_ec(const char* common_name, const dht_identity* ca);
OPENDHT_C_PUBLIC void dht_identity_delete(dht_identity*);
// dht::Value
@@ -109,6 +126,7 @@ OPENDHT_C_PUBLIC dht_value* dht_value_new(const uint8_t* data, size_t size);
OPENDHT_C_PUBLIC dht_value* dht_value_with_id_new(const uint8_t* data, size_t size, uint64_t value_id);
OPENDHT_C_PUBLIC dht_value* dht_value_new_from_string(const char* str);
OPENDHT_C_PUBLIC dht_value* dht_value_ref(const dht_value*);
OPENDHT_C_PUBLIC dht_value* dht_value_copy(const dht_value*);
OPENDHT_C_PUBLIC void dht_value_unref(dht_value*);
OPENDHT_C_PUBLIC dht_data_view dht_value_get_data(const dht_value* data);
OPENDHT_C_PUBLIC dht_value_id dht_value_get_id(const dht_value* data);
@@ -116,12 +134,53 @@ OPENDHT_C_PUBLIC dht_publickey* dht_value_get_owner(const dht_value* data);
OPENDHT_C_PUBLIC dht_infohash dht_value_get_recipient(const dht_value* data);
OPENDHT_C_PUBLIC const char* dht_value_get_user_type(const dht_value* data);
OPENDHT_C_PUBLIC void dht_value_set_user_type(dht_value* data, const char* user_type);
OPENDHT_C_PUBLIC uint16_t dht_value_get_type(const dht_value* value);
OPENDHT_C_PUBLIC void dht_value_set_type(dht_value* value, uint16_t type);
OPENDHT_C_PUBLIC uint16_t dht_value_get_seq(const dht_value* value);
OPENDHT_C_PUBLIC void dht_value_set_seq(dht_value* value, uint16_t seq);
OPENDHT_C_PUBLIC unsigned dht_value_get_priority(const dht_value* value);
OPENDHT_C_PUBLIC void dht_value_set_priority(dht_value* value, unsigned priority);
OPENDHT_C_PUBLIC const char* dht_value_get_push_type(const dht_value* value);
OPENDHT_C_PUBLIC void dht_value_set_push_type(dht_value* value, const char* push_type);
// callbacks
typedef bool (*dht_get_cb)(const dht_value* value, void* user_data);
typedef bool (*dht_value_cb)(const dht_value* value, bool expired, void* user_data);
typedef void (*dht_done_cb)(bool ok, void* user_data);
typedef void (*dht_shutdown_cb)(void* user_data);
typedef void (*dht_status_cb)(int ipv4_status, int ipv6_status, void* user_data);
typedef void (*dht_certificate_cb)(const dht_certificate* certificate, void* user_data);
typedef void (*dht_certificate_store_cb)(const dht_pkid* id, dht_certificate_cb emit, void* emit_data, void* user_data);
typedef struct dht_runner_context
{
dht_status_cb status_cb;
dht_certificate_store_cb certificate_store_cb;
dht_shutdown_cb released_cb;
void* user_data;
} dht_runner_context;
enum dht_query_field {
DHT_FIELD_ID = 1,
DHT_FIELD_TYPE = 2,
DHT_FIELD_SEQ = 4,
DHT_FIELD_OWNER = 8,
DHT_FIELD_USER_TYPE = 16
};
typedef struct dht_query_result
{
uint32_t fields;
uint64_t id;
uint16_t value_type;
uint16_t seq;
dht_infohash owner;
dht_data_view user_type;
} dht_query_result;
typedef bool (*dht_query_cb)(const dht_query_result* result, void* user_data);
struct dht_push_token;
typedef struct dht_push_token dht_push_token;
struct OPENDHT_C_PUBLIC dht_op_token;
typedef struct dht_op_token dht_op_token;
@@ -169,17 +228,50 @@ struct OPENDHT_C_PUBLIC dht_runner;
typedef struct dht_runner dht_runner;
OPENDHT_C_PUBLIC dht_runner* dht_runner_new(void);
OPENDHT_C_PUBLIC void dht_runner_delete(dht_runner* runner);
OPENDHT_C_PUBLIC int dht_runner_loop(dht_runner* runner);
OPENDHT_C_PUBLIC int dht_runner_join(dht_runner* runner);
/* Returns 0 on success, standard error code on failure */
OPENDHT_C_PUBLIC int dht_runner_run(dht_runner* runner, in_port_t port);
/* Returns 0 on success, standard error code on failure */
OPENDHT_C_PUBLIC int dht_runner_run_config(dht_runner* runner, in_port_t port, const dht_runner_config* config);
OPENDHT_C_PUBLIC int dht_runner_run_bind(dht_runner* runner,
const char* ipv4,
const char* ipv6,
in_port_t port,
const dht_runner_config* config,
const dht_runner_context* context);
OPENDHT_C_PUBLIC int dht_runner_connectivity_changed(dht_runner* runner);
OPENDHT_C_PUBLIC int dht_runner_register_type(
dht_runner* runner, uint16_t id, const char* name, uint64_t expiration_seconds, bool insecure);
OPENDHT_C_PUBLIC dht_publickey* dht_runner_get_publickey(const dht_runner* runner);
OPENDHT_C_PUBLIC void dht_runner_find_certificate(
dht_runner* runner, const dht_pkid* id, dht_certificate_cb callback, dht_shutdown_cb released_cb, void* user_data);
OPENDHT_C_PUBLIC int dht_runner_register_certificate(dht_runner* runner, const dht_certificate* certificate);
OPENDHT_C_PUBLIC void dht_runner_ping(
dht_runner* runner, struct sockaddr* addr, socklen_t addr_len, dht_done_cb done_cb, void* cb_user_data);
OPENDHT_C_PUBLIC void dht_runner_bootstrap(dht_runner* runner, const char* host, const char* service);
OPENDHT_C_PUBLIC void dht_runner_get(
dht_runner* runner, const dht_infohash* hash, dht_get_cb cb, dht_done_cb done_cb, void* cb_user_data);
OPENDHT_C_PUBLIC void dht_runner_get_filtered(dht_runner* runner,
const dht_infohash* hash,
const char* where,
dht_get_cb cb,
dht_done_cb done_cb,
void* user_data);
OPENDHT_C_PUBLIC void dht_runner_query(dht_runner* runner,
const dht_infohash* hash,
const char* query,
dht_query_cb cb,
dht_done_cb done_cb,
void* user_data);
OPENDHT_C_PUBLIC dht_op_token* dht_runner_listen(
dht_runner* runner, const dht_infohash* hash, dht_value_cb cb, dht_shutdown_cb done_cb, void* cb_user_data);
OPENDHT_C_PUBLIC dht_op_token* dht_runner_listen_filtered(dht_runner* runner,
const dht_infohash* hash,
const char* where,
dht_value_cb cb,
dht_shutdown_cb released_cb,
void* user_data);
OPENDHT_C_PUBLIC void dht_runner_cancel_listen(dht_runner* runner, const dht_infohash* hash, dht_op_token* token);
OPENDHT_C_PUBLIC void dht_runner_put(dht_runner* runner,
const dht_infohash* hash,
@@ -208,6 +300,16 @@ OPENDHT_C_PUBLIC bool dht_runner_is_running(const dht_runner* runner);
OPENDHT_C_PUBLIC in_port_t dht_runner_get_bound_port(const dht_runner* runner, sa_family_t af);
/** Returns null-terminated array that must be freed after use as well as each element */
OPENDHT_C_PUBLIC struct sockaddr** dht_runner_get_public_address(const dht_runner* runner);
OPENDHT_C_PUBLIC void dht_sockaddr_array_delete(struct sockaddr** addresses);
OPENDHT_C_PUBLIC int dht_runner_set_push_notification_token(dht_runner* runner, const char* value);
OPENDHT_C_PUBLIC int dht_runner_set_push_notification_topic(dht_runner* runner, const char* value);
OPENDHT_C_PUBLIC int dht_runner_set_push_notification_platform(dht_runner* runner, const char* value);
OPENDHT_C_PUBLIC dht_push_token* dht_runner_push_notification_received(dht_runner* runner,
const char* const* keys,
const char* const* values,
size_t count);
OPENDHT_C_PUBLIC int dht_push_token_poll(dht_push_token* token, int* result);
OPENDHT_C_PUBLIC void dht_push_token_delete(dht_push_token* token);
#ifdef __cplusplus
}
+59 -1
View File
@@ -27,6 +27,19 @@
[\fB\-y\fR \fIendpoint\fR]
[\fB\-z\fR \fIkey\fR]
[\fB\-u\fR \fIbundleid\fR]
[\fB\-\-no\-encryption\fR]
[\fB\-\-transport\-key\fR \fIfile\fR]
[\fB\-\-network\-authority\fR \fIkey\fR]
[\fB\-\-network\-credential\fR \fIfile\fR]
.br
.B dhtnode
\fB\-\-gen\-network\-authority\fR \fIprefix\fR
.br
.B dhtnode
\fB\-\-issue\-credential\fR \fIfile\fR
\fB\-\-authority\-key\fR \fIfile\fR
\fB\-\-transport\-key\fR \fIfile\fR
[\fB\-\-validity\fR \fIdays\fR]
.SH DESCRIPTION
Runs an OpenDHT node, with an interactive CLI (default) or as a daemon (with \fB'-d'\fP or \fB'-s'\fP).
Commands available in the interactive shell are:
@@ -122,7 +135,9 @@ Disable rate limiting (allows unlimited requests and storage).
Enable public stable mode.
.TP
\fB\-f\fP \fIpath\fP, \fB\-\-persist\fP \fIpath\fP
Specify a file path to persist/load the node state (routing table and values).
Specify a file path to persist/load the node state (routing table, values and
generated transport key). State is written to a temporary file and renamed
over the destination only after the write succeeds. Save failures are logged.
.TP
\fB\-n\fP \fInetwork_id\fP, \fB\-\-net\fP \fInetwork_id\fP
Specify the network id. This lets you connect to distinct networks and prevents
@@ -164,6 +179,40 @@ Device key for push notifications.
.TP
\fB\-u\fP \fIbundleid\fP, \fB\-\-bundleid\fP \fIbundleid\fP
Bundle ID for push notifications.
.SH TRANSPORT ENCRYPTION
The DHT protocol is encrypted with nodes supporting it, and stays in plaintext
with older nodes. The node ID derives from the transport key.
.TP
\fB\-\-no\-encryption\fP
Disable protocol encryption, using a random node ID like older versions.
.TP
\fB\-\-transport\-key\fP \fIfile\fP
Load the transport key from \fIfile\fP, or create it if it doesn't exist.
Keeps the same node ID across restarts. Otherwise, a key is generated and kept
in the node state if \fB\-\-persist\fP is used, or generated on each start.
.TP
\fB\-\-network\-authority\fP \fIkey\fP
Join a private network: only accept nodes with a credential issued by the
authority with public key \fIkey\fP (hexadecimal). Can be repeated.
Requires \fB\-\-network\-credential\fP and \fB\-\-transport\-key\fP.
.TP
\fB\-\-network\-credential\fP \fIfile\fP
Credential of our transport key, issued by a network authority.
.TP
\fB\-\-gen\-network\-authority\fP \fIprefix\fP
Generate a network authority, save its secret key to \fIprefix\fP.key and its
public key to \fIprefix\fP.pub, and exit. Refuses to overwrite either existing
output path, including a dangling symlink.
.TP
\fB\-\-issue\-credential\fP \fIfile\fP
Issue a credential for the key in \fB\-\-transport\-key\fP (created if needed),
signed with \fB\-\-authority\-key\fP, save it to \fIfile\fP and exit.
.TP
\fB\-\-authority\-key\fP \fIfile\fP
Network authority secret key used to issue credentials.
.TP
\fB\-\-validity\fP \fIdays\fP
Validity of issued credentials (default: 365 days).
.SH EXAMPLES
Run a node using bootstrap.jami.net as a bootstrap:
.PP
@@ -188,6 +237,15 @@ Use a proxy client to connect to a proxy server:
.EX
dhtnode --proxyclient dhtproxy.jami.net:80
.EE
.PP
Create a private network and run one of its nodes:
.PP
.EX
dhtnode --gen-network-authority net
dhtnode --issue-credential node1.cred --authority-key net.key --transport-key node1.key
dhtnode --transport-key node1.key --network-credential node1.cred \\
--network-authority $(cat net.pub) -b 192.0.2.1:4222
.EE
.SH AUTHORS
.TP
Program written by
+12 -11
View File
@@ -5,15 +5,16 @@ LABEL org.opencontainers.image.source=https://github.com/savoirfairelinux/opendh
COPY . opendht
RUN cd opendht && mkdir build && cd build \
&& cmake .. -DCMAKE_INSTALL_PREFIX=/usr \
-DBUILD_TESTING=Off \
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=On \
-DOPENDHT_C=On \
-DOPENDHT_PEER_DISCOVERY=On \
-DOPENDHT_PYTHON=On \
-DOPENDHT_TOOLS=On \
-DOPENDHT_PROXY_SERVER=On \
-DOPENDHT_PROXY_CLIENT=On \
-DOPENDHT_SYSTEMD=On \
&& make -j8 && make install \
&& cmake -GNinja .. -DCMAKE_INSTALL_PREFIX=/usr \
-DBUILD_TESTING=Off \
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=On \
-DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DOPENDHT_C=On \
-DOPENDHT_PEER_DISCOVERY=On \
-DOPENDHT_PYTHON=On \
-DOPENDHT_TOOLS=On \
-DOPENDHT_PROXY_SERVER=On \
-DOPENDHT_PROXY_CLIENT=On \
-DOPENDHT_SYSTEMD=On \
&& ninja && ninja install \
&& cd ../.. && rm -rf opendht
+2 -1
View File
@@ -10,6 +10,7 @@ ENV DESTDIR=/install
RUN cd opendht && mkdir build && cd build \
&& cmake .. -DCMAKE_INSTALL_PREFIX=/usr \
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=On \
-DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DBUILD_TESTING=Off \
-DOPENDHT_C=On \
-DOPENDHT_PEER_DISCOVERY=On \
@@ -20,7 +21,7 @@ RUN cd opendht && mkdir build && cd build \
-DOPENDHT_SYSTEMD=On \
&& make -j8 && make install
FROM alpine:3.23 AS install
FROM alpine:3.24 AS install
COPY --from=build /install /
RUN apk add --no-cache \
libstdc++ \
+8 -6
View File
@@ -1,4 +1,4 @@
FROM ubuntu:24.04
FROM ubuntu:26.04
LABEL maintainer="Adrien Béraud <adrien.beraud@savoirfairelinux.com>"
LABEL org.opencontainers.image.source=https://github.com/savoirfairelinux/opendht
@@ -8,23 +8,25 @@ RUN apt-get update && apt-get install -y \
&& echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections
RUN apt-get update && apt-get install -y \
build-essential pkg-config cmake git wget meson \
build-essential pkg-config cmake git wget meson ninja-build \
libtool \
python3-dev python3-setuptools python3-build python3-virtualenv cython3 \
libncurses5-dev libreadline-dev nettle-dev libcppunit-dev \
libgnutls28-dev libuv1-dev libjsoncpp-dev libargon2-dev \
libssl-dev libfmt-dev libasio-dev libmsgpack-dev \
libssl-dev libfmt-dev libasio-dev libmsgpack-cxx-dev \
libllhttp-dev libsimdutf-dev \
&& apt-get clean && rm -rf /var/lib/apt/lists/* /var/cache/apt/*
RUN mkdir /usr/include/nonstd \
&& wget https://raw.githubusercontent.com/martinmoene/expected-lite/master/include/nonstd/expected.hpp \
-O /usr/include/nonstd/expected.hpp
ARG RESTINIO_VERSION=0.7.10
RUN echo "*** Downloading RESTinio ***" \
&& mkdir restinio && cd restinio \
&& wget https://github.com/Stiffstream/restinio/releases/download/v0.7.8.1/restinio-0.7.8.1.tar.bz2 \
&& ls -l && tar -xjf restinio-0.7.8.1.tar.bz2 \
&& cd restinio-0.7.8.1/dev \
&& wget "https://github.com/Stiffstream/restinio/releases/download/v${RESTINIO_VERSION}/restinio-${RESTINIO_VERSION}.tar.bz2" \
&& ls -l && tar -xjf "restinio-${RESTINIO_VERSION}.tar.bz2" \
&& cd "restinio-${RESTINIO_VERSION}/dev" \
&& cmake -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=Off -DRESTINIO_SAMPLE=Off -DRESTINIO_BENCHMARK=Off \
-DRESTINIO_WITH_SOBJECTIZER=Off -DRESTINIO_DEP_STANDALONE_ASIO=system -DRESTINIO_DEP_LLHTTP=system \
-DRESTINIO_DEP_FMT=system -DRESTINIO_DEP_EXPECTED_LITE=system . \
+6 -5
View File
@@ -1,4 +1,4 @@
FROM alpine:3.23
FROM alpine:3.24
LABEL maintainer="Adrien Béraud <adrien.beraud@savoirfairelinux.com>"
LABEL org.opencontainers.image.source=https://github.com/savoirfairelinux/opendht
@@ -7,7 +7,7 @@ RUN apk add --no-cache \
cython python3-dev py3-setuptools \
ncurses-dev readline-dev nettle-dev \
cppunit-dev gnutls-dev jsoncpp-dev \
argon2-dev openssl-dev fmt-dev \
argon2-dev openssl-dev fmt-dev simdutf-dev \
llhttp-dev asio-dev msgpack-cxx-dev \
&& rm -rf /var/cache/apk/*
@@ -15,11 +15,12 @@ RUN mkdir /usr/include/nonstd \
&& wget https://raw.githubusercontent.com/martinmoene/expected-lite/master/include/nonstd/expected.hpp \
-O /usr/include/nonstd/expected.hpp
ARG RESTINIO_VERSION=0.7.10
RUN echo "*** Downloading RESTinio ***" \
&& mkdir restinio && cd restinio \
&& wget https://github.com/Stiffstream/restinio/releases/download/v0.7.8.1/restinio-0.7.8.1.tar.bz2 \
&& ls -l && tar -xjf restinio-0.7.8.1.tar.bz2 \
&& cd restinio-0.7.8.1/dev \
&& wget "https://github.com/Stiffstream/restinio/releases/download/v${RESTINIO_VERSION}/restinio-${RESTINIO_VERSION}.tar.bz2" \
&& ls -l && tar -xjf "restinio-${RESTINIO_VERSION}.tar.bz2" \
&& cd "restinio-${RESTINIO_VERSION}/dev" \
&& cmake -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=Off -DRESTINIO_SAMPLE=Off -DRESTINIO_BENCHMARK=Off \
-DRESTINIO_WITH_SOBJECTIZER=Off -DRESTINIO_DEP_STANDALONE_ASIO=system -DRESTINIO_DEP_LLHTTP=system \
-DRESTINIO_DEP_FMT=system -DRESTINIO_DEP_EXPECTED_LITE=system . \
+4 -3
View File
@@ -15,11 +15,12 @@ RUN mkdir /usr/include/nonstd \
&& wget https://raw.githubusercontent.com/martinmoene/expected-lite/master/include/nonstd/expected.hpp \
-O /usr/include/nonstd/expected.hpp
ARG RESTINIO_VERSION=0.7.10
RUN echo "*** Downloading RESTinio ***" \
&& mkdir restinio && cd restinio \
&& wget https://github.com/Stiffstream/restinio/releases/download/v0.7.8.1/restinio-0.7.8.1.tar.bz2 \
&& ls -l && tar -xjf restinio-0.7.8.1.tar.bz2 \
&& cd restinio-0.7.8.1/dev \
&& wget "https://github.com/Stiffstream/restinio/releases/download/v${RESTINIO_VERSION}/restinio-${RESTINIO_VERSION}.tar.bz2" \
&& ls -l && tar -xjf "restinio-${RESTINIO_VERSION}.tar.bz2" \
&& cd "restinio-${RESTINIO_VERSION}/dev" \
&& cmake -GNinja -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=Off -DRESTINIO_SAMPLE=Off -DRESTINIO_BENCHMARK=Off \
-DRESTINIO_WITH_SOBJECTIZER=Off -DRESTINIO_DEP_STANDALONE_ASIO=system -DRESTINIO_DEP_LLHTTP=system \
-DRESTINIO_DEP_FMT=system -DRESTINIO_DEP_EXPECTED_LITE=system . \
+17 -3
View File
@@ -18,15 +18,29 @@ RUN apt-get update && apt-get install -y \
RUN pip3 install meson Cython
ARG SIMDUTF_VERSION=8.0.0
RUN mkdir simdutf \
&& wget -q "https://github.com/simdutf/simdutf/archive/refs/tags/v${SIMDUTF_VERSION}.tar.gz" -O simdutf.tar.gz \
&& tar -xzf simdutf.tar.gz -C simdutf --strip-components=1 \
&& cmake -S simdutf -B simdutf/build -DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=/usr/local -DCMAKE_POSITION_INDEPENDENT_CODE=ON \
-DBUILD_SHARED_LIBS=OFF -DSIMDUTF_TESTS=OFF -DSIMDUTF_TOOLS=OFF \
&& cmake --build simdutf/build --parallel 2 \
&& cmake --install simdutf/build \
&& install -D simdutf/LICENSE-MIT /usr/share/doc/simdutf/LICENSE-MIT \
&& install -D simdutf/LICENSE-APACHE /usr/share/doc/simdutf/LICENSE-APACHE \
&& rm -rf simdutf simdutf.tar.gz
RUN mkdir /usr/include/nonstd \
&& wget https://raw.githubusercontent.com/martinmoene/expected-lite/master/include/nonstd/expected.hpp \
-O /usr/include/nonstd/expected.hpp
ARG RESTINIO_VERSION=0.7.10
RUN echo "*** Downloading RESTinio ***" \
&& mkdir restinio && cd restinio \
&& wget https://github.com/Stiffstream/restinio/releases/download/v.0.7.3/restinio-0.7.3.tar.bz2 \
&& ls -l && tar -xjf restinio-0.7.3.tar.bz2 \
&& cd restinio-0.7.3/dev \
&& wget "https://github.com/Stiffstream/restinio/releases/download/v${RESTINIO_VERSION}/restinio-${RESTINIO_VERSION}.tar.bz2" \
&& ls -l && tar -xjf "restinio-${RESTINIO_VERSION}.tar.bz2" \
&& cd "restinio-${RESTINIO_VERSION}/dev" \
&& cmake -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=Off -DRESTINIO_SAMPLE=Off -DRESTINIO_BENCHMARK=Off \
-DRESTINIO_WITH_SOBJECTIZER=Off -DRESTINIO_DEP_STANDALONE_ASIO=system -DRESTINIO_DEP_LLHTTP=system \
-DRESTINIO_DEP_FMT=system -DRESTINIO_DEP_EXPECTED_LITE=system . \
+7 -6
View File
@@ -1,4 +1,4 @@
FROM ubuntu:24.04
FROM ubuntu:26.04
LABEL maintainer="Adrien Béraud <adrien.beraud@savoirfairelinux.com>"
LABEL org.opencontainers.image.source=https://github.com/savoirfairelinux/opendht
@@ -10,9 +10,9 @@ RUN apt-get update && apt-get install -y \
RUN apt-get update \
&& apt-get install -y llvm llvm-dev lldb clang gdb make cmake meson pkg-config \
libtool git wget libncurses5-dev libreadline-dev \
nettle-dev libgnutls28-dev libuv1-dev libmsgpack-dev libjsoncpp-dev python3-dev \
nettle-dev libgnutls28-dev libuv1-dev libmsgpack-cxx-dev libjsoncpp-dev python3-dev \
python3-setuptools libcppunit-dev python3-pip python3-build python3-virtualenv \
libssl-dev libargon2-dev \
libssl-dev libargon2-dev libllhttp-dev libsimdutf-dev \
libfmt-dev libasio-dev \
&& apt-get remove -y gcc g++ && apt-get autoremove -y \
&& apt-get clean && rm -rf /var/lib/apt/lists/* /var/cache/apt/*
@@ -26,11 +26,12 @@ RUN mkdir /usr/include/nonstd \
&& wget https://raw.githubusercontent.com/martinmoene/expected-lite/master/include/nonstd/expected.hpp \
-O /usr/include/nonstd/expected.hpp
ARG RESTINIO_VERSION=0.7.10
RUN echo "*** Downloading RESTinio ***" \
&& mkdir restinio && cd restinio \
&& wget https://github.com/Stiffstream/restinio/releases/download/v0.7.8.1/restinio-0.7.8.1.tar.bz2 \
&& ls -l && tar -xjf restinio-0.7.8.1.tar.bz2 \
&& cd restinio-0.7.8.1/dev \
&& wget "https://github.com/Stiffstream/restinio/releases/download/v${RESTINIO_VERSION}/restinio-${RESTINIO_VERSION}.tar.bz2" \
&& ls -l && tar -xjf "restinio-${RESTINIO_VERSION}.tar.bz2" \
&& cd "restinio-${RESTINIO_VERSION}/dev" \
&& cmake -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=Off -DRESTINIO_SAMPLE=Off -DRESTINIO_BENCHMARK=Off \
-DRESTINIO_WITH_SOBJECTIZER=Off -DRESTINIO_DEP_STANDALONE_ASIO=system -DRESTINIO_DEP_LLHTTP=system \
-DRESTINIO_DEP_FMT=system -DRESTINIO_DEP_EXPECTED_LITE=system . \
+56
View File
@@ -0,0 +1,56 @@
FROM ubuntu:24.04
LABEL maintainer="Adrien Béraud <adrien.beraud@savoirfairelinux.com>"
LABEL org.opencontainers.image.source=https://github.com/savoirfairelinux/opendht
RUN apt-get update && apt-get install -y \
dialog apt-utils \
&& apt-get clean \
&& echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections
RUN apt-get update && apt-get install -y \
build-essential pkg-config cmake git wget meson ninja-build patchelf \
libtool \
python3-dev python3-setuptools python3-build python3-virtualenv cython3 \
libncurses5-dev libreadline-dev nettle-dev libcppunit-dev \
libgnutls28-dev libuv1-dev libjsoncpp-dev libargon2-dev \
libssl-dev libfmt-dev libasio-dev libmsgpack-cxx-dev \
&& apt-get clean && rm -rf /var/lib/apt/lists/* /var/cache/apt/*
ARG LLHTTP_VERSION=9.2.1
RUN mkdir llhttp \
&& wget -q "https://github.com/nodejs/llhttp/archive/refs/tags/release/v${LLHTTP_VERSION}.tar.gz" -O llhttp.tar.gz \
&& tar -xzf llhttp.tar.gz -C llhttp --strip-components=1 \
&& cmake -S llhttp -B llhttp/build -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr \
&& cmake --build llhttp/build --parallel 2 \
&& cmake --install llhttp/build \
&& rm -rf llhttp llhttp.tar.gz
ARG SIMDUTF_VERSION=8.0.0
RUN mkdir simdutf \
&& wget -q "https://github.com/simdutf/simdutf/archive/refs/tags/v${SIMDUTF_VERSION}.tar.gz" -O simdutf.tar.gz \
&& tar -xzf simdutf.tar.gz -C simdutf --strip-components=1 \
&& cmake -S simdutf -B simdutf/build -DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=/usr/local -DCMAKE_POSITION_INDEPENDENT_CODE=ON \
-DBUILD_SHARED_LIBS=OFF -DSIMDUTF_TESTS=OFF -DSIMDUTF_TOOLS=OFF \
&& cmake --build simdutf/build --parallel 2 \
&& cmake --install simdutf/build \
&& install -D simdutf/LICENSE-MIT /usr/share/doc/simdutf/LICENSE-MIT \
&& install -D simdutf/LICENSE-APACHE /usr/share/doc/simdutf/LICENSE-APACHE \
&& rm -rf simdutf simdutf.tar.gz
RUN mkdir /usr/include/nonstd \
&& wget https://raw.githubusercontent.com/martinmoene/expected-lite/master/include/nonstd/expected.hpp \
-O /usr/include/nonstd/expected.hpp
ARG RESTINIO_VERSION=0.7.10
RUN echo "*** Downloading RESTinio ***" \
&& mkdir restinio && cd restinio \
&& wget "https://github.com/Stiffstream/restinio/releases/download/v${RESTINIO_VERSION}/restinio-${RESTINIO_VERSION}.tar.bz2" \
&& ls -l && tar -xjf "restinio-${RESTINIO_VERSION}.tar.bz2" \
&& cd "restinio-${RESTINIO_VERSION}/dev" \
&& cmake -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=Off -DRESTINIO_SAMPLE=Off -DRESTINIO_BENCHMARK=Off \
-DRESTINIO_WITH_SOBJECTIZER=Off -DRESTINIO_DEP_STANDALONE_ASIO=system -DRESTINIO_DEP_LLHTTP=system \
-DRESTINIO_DEP_FMT=system -DRESTINIO_DEP_EXPECTED_LITE=system . \
&& make -j2 && make install \
&& cd ../../ && rm -rf restinio* \
&& ldconfig
+1
View File
@@ -8,6 +8,7 @@ RUN cd opendht && mkdir build && cd build \
&& cmake .. -DCMAKE_INSTALL_PREFIX=/usr \
-DBUILD_TESTING=Off \
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=On \
-DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DOPENDHT_C=On \
-DOPENDHT_PEER_DISCOVERY=On \
-DOPENDHT_PYTHON=On \
+1
View File
@@ -7,6 +7,7 @@ COPY . opendht
RUN cd opendht && mkdir build && cd build \
&& cmake .. -DCMAKE_INSTALL_PREFIX=/usr \
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=On \
-DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DOPENDHT_C=On \
-DOPENDHT_PEER_DISCOVERY=On \
-DOPENDHT_PYTHON=On \
+20
View File
@@ -0,0 +1,20 @@
FROM ghcr.io/savoirfairelinux/opendht/opendht-deps-noble:latest
LABEL maintainer="Adrien Béraud <adrien.beraud@savoirfairelinux.com>"
LABEL org.opencontainers.image.source=https://github.com/savoirfairelinux/opendht
COPY . opendht
RUN cd opendht && mkdir build && cd build \
&& cmake .. -DCMAKE_INSTALL_PREFIX=/usr \
-DBUILD_TESTING=Off \
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=On \
-DCMAKE_BUILD_TYPE=RelWithDebInfo \
-DOPENDHT_C=On \
-DOPENDHT_PEER_DISCOVERY=On \
-DOPENDHT_PYTHON=On \
-DOPENDHT_TOOLS=On \
-DOPENDHT_PROXY_SERVER=On \
-DOPENDHT_PROXY_CLIENT=On \
-DOPENDHT_SYSTEMD=On \
&& make -j8 && make install \
&& cd ../.. && rm -rf opendht
+564
View File
@@ -0,0 +1,564 @@
# Transport encryption: cryptographic design
This document specifies how OpenDHT encrypts and authenticates the UDP DHT protocol between nodes (the "transport"), and how private networks restrict participation with a small public-key infrastructure.
Implementation: [include/opendht/transport.h](../include/opendht/transport.h), [src/transport.cpp](../src/transport.cpp), and the send/receive paths in [src/network_engine.cpp](../src/network_engine.cpp).
## 1. Scope
### Goals
- **Confidentiality and integrity** of DHT protocol messages exchanged between two nodes that both support encryption.
- **Authenticated node IDs**: a node can only send encrypted messages under the node ID that derives from its own key.
- **Backward compatibility**: nodes without this feature keep working with new nodes, in plaintext.
- **Optional private networks**: when configured, a node only talks to nodes holding a credential signed by a trusted authority.
- **Permanent membership**: grants do not expire. Without a compromise or an intentional membership change, no grant renewal, authority rotation, or operator maintenance is required.
- **Forward secrecy**: recorded traffic stays confidential after a node's long-term key is compromised, once the ephemeral keys used for it are erased (at most one hour later). The first packet to a peer has weaker guarantees ([§5.2](#52-key-schedule)).
- **Address validation**: a Session or Data envelope proves that its sender receives packets at its source address. It replaces the legacy write tokens between encrypted nodes ([§6.6](#66-write-authorization)).
- **Low cost**: routing discovery also establishes the session, with a few X25519 operations per peer every 30 minutes. Once a session is established, 30 bytes of overhead per packet, minus the 40-byte token in messages that carried one.
### Non-goals
- **End-to-end protection of values.** Encryption is hop-by-hop. Nodes storing a value see it in clear. Use `SecureDht` (signed or encrypted values) for end-to-end protection. `SecureDht` identities are unrelated to transport keys.
- **Hiding metadata**: packet sizes, timing and addresses are visible, and so are the sender's public keys until a session is established.
- The REST proxy (`DhtProxyServer`/`DhtProxyClient`) and local peer discovery are not covered.
## 2. Notation
| Notation | Meaning |
|---|---|
| `a ‖ b` | Concatenation of byte strings |
| `"text"` | ASCII bytes, without terminator |
| `u32be(x)` | Unsigned integer, big-endian, 4 bytes |
| `sk_A`, `pk_A` | Static transport secret and public key of node A |
| `esk_A`, `epk_A` | Ephemeral secret and public key of node A for the address of its peer |
| `DH(x, Y)` | `X25519(x, Y)` |
| `N` | Network ID (`Config::network`, 32-bit, `0` by default) |
| `K_A→B` | Key encrypting envelopes from A to B |
All keys are 32 bytes. Integers on the wire are big-endian.
## 3. Primitives
| Purpose | Primitive | Reference |
|---|---|---|
| Key agreement | X25519 | RFC 7748 |
| Key derivation | HKDF-SHA256 | RFC 5869 |
| Packet encryption | ChaCha20-Poly1305 AEAD, 96-bit nonce, 128-bit tag | RFC 8439 |
| Node ID | SHA-256, truncated to 160 bits | FIPS 180-4 |
| Credential signature | Ed25519 | RFC 8032 |
| Randomness | `gnutls_rnd` (`GNUTLS_RND_KEY` for keys, `GNUTLS_RND_NONCE` for nonces) | |
Primitives come from Nettle (libnettle and libhogweed) and GnuTLS, which OpenDHT already depends on. All primitives are used through their standard interfaces with no modification. The authentication tag is compared in constant time. Secret keys are erased with `gnutls_memset`.
## 4. Keys and identities
### 4.1 Transport key
Each node has a static X25519 key pair, its long-term identity:
```
sk = 32 random bytes
pk = X25519(sk, 9)
```
The key is either given in `Config::transport.key`, or generated on start. A generated key is saved in the node state when `Config::persist_path` is set (see [§8](#8-key-persistence)). Otherwise, it changes on every restart, and so does the node ID.
The transport key is used only for the transport. It is independent of the `SecureDht` identity (an RSA certificate).
### 4.2 Node ID
When transport encryption is enabled, the node ID is derived from the public key:
```
nodeId(pk) = SHA-256("opendht-node-id-v1" ‖ pk)[0..20)
```
`Config::node_id` is ignored in that case. This binding is what authenticates node IDs: a receiver checks that the ID claimed inside a decrypted message equals `nodeId(pk)` of the static key the envelope was authenticated with (see [§6.5](#65-receiving)). Holding the ID therefore requires holding `sk`.
When encryption is disabled (`TransportConfig::enabled = false`), the node uses the legacy behavior: a random ID, or `Config::node_id`.
### 4.3 Network authority (private networks)
An authority is an Ed25519 key pair. Its secret is the 32-byte Ed25519 seed. Its public key identifies the private network, and nodes are configured with a list of trusted authority public keys (`TransportConfig::authorities`). A network is private if and only if this list is non-empty.
Any one trusted authority can grant membership. A node can trust multiple authorities simultaneously, allowing old and replacement grants to coexist during an explicit transition. There is no scheduled authority rotation.
### 4.4 Credential (private networks)
A membership grant, called a credential in the API and CLI, is an authority's signature over a transport public key. It never expires and contains no timestamp. It is 97 bytes:
```
offset size field
0 1 version = 0x02
1 32 pk transport public key of the node
33 64 signature Ed25519 signature by the authority
```
The signature covers:
```
"opendht-credential-v2" ‖ 0x02 ‖ pk
```
A credential is **valid** when its signature verifies under at least one trusted authority. Admission does not depend on a system clock. The same grant can be reused across restarts and session-key rotations without an update.
Version 1 expiring credentials are rejected, not reinterpreted as permanent grants. Migrating from that format requires issuing a version 2 grant once.
Credentials are not bound to a network ID. An authority key therefore defines a network: use distinct authorities for distinct networks.
## 5. Key agreement
### 5.1 Ephemeral keys
Each node has an epoch secret `S`: 32 random bytes, replaced every 30 minutes (an *epoch*). Its ephemeral key for a peer at address `addr` derives from it:
```
addr = IP (4 bytes for IPv4, including IPv4-mapped IPv6, else 16 bytes) ‖ port (2 bytes, network order)
esk(addr) = HMAC-SHA256(key = S, "opendht-address-key-v1" ‖ addr)
epk(addr) = X25519(esk(addr), 9)
```
- The node always sends to `addr` with `epk(addr)` from its **current** epoch.
- It still accepts envelopes addressed to its key from the **previous** epoch, so peers have one epoch to learn the new key.
- On rotation, the previous epoch secret and every key derived from it are erased.
An ephemeral secret therefore lives at most one hour. Epoch secrets are never persisted, and a restart starts with a new one.
Nodes learn each other's ephemeral key from the header of every envelope ([§6.1](#61-envelope)). A node records a peer's ephemeral key only from an envelope it authenticated.
`epk(addr)` is only ever sent to `addr`. Learning it therefore requires receiving packets at `addr`, or observing the path to it. This is what makes a Session envelope a proof of address ([§6.6](#66-write-authorization)). Deriving keys per address costs one more X25519 base-point multiplication per new address, and nothing per packet.
### 5.2 Key schedule
Each envelope from a sender A to a receiver B is encrypted with a key `K_A→B`, derived with HKDF-SHA256 (RFC 5869):
```
prk = HMAC-SHA256(key = "opendht-transport-v2", ikm)
K_A→B = HMAC-SHA256(key = prk, info ‖ 0x01)
```
The input keying material `ikm` and `info` depend on what A knows about B.
**Session**: A knows B's ephemeral key `epk_B`:
```
ikm = DH(esk_A, epk_B) ‖ DH(esk_A, pk_B) ‖ DH(sk_A, epk_B) ‖ DH(sk_A, pk_B)
info = u32be(N) ‖ 0x02 ‖ pk_A ‖ epk_A ‖ pk_B ‖ epk_B
```
**Initial**: A doesn't know B's ephemeral key, typically on first contact:
```
ikm = DH(esk_A, pk_B) ‖ DH(sk_A, pk_B)
info = u32be(N) ‖ 0x01 ‖ pk_A ‖ epk_A ‖ pk_B
```
B computes the same values with its own secrets, since `DH(x_A, Y_B) = DH(y_B, X_A)`. Any DH result equal to zero (a low-order public key) is rejected.
Properties:
- **Directional keys.** `info` orders the keys as sender, then receiver, so `K_A→B ≠ K_B→A`. Each node derives both from the same four DH values.
- **Authentication.** Every key includes `DH(sk_A, pk_B)`, so only the holders of `sk_A` or `sk_B` can produce or read the envelope. Every envelope thus proves possession of the sender's static key.
- **Forward secrecy.** A Session key includes `DH(esk_A, epk_B)`. Once both ephemeral secrets are erased, the key cannot be recomputed, even with both static secrets.
- **Key-compromise impersonation resistance.** With `sk_B`, an attacker still cannot impersonate A to B in a Session envelope: it cannot compute `DH(sk_A, epk_B)` without `sk_A` or `esk_B`.
- **Initial keys are weaker**, as for the first message of Noise IK. Without `epk_B`, an Initial key is only forward-secret against a later compromise of `sk_A`, not of `sk_B`. It also allows impersonating A to B with `sk_B`. Initial envelopes are only sent until A learns `epk_B`, usually for a single packet.
- **Binding.** `info` includes `N`, the envelope type and all public keys involved. Nodes with different network IDs cannot decrypt each other's packets.
- **Cost.** A Session key costs four X25519 operations, plus one for our ephemeral key. It is cached in the epoch, so it is derived once per peer address and ephemeral key pair, once the peer has proven to receive at that address ([§6.5](#65-receiving)). An Initial key costs two X25519 operations, plus one for our ephemeral key, and is not cached. Each epoch caches at most 16384 sessions, evicting the least recently used; beyond that, keys are derived for each packet.
## 6. Wire protocol
### 6.1 Envelope
Initial and Session envelopes carry the sender keys:
```
offset size field
0 1 marker = 0xC1
1 1 type: 1 = Initial, 2 = Session, 3 = Hello; | 0x80 if a credential is included
2 32 pk_sender
34 32 epk_sender
66 12 nonce (random)
78 len ciphertext
78+len 16 Poly1305 tag
```
Once the session is confirmed ([§6.4](#64-sending)), Data envelopes leave them out. The receiver finds the session keys from the source address:
```
offset size field
0 1 marker = 0xC1
1 1 type = 4 (Data), never with a credential
2 12 nonce (random)
14 len ciphertext
14+len 16 Poly1305 tag
```
- The ciphertext is `ChaCha20-Poly1305(K_sender→receiver, nonce, plaintext, aad = header)`.
- The header is bytes `[0, 66)`, or `[0, 2)` for Data: marker, type, and the sender keys. Because it is the associated data, an attacker cannot change the type, the credential flag or the sender keys without the tag check failing.
- The plaintext is the msgpack-encoded DHT message. With the credential flag, it is preceded by the sender's 97-byte credential.
- The overhead is 30 bytes for Data, 94 bytes for Initial and Session, or 191 bytes with a credential.
- Nonces are 96 bits from the system CSPRNG. For `n` packets under one key, the probability of a nonce collision is about $n^2 / 2^{97}$. Keys change at least every epoch, so this is negligible.
`0xC1` is the one byte value that msgpack never uses. Every legacy OpenDHT message is a msgpack map, which starts with `0x80`–`0x8F` or `0xDE`–`0xDF`. This marker therefore unambiguously separates envelopes from plaintext messages:
- legacy nodes fail to parse an envelope and drop it, without blacklisting the sender;
- new nodes still parse plaintext messages normally.
### 6.2 Hello (private networks only)
`Hello` introduces a node to a peer whose key is unknown. This happens when pinging a bare address, typically a bootstrap node. A Hello is **not encrypted**:
```
0xC1 ‖ 0x03 ‖ pk_sender ‖ epk_sender ‖ credential (97 bytes) ‖ msgpack ping
```
The sender cannot encrypt to a key it doesn't know, and X25519 keys cannot sign, so **a Hello proves nothing about its sender**. Its credential travels in clear and can be copied. The receiver therefore:
- answers only a ping, and only if the credential is valid for `pk_sender`;
- answers with a Session envelope to `pk_sender` and `epk_sender`, so only the holder of `sk_sender` can read the answer;
- keeps no state from the Hello: it does not add, move or authorize the claimed node.
The sender proves possession of its key with its next envelope.
### 6.3 Key advertisement in plaintext messages
New nodes add fields to plaintext msgpack messages. Legacy nodes ignore unknown fields.
| Field | Location | Content |
|---|---|---|
| `k` | top level | Sender's transport public key (bin, 32 bytes) |
| `k4` | reply body, next to `n4` | Concatenated public keys of the listed IPv4 nodes, 32 bytes per node, all-zero if unknown |
| `k6` | reply body, next to `n6` | Same for IPv6 |
Rules for using these fields:
- **`k` is sent only when it is needed**: in public mode, in plaintext messages to nodes whose key is not yet known. Encrypted messages do not need it, because the sender key is in the envelope header.
- **Listed keys are checked before use.** A key from `k`, `k4` or `k6` is attached to a node only if `nodeId(key)` equals the node's ID. Otherwise it is ignored.
- **The binding limits what a dishonest node can do with a node list.** It cannot attach a forged key to an existing node ID. It can still list a wrong address, which at worst causes encrypted packets to be sent to the wrong host. That host cannot read them.
- `k4` and `k6` are omitted if no listed node has a known key. They are ignored if their length is not exactly 32 × the number of listed nodes.
### 6.4 Sending
To send a message to node B:
1. If transport encryption is disabled, send plaintext.
2. If B's static key is known (B has a peer record), send an envelope with our ephemeral key for B's address:
- a Data envelope if our session with B's address is **confirmed**;
- otherwise a Session envelope if B's ephemeral key is known, or else an Initial envelope. In a private network, both include our credential.
3. Else, in a private network:
- if B's ID is unknown (a ping to a bare address), send a `Hello`;
- otherwise refuse to send (`EPERM`).
4. Else (public network, key unknown), send plaintext, including `k`.
Once a node's key is known, nothing is ever sent to it in plaintext. Failed key agreement, including a low-order public key, refuses the send (`EPERM`); it never falls back to plaintext.
**Confirmation.** A session with an address is confirmed when B shows that it holds the session keys and accepted us:
- B answers, in a Session or Data envelope, a request we sent to its node ID **in a Session or Data envelope**. B cached the session when it opened our envelope ([§6.5](#65-receiving)), and answered with keys of our current epoch. Requests sent in an Initial envelope or a Hello are excluded: B answered them without caching a session, so our next request is a Session envelope, and its answer confirms;
- or we receive a Data envelope under our current epoch. B only sends one after we confirmed it the same way.
The confirmation is lost when the session keys change: on epoch rotation, or when B's ephemeral key changes through accepted traffic. An Initial request leaves existing state untouched and is answered without reusing a cached session or its confirmation.
**Establishment.** Only `ping` and `find_node` requests may precede address validation. Application requests wait until B's ephemeral key is known, or a successful control reply identifies a genuinely keyless legacy peer. Learning the ephemeral from an accepted Initial reply is enough to send a Session: neither address-indexed cache admission nor Data confirmation requires another handshake. Direct NetworkEngine application requests share one engine-owned ping; canceling a caller's ping cannot strand them. Private bare-address bootstrap remains a ping-only Hello.
**Recovery.** At the first response timeout, a request forgets B's ephemeral key and starts recovering the session, for example after B restarts. Application requests to B are withheld, including new requests, while one internal ping retries in Initial envelopes (with our credential in a private network). `find_node` remains available and its successful reply can also release the waiters. An accepted reply completes its own request before held work is released, so a delayed original reply is not retransmitted during attachment. Released requests receive fresh response timers; withholding consumes no transmission attempts. The internal probe's first timeout notifies waiting operations so searches can replace unresponsive peers promptly. Probe exhaustion or rejection terminates the waiters. Their payloads never enter Initial envelopes, and no `401` challenge/retry exchange is used.
**Value searches.** A cold peer first receives `find_node(search_target)`. Its reply supplies its ephemeral and closer candidates in the same exchange. The existing distance-ordered search dispatch contacts closer candidates immediately, within the existing four-active-peer limit; it does not wait for a SELECT reply from the farther responder. This routing probe has separate state: it cannot satisfy a GET, pagination, or an empty-result callback. Responders remain eligible for subsequent value retrieval. A cold holder typically needs three exchanges (`find_node`, SELECT IDs, GET by ID), while a session-ready holder retains its usual two. A routing probe's first timeout frees its active search slot while protocol retries continue, rather than waiting for final expiration.
Value data is fragmented so that each packet fits in the 1280-byte IPv6 minimum MTU: `1280 − 40 − 8 = 1232` bytes of UDP payload, less 48 bytes reserved for the msgpack framing of a fragment, less the overhead of the envelope the fragments are sent in. Fragments carry `1184` bytes of value data in plaintext, `1154` in Data envelopes, `1090` in Initial or Session envelopes, and `993` with a credential.
This sizes value fragments only. Complete replies and node lists are not yet subject to a whole-datagram MTU budget; node-list trimming and packet-size enforcement remain separate work.
### 6.5 Receiving
For each received packet, a node applies the following checks in order:
1. **Address filters.** Drop the packet if it comes from a martian or blacklisted address. This is unchanged.
2. **Envelopes.** If the packet starts with `0xC1`:
1. If encryption is disabled, drop it.
2. Drop it if its type is unknown, or if `pk_sender` is our own key. This rejects reflected packets.
3. **Hello:**
- Drop it outside private networks, or if it carries no message.
- Apply the public-key limit, then verify the credential: its key must equal `pk_sender` and it must be valid. Drop the packet on failure.
- Continue with steps 4 and 5, then answer as described in [§6.2](#62-hello-private-networks-only). The Hello is not processed any further.
4. **Initial:** apply the public-key limit, derive the Initial key from `pk_sender` and `epk_sender`, and decrypt.
5. **Session:** first try the keys cached in each epoch for (source address, `pk_sender`, `epk_sender`). Otherwise, apply the public-key limit, then for our current epoch then the previous one: derive the keys with our ephemeral key for the source address, and decrypt. Keep newly derived keys pending until the packet passes authorization and message validation.
6. **Data:** decrypt with the session cached for the source address in our current epoch, then in the previous one. Its `pk_sender` and `epk_sender` are those of the session. This needs no key derivation and no public-key limit.
7. Drop the packet if it was not decrypted.
8. **Credential flag:** strip the 97-byte credential. In a private network, verify it as for Hello, unless the peer is already authorized; on success, record the peer as authorized, without an expiry.
**No credential:** in a private network, drop the packet unless the peer is already authorized.
9. Record `epk_sender` on a temporary peer record. It is **validated for the source address** if the envelope was a Session or Data envelope: opening it required our ephemeral key for that address, which was only ever sent there. Existing peer state is updated only after parsing, ID binding and the processing checks below.
**Session cache.** The epoch caches one session per source address, including its verified admission in private networks. Authorization therefore survives the release of a client-mode peer's node record. Newly derived Session keys enter the cache only after credential authorization, parsing, network and node ID checks, and basic fragment validation. Replies must match a pending request or listener. Admission is per packet, so subsequent Data fragments can decrypt before the whole message is assembled. Sending to the exact address for which the peer's ephemeral key is validated can also cache a session. Answers to Initial and Hello envelopes never use or change cached sessions: their source address may be spoofed. If both the peer record and cached admission are lost, the same permanent credential is verified again; no replacement grant is needed.
3. **Plaintext in a private network.** Drop any packet that is not an envelope.
4. **Parsing.** Parse the msgpack message and apply the network ID check. This is unchanged.
5. **Envelope ID binding.** For an envelope, the message's node ID must equal `nodeId(pk_sender)`, otherwise drop the message. Value data fragments are exempt because they carry no node ID. Fragment buffers are keyed by source address, transaction ID and sender key, with a separate keyless buffer for plaintext. Only parts from the same sender can join a header, and a response header must match a pending request or listener. Foreign-key or plaintext parts cannot reserve the expected sender's buffer, even when they arrive first. Initial envelopes carrying value fragments or fragmented-message headers are dropped: encrypted value data requires a Session or Data envelope.
6. **Plaintext with `k`.** For a plaintext message carrying `k`, attach the key to the sender if it matches the sender's ID.
7. **Processing.**
- A Session or Data envelope can move the sender's node record to the packet's source address.
- Initial requests use transient node records. Only `ping` and `find_node` receive successful replies; GET (including SELECT) and writes receive a short `401` error, and value updates are dropped. Rejected GETs never invoke storage or serialize values. These requests do not add or move an established node, count as activity, or change its ephemeral key or session confirmation.
- Initial replies and errors are accepted only for a pending request at the expected source address, never for a listener. They cannot move a known keyed node to a different address.
- Replies and Data envelopes can confirm the session ([§6.4](#64-sending)).
A **plaintext** message claiming the ID of a node whose key is known is untrusted, even from that node's recorded address:
- it does not move the node record and does not count as activity from that node;
- replies, errors, value updates and writes are dropped;
- `ping` and `find_node` receive control replies using a transient node record and an Initial envelope to the known key; GET receives a short `401`, even when the plaintext packet is the first advertisement of a matching `k`. Truly keyless legacy GETs remain unchanged.
**Rate limits.** A node keeps separate budgets, each with a global quota and a quota per source address (per /64 prefix for IPv6):
- **Session requests:** requests in Session or Data envelopes, whose source address is validated. `Config::max_req_per_sec`, by default 8192 per second, and 1024 per address.
- **Other requests:** plaintext requests and requests in Initial or Hello envelopes, whose source address can be spoofed. `Config::max_unauth_req_per_sec`, with the same defaults. A flood of them does not affect requests from established peers.
- **Public-key work:** one unit per packet needing a key agreement or a signature verification, charged before that work. `Config::max_crypto_per_sec`, by default 512 per second, and 64 per address. A unit covers the packet and its answer, including credential verification:
- each Initial envelope (two X25519 operations, and five more to answer a request);
- each Session envelope from an uncached (address, `pk_sender`, `epk_sender`) triple (five X25519 operations per epoch tried);
- each cached Session envelope whose peer has not yet been authorized and needs credential verification (one charge, not an additional charge for packets already admitted for key agreement);
- each Hello (one Ed25519 verification, and the key derivation of its answer);
- each plaintext request from a sender whose key is known or advertised, answered in an Initial envelope (three X25519 operations).
Initial and uncached Session envelopes from an address where we await an answer, from the sender's node record or from a request sent by address, use a separate budget of the same size. Our own key exchanges thus complete during a flood.
Packets from peers with cached session keys cost one AEAD operation, except when a private-network credential needs verification. Each epoch caches at most 16384 sessions, evicting the least recently used one to store a new one. An evicted peer recovers after one retransmission. Only a peer that received a packet from us at its address can take a cache slot, so a slot costs the attacker a reachable address and a round trip.
### 6.6 Write authorization
`put`, `refresh` and `listen` create state on the receiver, and `listen` makes it send value updates to the requester's address. The receiver therefore requires proof that the requester receives packets at its source address. Without it, an attacker could spoof a victim's address to fill its storage quota, or direct update traffic to it.
- **Encrypted requests** must arrive in a **Session** or **Data** envelope. Our ephemeral key for the source address is only sent to that address ([§5.1](#51-ephemeral-keys)), so opening the envelope proves the sender received one of our packets there. Requests in Initial envelopes are refused with error `401`; normal senders establish the session through `ping` or `find_node` before sending application requests.
- **Plaintext requests** must carry a token, as in the legacy protocol. The token is sent in answers to `find`, `get` and `listen`, and in value updates, to the address of the node record:
```
token(addr) = HMAC-SHA256(key = S, "opendht-address-token-v1" ‖ addr)
```
A token is accepted in the epoch it was issued and the next one. Legacy nodes treat tokens as opaque, so changing their derivation keeps compatibility.
Messages to and from encrypted nodes carry no token. This saves 40 bytes in each answer to `find`, `get` and `listen`, and in each `put`, `refresh`, `listen` and value update. A node that receives an encrypted answer may write to its sender.
## 7. Message flows
### 7.1 Public network
Encryption is opportunistic, and keys travel with node IDs.
```mermaid
sequenceDiagram
participant A as A (new)
participant B as B (new)
participant L as L (legacy)
A->>B: plaintext ping + k = pk_A (B's key unknown)
Note over B: attaches pk_A to A (nodeId check)
B->>A: Initial envelope (pong)
Note over A: decrypts, checks ID = nodeId(pk_B), learns epk_B
A->>B: Session envelope (request)
B->>A: Session envelope (reply)
Note over A: session confirmed
A->>B: Data envelope (request)
Note over B: session confirmed
B->>A: Data envelope (all further traffic)
A->>L: plaintext + k (ignored by L)
L->>A: plaintext
```
- The first request to a node with an unknown key, typically a bootstrap ping, is in plaintext. Its reply is an Initial envelope. The next request and its reply use Session envelopes, and all later traffic uses Data envelopes.
- Nodes learned through `k4`/`k6` get an Initial envelope as the first packet. Its answer is a Session envelope, but the answerer caches nothing yet: the next request is a Session envelope, and its answer confirms the session. Data envelopes follow.
- Legacy nodes see plaintext from new nodes and are unaffected.
### 7.2 Private network
Only nodes with a credential participate. Nodes never send or accept plaintext.
```mermaid
sequenceDiagram
participant A
participant B as B (bootstrap)
A->>B: Hello(pk_A, epk_A, cred_A, ping)
Note over B: verifies cred_A, keeps no state
B->>A: Session(cred_B, pong)
Note over A: verifies cred_B, authorizes B
A->>B: Session(cred_A, request)
Note over B: verifies cred_A, authorizes A, caches the session
B->>A: Session(cred_B, reply)
Note over A: session confirmed
A->>B: Data (without credential)
```
- **Credential exchange.** Each side attaches its credential to its Initial and Session envelopes. Data envelopes, sent once the peer has accepted us, never carry it.
- **Authorization lifetime.** Admission does not expire. Session keys still rotate for forward secrecy, independently of membership. A lost admission record is recovered using the same grant, without operator action.
- **Node lists.** Listed nodes without a valid matching key are discarded.
- **Startup checks.** The node refuses to start (`DhtException`) if:
- encryption is disabled while authorities are configured;
- the configured credential does not match the transport key;
- the credential is not valid for the configured authorities.
The credential and trusted authorities are read on start. Restart with the updated configuration when explicitly changing a grant or the trust set; this also discards cached admissions.
## 8. Key persistence
If the following conditions all hold, the static secret key is saved in the state file as the msgpack field `transport_key` (32 bytes):
- encryption is enabled;
- no key was configured;
- `Config::persist_path` is set.
When the key is saved:
- the state is written to a randomly named temporary file in the destination directory;
- owner-only read/write permissions are requested before any secret data is written;
- the temporary file is closed successfully before it is renamed over the state file;
- a save failure is logged, the temporary file is removed, and the previous state file is left intact;
- on start, the key is restored before any packet is exchanged, so the node keeps its ID across restarts.
The node also stores its ID in the state file, so older versions reading this state keep the same ID. A configured key is never written to the state file. Ephemeral keys and session keys are never persisted.
`dhtnode --transport-key <file>` loads the raw 32-byte secret from `<file>`. If the file does not exist, `dhtnode` creates it using the same temporary-file and permission handling. Unreadable or invalid existing keys are not replaced.
File operations use portable C++ filesystem APIs. Permission semantics depend on the platform; exclusive file creation and power-loss durability are not guaranteed. Use an owner-controlled directory and a single writer for each path.
## 9. Security analysis
### 9.1 Properties
Against an attacker who controls the network but holds no node's secret key:
- **Confidentiality and integrity.** Traffic between two nodes that know each other's keys is confidential and integrity-protected. The protection is IND-CCA2 and INT-CTXT, from ChaCha20-Poly1305 under keys derived from X25519.
- **ID authentication.** An encrypted message under node ID `X` can only be produced by the holder of the static secret key behind `X`.
- Producing a key for a given ID requires a SHA-256 preimage on 160 bits.
- Choosing the first `b` bits of one's own ID costs about $2^b$ key generations. Legacy nodes choose their ID freely.
- **Forward secrecy.** Session envelopes recorded by the attacker stay confidential if static keys are compromised later, once both ephemeral secrets are erased (at most one hour after use). Initial envelopes stay confidential if the sender's static key is compromised, but not the receiver's.
- **Key-compromise impersonation.** With a node's static secret, an attacker can decrypt the Initial envelopes sent to it and impersonate other nodes to it in Initial envelopes, but not in Session envelopes.
- **Address stickiness.** Spoofed plaintext, Hellos and replayed Initial envelopes cannot redirect a node record whose key is known. Moving it requires a Session or Data envelope from the new address (see [§6.2](#62-hello-private-networks-only) and [§6.5](#65-receiving)).
- **Fragment integrity.** Once a fragmented message arrives in envelopes, only parts from the same sender key are merged. Spoofed plaintext or foreign-key parts cannot alter or block it ([§6.5](#65-receiving), step 5).
- **Address validation.** Session and Data envelopes only open from the address our ephemeral key was sent to. An off-path attacker cannot send one from a spoofed address, even with its own valid static key, so it cannot `put`, `refresh` or `listen` on behalf of another address.
- **Private-network admission.** Without a valid credential and its matching secret key, a node:
- cannot read any traffic;
- cannot get answers it can decrypt;
- is never listed by honest nodes.
- **Network separation.** Different network IDs derive different keys.
- **DoS cost.** An unauthenticated Data envelope costs a lookup and at most two AEAD operations. Other envelopes pass the public-key limit before any key derivation or signature verification.
### 9.2 Limitations
Each item states who it applies to, what that party can do, and what it cannot do.
- **Encryption is hop-by-hop.** Nodes that store values read them. Only `SecureDht` protects values end to end ([§1](#1-scope)).
- **Public mode is trust on first use.** Applies in public networks, to an attacker who modifies traffic on the path between two nodes A and B before A knows B's key.
- A node learns a peer's key from the first plaintext message that carries it with the peer's ID: `k` in a message from the peer, or `k4`/`k6` in a reply from any node. An attacker that removes these fields from every such message keeps A and B in plaintext, and reads and modifies their traffic. The downgrade ends with the first message carrying a key that the attacker misses: the node that receives it then sends only envelopes to that peer, which the attacker cannot alter, and treats plaintext claiming the peer's ID as untrusted ([§6.5](#65-receiving), step 7). The attacker can still drop their traffic.
- The attacker cannot bind its own key to B's ID. It can rewrite B's plaintext messages under an ID and key of its own, so that A talks to the attacker's node at B's address, and the attacker relays to B. A then knows the attacker's node at that address, not B.
- Until A knows a key for an ID, any plaintext sender can claim that ID from any address. The claim lasts until A learns the key: plaintext claiming the ID is untrusted from then on, and the first Session or Data envelope under it moves the node record to its sender's address.
- A key received in an encrypted reply cannot be stripped. One received in a plaintext reply can, by an attacker on that reply's path.
- The first request to a node with an unknown key, for example a bootstrap ping, is in plaintext.
- Private networks are not affected: they never send or accept plaintext, and credentials authenticate keys.
- **Compromise of a static secret key.** Applies to whoever obtains `sk_B`, for example from B's key file or state file, for as long as B keeps that key.
- It can act as B from any address, and talk to any node under B's ID.
- It can read every Initial envelope ever sent to B, including recorded ones. Initial keys depend only on `sk_B` and the sender's public keys.
- It can send Initial envelopes to B under any other node's ID. B answers `ping` and `find_node` in them, under keys the attacker cannot derive, and changes no state ([§6.5](#65-receiving), step 7). GET and writes are refused with `401`.
- It cannot read Session or Data envelopes between B and other nodes, recorded or live, nor impersonate another node to B in them. That requires B's ephemeral secret, or the other node's static secret.
- In a private network, it stays admitted for as long as B's granting authority is trusted. The credential is not secret: Hellos carry it in clear.
- Compromise of a running node also exposes its epoch secrets, and with them all sessions of its current and previous epoch.
Recover by changing the key. This changes the node ID, and in a private network needs a new credential. Changing B's key alone does not exclude the compromised key: its grant remains valid until its authority is distrusted, as described below. The old key cannot be revoked in a public network.
- **Initial envelopes are not forward-secret against the receiver.** An Initial envelope stays confidential if the sender's static key is compromised later, but not if the receiver's is. Initial envelopes carry:
- `ping` or `find_node` to a node whose ephemeral key is unknown, including their retransmissions;
- control replies to plaintext requests advertising a key. Application requests wait for a Session ([§6.4](#64-sending)).
Routing targets and control traffic remain exposed to a later compromise of the receiver's static key. Application GET/SELECT and write payloads do not enter Initial envelopes. Answers to Initial envelopes are Session envelopes, and are forward-secret.
- **No replay protection.** Nodes keep no record of received nonces. A captured envelope is accepted again as long as the receiver holds the keys that open it:
- **Initial envelopes** open from any address, for as long as the receiver's static key is unchanged. A replayed `ping` or `find_node` is answered again, to the replayer's source address, under keys the replayer cannot derive. GET and writes are refused with `401`, value updates are dropped, and replies and errors are accepted only for a request pending toward that address ([§6.5](#65-receiving), step 7). Reflection remains for control replies: anyone holding a captured Initial request, or a Hello, can make the receiver send an encrypted answer to an address of its choice. In a private network this includes outsiders, because the captured packet carries its sender's credential. Rate limits bound request frequency, not reply size; node-list packet-size limits remain separate work.
- **Session and Data envelopes** open only from their original source address, which the replayer must spoof, and only until the epoch that opened them is erased: at most one hour. A replayed request is processed again as its sender, including a write the sender did make: a repeated `put`, `refresh` or `listen` extends the life of a value or of a listener. A replayed value update is accepted while its listen is open, and repeats it: the values it lists are added or refreshed again, and the expirations it announces are signaled again. A replayed reply or error is accepted only if its transaction ID is pending. Transaction IDs are 32-bit counters per peer with a random start, so an old ID is pending again only after $2^{32}$ requests to that peer.
- **On-path attackers get write authorization for the addresses they observe.** Applies to an attacker that reads the packets we send to an address `addr`. Our ephemeral key `epk(addr)` is in the clear header of every Initial and Session envelope we send there: at each first contact, retransmission and epoch rotation. Data envelopes do not carry it.
- With its own key pair, the attacker can send us Session envelopes with source address `addr`, and they open. It then has what the owner of `addr` has: write authorization ([§6.6](#66-write-authorization)). Its `put` and `refresh` consume the storage quota of `addr`, and its `listen` directs value updates to `addr`, all under the attacker's own node ID. Each such envelope also replaces the session cached for `addr`, so Data envelopes from the real owner are dropped until its next retransmission establishes a new session ([§6.4](#64-sending), recovery).
- It cannot impersonate the node at `addr`, nor read or alter its traffic. That requires the static secret of that node.
- A captured `epk(addr)` is usable until the epoch that produced it is erased: at most one hour.
- In a private network, the attacker's key must also hold a valid credential.
This is the same exposure as legacy tokens, which are sent in clear to the address they authorize. Address validation defends against off-path spoofing only.
- **Address changes.** Our ephemeral key for a peer is derived from its address. When a peer's address or port changes, for example after a NAT rebinding, its Session and Data envelopes to us no longer open, and ours no longer reach it. Its next retransmission is a ping in an Initial envelope, which starts a new session for the new address, and the request follows in a Session envelope ([§6.4](#64-sending), recovery).
- **No Sybil resistance.** An identity costs one key generation. Encryption prevents choosing a node ID ([§9.1](#91-properties)), not creating many of them. In public networks, legacy nodes still choose their IDs freely.
- **Denial of service.** Public-key operations on unauthenticated packets are bounded by the public-key limit ([§6.5](#65-receiving)): by default 512 packets per second globally, and 64 per source address or IPv6 /64 prefix.
- An attacker spending that budget from many or spoofed addresses drops Initial, Hello and uncached Session envelopes from new peers, and plaintext requests from keyed senders. Answers to our own requests have a separate budget, which only an attacker spoofing the addresses of the peers we contact can spend. Data envelopes, cached sessions and requests from established peers are unaffected.
- Likewise, a flood of plaintext or Initial requests drops other requests without a session, as in the legacy protocol, but not those of established peers.
- A unit stands for three to ten X25519 operations, so the default allows about 5000 per second. `Config::max_crypto_per_sec` adapts it to the CPU available.
- An attacker reachable at many addresses, for example an IPv6 prefix, can fill the session cache of an epoch: one slot costs it one round trip. Evicted peers have their Data envelopes dropped until their next retransmission, which starts a new session.
- **No per-grant revocation.** Applies to private networks. A grant is permanent: a compromised or excluded node can participate for as long as any authority that signed a grant for its key remains trusted. Compromise of an authority allows issuing arbitrary new grants. Exclusion requires every remaining node to remove the affected authority from its trust set and restart, discarding cached admissions. This also excludes every member relying only on that authority; retained members need grants from another trusted authority. Multiple accepted authorities provide a transition path, but do not neutralize a compromised authority while it is still trusted. There is no clock-based fallback, revocation list, or automatic expiry.
- **Metadata.** Anyone observing the traffic sees:
- in Initial and Session envelopes, the sender's static key, hence its node ID, and its ephemeral key: who talks to whom, at each first contact, retransmission and epoch rotation;
- in Data envelopes, only the type; the sender is still identified by its address, from the Session envelopes that preceded;
- in every envelope, the type, the credential flag, the size and the timing, which can reveal the kind of message, for example fragmented value data;
- Hello messages entirely, including the credential, which reveals the node's key. A copied credential is useless without the matching secret key;
- in public networks, plaintext messages entirely, including node lists.
Node addresses are not hidden: they are in node lists, and on the packets themselves.
- **Node state.** The saved node list holds IDs and addresses, but no keys. In a public network, saved nodes are contacted again in plaintext, with the first-contact exposure above. In a private network, a saved node with an ID but no key cannot be contacted ([§6.4](#64-sending), step 3): a restarted node needs a bootstrap address to rejoin, and reuses saved nodes once it learns their keys from node lists.
## 10. Operating a private network
With `dhtnode`:
```sh
# Once: create the authority (net.key = secret seed, net.pub = hex public key)
dhtnode --gen-network-authority net
# Once per node: create its transport key and permanent membership grant
dhtnode --transport-key node1.key --issue-credential node1.cred --authority-key net.key
# Run a node
dhtnode --transport-key node1.key --network-credential node1.cred --network-authority $(cat net.pub)
```
Authority generation refuses to run if either output path already exists, including a dangling symlink.
Keep the authority secret offline. Distribute only `net.pub` and the per-node credentials. Without a compromise or an intentional membership change, keep these keys, grants, and trust sets unchanged: there is no scheduled renewal or maintenance.
`--network-authority` can be repeated to trust several authorities during an explicit transition:
```sh
dhtnode --transport-key node1.key --network-credential node1.cred \
--network-authority $(cat net.pub) --network-authority $(cat replacement.pub)
```
To move away from an authority:
1. Create a replacement authority and configure retained members to trust both old and replacement public keys. Restart them with this overlapping trust set.
2. Issue replacement grants only to retained, uncompromised transport keys and restart those nodes with their new grants. Old and replacement grants interoperate while both authorities are trusted. A compromised transport key must be replaced as well.
3. Remove the old authority from every retained node's configuration and restart them, clearing prior admissions. Only grants from the remaining trusted authorities are then accepted.
A compromised authority or a compromised key's old grant remains usable wherever the old authority is still trusted. The overlap is a transition mechanism, not immediate exclusion; it ends only when all retained nodes stop trusting the affected authority. Uncompromised grants from other authorities do not need updating.
From the library, set `Config::transport` before starting:
- `authorities`;
- `key`, which must be the key the credential was issued for;
- `credential`.
## 11. Constants
| Name | Value |
|---|---|
| Envelope marker | `0xC1` |
| Envelope types | `Initial = 1`, `Session = 2`, `Hello = 3`, `Data = 4` |
| Credential flag | `0x80` |
| Key, nonce, tag sizes | 32, 12, 16 bytes |
| Envelope header | 66 bytes, 2 for Data |
| Envelope overhead | 30 bytes for Data, 94 for Initial and Session (191 with credential) |
| Epoch (ephemeral keys and tokens) | 30 minutes, previous epoch accepted for one more period |
| Cached sessions per epoch | 16384, least recently used evicted |
| Credential size, version | 97 bytes, `0x02` |
| Node ID context | `"opendht-node-id-v1"` |
| HKDF salt | `"opendht-transport-v2"` |
| Credential signature context | `"opendht-credential-v2"` |
| Ephemeral key, token contexts | `"opendht-address-key-v1"`, `"opendht-address-token-v1"` |
| Token size | 32 bytes |
| Value fragment size | 1184 bytes, less the envelope overhead (1154 to 993) |
+19 -1
View File
@@ -4,6 +4,7 @@
#include "infohash.h"
#include "value.h"
#include "transport.h"
#include <vector>
#include <memory>
@@ -97,7 +98,7 @@ struct OPENDHT_PUBLIC NodeInfo
*/
struct OPENDHT_PUBLIC Config
{
/** DHT node ID */
/** DHT node ID. Ignored if transport encryption is enabled: the node ID derives from the transport key. */
InfoHash node_id {};
/**
@@ -122,6 +123,20 @@ struct OPENDHT_PUBLIC Config
/** If non-0, overrides the default per-IP address rate-limit. -1 means no limit. */
ssize_t max_peer_req_per_sec {0};
/**
* If non-0, override the rate-limits of requests from senders without an encrypted session
* (default: same as above). -1 means no limit.
*/
ssize_t max_unauth_req_per_sec {0};
ssize_t max_peer_unauth_req_per_sec {0};
/**
* If non-0, override the rate-limits of packets needing a transport key agreement
* or credential check. -1 means no limit.
*/
ssize_t max_crypto_per_sec {0};
ssize_t max_peer_crypto_per_sec {0};
/* If non-0, overrides the default maximum number of searches. -1 means no limit. */
ssize_t max_searches {0};
@@ -143,6 +158,9 @@ struct OPENDHT_PUBLIC Config
/* Client mode, node will not be used by other nodes to store data. */
bool client_mode {false};
/** Encryption of the DHT protocol, and optional private network */
TransportConfig transport {};
};
/**
+15 -3
View File
@@ -204,7 +204,13 @@ struct OPENDHT_PUBLIC PrivateKey
* @param algo : the public key algorithm to use, GNUTLS_PK_RSA or GNUTLS_PK_RSA_OAEP
*/
static PrivateKey generate(unsigned key_length = 4096, gnutls_pk_algorithm_t algo = GNUTLS_PK_RSA);
static PrivateKey generateEC();
/**
* Generate a new EC key pair
* @param curve : the elliptic curve to use. If GNUTLS_ECC_CURVE_INVALID is provided, the default curve will be used
* (currently GNUTLS_ECC_CURVE_SECP384R1).
*/
static PrivateKey generateEC(gnutls_ecc_curve_t curve = GNUTLS_ECC_CURVE_INVALID);
gnutls_privkey_t key {};
gnutls_x509_privkey_t x509_key {};
@@ -267,9 +273,15 @@ public:
/**
* Sign this revocation list using provided key and certificate.
* Validity_period sets the duration until next update (default to no next update).
* If `number` is non-zero, it is stored verbatim (big-endian) as the CRL Number
* extension. Otherwise the number is initialized randomly on the first signature
* and incremented on subsequent ones.
*/
void sign(const PrivateKey&, const Certificate&, duration validity_period = {});
void sign(const Identity& id) { sign(*id.first, *id.second); }
void sign(const PrivateKey&, const Certificate&, duration validity_period = {}, uint64_t number = 0);
void sign(const Identity& id, duration validity_period = {}, uint64_t number = 0)
{
sign(*id.first, *id.second, validity_period, number);
}
bool isSignedBy(const Certificate& issuer) const;
+6
View File
@@ -159,7 +159,13 @@ public:
{}
static Value::Filter getFilter() { return EncryptedValue::getFilter(); }
virtual void unpackValue(const dht::Value& v) override
{
BaseClass::unpackValue(v);
id = v.id;
}
dht::Value::Id id {0}; /// Value ID of the trust request, used for deserialization only.
std::string service;
std::string conversationId;
Blob payload;
+10 -16
View File
@@ -277,9 +277,6 @@ public:
std::vector<ValuesExport> exportValues() const override;
void importValues(const std::vector<ValuesExport>&) override;
void saveState(const std::string& path) const;
void loadState(const std::string& path);
NodeStats getNodesStats(sa_family_t af) const override;
std::string getStorageLog() const override;
@@ -367,8 +364,6 @@ private:
static constexpr std::chrono::seconds BOOTSTRAP_PERIOD {10};
static constexpr size_t TOKEN_SIZE {32};
// internal structures
struct SearchNode;
struct Get;
@@ -379,13 +374,14 @@ private:
Dht(const Dht&) = delete;
Dht& operator=(const Dht&) = delete;
void saveState(const std::string& path) const;
/* Only called on construction: may replace the transport key */
void loadState(const std::string& path);
std::mt19937_64 rd;
InfoHash myid {};
uint64_t secret {};
uint64_t oldsecret {};
// registred types
TypeStore types;
@@ -438,6 +434,8 @@ private:
net::NetworkEngine network_engine;
std::string persistPath;
/* The transport key was generated: keep it in the persisted state */
bool persistTransportKey {false};
// are we a bootstrap node ?
// note: Any running node can be used as a bootstrap node.
@@ -452,11 +450,6 @@ private:
return public_stable ? LISTEN_EXPIRE_TIME_PUBLIC : LISTEN_EXPIRE_TIME;
}
void rotateSecrets();
Blob makeToken(const SockAddr&, bool old) const;
bool tokenMatch(const Blob& token, const SockAddr&) const;
void reportedAddr(const SockAddr&);
// Storage
@@ -576,6 +569,8 @@ private:
* @param query The query sent to the node.
*/
void searchNodeGetExpired(const net::Request& status, bool over, std::weak_ptr<Search> ws, Sp<Query> query);
void searchNodeRoutingDone(const net::Request& req, net::RequestAnswer&& answer, std::weak_ptr<Search> ws);
void searchNodeRoutingExpired(const net::Request& req, bool over, std::weak_ptr<Search> ws);
/**
* This method recovers sends individual request for values per id.
@@ -627,15 +622,14 @@ private:
void onGetValuesDone(const Sp<Node>& status, net::RequestAnswer& a, Sp<Search>& sr, const Sp<Query>& orig_query);
/* when we receive a listen request */
net::RequestAnswer onListen(
Sp<Node> node, const InfoHash& hash, const Blob& token, size_t socket_id, const Query& query, int version = 0);
Sp<Node> node, const InfoHash& hash, size_t socket_id, const Query& query, int version = 0);
void onListenDone(const Sp<Node>& node, net::RequestAnswer& a, Sp<Search>& sr);
/* when we receive an announce request */
net::RequestAnswer onAnnounce(Sp<Node> node,
const InfoHash& hash,
const Blob& token,
const std::vector<Sp<Value>>& v,
const time_point& created);
net::RequestAnswer onRefresh(Sp<Node> node, const InfoHash& hash, const Blob& token, const Value::Id& vid);
net::RequestAnswer onRefresh(Sp<Node> node, const InfoHash& hash, const Value::Id& vid);
void onAnnounceDone(const Sp<Node>& status, net::RequestAnswer& a, Sp<Search>& sr);
};
+23 -14
View File
@@ -19,7 +19,9 @@
#include <mutex>
namespace dht {
enum class PushType { None = 0, Android, iOS, UnifiedPush };
// Appended only: the values are serialized as integers by MSGPACK_ADD_ENUM,
// so reordering would break compatibility with deployed proxies.
enum class PushType { None = 0, Android, iOS, UnifiedPush, Huawei };
}
MSGPACK_ADD_ENUM(dht::PushType)
@@ -76,8 +78,14 @@ public:
asio::io_context& io_context() const;
// Monotonic clock for elapsed-time statistics (request rate, uptime).
// Must stay on steady_clock so NTP/wall-clock corrections don't skew durations.
using clock = std::chrono::steady_clock;
using time_point = clock::time_point;
// Wall clock for persisted subscribe/permanent-put lease expirations,
// so they map directly to epoch time and survive restarts.
using sys_clock = std::chrono::system_clock;
using sys_time_point = sys_clock::time_point;
struct PushStats
{
@@ -126,6 +134,7 @@ public:
PushStats androidPush;
PushStats iosPush;
PushStats unifiedPush;
PushStats huaweiPush;
/** Average requests per second */
double requestRate {0};
@@ -266,6 +275,9 @@ private:
{
std::mutex lock;
std::string sessionId;
/** Where to push: follows the latest subscription refresh of the device. */
PushType type {PushType::None};
std::string topic;
PushSessionContext(const std::string& id)
: sessionId(id)
{}
@@ -335,20 +347,16 @@ private:
*
* @param infoHash The information hash associated with the push listen request.
* @param pushToken The push token associated with the push listen request.
* @param type The type of the push listen request.
* @param clientId The client ID associated with the push listen request.
* @param sessionCtx The shared pointer to the push session context associated with the push listen request.
* @param topic The topic associated with the push listen request.
* @param sessionCtx The push session context of the listener, giving the session, push type and topic to use.
* @param values The vector of shared pointers to values associated with the push listen request.
* @param expired A boolean indicating whether the push listen request has expired.
* @return true.
*/
bool handlePushListen(const InfoHash& infoHash,
const std::string& pushToken,
PushType type,
const std::string& clientId,
const std::shared_ptr<DhtProxyServer::PushSessionContext>& sessionCtx,
const std::string& topic,
const std::vector<std::shared_ptr<Value>>& values,
bool expired);
@@ -392,6 +400,7 @@ private:
PushStats androidPush_;
PushStats iosPush_;
PushStats unifiedPush_;
PushStats huaweiPush_;
// Thread-safe access to listeners map.
std::mutex lockListener_;
@@ -401,12 +410,12 @@ private:
std::shared_ptr<ConnectionListener> connListener_;
struct PermanentPut
{
time_point expiration;
sys_time_point expiration;
std::string pushToken;
std::string clientId;
std::shared_ptr<PushSessionContext> sessionCtx;
std::unique_ptr<asio::steady_timer> expireTimer;
std::unique_ptr<asio::steady_timer> expireNotifyTimer;
std::unique_ptr<asio::system_timer> expireTimer;
std::unique_ptr<asio::system_timer> expireNotifyTimer;
Sp<Value> value;
PushType type;
std::string topic;
@@ -419,7 +428,7 @@ private:
p.pack("value");
p.pack(value);
p.pack("exp");
p.pack(to_time_t(expiration));
p.pack(sys_clock::to_time_t(expiration));
if (not clientId.empty()) {
p.pack("cid");
p.pack(clientId);
@@ -460,12 +469,12 @@ private:
#ifdef OPENDHT_PUSH_NOTIFICATIONS
struct Listener
{
time_point expiration;
sys_time_point expiration;
std::string clientId;
std::shared_ptr<PushSessionContext> sessionCtx;
std::future<size_t> internalToken;
std::unique_ptr<asio::steady_timer> expireTimer;
std::unique_ptr<asio::steady_timer> expireNotifyTimer;
std::unique_ptr<asio::system_timer> expireTimer;
std::unique_ptr<asio::system_timer> expireNotifyTimer;
PushType type;
std::string topic;
@@ -476,7 +485,7 @@ private:
p.pack("cid");
p.pack(clientId);
p.pack("exp");
p.pack(to_time_t(expiration));
p.pack(sys_clock::to_time_t(expiration));
if (sessionCtx) {
std::lock_guard l(sessionCtx->lock);
p.pack("sid");
+2
View File
@@ -532,6 +532,8 @@ public:
/**
* Insert a push notification to process for OpenDHT
* Returns an immediately ready future with IgnoredStopped if the runner is idle,
* or IgnoredDisabled if push notification support is disabled at build time.
*/
std::future<PushNotificationResult> pushNotificationReceived(const std::map<std::string, std::string>& data);
+15 -14
View File
@@ -51,11 +51,11 @@ using ConnectHandlerCb = std::function<void(const asio::error_code& ec, const as
using ssl_socket_t = restinio::impl::tls_socket_t;
using socket_t = asio::ip::tcp::socket;
class OPENDHT_PUBLIC Url
struct OPENDHT_PUBLIC Url
{
public:
Url() = default;
Url(std::string_view url);
Url(std::string url);
std::string url;
std::string protocol {"http"};
std::string_view user;
@@ -106,9 +106,16 @@ public:
void timeout(const std::chrono::seconds& timeout, HandlerCb cb = {});
/**
* Close the underlying socket.
* Safe to call from any thread: the actual close is dispatched to
* the thread running the io_context.
*/
void close();
private:
void doClose();
template<typename T>
T wrapCallback(T cb) const
{
@@ -149,7 +156,7 @@ public:
using ResolverCb
= std::function<void(const asio::error_code& ec, const std::vector<asio::ip::tcp::endpoint>& endpoints)>;
Resolver(asio::io_context& ctx, const std::string& url, std::shared_ptr<log::Logger> logger = {});
Resolver(asio::io_context& ctx, std::string url, std::shared_ptr<log::Logger> logger = {});
Resolver(asio::io_context& ctx,
std::string_view host,
std::string_view service,
@@ -161,10 +168,6 @@ public:
std::vector<asio::ip::tcp::endpoint> endpoints,
const bool ssl = false,
std::shared_ptr<log::Logger> logger = {});
Resolver(asio::io_context& ctx,
const std::string& url,
std::vector<asio::ip::tcp::endpoint> endpoints,
std::shared_ptr<log::Logger> logger = {});
~Resolver();
@@ -216,19 +219,19 @@ public:
// resolves implicitly
Request(asio::io_context& ctx,
const std::string& url,
std::string url,
const Json::Value& json,
OnJsonCb jsoncb,
std::shared_ptr<log::Logger> logger = {});
Request(asio::io_context& ctx, const std::string& url, OnJsonCb jsoncb, std::shared_ptr<log::Logger> logger = {});
Request(asio::io_context& ctx, std::string url, OnJsonCb jsoncb, std::shared_ptr<log::Logger> logger = {});
Request(asio::io_context& ctx, const std::string& url, std::shared_ptr<log::Logger> logger = {});
Request(asio::io_context& ctx, std::string url, std::shared_ptr<log::Logger> logger = {});
Request(asio::io_context& ctx,
std::string_view host,
std::string_view service,
const bool ssl = false,
std::shared_ptr<log::Logger> logger = {});
Request(asio::io_context& ctx, const std::string& url, OnDoneCb onDone, std::shared_ptr<log::Logger> logger = {});
Request(asio::io_context& ctx, std::string url, OnDoneCb onDone, std::shared_ptr<log::Logger> logger = {});
// user defined resolver
Request(asio::io_context& ctx, std::shared_ptr<Resolver> resolver, sa_family_t family = AF_UNSPEC);
@@ -344,8 +347,6 @@ private:
dht::crypto::Identity client_identity_;
std::shared_ptr<dht::crypto::Certificate> server_ca_;
std::string service_;
std::string host_;
unsigned int id_;
static std::atomic_uint ids_;
+12 -20
View File
@@ -123,28 +123,19 @@ ee *
*/
static inline unsigned commonBits(const Prefix& p1, const Prefix& p2)
{
unsigned i, j;
uint8_t x;
/* size_ counts bits, so the comparison walks bits: reading it as a
byte count would run past the end of the shorter content. */
auto longest_prefix_size = std::min(p1.size_, p2.size_);
for (i = 0; i < longest_prefix_size; i++) {
if (p1.content_.data()[i] != p2.content_.data()[i] or not p1.isFlagActive(i) or not p2.isFlagActive(i)) {
unsigned common = 0;
while (common < longest_prefix_size) {
if (not p1.isFlagActive(common) or not p2.isFlagActive(common)
or p1.isContentBitActive(common) != p2.isContentBitActive(common))
break;
}
common++;
}
if (i == longest_prefix_size)
return 8 * longest_prefix_size;
x = p1.content_.data()[i] ^ p2.content_.data()[i];
j = 0;
while ((x & 0x80) == 0) {
x <<= 1;
j++;
}
return 8 * i + j;
return common;
}
/**
@@ -205,7 +196,7 @@ private:
for (auto i = copy.size(); i < size; i++)
copy.push_back(0);
swapBit(copy, size_ + 1);
swapBit(copy, size_);
return copy;
}
@@ -242,8 +233,9 @@ private:
if (bit >= b.size() * 8)
throw std::out_of_range("bit larger than prefix size.");
size_t offset_bit = (8 - bit) % 8;
b[bit / 8] ^= (1 << offset_bit);
/* Bit 0 is the most significant one of the first byte, as
isActiveBit reads it. */
b[bit / 8] ^= (1 << (7 - (bit % 8)));
}
};
+195 -37
View File
@@ -12,13 +12,16 @@
#include "rate_limiter.h"
#include "logger.h"
#include "network_utils.h"
#include "transport.h"
#include <vector>
#include <string>
#include <functional>
#include <algorithm>
#include <list>
#include <memory>
#include <queue>
#include <tuple>
namespace dht {
namespace net {
@@ -34,9 +37,18 @@ struct TransId;
struct NetworkConfig
{
NetId network {0};
/* Requests in Session or Data envelopes, from a validated address */
ssize_t max_req_per_sec {0};
ssize_t max_peer_req_per_sec {0};
/* Other requests. 0: same as above */
ssize_t max_unauth_req_per_sec {0};
ssize_t max_peer_unauth_req_per_sec {0};
/* Packets needing a key agreement or signature check. 0: same as max_req_per_sec */
ssize_t max_crypto_per_sec {0};
ssize_t max_peer_crypto_per_sec {0};
bool is_client {false};
/** Transport encryption is used only if enabled and a key is set */
TransportConfig transport {};
};
class DhtProtocolException : public DhtException
@@ -83,7 +95,8 @@ struct ParsedMessage;
*/
struct RequestAnswer
{
Blob ntoken {};
/* Write token of the node; empty if it needs none (encrypted), nullopt if not granted. */
std::optional<Blob> ntoken {};
Value::Id vid {};
std::vector<Sp<Value>> values {};
std::vector<Value::Id> refreshed_values {};
@@ -160,34 +173,31 @@ private:
*/
std::function<RequestAnswer(Sp<Node>, const InfoHash&, want_t, const Query&)> onGetValues {};
/**
* Called on listen request.
* Called on listen request, from a node proven to receive at its address.
*
* @param node (type: Sp<Node>) the requesting node.
* @param h (type: InfoHash) hash of the value of interest.
* @param token (type: Blob) security token.
* @param rid (type: uint16_t) request id.
*/
std::function<RequestAnswer(Sp<Node>, const InfoHash&, const Blob&, Tid, const Query&, int)> onListen {};
std::function<RequestAnswer(Sp<Node>, const InfoHash&, Tid, const Query&, int)> onListen {};
/**
* Called on announce request.
* Called on announce request, from a node proven to receive at its address.
*
* @param node (type: Sp<Node>) the requesting node.
* @param h (type: InfoHash) hash of the value of interest.
* @param token (type: Blob) security token.
* @param values (type: std::vector<Sp<Value>>) values to store.
* @param created (type: time_point) time when the value was created.
*/
std::function<RequestAnswer(Sp<Node>, const InfoHash&, const Blob&, const std::vector<Sp<Value>>&, const time_point&)>
std::function<RequestAnswer(Sp<Node>, const InfoHash&, const std::vector<Sp<Value>>&, const time_point&)>
onAnnounce {};
/**
* Called on refresh request.
* Called on refresh request, from a node proven to receive at its address.
*
* @param node (type: Sp<Node>) the requesting node.
* @param h (type: InfoHash) hash of the value of interest.
* @param token (type: Blob) security token.
* @param vid (type: Value::id) the value id.
*/
std::function<RequestAnswer(Sp<Node>, const InfoHash&, const Blob&, const Value::Id&)> onRefresh {};
std::function<RequestAnswer(Sp<Node>, const InfoHash&, const Value::Id&)> onRefresh {};
public:
using RequestCb = std::function<void(const Request&, RequestAnswer&&)>;
@@ -214,6 +224,15 @@ public:
net::DatagramSocket* getSocket() const { return dht_socket.get(); };
/** Our transport key, or nullptr if transport encryption is disabled */
const transport::KeyPair* getTransportKey() const { return transport_ ? &config.transport.key : nullptr; }
/** Replaces our transport key. Only valid before any message is exchanged. */
void setTransportKey(const transport::KeyPair& key)
{
if (transport_)
config.transport.key = key;
}
void clear();
/**
@@ -225,7 +244,6 @@ public:
* @param socket_id The tid to use to write to the request socket.
* @param hash The hash key of the value.
* @param want Wether to send ipv4 and/or ipv6 nodes.
* @param ntoken Listen security token.
* @param nodes The ipv4 closest nodes.
* @param nodes6 The ipv6 closest nodes.
* @param values The values to send.
@@ -235,7 +253,6 @@ public:
Tid socket_id,
const InfoHash& hash,
want_t want,
const Blob& ntoken,
std::vector<Sp<Node>>&& nodes,
std::vector<Sp<Node>>&& nodes6,
std::vector<Sp<Value>>&& values,
@@ -245,13 +262,11 @@ public:
void tellListenerRefreshed(const Sp<Node>& n,
Tid socket_id,
const InfoHash& hash,
const Blob& ntoken,
const std::vector<Value::Id>& values,
int version);
void tellListenerExpired(const Sp<Node>& n,
Tid socket_id,
const InfoHash& hash,
const Blob& ntoken,
const std::vector<Value::Id>& values,
int version);
@@ -264,6 +279,8 @@ public:
* Requests *
**************/
bool needsSession(const Node& node) const;
/**
* Send a "ping" request to a given node.
*
@@ -461,7 +478,10 @@ public:
size_t getNodeCacheSize() const { return cache.size(); }
size_t getNodeCacheSize(sa_family_t af) const { return cache.size(af); }
size_t getRateLimiterSize() const { return address_rate_limiter.size(); }
size_t getRateLimiterSize() const
{
return requestLimiter_.size() + unauthLimiter_.size() + cryptoLimiter_.size() + replyCryptoLimiter_.size();
}
size_t getPartialCount() const { return partial_messages.size(); }
@@ -486,18 +506,136 @@ private:
reason to increase this value. */
static constexpr unsigned BLACKLISTED_MAX {10};
static constexpr size_t MTU {1280};
/* IPv6 minimum MTU, less the IPv6 and UDP headers */
static constexpr size_t MAX_PACKET_SIZE {1280 - 40 - 8};
/* Upper bound of the msgpack framing around a value fragment */
static constexpr size_t FRAGMENT_HEADER_SIZE {48};
/* Value fragment sizes keeping each packet within the minimum MTU, less the envelope */
static constexpr size_t FRAGMENT_SIZE {MAX_PACKET_SIZE - FRAGMENT_HEADER_SIZE};
static constexpr size_t ENCRYPTED_FRAGMENT_SIZE {FRAGMENT_SIZE - transport::MAX_ENVELOPE_OVERHEAD};
static constexpr size_t MAX_PACKET_VALUE_SIZE {600};
static constexpr size_t MAX_MESSAGE_VALUE_SIZE {56 * 1024};
static constexpr size_t RX_MAX_EARLY_BYTES {MAX_MESSAGE_VALUE_SIZE + MAX_VALUE_SIZE + 32};
static constexpr size_t RX_MAX_EARLY_PARTS {RX_MAX_EARLY_BYTES / ENCRYPTED_FRAGMENT_SIZE + 2};
/* Lifetime of ephemeral transport keys and write tokens: in use for one period, then accepted for another */
static constexpr std::chrono::minutes EPOCH_DURATION {30};
/* Cached session keys per epoch */
static constexpr size_t MAX_SESSIONS {16 * 1024};
static constexpr size_t TOKEN_SIZE {transport::KEY_SIZE};
struct Session
{
/* our ephemeral key for the peer address */
transport::Key ephemeral;
transport::Key peer;
transport::Key peerEphemeral;
transport::SessionKeys keys;
/* the peer has these keys and accepted us: send Data envelopes */
bool confirmed {false};
bool authorized {false};
};
/* The secret from which ephemeral keys and tokens derive, and session keys, all erased together.
* A session is cached for an address only once its peer proved to receive there: it opened an envelope
* encrypted with our key for that address. Replies to Initial and Hello envelopes never cache one,
* since their source address can be spoofed. */
struct Epoch
{
struct Entry
{
Session session;
std::list<SockAddr>::iterator used;
};
transport::Key secret {};
std::map<SockAddr, Entry> sessions {};
/* most recently used first */
std::list<SockAddr> used {};
explicit operator bool() const { return secret != transport::Key {}; }
transport::KeyPair ephemeral(const SockAddr& addr) const;
Blob token(const SockAddr& addr) const;
/* Marks the session as used */
Session* find(const SockAddr& addr);
Session* find(const SockAddr& addr, const transport::Key& peer, const transport::Key& peerEphemeral);
/* Evicts the least recently used session when full */
void store(const SockAddr& addr, const Session& session);
void wipe();
};
struct PendingSession
{
Epoch* epoch;
Session session;
};
void rotateEpoch();
/* Session with the peer at `addr`, or nullopt for an invalid peer key. Cached if the peer ephemeral key is validated. */
std::optional<Session> session(const transport::Peer& peer, const SockAddr& addr);
std::optional<Blob> openSession(const transport::Header& header,
const uint8_t* buf,
size_t buflen,
const SockAddr& from,
std::optional<PendingSession>& pending,
Session*& cached);
/* Fills the sender keys of `header` from the session */
std::optional<Blob> openData(
transport::Header& header, const uint8_t* buf, size_t buflen, const SockAddr& from, Session*& cached);
struct OpenedEnvelope
{
Sp<transport::Peer> peer;
Blob payload;
/* unauthenticated */
bool hello {false};
/* Session or Data envelope: the sender got our ephemeral key for its address */
bool session {false};
std::optional<PendingSession> pending {};
Session* cached {};
};
std::optional<OpenedEnvelope> openEnvelope(const uint8_t* buf, size_t buflen, const SockAddr& from);
bool authorize(transport::Peer& peer, const uint8_t* credential, size_t size) const;
void attachPeer(Node& node, const Sp<transport::Peer>& peer);
Sp<transport::Peer> makePeer(const InfoHash& id, const transport::Key& key) const;
void processHello(const ParsedMessage& msg, const Sp<transport::Peer>& peer, const SockAddr& from);
/* Advertise our transport key in plaintext messages to nodes not known to support encryption */
bool advertiseKey(const Node& node) const
{
return transport_ and not config.transport.isPrivate() and not node.getTransport();
}
/* Messages to the node are encrypted */
bool encrypts(const Node& node) const { return transport_ and node.getTransport(); }
/* Token granting writes to a plaintext node. Encrypted nodes need none. */
Blob writeToken(const Node& node) const { return encrypts(node) ? Blob {} : epoch_.token(node.getAddr()); }
bool tokenMatch(const Blob& token, const SockAddr& addr) const;
/* Writes need proof that the sender receives at its address: a Session envelope, or else a token */
void authorizeWrite(const ParsedMessage& msg, const Node& node, const std::string& error) const;
/* Number of optional top-level fields packed by packOptionalFields */
unsigned optionalFields(bool key) const
{
return (config.network ? 1 : 0) + (config.is_client ? 1 : 0) + (key ? 1 : 0);
}
void packOptionalFields(msgpack::packer<msgpack::sbuffer>& pk, bool key) const;
void process(std::unique_ptr<ParsedMessage>&&, const SockAddr& from);
bool isExpectedReply(const ParsedMessage&, const SockAddr& from);
bool tryCompleteMessage(PartialMessage&, const SockAddr& from);
bool rateLimit(const SockAddr& addr);
/* Admits a request, unless `charged` already: Session and Data senders have their own budget.
* A plaintext request whose answer takes an Initial envelope also needs crypto budget. */
bool limitRequest(const ParsedMessage& msg, const SockAddr& from, bool charged = false);
/* Admits a packet needing a key agreement, from a separate budget if we await a reply from there */
bool limitCrypto(const transport::Key& sender, const SockAddr& from);
bool expectsReply(const transport::Key& sender, const SockAddr& from);
static bool isMartian(const SockAddr& addr);
bool isNodeBlacklisted(const SockAddr& addr) const;
void requestStep(Sp<Request> req);
void transmitRequest(const Sp<Request>& req);
void notifyRequestTimeout(Request& req);
void completeSession(Node& node);
/* Re-establishes the session with a ping in an Initial envelope, unless one is pending */
void recoverSession(const Sp<Node>& node);
/* Sends the requests withheld during the recovery, under the new session */
void resumeRequests(Node& node);
/**
* Sends a request to a node. Request::MAX_ATTEMPT_COUNT attempts will
@@ -517,9 +655,13 @@ private:
};
// basic wrapper for socket sendto function
int send(const SockAddr& addr, const char* buf, size_t len, bool confirmed = false);
int sendRaw(const SockAddr& addr, const uint8_t* buf, size_t len, bool confirmed = false);
// send to a node, encrypted if supported
int send(const Node& node, const SockAddr& addr, const char* buf, size_t len, bool confirmed = false);
/* Bytes added by the envelope that send() uses for the node at `addr` */
size_t envelopeOverhead(const Node& node, const SockAddr& addr);
void sendValueParts(Tid tid, const std::vector<Blob>& svals, const SockAddr& addr);
void sendValueParts(Tid tid, const std::vector<Blob>& svals, const Node& node, const SockAddr& addr);
std::vector<Blob> packValueHeader(msgpack::sbuffer&,
std::vector<Sp<Value>>::const_iterator,
std::vector<Sp<Value>>::const_iterator) const;
@@ -527,53 +669,69 @@ private:
{
return packValueHeader(buf, values.begin(), values.end());
}
void maintainRxBuffer(Tid tid);
using PartialMessageKey = std::tuple<SockAddr, Tid, transport::Key>;
void maintainRxBuffer(const PartialMessageKey& key);
/*************
* Answers *
*************/
/* answer to a ping request */
void sendPong(const SockAddr& addr, Tid tid);
void sendPong(const Node& node, const SockAddr& addr, Tid tid);
struct BufferedNodes
{
Blob nodes4, nodes6;
/* transport keys, aligned with nodes, zeros if unknown. Empty if none is known. */
Blob keys4, keys6;
};
/* answer to findnodes/getvalues request */
void sendNodesValues(const SockAddr& addr,
void sendNodesValues(const Node& node,
const SockAddr& addr,
Tid tid,
const Blob& nodes,
const Blob& nodes6,
const BufferedNodes& nodes,
const std::vector<Sp<Value>>& st,
const Query& query,
const Blob& token);
Blob bufferNodes(sa_family_t af, const InfoHash& id, std::vector<Sp<Node>>& nodes);
std::pair<Blob, Blob> bufferNodes(sa_family_t af, const InfoHash& id, std::vector<Sp<Node>>& nodes);
std::pair<Blob, Blob> bufferNodes(
BufferedNodes bufferNodes(
sa_family_t af, const InfoHash& id, want_t want, std::vector<Sp<Node>>& nodes, std::vector<Sp<Node>>& nodes6);
/* answer to a listen request */
void sendListenConfirmation(const SockAddr& addr, Tid tid, const Blob& token = {});
void sendListenConfirmation(const Node& node, const SockAddr& addr, Tid tid, const Blob& token = {});
/* answer to put request */
void sendValueAnnounced(const SockAddr& addr, Tid, Value::Id);
void sendValueAnnounced(const Node& node, const SockAddr& addr, Tid, Value::Id);
/* answer in case of error */
void sendError(const SockAddr& addr, Tid tid, uint16_t code, const std::string& message, bool include_id = false);
void sendError(const Node& node,
const SockAddr& addr,
Tid tid,
uint16_t code,
const std::string& message,
bool include_id = false);
void deserializeNodes(ParsedMessage& msg, const SockAddr& from);
/* DHT info */
const InfoHash& myid;
const NetworkConfig config {};
NetworkConfig config {};
const bool transport_ {false};
const Blob credential_ {};
Epoch epoch_ {};
Epoch previousEpoch_ {};
Sp<Scheduler::Job> epochRotation_ {};
const std::unique_ptr<DatagramSocket> dht_socket;
Sp<Logger> logger_;
std::mt19937_64& rd;
NodeCache cache;
// global limiting should be triggered by at least 8 different IPs
using IpLimiter = RateLimiter;
using IpLimiterMap = std::map<SockAddr, IpLimiter, SockAddr::ipCmp>;
IpLimiterMap address_rate_limiter;
RateLimiter rate_limiter;
ssize_t limiter_maintenance {0};
/* Requests from Session/Data senders, other requests, public-key work for unexpected and expected senders */
AddressRateLimiter requestLimiter_;
AddressRateLimiter unauthLimiter_;
AddressRateLimiter cryptoLimiter_;
AddressRateLimiter replyCryptoLimiter_;
// requests handling
std::map<Tid, Sp<Request>> requests {};
std::map<Tid, PartialMessage> partial_messages;
std::map<PartialMessageKey, PartialMessage> partial_messages;
MessageStats in_stats {}, out_stats {};
std::set<SockAddr> blacklist {};
+9
View File
@@ -18,6 +18,9 @@ struct Request;
struct Socket;
struct RequestAnswer;
} /* namespace net */
namespace transport {
struct Peer;
}
using Tid = uint32_t;
using SocketCb = std::function<void(const Sp<Node>&, net::RequestAnswer&&)>;
@@ -46,6 +49,10 @@ struct Node
bool isClient() const { return is_client; }
bool isIncoming() { return time > reply_time; }
/** Transport encryption state, if the node is known to support it */
const Sp<transport::Peer>& getTransport() const { return transport_; }
void setTransport(const Sp<transport::Peer>& peer) { transport_ = peer; }
const time_point& getTime() const { return time; }
const time_point& getReplyTime() const { return reply_time; }
void setTime(const time_point& t) { time = t; }
@@ -84,6 +91,7 @@ struct Node
void requested(const Sp<net::Request>& req);
void received(time_point now, const Sp<net::Request>& req);
Sp<net::Request> getRequest(Tid tid);
const std::map<Tid, Sp<net::Request>>& getRequests() const { return requests_; }
void cancelRequest(const Sp<net::Request>& req);
void setExpired();
@@ -155,6 +163,7 @@ private:
std::map<Tid, Sp<net::Request>> requests_ {};
std::map<Tid, Sp<Socket>> sockets_;
Sp<transport::Peer> transport_ {};
};
} // namespace dht
+1
View File
@@ -24,6 +24,7 @@ struct NodeCache
* To use in case of connectivity change etc.
*/
void clearBadNodes(sa_family_t family = 0);
void clear();
NodeCache(std::mt19937_64& r)
: rd(r) {};
+1 -1
View File
@@ -8,7 +8,7 @@ namespace dht {
namespace proxy {
constexpr const std::chrono::hours OP_TIMEOUT {24}; // one day
constexpr const std::chrono::hours OP_MARGIN {2}; // two hours
constexpr const std::chrono::hours OP_MARGIN {8}; // eight hours
using ListenToken = uint64_t;
+56
View File
@@ -3,6 +3,9 @@
#pragma once
#include "utils.h"
#include "sockaddr.h"
#include <map>
#include <queue>
namespace dht {
@@ -23,6 +26,11 @@ public:
records.pop();
return records.size();
}
/** Whether quota remains, without using it */
bool available(const time_point& now)
{
return quota_ == std::numeric_limits<size_t>::max() or maintain(now) < quota_;
}
/** Return false if quota is reached, insert record and return true otherwise. */
bool limit(const time_point& now)
{
@@ -41,4 +49,52 @@ private:
std::queue<time_point> records {};
};
/**
* A global quota and a quota per address. IPv6 addresses share the quota of their /64 prefix.
* Negative quotas are unlimited.
*/
class AddressRateLimiter
{
public:
AddressRateLimiter(ssize_t quota, ssize_t peerQuota, const duration& period = std::chrono::seconds(1))
: global_(quota, period)
, peerQuota_(peerQuota)
, period_(period)
{}
bool limit(const SockAddr& addr, const time_point& now)
{
// Checked first so that dropped packets create no per-address record
if (not global_.available(now))
return false;
if (peerQuota_ >= 0) {
maintain(now);
auto it = peers_.find(addr);
if (it == peers_.end())
it = peers_.emplace(addr, RateLimiter(peerQuota_, period_)).first;
if (not it->second.limit(now))
return false;
}
return global_.limit(now);
}
/** Number of addresses with a record */
size_t size() const { return peers_.size(); }
private:
void maintain(const time_point& now)
{
if (now < nextMaintenance_)
return;
for (auto it = peers_.begin(); it != peers_.end();)
it = it->second.maintain(now) ? std::next(it) : peers_.erase(it);
nextMaintenance_ = now + period_;
}
RateLimiter global_;
const ssize_t peerQuota_;
const duration period_;
std::map<SockAddr, RateLimiter, SockAddr::ipCmp> peers_ {};
time_point nextMaintenance_ {time_point::min()};
};
} // namespace dht
+5
View File
@@ -284,6 +284,11 @@ public:
void forwardAllMessages(bool forward) { forward_all_ = forward; }
void setPushNotificationToken(const std::string& token = "") override { dht_->setPushNotificationToken(token); }
void setPushNotificationTopic(const std::string& topic) override { dht_->setPushNotificationTopic(topic); }
void setPushNotificationPlatform(const std::string& platform) override
{
dht_->setPushNotificationPlatform(platform);
}
/**
* Call linked callback with push_notification
+3 -1
View File
@@ -10,7 +10,9 @@
#include <future>
#include <functional>
#include <ciso646> // fix windows compiler bug
#if __cplusplus < 202002L
#include <ciso646>
#endif
namespace dht {
+219
View File
@@ -0,0 +1,219 @@
// Copyright (c) 2014-2026 Savoir-faire Linux Inc.
// SPDX-License-Identifier: MIT
#pragma once
#include "def.h"
#include "infohash.h"
#include "sockaddr.h"
#include "utils.h"
#include <array>
#include <optional>
#include <vector>
namespace dht {
namespace transport {
constexpr size_t KEY_SIZE {32};
constexpr size_t SIGNATURE_SIZE {64};
using Key = std::array<uint8_t, KEY_SIZE>;
using Signature = std::array<uint8_t, SIGNATURE_SIZE>;
/** Erases secret key material. */
OPENDHT_PUBLIC void wipe(Key& key);
/**
* X25519 key pair used to encrypt the DHT protocol: the static key of a node, from which its node ID derives,
* or a periodically rotated ephemeral key.
*/
struct OPENDHT_PUBLIC KeyPair
{
Key secret {};
Key pub {};
static KeyPair generate();
static KeyPair fromSecret(const Key& secret);
explicit operator bool() const { return pub != Key {}; }
};
/** Node ID of a node using the transport public key `pub`. */
OPENDHT_PUBLIC InfoHash nodeId(const Key& pub);
/** Secret from the system CSPRNG. */
OPENDHT_PUBLIC Key randomSecret();
/** Ephemeral key pair used with the peer at `address` (IP and port), derived from an epoch secret. */
OPENDHT_PUBLIC KeyPair addressKey(const Key& epochSecret, const uint8_t* address, size_t size);
/** Write token of the plaintext peer at `address` (IP and port), derived from an epoch secret. */
OPENDHT_PUBLIC Key addressToken(const Key& epochSecret, const uint8_t* address, size_t size);
/** Non-expiring authority-signed grant binding a transport public key to a private network. */
struct OPENDHT_PUBLIC Credential
{
static constexpr size_t PACKED_SIZE {1 + KEY_SIZE + SIGNATURE_SIZE};
Key key {};
Signature signature {};
Blob pack() const;
/** @throws std::invalid_argument if the data is not a packed credential */
static Credential unpack(const uint8_t* data, size_t size);
static Credential unpack(const Blob& b) { return unpack(b.data(), b.size()); }
/** True if signed by one of the trusted authorities. */
bool verify(const std::vector<Key>& authorities) const;
explicit operator bool() const { return key != Key {}; }
};
/** Ed25519 key of a private network authority, issuing node credentials. */
struct OPENDHT_PUBLIC Authority
{
Key secret {};
Key pub {};
static Authority generate();
static Authority fromSecret(const Key& secret);
Credential issue(const Key& transportKey) const;
};
/** Transport state of a remote node, attached to its Node. */
struct Peer
{
explicit Peer(const Key& k)
: key(k)
{}
const Key key;
/** Last authenticated ephemeral key of the peer */
std::optional<Key> ephemeral {};
/** Address proven by the Session or Data envelope carrying the ephemeral key */
SockAddr validatedAddress {};
/** Private networks: the peer presented a grant from a trusted authority */
bool authorized {false};
/** Our session was lost: requests other than pings wait for a new ephemeral key */
bool recovering {false};
void setEphemeral(const Key& k, const SockAddr& address = {})
{
ephemeral = k;
validatedAddress = address;
}
void resetEphemeral()
{
ephemeral.reset();
validatedAddress = {};
}
};
/*
* Envelope: marker | type | sender static key | sender ephemeral key | nonce | ChaCha20-Poly1305(payload) | tag
* Data envelope: marker | type | nonce | ChaCha20-Poly1305(payload) | tag, keys found from the source address.
* The marker is never a valid msgpack first byte, so legacy nodes drop envelopes.
* Hello (private networks only, plaintext): marker | type | sender static key | sender ephemeral key | credential | message
*/
constexpr uint8_t ENVELOPE_MARKER {0xC1};
/** Initial: the sender doesn't know the receiver ephemeral key. Session: it does. Data: the receiver has the session. */
enum class EnvelopeType : uint8_t { Initial = 1, Session = 2, Hello = 3, Data = 4 };
/** Type flag: the payload starts with the sender credential */
constexpr uint8_t CREDENTIAL_FLAG {0x80};
constexpr size_t NONCE_SIZE {12};
constexpr size_t TAG_SIZE {16};
constexpr size_t HEADER_SIZE {2 + 2 * KEY_SIZE};
constexpr size_t DATA_HEADER_SIZE {2};
constexpr size_t ENVELOPE_OVERHEAD {HEADER_SIZE + NONCE_SIZE + TAG_SIZE};
constexpr size_t DATA_ENVELOPE_OVERHEAD {DATA_HEADER_SIZE + NONCE_SIZE + TAG_SIZE};
constexpr size_t MAX_ENVELOPE_OVERHEAD {ENVELOPE_OVERHEAD + Credential::PACKED_SIZE};
inline bool
isEnvelope(const uint8_t* data, size_t size)
{
return size >= DATA_HEADER_SIZE and data[0] == ENVELOPE_MARKER;
}
struct Header
{
EnvelopeType type;
bool credential;
/* not in Data envelopes */
Key sender;
Key ephemeral;
};
/** Envelope header, or nullopt if not an envelope of a known type. */
OPENDHT_PUBLIC std::optional<Header> parseHeader(const uint8_t* data, size_t size);
/** Keys to send to and receive from a peer. */
struct SessionKeys
{
Key tx;
Key rx;
/** Erases secret key material. */
void wipe()
{
transport::wipe(tx);
transport::wipe(rx);
}
/** Destructor that wipes secret key material. */
~SessionKeys()
{
wipe();
}
};
/** Session keys with a peer whose ephemeral key is known, or nullopt for an invalid peer key. */
OPENDHT_PUBLIC std::optional<SessionKeys> sessionKeys(
const KeyPair& own, const KeyPair& ephemeral, const Key& peer, const Key& peerEphemeral, NetId network);
/** Key of an Initial envelope to `peer`, or nullopt for an invalid peer key. */
OPENDHT_PUBLIC std::optional<Key> initialSendKey(const KeyPair& own,
const KeyPair& ephemeral,
const Key& peer,
NetId network);
/** Key of an Initial envelope from `peer`, or nullopt for an invalid peer key. */
OPENDHT_PUBLIC std::optional<Key> initialReceiveKey(const KeyPair& own,
const Key& peer,
const Key& peerEphemeral,
NetId network);
/** Encrypts `credential || data`, authenticating the envelope header. */
OPENDHT_PUBLIC Blob seal(const Key& key,
EnvelopeType type,
const Key& sender,
const Key& ephemeral,
const uint8_t* data,
size_t size,
const Blob& credential = {});
/** Encrypts `data` in a Data envelope. */
OPENDHT_PUBLIC Blob sealData(const Key& key, const uint8_t* data, size_t size);
/** Decrypted payload, or nullopt if the envelope is not authentic. */
OPENDHT_PUBLIC std::optional<Blob> open(const Key& key, const uint8_t* data, size_t size);
OPENDHT_PUBLIC Blob
hello(const Key& sender, const Key& ephemeral, const Blob& credential, const uint8_t* data, size_t size);
} // namespace transport
/**
* Encryption of the DHT protocol. Peers that don't support it keep talking in plaintext,
* unless the node is part of a private network.
*/
struct TransportConfig
{
/** Advertise our key and encrypt traffic with nodes supporting it. Disabling uses a legacy random node ID. */
bool enabled {true};
/** Transport key. If empty, generated on start, and kept in the persisted state if Config::persist_path is set. */
transport::KeyPair key {};
/** Network authorities. If set, only nodes with a credential issued by one of them are accepted. */
std::vector<transport::Key> authorities {};
/** Credential of `key`, required for private networks. */
transport::Credential credential {};
bool isPrivate() const { return not authorities.empty(); }
};
} // namespace dht
+8
View File
@@ -85,6 +85,14 @@ using system_clock = std::chrono::system_clock;
using time_point = clock::time_point;
using duration = clock::duration;
OPENDHT_PUBLIC time_point from_system_time(system_clock::time_point t,
time_point steady_now,
system_clock::time_point system_now);
OPENDHT_PUBLIC time_point from_system_time(system_clock::time_point t);
OPENDHT_PUBLIC system_clock::time_point to_system_time(time_point t,
time_point steady_now,
system_clock::time_point system_now);
OPENDHT_PUBLIC system_clock::time_point to_system_time(time_point t);
OPENDHT_PUBLIC time_point from_time_t(std::time_t t);
OPENDHT_PUBLIC std::time_t to_time_t(time_point t);
+17 -1
View File
@@ -742,6 +742,7 @@ struct OPENDHT_PUBLIC FieldValue
switch (field) {
case Value::Field::Id:
case Value::Field::ValueType:
case Value::Field::SeqNum:
p.pack(intValue);
break;
case Value::Field::OwnerPk:
@@ -773,6 +774,7 @@ struct OPENDHT_PUBLIC FieldValue
switch (field) {
case Value::Field::Id:
case Value::Field::ValueType:
case Value::Field::SeqNum:
intValue = v->as<decltype(intValue)>();
break;
case Value::Field::OwnerPk:
@@ -1015,7 +1017,7 @@ struct OPENDHT_PUBLIC Query
*
* where
*
* - $field$ = *|id|value_type|owner_pk|user_type
* - $field$ = *|id|value_type|seq|owner_pk|user_type
* - $value$ = $string$|$integer$
* - $string$: a simple string WITHOUT SPACES.
* - $integer$: a simple integer.
@@ -1032,9 +1034,23 @@ struct OPENDHT_PUBLIC Query
/**
* Tell if the query is satisfied by another query.
*
* The answer to `q` contains every value this query asks for, but it may
* contain more: the caller is responsible for applying this query's own
* filters to the values it receives.
*/
bool isSatisfiedBy(const Query& q) const;
/**
* Tell if the projected answer to another query can be reported for this
* query without any further filtering.
*
* A field-projected answer only carries the fields selected by `q`, so a
* narrower filter can no longer be evaluated on it. Both queries must
* therefore apply exactly the same filters.
*/
bool isProjectionSatisfiedBy(const Query& q) const;
template<typename Packer>
void msgpack_pack(Packer& pk) const
{
+113 -8
View File
@@ -5,26 +5,41 @@ project(
'cython',
license: 'MIT',
license_files: 'LICENSE',
version: '4.0.0',
version: '4.4.99',
meson_version: '>=1.1.0',
default_options: ['cpp_std=c++20', 'warning_level=3'],
)
gnutls = dependency('gnutls')
nettle = dependency('nettle')
hogweed = dependency('hogweed', version: '>=3.1')
msgpack = dependency('msgpack-cxx', required: false)
argon2 = dependency('libargon2')
fmt = dependency('fmt')
openssl = dependency('openssl', required: get_option('proxy_client'))
jsoncpp = dependency('jsoncpp', required: get_option('proxy_client'))
llhttp = dependency('llhttp', 'libllhttp', required: get_option('proxy_client'))
simdutf = dependency('simdutf', required: get_option('proxy_client'))
proxy_enabled = get_option('proxy_client').enabled() or get_option('proxy_server').enabled()
openssl = dependency('openssl', required: proxy_enabled)
jsoncpp = dependency('jsoncpp', required: proxy_enabled)
llhttp = dependency('llhttp', 'libllhttp', required: proxy_enabled)
simdutf = dependency('simdutf', required: proxy_enabled)
# restinio is a header-only library that may not ship a pkg-config file
restinio_dep = dependency('restinio', required: false)
if not restinio_dep.found()
cpp = meson.get_compiler('cpp')
if cpp.has_header('restinio/all.hpp', required: proxy_enabled)
restinio_dep = declare_dependency()
endif
endif
io_uring = host_machine.system() == 'linux' ? dependency('liburing', required: false) : disabler()
deps = [
fmt,
gnutls,
nettle,
hogweed,
msgpack,
argon2,
openssl,
@@ -33,6 +48,10 @@ deps = [
llhttp,
]
if restinio_dep.found()
deps += restinio_dep
endif
if host_machine.system() == 'linux' and io_uring.found()
deps += io_uring
endif
@@ -67,6 +86,7 @@ opendht_src = [
'src/routing_table.cpp',
'src/node_cache.cpp',
'src/network_engine.cpp',
'src/transport.cpp',
'src/securedht.cpp',
'src/dhtrunner.cpp',
'src/log.cpp',
@@ -79,7 +99,7 @@ if get_option('indexation').enabled()
opendht_src += 'src/indexation/pht.cpp'
add_project_arguments('-DOPENDHT_INDEXATION', language: 'cpp')
endif
if jsoncpp.found()
if jsoncpp.found() and simdutf.found()
opendht_src += ['src/base64.cpp']
add_project_arguments('-DOPENDHT_JSONCPP', language: 'cpp')
conf_data.set('jsoncpp_lib', ', jsoncpp')
@@ -188,19 +208,68 @@ if get_option('c').enabled()
endif
endif
rust_option = get_option('rust').require(
get_option('c').enabled(),
error_message: 'rust requires c',
).require(
get_option('default_library') != 'static',
error_message: 'rust requires shared libraries',
)
if rust_option.allowed()
cargo = find_program('cargo', required: rust_option)
if cargo.found()
rust_env = environment()
rust_env.prepend('LIBRARY_PATH', meson.current_build_dir())
rust_env.prepend('LD_LIBRARY_PATH', meson.current_build_dir())
rust_env.prepend('DYLD_LIBRARY_PATH', meson.current_build_dir())
rust_env.prepend('PATH', meson.current_build_dir())
cargo_args = [
'--manifest-path', meson.current_source_dir() / 'rust/Cargo.toml',
'--target-dir', meson.current_build_dir() / 'rust',
]
rust_build = custom_target(
'rust-bindings',
output: 'rust-build.json',
command: [cargo, 'build', '--lib', '--examples', '--message-format=json-render-diagnostics'] + cargo_args,
env: rust_env,
depends: opendht_c,
capture: true,
build_by_default: true,
build_always_stale: true,
)
if get_option('tests').enabled()
test(
'Rust',
cargo,
args: ['test'] + cargo_args,
env: rust_env,
depends: rust_build,
timeout: 120,
)
endif
endif
endif
if get_option('python').require(
get_option('indexation').enabled(),
error_message: 'python requires indexation',
).allowed()
py = import('python').find_installation()
py = import('python').find_installation(pure: false)
dep_py = py.dependency()
py.extension_module(
'opendht',
'_core',
'python/opendht.pyx',
subdir: 'opendht',
cython_args: ['--module-name', 'opendht._core'],
override_options: ['cython_language=cpp'],
dependencies: [opendht_dep, msgpack, jsoncpp, fmt, dep_py],
install: true,
)
py.install_sources(
'python/opendht/__init__.py',
'python/opendht/aio.py',
subdir: 'opendht',
)
endif
if get_option('tools').enabled()
@@ -239,6 +308,16 @@ endif
# Unit tests
if get_option('tests').enabled()
if get_option('c').enabled()
test_c = executable(
'test_c',
'tests/test_c.c',
include_directories: ['c', opendht_interface_inc],
dependencies: [gnutls],
link_with: opendht_c,
)
test('CBindings', test_c)
endif
cppunit = dependency('cppunit')
test_infohash = executable(
@@ -275,6 +354,24 @@ if get_option('tests').enabled()
)
test('NetworkEngine', test_networkengine)
test_opcache = executable(
'test_opcache',
'tests/test_opcache.cpp',
'tests/tests_runner.cpp',
include_directories: opendht_inc,
dependencies: [opendht_dep, cppunit, jsoncpp, fmt, openssl, msgpack],
)
test('OpCache', test_opcache)
test_valuecache = executable(
'test_valuecache',
'tests/test_valuecache.cpp',
'tests/tests_runner.cpp',
include_directories: opendht_inc,
dependencies: [opendht_dep, cppunit, jsoncpp, fmt, openssl, msgpack],
)
test('ValueCache', test_valuecache)
test_crypto = executable(
'test_crypto',
'tests/test_crypto.cpp',
@@ -283,6 +380,14 @@ if get_option('tests').enabled()
)
test('Crypto', test_crypto, timeout: 60)
test_transport = executable(
'test_transport',
'tests/test_transport.cpp',
'tests/tests_runner.cpp',
dependencies: [opendht_dep, cppunit, jsoncpp, fmt, openssl, msgpack],
)
test('Transport', test_transport)
test_dhtrunner = executable(
'test_dhtrunner',
'tests/test_dhtrunner.cpp',
+1
View File
@@ -4,6 +4,7 @@ option('push_notifications', type : 'feature', value : 'disabled')
option('peer_discovery', type : 'feature', value : 'enabled')
option('tools', type : 'feature', value : 'enabled')
option('c', type : 'feature', value : 'enabled')
option('rust', type : 'feature', value : 'disabled')
option('indexation', type : 'feature', value : 'enabled')
option('python', type : 'feature', value : 'enabled')
option('tests', type : 'feature', value : 'enabled')
+5
View File
@@ -0,0 +1,5 @@
node_modules/
build*/
prebuilds/
dist/
examples/dhtnode.js
+64
View File
@@ -0,0 +1,64 @@
cmake_minimum_required(VERSION 3.21)
project(opendht_node LANGUAGES C CXX)
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
set(OPENDHT_TOOLS OFF CACHE BOOL "Build DHT tools")
set(OPENDHT_PYTHON OFF CACHE BOOL "Build Python bindings")
set(OPENDHT_PROXY_CLIENT ON CACHE BOOL "Enable DHT proxy client")
set(OPENDHT_PEER_DISCOVERY OFF CACHE BOOL "Enable multicast peer discovery")
set(OPENDHT_DOCUMENTATION OFF CACHE BOOL "Build documentation")
set(BUILD_TESTING OFF CACHE BOOL "Build native tests")
add_subdirectory(.. native EXCLUDE_FROM_ALL)
find_program(NODE_EXECUTABLE node REQUIRED)
execute_process(
COMMAND "${NODE_EXECUTABLE}" --input-type=module -e "import addon from 'node-addon-api'; process.stdout.write(addon.include_dir)"
WORKING_DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}"
OUTPUT_VARIABLE NODE_ADDON_API_DIR
OUTPUT_STRIP_TRAILING_WHITESPACE
COMMAND_ERROR_IS_FATAL ANY
)
add_library(opendht_node MODULE addon.cpp crypto.cpp ${CMAKE_JS_SRC})
target_compile_definitions(opendht_node PRIVATE NAPI_VERSION=8 NAPI_CPP_EXCEPTIONS)
target_include_directories(opendht_node PRIVATE
${CMAKE_JS_INC}
"${CMAKE_CURRENT_SOURCE_DIR}/${NODE_ADDON_API_DIR}"
"${CMAKE_CURRENT_SOURCE_DIR}/../include"
"${CMAKE_CURRENT_BINARY_DIR}/native/include"
)
target_link_libraries(opendht_node PRIVATE opendht ${CMAKE_JS_LIB})
if(OPENDHT_SANITIZE AND CMAKE_CXX_COMPILER_ID MATCHES "GNU|Clang")
target_compile_options(opendht_node PRIVATE -fsanitize=address -fstack-protector-strong)
target_link_options(opendht_node PRIVATE -fsanitize=address)
endif()
set_target_properties(opendht_node PROPERTIES
PREFIX ""
SUFFIX ".node"
OUTPUT_NAME "opendht"
LIBRARY_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}/$<CONFIG>"
RUNTIME_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}/$<CONFIG>"
)
if(APPLE)
target_link_options(opendht_node PRIVATE "-undefined" "dynamic_lookup")
endif()
if(WIN32)
add_custom_command(TARGET opendht_node POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different
$<TARGET_RUNTIME_DLLS:opendht_node> $<TARGET_FILE_DIR:opendht_node>
COMMAND_EXPAND_LISTS
)
endif()
if(MSVC)
target_compile_options(opendht_node PRIVATE /EHsc)
target_include_directories(opendht_node PRIVATE
"${CMAKE_CURRENT_SOURCE_DIR}/../src/compat/msvc"
"${CMAKE_CURRENT_SOURCE_DIR}/../src/compat/win32"
)
if(CMAKE_JS_NODELIB_DEF AND CMAKE_JS_NODELIB_TARGET)
execute_process(COMMAND ${CMAKE_AR} /def:${CMAKE_JS_NODELIB_DEF}
/out:${CMAKE_JS_NODELIB_TARGET} ${CMAKE_STATIC_LINKER_FLAGS})
endif()
endif()
+243
View File
@@ -0,0 +1,243 @@
# OpenDHT Node.js Bindings
Initial Node-API bindings using `node-addon-api` directly over the C++ library.
Node.js 22.9 or later is required; the addon targets Node-API version 8.
## Install
```sh
npm install opendht
```
## Build
Install the native dependencies described in [BUILD.md](../BUILD.md), plus Node.js,
a C++17 compiler, and CMake 3.21 or later. Ninja is optional. Proxy-client support
defaults to enabled and requires ASIO, llhttp, RESTinio, and expected-lite headers.
```sh
cd nodejs
npm ci
npm run build
npm test
npm run typecheck
```
CMake.js builds OpenDHT from the parent source directory in `nodejs/build/native`.
It does not change the repository's existing build directory. Tools, Python,
peer discovery, documentation, and native tests default to disabled in this build.
Native CMake options can be passed through, for example:
```sh
npm run build -- --CDOPENDHT_PEER_DISCOVERY=ON
```
For an existing build, change cached options directly with CMake, for example
`cmake -S . -B build -DOPENDHT_PROXY_CLIENT=ON`, then rebuild. A custom build can
disable proxy support with `-DOPENDHT_PROXY_CLIENT=OFF`; requesting a proxy then
throws an explicit unsupported-build error.
The loader checks local builds, then compatible Node-API binaries under `prebuilds/`.
Set `OPENDHT_NODE_BINDING` to an absolute addon path to select an explicit build.
Packaged binaries include OpenDHT and non-system native dependencies, so archive
installation does not require a compiler or a system OpenDHT installation.
## Usage
```js
import { DhtRunner, hash } from 'opendht';
async function main() {
const writer = new DhtRunner();
const reader = new DhtRunner();
try {
writer.run({ port: 0, ipv4: '127.0.0.1', ipv6: '::1' });
reader.run({ port: 0, ipv4: '127.0.0.1', ipv6: '::1' });
writer.bootstrap('127.0.0.1', reader.getBoundPort());
reader.bootstrap('127.0.0.1', writer.getBoundPort());
const key = hash('example');
const stored = await writer.put(key, Buffer.from('hello'), { id: 42n });
if (!stored) throw new Error('DHT put failed');
for await (const value of reader.get(key)) {
console.log(value.id, value.data.toString());
}
} finally {
await Promise.all([writer.close(), reader.close()]);
}
}
main().catch(console.error);
```
## API
- `hash(string | Uint8Array)` returns a SHA-1 hash as a 40-character hex string.
All get/put/listen keys must be hex hashes, not unhashed application keys.
- `version` is the underlying OpenDHT version.
- `run({port, ipv4, ipv6, network, persistPath, identity})` starts a threaded node. Port 0
selects a free port. The defaults bind both IP families on all interfaces.
`persistPath` is optional; use an empty string to disable persistence.
`proxyServer` and `proxyUserAgent` configure proxy mode (see below).
- `getBoundPort(4 | 6)` defaults to IPv4; `getNodeId()` returns the node's hex ID.
- `getStatus()` returns the latest JS-dispatched `{ipv4, ipv6}` connectivity
snapshot. Each family is `'disconnected'`, `'connecting'`, or `'connected'`.
`runner.on('statusChanged', status => {})` receives changes on the JS thread.
New and closing/closed runners report both families as disconnected. No status
events are delivered after close begins; close does not emit a final event.
- `bootstrap(host, port)` queues bootstrap discovery. Returning does not mean the
peer is connected; subsequent DHT operations report their own completion.
- `put(key, Uint8Array, {id, type, userType, permanent})` copies the input and
resolves to the native success boolean. IDs are unsigned 64-bit `bigint` values.
- `get(key)` returns an async iterator of values as they arrive. The iterator
ends when the lookup completes, or rejects with `ERR_DHT_GET_FAILED` on failure.
`getAll(key)` returns a single promise resolving to the complete value array.
Values contain `id`, `type`, `seq`, `userType`, copied `Buffer` data, `isSigned`,
`isEncrypted`, and an `owner` long fingerprint (or null for unsigned values).
- `listen(key, (value, expired) => {})` returns a listener with idempotent
`cancel()`. Queued deliveries are suppressed after cancellation. Callback
return values are ignored; callbacks may cancel or close the runner.
- `listen(key)` without a callback returns an async iterator of `{value, expired}`
events. It continues until cancelled or the runner closes.
- `close()` is idempotent and returns the same promise on repeated calls. It
waits off the JS thread for graceful shutdown, then joins the native threads.
After two seconds it forces the join if graceful shutdown has not completed.
Pending operations reject with `ERR_DHT_CLOSED`. A closed runner cannot restart.
- `ref()` and `unref()` control whether the runner keeps Node's event loop alive.
Running nodes are referenced by default. `Symbol.asyncDispose` calls `close()`.
Callbacks run only on their owning JS thread. There is no polling timer or worker
thread waiting on each get/put. Explicitly close runners; GC cleanup and Node
environment cleanup hooks are fallbacks, including forced worker termination.
Pending operations, active listeners, and registered status handlers keep their
runner alive until completion, cancellation, handler removal, or explicit close.
Graceful persistence cannot be guaranteed if the process is killed.
The binding does not cap the callback queue, accumulated `getAll()` results, or
iterator buffers. Slow iterator consumers can accumulate values without
backpressure. Queued callbacks retain their native values until delivery or cleanup.
## Async Iteration
`get()` and callback-free `listen()` start immediately and return iterators with
`next()`, `return()`, `throw()`, `cancel()`, and `Symbol.asyncDispose`.
Leaving a `for await` loop with `break` or an exception cancels the subscription.
Explicit cancellation discards buffered values and settles pending reads as done.
Closing the runner rejects active get iterators with `ERR_DHT_CLOSED`; listener
iterators end normally. Successful get completion preserves unread buffered values.
```js
for await (const value of runner.get(hash('example'))) {
console.log(value.id, value.data);
}
const snapshots = await runner.getAll(hash('example'));
for await (const { value, expired } of runner.listen(hash('updates'))) {
console.log(value.id, expired);
if (expired) break;
}
```
## Proxy Configuration
Pass the URL of an OpenDHT HTTP(S) proxy at startup:
```js
runner.run({
proxyServer: 'https://proxy.example.com',
proxyUserAgent: 'my-app/1.0',
});
```
This selects the native OpenDHT proxy client, not an HTTP CONNECT or SOCKS proxy.
The node does not bind UDP sockets in this mode; `port`, `ipv4`, `ipv6`, and
`bootstrap()` are for direct DHT nodes. Omit `proxyServer` or use an empty string
for direct networking. Proxy settings are startup-only; create a new runner to
change transports or servers. Packaged binaries include proxy-client support.
## Crypto
`PrivateKey.generate(options)` and `Identity.generate(name, options)` return
promises and generate keys off the JS thread. Options accept `algorithm: 'ec'`
(the default, using OpenDHT's default EC curve) or `'rsa'`. RSA `bits` must be an
integer between 2048 and 8192; the default is 2048.
- `new PrivateKey(bytes, password?)` imports PEM/DER; `serialize(password?)`
exports PKCS#8 PEM. A nonempty password encrypts the exported key.
- `privateKey.getPublicKey()` returns an owned public key. `sign(bytes)` and
`decrypt(bytes)` return copied Buffers.
- `new PublicKey(bytes)` imports a public key; `serialize()` exports DER.
`checkSignature(data, signature)` returns a boolean. `encrypt(bytes)` requires
an RSA key; EC keys support signatures, not this encryption scheme.
- `new Certificate(bytes)` imports PEM/DER or a PEM certificate chain;
`serialize()` exports a PEM chain. Import does not establish trust or validate
the chain against a trust store. `getPublicKey()` returns its public key.
- Public keys and certificates expose `getId()` (40 hex characters) and
`getLongId()` (64 hex characters).
- `new Identity(privateKey, certificate)` requires matching fingerprints.
Its read-only `privateKey` and `certificate` properties return owned wrappers.
- Pass `identity` to `run()` to enable `putSigned(key, data, options?)` and
`putEncrypted(key, data, recipientPublicKey, options?)`. Both return the same
success boolean as `put()`. Encrypted values are signed by the sender and
transparently decrypted for the recipient by OpenDHT's get/listen layer.
```js
import { Identity, DhtRunner } from 'opendht';
async function createAuthenticatedNode() {
const identity = await Identity.generate('example', { algorithm: 'rsa', bits: 2048 });
const runner = new DhtRunner();
runner.run({ port: 0, identity });
return { runner, identity };
}
```
Crypto methods take Uint8Array/Buffer inputs, not implicit text encodings. Imports
must be nonempty. Malformed handles/data,
mismatched identities, and unsupported encryption keys raise JS exceptions.
Only generation is asynchronous; import/export/sign/encrypt/decrypt are
synchronous and should be used from worker threads for large workloads.
## Packaging And CI
From a successful Release build:
```sh
npm run package
npm run test:package
```
`bundle` copies the addon into `prebuilds/<platform>-<arch>/`, gathers its runtime
dependencies, rewrites native loader paths, and copies dependency license
notices. macOS needs `dylibbundler` and Apple developer tools; Linux needs
`patchelf` and Debian/Ubuntu dependency copyright files. Windows uses the vcpkg
installed tree (`VCPKG_INSTALLED_DIR`, `VCPKG_DEFAULT_TRIPLET`) and MSVC tools.
`OPENDHT_LIBRARY_PATH` adds native library search directories when bundling.
An explicit addon path may be passed to `npm run bundle -- /absolute/addon.node`.
Bundling replaces only the current platform's generated prebuild directory.
`test:package` creates an npm archive, installs it in a temporary consumer with
install scripts disabled and no dev dependencies, forces prebuild-only loading,
and runs the full runtime suite. Pass an archive file or a directory containing
one archive to test it directly, for example `npm run test:package -- dist`.
Only runtime JS, declarations, the example's TypeScript source, documentation,
and prebuilds are included in the archive. `prepack` compiles the example, so
archive creation needs the development dependencies; archive installation does not.
The [Node.js package workflow](../.github/workflows/nodejs-package.yml) builds:
| Target | Build Baseline |
| --- | --- |
| Linux x64 | Ubuntu 24.04 / glibc 2.39 (Noble dependency image) |
| macOS ARM64 and x64 | macOS 15 |
| Windows x64 | Windows Server 2022 / MSVC |
CI combines platform prebuilds into one npm archive and verifies that archive on
Node 22, 24, and 26 on each platform, including Ubuntu 24.04 and 26.04. Branch
pushes, pull requests, and manual runs build and test without publishing.
musl/Alpine and other architectures are not currently built. Node-API stability
does not remove OS/libc or native dependency compatibility requirements. Review
the included dependency licenses and source-distribution obligations before
redistributing binaries.
Query filters, AbortSignal support, certificate trust-store validation, CA
issuance controls, and runtime proxy switching remain future work.
+734
View File
@@ -0,0 +1,734 @@
#include <napi.h>
#include <opendht.h>
#include <gnutls/gnutls.h>
#include "crypto.h"
#include <algorithm>
#include <atomic>
#include <chrono>
#include <cmath>
#include <condition_variable>
#include <functional>
#include <mutex>
#include <optional>
#include <unordered_map>
namespace {
struct GnuTlsContext
{
const int result = gnutls_global_init();
~GnuTlsContext()
{
if (result == GNUTLS_E_SUCCESS)
gnutls_global_deinit();
}
};
using opendht_node::guarded;
uint32_t
unsignedNumber(Napi::Value value, uint32_t maximum, const char* name)
{
if (!value.IsNumber())
throw Napi::TypeError::New(value.Env(), std::string(name) + " must be a number");
const auto number = value.As<Napi::Number>().DoubleValue();
if (!std::isfinite(number) || number < 0 || number > maximum || std::floor(number) != number)
throw Napi::RangeError::New(value.Env(), std::string(name) + " is out of range");
return static_cast<uint32_t>(number);
}
uint64_t
unsignedBigInt(Napi::Value value)
{
if (!value.IsBigInt())
throw Napi::TypeError::New(value.Env(), "id must be a bigint");
int sign = 0;
size_t count = 1;
uint64_t word = 0;
value.As<Napi::BigInt>().ToWords(&sign, &count, &word);
if (sign || count > 1)
throw Napi::RangeError::New(value.Env(), "id must fit in an unsigned 64-bit integer");
return word;
}
std::string
text(Napi::Value value, const char* name)
{
if (!value.IsString())
throw Napi::TypeError::New(value.Env(), std::string(name) + " must be a string");
auto result = value.As<Napi::String>().Utf8Value();
if (result.find('\0') != std::string::npos)
throw Napi::TypeError::New(value.Env(), std::string(name) + " must not contain NUL bytes");
return result;
}
dht::InfoHash
infoHash(Napi::Value value)
{
const auto result = text(value, "hash");
if (result.size() != 40 || result.find_first_not_of("0123456789abcdefABCDEF") != std::string::npos)
throw Napi::TypeError::New(value.Env(), "Expected a 40-character hexadecimal hash");
return dht::InfoHash(result);
}
Napi::Object
optionsObject(Napi::Value value)
{
if (!value.IsObject() || value.IsArray() || value.IsNull())
throw Napi::TypeError::New(value.Env(), "options must be an object");
return value.As<Napi::Object>();
}
Napi::Object
valueToJS(Napi::Env env, const dht::Value& value)
{
auto result = Napi::Object::New(env);
result.Set("id", Napi::BigInt::New(env, value.id));
result.Set("type", Napi::Number::New(env, value.type));
result.Set("seq", Napi::Number::New(env, value.seq));
result.Set("data", Napi::Buffer<uint8_t>::Copy(env, value.data.data(), value.data.size()));
result.Set("userType", Napi::String::New(env, value.user_type));
result.Set("owner",
value.owner ? Napi::Value(Napi::String::New(env, value.owner->getLongId().to_view()))
: Napi::Value(env.Null()));
result.Set("isSigned", Napi::Boolean::New(env, value.isSigned()));
result.Set("isEncrypted", Napi::Boolean::New(env, value.isEncrypted()));
return result;
}
struct PendingOperation
{
Napi::Promise::Deferred deferred;
bool get;
std::vector<std::shared_ptr<const dht::Value>> values;
Napi::FunctionReference onValue;
std::shared_ptr<std::atomic_bool> cancelled;
};
struct Subscription
{
dht::InfoHash key;
std::shared_future<size_t> token;
Napi::FunctionReference callback;
};
struct RunnerState
{
using Callback = std::function<void(RunnerState&)>;
explicit RunnerState(Napi::Env environment)
: env(environment)
{}
static std::shared_ptr<RunnerState> create(Napi::Env env)
{
auto state = std::make_shared<RunnerState>(env);
auto dispatchOwner = std::make_unique<std::shared_ptr<RunnerState>>(state);
napi_threadsafe_function function;
if (napi_create_threadsafe_function(env,
nullptr,
nullptr,
Napi::String::New(env, "OpenDHT callbacks"),
0,
1,
dispatchOwner.get(),
finalizeDispatcher,
state.get(),
dispatch,
&function)
!= napi_ok)
throw Napi::Error::New(env, "Could not create OpenDHT callback dispatcher");
dispatchOwner.release();
state->dispatcher = Napi::ThreadSafeFunction(function);
state->dispatcher.Unref(env);
auto closeOwner = std::make_unique<std::shared_ptr<RunnerState>>(state);
if (napi_create_async_work(env,
nullptr,
Napi::String::New(env, "OpenDHT close"),
executeClose,
completeClose,
closeOwner.get(),
&state->closeWork)
!= napi_ok) {
state->dispatcher.Release();
throw Napi::Error::New(env, "Could not create OpenDHT cleanup work");
}
if (napi_add_async_cleanup_hook(env, cleanup, state.get(), &state->cleanupHook) != napi_ok) {
napi_delete_async_work(env, state->closeWork);
state->dispatcher.Release();
throw Napi::Error::New(env, "Could not register OpenDHT cleanup hook");
}
state->closeOwner = closeOwner.release();
return state;
}
void requireRunning() const
{
if (closing)
throw Napi::Error::New(env, "DHT runner is closed");
if (!started)
throw Napi::Error::New(env, "DHT runner is not running");
}
void startClose()
{
if (closing.exchange(true))
return;
if (napi_queue_async_work(env, closeWork) != napi_ok) {
executeClose(env, closeOwner);
completeClose(env, napi_ok, closeOwner);
}
}
void send(Callback callback)
{
if (closing)
return;
auto task = std::make_unique<Callback>(std::move(callback));
if (dispatcher.NonBlockingCall(static_cast<void*>(task.get())) == napi_ok)
task.release();
}
void sendValue(uint64_t operation, std::shared_ptr<const dht::Value> value, bool expired = false)
{
send([operation, value = std::move(value), expired](RunnerState& state) mutable {
const auto listener = state.listeners.find(operation);
if (listener != state.listeners.end()) {
listener->second.callback.Call({valueToJS(state.env, *value), Napi::Boolean::New(state.env, expired)});
} else if (const auto pending = state.operations.find(operation); pending != state.operations.end()) {
if (!pending->second.onValue.IsEmpty())
pending->second.onValue.Call({valueToJS(state.env, *value)});
else
pending->second.values.emplace_back(std::move(value));
}
});
}
void completeOperation(uint64_t identifier, bool ok)
{
const auto pending = operations.find(identifier);
if (pending == operations.end())
return;
auto operation = std::move(pending->second);
operations.erase(pending);
if (!operation.get) {
operation.deferred.Resolve(Napi::Boolean::New(env, ok));
} else if (!ok) {
auto error = Napi::Error::New(env, "DHT get failed");
error.Set("code", "ERR_DHT_GET_FAILED");
operation.deferred.Reject(error.Value());
} else if (!operation.onValue.IsEmpty()) {
operation.deferred.Resolve(env.Undefined());
} else {
auto values = Napi::Array::New(env, operation.values.size());
for (size_t index = 0; index < operation.values.size(); ++index)
values.Set(static_cast<uint32_t>(index), valueToJS(env, *operation.values[index]));
operation.deferred.Resolve(values);
}
}
Napi::Object status() const
{
auto result = Napi::Object::New(env);
result.Set("ipv4", Napi::String::New(env, dht::statusToStr(closing ? dht::NodeStatus::Disconnected : status4)));
result.Set("ipv6", Napi::String::New(env, dht::statusToStr(closing ? dht::NodeStatus::Disconnected : status6)));
return result;
}
static void dispatch(napi_env environment, napi_value, void* context, void* data)
{
auto state = static_cast<RunnerState*>(context);
std::unique_ptr<Callback> callback(static_cast<Callback*>(data));
if (!environment || state->envClosing || state->closing)
return;
try {
(*callback)(*state);
} catch (const Napi::Error& error) {
napi_fatal_exception(environment, error.Value());
} catch (const std::exception& error) {
napi_fatal_exception(environment, Napi::Error::New(environment, error.what()).Value());
}
}
static void finalizeDispatcher(napi_env, void* data, void*)
{
delete static_cast<std::shared_ptr<RunnerState>*>(data);
}
static void cleanup(napi_async_cleanup_hook_handle, void* data)
{
auto state = static_cast<RunnerState*>(data);
state->envClosing = true;
state->startClose();
}
static void executeClose(napi_env, void* data)
{
auto state = *static_cast<std::shared_ptr<RunnerState>*>(data);
struct ShutdownSignal
{
std::mutex mutex;
std::condition_variable condition;
bool done = false;
};
try {
auto signal = std::make_shared<ShutdownSignal>();
state->runner.shutdown(
[signal] {
std::lock_guard lock(signal->mutex);
signal->done = true;
signal->condition.notify_one();
},
true);
std::unique_lock lock(signal->mutex);
signal->condition.wait_for(lock, std::chrono::seconds(2), [&] { return signal->done; });
} catch (const std::exception& error) {
state->closeError = error.what();
}
state->runner.join();
}
Napi::Error makeCloseError() const
{
auto error = Napi::Error::New(env, closeError.empty() ? "DHT runner is closed" : closeError);
error.Set("code", "ERR_DHT_CLOSED");
return error;
}
static void completeClose(napi_env environment, napi_status, void* data)
{
std::unique_ptr<std::shared_ptr<RunnerState>> owner(static_cast<std::shared_ptr<RunnerState>*>(data));
auto state = *owner;
state->finished = true;
if (!state->envClosing) {
state->dispatcher.Release();
auto error = state->makeCloseError();
for (auto& pending : state->operations)
pending.second.deferred.Reject(error.Value());
if (state->closeDeferred) {
if (state->closeError.empty())
state->closeDeferred->Resolve(Napi::Env(environment).Undefined());
else
state->closeDeferred->Reject(error.Value());
}
}
state->operations.clear();
state->listeners.clear();
state->closeDeferred.reset();
state->closePromise.Reset();
state->onStatus.Reset();
state->owner.Reset();
napi_delete_async_work(environment, state->closeWork);
napi_remove_async_cleanup_hook(state->cleanupHook);
state->closeOwner = nullptr;
}
Napi::Env env;
dht::DhtRunner runner;
Napi::ThreadSafeFunction dispatcher;
napi_async_work closeWork {};
napi_async_cleanup_hook_handle cleanupHook {};
std::shared_ptr<RunnerState>* closeOwner {};
std::atomic_bool closing {false};
bool envClosing = false;
bool started = false;
bool hasIdentity = false;
bool finished = false;
uint64_t nextOperation = 1;
std::string closeError;
std::unordered_map<uint64_t, PendingOperation> operations;
std::unordered_map<uint64_t, Subscription> listeners;
std::optional<Napi::Promise::Deferred> closeDeferred;
Napi::Reference<Napi::Promise> closePromise;
Napi::FunctionReference onStatus;
Napi::ObjectReference owner;
dht::NodeStatus status4 = dht::NodeStatus::Disconnected;
dht::NodeStatus status6 = dht::NodeStatus::Disconnected;
};
class Runner : public Napi::ObjectWrap<Runner>
{
public:
explicit Runner(const Napi::CallbackInfo& info)
: Napi::ObjectWrap<Runner>(info)
, state_(RunnerState::create(info.Env()))
{
if (info.Length() && info[0].IsFunction())
state_->onStatus = Napi::Persistent(info[0].As<Napi::Function>());
}
~Runner() override { state_->startClose(); }
static Napi::Function define(Napi::Env env)
{
return DefineClass(env,
"NativeRunner",
{
InstanceMethod("run", &Runner::run),
InstanceMethod("close", &Runner::close),
InstanceMethod("isRunning", &Runner::isRunning),
InstanceMethod("getBoundPort", &Runner::getBoundPort),
InstanceMethod("getNodeId", &Runner::getNodeId),
InstanceMethod("getStatus", &Runner::getStatus),
InstanceMethod("bootstrap", &Runner::bootstrap),
InstanceMethod("get", &Runner::get),
InstanceMethod("getAll", &Runner::getAll),
InstanceMethod("cancelGet", &Runner::cancelGet),
InstanceMethod("put", &Runner::put),
InstanceMethod("putSigned", &Runner::putSigned),
InstanceMethod("putEncrypted", &Runner::putEncrypted),
InstanceMethod("listen", &Runner::listen),
InstanceMethod("cancelListen", &Runner::cancelListen),
InstanceMethod("ref", &Runner::ref),
InstanceMethod("unref", &Runner::unref),
InstanceMethod("setOwner", &Runner::setOwner),
});
}
private:
Napi::Value run(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
if (state_->closing)
throw Napi::Error::New(info.Env(), "DHT runner is closed");
if (state_->started)
throw Napi::Error::New(info.Env(), "DHT runner is already running");
auto options = info.Length() && !info[0].IsUndefined() ? optionsObject(info[0])
: Napi::Object::New(info.Env());
const auto port = options.Has("port") ? unsignedNumber(options.Get("port"), 65535, "port") : 0;
const auto ipv4 = options.Has("ipv4") ? text(options.Get("ipv4"), "ipv4") : "0.0.0.0";
const auto ipv6 = options.Has("ipv6") ? text(options.Get("ipv6"), "ipv6") : "::";
dht::DhtRunner::Config config;
if (options.Has("network"))
config.dht_config.node_config.network = unsignedNumber(options.Get("network"), UINT32_MAX, "network");
if (options.Has("persistPath"))
config.dht_config.node_config.persist_path = text(options.Get("persistPath"), "persistPath");
if (options.Has("identity"))
config.dht_config.id = opendht_node::readIdentity(options.Get("identity"));
if (options.Has("proxyServer"))
config.proxy_server = text(options.Get("proxyServer"), "proxyServer");
if (options.Has("proxyUserAgent")) {
config.proxy_user_agent = text(options.Get("proxyUserAgent"), "proxyUserAgent");
if (config.proxy_user_agent.find_first_of("\r\n") != std::string::npos)
throw Napi::TypeError::New(info.Env(), "proxyUserAgent must not contain line breaks");
}
dht::DhtRunner::Context context;
const std::weak_ptr<RunnerState> state = state_;
context.statusChangedCallback = [state](dht::NodeStatus ipv4, dht::NodeStatus ipv6) {
if (auto owner = state.lock())
owner->send([ipv4, ipv6](RunnerState& state) {
state.status4 = ipv4;
state.status6 = ipv6;
if (!state.onStatus.IsEmpty())
state.onStatus.Call({state.status()});
});
};
const auto service = std::to_string(port);
state_->runner.run(ipv4.c_str(), ipv6.c_str(), service.c_str(), config, std::move(context));
state_->started = true;
state_->hasIdentity = bool(config.dht_config.id.first);
state_->dispatcher.Ref(info.Env());
return info.Env().Undefined();
});
}
Napi::Value close(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
if (!state_->closePromise.IsEmpty())
return state_->closePromise.Value();
auto deferred = Napi::Promise::Deferred::New(info.Env());
if (state_->finished) {
if (!state_->closeError.empty())
deferred.Reject(state_->makeCloseError().Value());
else
deferred.Resolve(info.Env().Undefined());
} else {
state_->closeDeferred = deferred;
state_->closePromise = Napi::Persistent(deferred.Promise());
state_->startClose();
}
return deferred.Promise();
});
}
Napi::Value isRunning(const Napi::CallbackInfo& info)
{
return Napi::Boolean::New(info.Env(), !state_->closing && state_->runner.isRunning());
}
Napi::Value getStatus(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value { return state_->status(); });
}
Napi::Value getBoundPort(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
state_->requireRunning();
const auto family = info.Length() ? unsignedNumber(info[0], 6, "family") : 4;
if (family != 4 && family != 6)
throw Napi::RangeError::New(info.Env(), "family must be 4 or 6");
return Napi::Number::New(info.Env(), state_->runner.getBoundPort(family == 4 ? AF_INET : AF_INET6));
});
}
Napi::Value getNodeId(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
state_->requireRunning();
return Napi::String::New(info.Env(), state_->runner.getNodeId().to_view());
});
}
Napi::Value bootstrap(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
state_->requireRunning();
const auto host = text(info[0], "host");
const auto port = unsignedNumber(info[1], 65535, "port");
if (host.empty() || !port)
throw Napi::RangeError::New(info.Env(), "bootstrap requires a host and nonzero port");
state_->runner.bootstrap(host, std::to_string(port));
return info.Env().Undefined();
});
}
Napi::Value get(const Napi::CallbackInfo& info) { return getValues(info, true); }
Napi::Value getAll(const Napi::CallbackInfo& info) { return getValues(info, false); }
Napi::Value getValues(const Napi::CallbackInfo& info, bool stream)
{
return guarded(info, [&]() -> Napi::Value {
state_->requireRunning();
const auto key = infoHash(info[0]);
if (stream && !info[1].IsFunction())
throw Napi::TypeError::New(info.Env(), "callback must be a function");
const auto operation = state_->nextOperation++;
auto deferred = Napi::Promise::Deferred::New(info.Env());
auto cancelled = std::make_shared<std::atomic_bool>(false);
PendingOperation pending {deferred, true, {}, {}, cancelled};
if (stream)
pending.onValue = Napi::Persistent(info[1].As<Napi::Function>());
state_->operations.emplace(operation, std::move(pending));
const std::weak_ptr<RunnerState> state = state_;
try {
state_->runner.get(key,
dht::GetCallback([state, operation, cancelled](
const std::vector<std::shared_ptr<dht::Value>>& values) {
const auto owner = state.lock();
if (!owner || owner->closing || *cancelled)
return false;
for (const auto& value : values)
owner->sendValue(operation, value);
return !owner->closing && !*cancelled;
}),
dht::DoneCallbackSimple([state, operation](bool ok) {
if (const auto owner = state.lock())
owner->send([operation, ok](RunnerState& state) {
state.completeOperation(operation, ok);
});
}));
} catch (...) {
state_->operations.erase(operation);
throw;
}
if (stream) {
auto result = Napi::Object::New(info.Env());
result.Set("token", Napi::BigInt::New(info.Env(), operation));
result.Set("done", deferred.Promise());
return result;
}
return deferred.Promise();
});
}
Napi::Value cancelGet(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
const auto operation = state_->operations.find(unsignedBigInt(info[0]));
if (operation != state_->operations.end() && !operation->second.onValue.IsEmpty()) {
*operation->second.cancelled = true;
operation->second.deferred.Resolve(info.Env().Undefined());
state_->operations.erase(operation);
}
return info.Env().Undefined();
});
}
Napi::Value put(const Napi::CallbackInfo& info) { return putValue(info, PutMode::Plain); }
Napi::Value putSigned(const Napi::CallbackInfo& info) { return putValue(info, PutMode::Signed); }
Napi::Value putEncrypted(const Napi::CallbackInfo& info) { return putValue(info, PutMode::Encrypted); }
enum class PutMode { Plain, Signed, Encrypted };
Napi::Value putValue(const Napi::CallbackInfo& info, PutMode mode)
{
return guarded(info, [&]() -> Napi::Value {
state_->requireRunning();
if (mode != PutMode::Plain && !state_->hasIdentity)
throw Napi::Error::New(info.Env(), "Signed and encrypted puts require an identity");
std::shared_ptr<dht::crypto::PublicKey> recipient;
if (mode == PutMode::Encrypted) {
recipient = opendht_node::readEncryptionKey(info[3]);
}
const auto key = infoHash(info[0]);
if (!info[1].IsTypedArray() || info[1].As<Napi::TypedArray>().TypedArrayType() != napi_uint8_array)
throw Napi::TypeError::New(info.Env(), "data must be a Uint8Array");
const auto data = info[1].As<Napi::Uint8Array>();
dht::Blob bytes(data.ElementLength());
if (!bytes.empty())
std::copy_n(data.Data(), bytes.size(), bytes.data());
auto value = std::make_shared<dht::Value>(std::move(bytes));
auto options = info.Length() > 2 && !info[2].IsUndefined() ? optionsObject(info[2])
: Napi::Object::New(info.Env());
if (options.Has("id"))
value->id = unsignedBigInt(options.Get("id"));
if (options.Has("type"))
value->type = unsignedNumber(options.Get("type"), 65535, "type");
if (options.Has("userType")) {
if (!options.Get("userType").IsString())
throw Napi::TypeError::New(info.Env(), "userType must be a string");
value->user_type = options.Get("userType").As<Napi::String>().Utf8Value();
}
bool permanent = false;
if (options.Has("permanent")) {
if (!options.Get("permanent").IsBoolean())
throw Napi::TypeError::New(info.Env(), "permanent must be a boolean");
permanent = options.Get("permanent").As<Napi::Boolean>().Value();
}
const auto operation = state_->nextOperation++;
auto deferred = Napi::Promise::Deferred::New(info.Env());
state_->operations.emplace(operation, PendingOperation {deferred, false, {}, {}, {}});
const std::weak_ptr<RunnerState> state = state_;
try {
dht::DoneCallbackSimple done = [state, operation](bool ok) {
if (const auto owner = state.lock())
owner->send([operation, ok](RunnerState& state) { state.completeOperation(operation, ok); });
};
if (mode == PutMode::Signed)
state_->runner.putSigned(key, value, std::move(done), permanent);
else if (mode == PutMode::Encrypted)
state_->runner.putEncrypted(key, recipient, value, std::move(done), permanent);
else
state_->runner.put(key, value, std::move(done), dht::time_point::max(), permanent);
} catch (...) {
state_->operations.erase(operation);
throw;
}
return deferred.Promise();
});
}
Napi::Value listen(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
state_->requireRunning();
const auto key = infoHash(info[0]);
if (!info[1].IsFunction())
throw Napi::TypeError::New(info.Env(), "callback must be a function");
const auto operation = state_->nextOperation++;
const std::weak_ptr<RunnerState> state = state_;
auto token = state_->runner
.listen(key,
dht::ValueCallback(
[state, operation](const std::vector<std::shared_ptr<dht::Value>>& values,
bool expired) {
const auto owner = state.lock();
if (!owner || owner->closing)
return false;
for (const auto& value : values)
owner->sendValue(operation, value, expired);
return !owner->closing;
}))
.share();
state_->listeners.emplace(operation,
Subscription {key,
std::move(token),
Napi::Persistent(info[1].As<Napi::Function>())});
return Napi::BigInt::New(info.Env(), operation);
});
}
Napi::Value cancelListen(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
if (state_->closing)
return info.Env().Undefined();
const auto listener = state_->listeners.find(unsignedBigInt(info[0]));
if (listener != state_->listeners.end()) {
state_->runner.cancelListen(listener->second.key, listener->second.token);
state_->listeners.erase(listener);
}
return info.Env().Undefined();
});
}
Napi::Value ref(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
state_->requireRunning();
state_->dispatcher.Ref(info.Env());
return info.This();
});
}
Napi::Value unref(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
state_->requireRunning();
state_->dispatcher.Unref(info.Env());
return info.This();
});
}
Napi::Value setOwner(const Napi::CallbackInfo& info)
{
if (!state_->closing) {
state_->owner.Reset();
if (info[0].IsObject())
state_->owner = Napi::Persistent(info[0].As<Napi::Object>());
}
return info.Env().Undefined();
}
std::shared_ptr<RunnerState> state_;
};
Napi::Value
hash(const Napi::CallbackInfo& info)
{
const auto env = info.Env();
dht::InfoHash result;
if (info.Length() == 1 && info[0].IsString()) {
result = dht::InfoHash::get(info[0].As<Napi::String>().Utf8Value());
} else if (info.Length() == 1 && info[0].IsTypedArray()
&& info[0].As<Napi::TypedArray>().TypedArrayType() == napi_uint8_array) {
const auto data = info[0].As<Napi::Uint8Array>();
result = dht::InfoHash::get(data.Data(), data.ElementLength());
} else {
throw Napi::TypeError::New(env, "Expected a string or Uint8Array");
}
return Napi::String::New(env, result.to_view());
}
Napi::Object
initialize(Napi::Env env, Napi::Object exports)
{
static const GnuTlsContext crypto;
if (crypto.result != GNUTLS_E_SUCCESS)
throw Napi::Error::New(env, std::string("Unable to initialize GnuTLS: ") + gnutls_strerror(crypto.result));
opendht_node::initializeCrypto(env, exports);
exports.Set("NativeRunner", Runner::define(env));
exports.Set("hash", Napi::Function::New(env, hash));
exports.Set("version", Napi::String::New(env, dht::version()));
return exports;
}
} // namespace
NODE_API_MODULE(opendht, initialize)
+304
View File
@@ -0,0 +1,304 @@
#include "crypto.h"
#include <algorithm>
#include <cmath>
#include <variant>
namespace opendht_node {
namespace {
using PublicKey = std::shared_ptr<dht::crypto::PublicKey>;
using PrivateKey = std::shared_ptr<dht::crypto::PrivateKey>;
using Certificate = std::shared_ptr<dht::crypto::Certificate>;
using Identity = dht::crypto::Identity;
using Handle = std::variant<PublicKey, PrivateKey, Certificate, Identity>;
constexpr napi_type_tag cryptoTag {0x37cdf3e4648e4cb5ULL, 0x964fb031062c245aULL};
Handle&
handle(Napi::Value value)
{
bool matches = false;
if (!value.IsExternal() || napi_check_object_type_tag(value.Env(), value, &cryptoTag, &matches) != napi_ok
|| !matches)
throw Napi::TypeError::New(value.Env(), "Invalid OpenDHT crypto handle");
return *value.As<Napi::External<Handle>>().Data();
}
template<typename Type>
Type
read(Napi::Value value)
{
if (auto result = std::get_if<Type>(&handle(value)))
return *result;
throw Napi::TypeError::New(value.Env(), "Wrong OpenDHT crypto handle type");
}
Napi::Value
wrap(Napi::Env env, Handle value)
{
auto owner = std::make_unique<Handle>(std::move(value));
auto result = Napi::External<Handle>::New(env, owner.get(), [](Napi::Env, Handle* data) { delete data; });
owner.release();
if (napi_type_tag_object(env, result, &cryptoTag) != napi_ok)
throw Napi::Error::New(env, "Could not tag OpenDHT crypto handle");
return result;
}
dht::Blob
bytes(Napi::Value value)
{
if (!value.IsTypedArray() || value.As<Napi::TypedArray>().TypedArrayType() != napi_uint8_array)
throw Napi::TypeError::New(value.Env(), "Expected a Uint8Array");
auto array = value.As<Napi::Uint8Array>();
dht::Blob result(array.ElementLength());
if (!result.empty())
std::copy_n(array.Data(), result.size(), result.data());
return result;
}
std::string
string(Napi::Value value)
{
if (!value.IsString())
throw Napi::TypeError::New(value.Env(), "Expected a string");
auto result = value.As<Napi::String>().Utf8Value();
if (result.find('\0') != std::string::npos)
throw Napi::TypeError::New(value.Env(), "Crypto strings must not contain NUL bytes");
return result;
}
Napi::Buffer<uint8_t>
buffer(Napi::Env env, const dht::Blob& value)
{
return Napi::Buffer<uint8_t>::Copy(env, value.data(), value.size());
}
PublicKey
publicKey(const Handle& value)
{
if (auto key = std::get_if<PublicKey>(&value))
return *key;
if (auto key = std::get_if<PrivateKey>(&value))
return (*key)->getSharedPublicKey();
if (auto certificate = std::get_if<Certificate>(&value))
return (*certificate)->getSharedPublicKey();
return std::get<Identity>(value).second->getSharedPublicKey();
}
class GenerateWorker : public Napi::AsyncWorker
{
public:
GenerateWorker(Napi::Env env, bool identity, std::string name, bool rsa, unsigned bits)
: Napi::AsyncWorker(env, "OpenDHT crypto generation")
, deferred_(Napi::Promise::Deferred::New(env))
, identity_(identity)
, name_(std::move(name))
, rsa_(rsa)
, bits_(bits)
{}
Napi::Promise promise() const { return deferred_.Promise(); }
void Execute() override
{
try {
if (identity_)
result_ = rsa_ ? dht::crypto::generateIdentity(name_, {}, bits_)
: dht::crypto::generateEcIdentity(name_);
else
result_ = std::make_shared<dht::crypto::PrivateKey>(rsa_ ? dht::crypto::PrivateKey::generate(bits_)
: dht::crypto::PrivateKey::generateEC());
} catch (const std::exception& error) {
SetError(error.what());
}
}
void OnOK() override { deferred_.Resolve(wrap(Env(), std::move(result_))); }
void OnError(const Napi::Error& error) override { deferred_.Reject(error.Value()); }
private:
Napi::Promise::Deferred deferred_;
bool identity_;
std::string name_;
bool rsa_;
unsigned bits_;
Handle result_;
};
Napi::Value
generate(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
const auto kind = string(info[0]);
const auto name = string(info[1]);
const auto algorithm = string(info[2]);
if (algorithm != "rsa" && algorithm != "ec")
throw Napi::TypeError::New(info.Env(), "algorithm must be rsa or ec");
if (!info[3].IsNumber())
throw Napi::TypeError::New(info.Env(), "bits must be a number");
const auto bits = info[3].As<Napi::Number>().DoubleValue();
if (!std::isfinite(bits) || std::floor(bits) != bits || bits < 2048 || bits > 8192)
throw Napi::RangeError::New(info.Env(), "RSA bits must be an integer between 2048 and 8192");
if (kind != "private" && kind != "identity")
throw Napi::TypeError::New(info.Env(), "Invalid crypto generation type");
auto worker = std::make_unique<GenerateWorker>(info.Env(), kind == "identity", name, algorithm == "rsa", bits);
auto promise = worker->promise();
worker->Queue();
worker.release();
return promise;
});
}
Napi::Value
import(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
const auto kind = string(info[0]);
auto data = bytes(info[1]);
if (data.empty())
throw Napi::TypeError::New(info.Env(), "Crypto imports must not be empty");
if (kind == "private")
return wrap(info.Env(), std::make_shared<dht::crypto::PrivateKey>(data, string(info[2])));
if (kind == "public")
return wrap(info.Env(), std::make_shared<dht::crypto::PublicKey>(data));
if (kind == "certificate")
return wrap(info.Env(), std::make_shared<dht::crypto::Certificate>(data));
throw Napi::TypeError::New(info.Env(), "Invalid crypto import type");
});
}
Napi::Value
serialize(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
auto& value = handle(info[0]);
if (auto key = std::get_if<PrivateKey>(&value))
return buffer(info.Env(), (*key)->serialize(string(info[1])));
if (auto certificate = std::get_if<Certificate>(&value))
return buffer(info.Env(), (*certificate)->getPacked());
dht::Blob result;
readPublicKey(info[0])->pack(result);
return buffer(info.Env(), result);
});
}
Napi::Value
getPublicKey(const Napi::CallbackInfo& info)
{
return guarded(info, [&] { return wrap(info.Env(), publicKey(handle(info[0]))); });
}
Napi::Value
getId(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
const auto key = publicKey(handle(info[0]));
if (!info[1].IsBoolean())
throw Napi::TypeError::New(info.Env(), "Expected a boolean");
return Napi::String::New(info.Env(),
info[1].As<Napi::Boolean>().Value() ? key->getLongId().toString()
: key->getId().toString());
});
}
Napi::Value
sign(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
return buffer(info.Env(), read<PrivateKey>(info[0])->sign(bytes(info[1])));
});
}
Napi::Value
verify(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
return Napi::Boolean::New(info.Env(), readPublicKey(info[0])->checkSignature(bytes(info[1]), bytes(info[2])));
});
}
Napi::Value
encrypt(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
const auto key = readEncryptionKey(info[0]);
return buffer(info.Env(), key->encrypt(bytes(info[1])));
});
}
Napi::Value
decrypt(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
return buffer(info.Env(), read<PrivateKey>(info[0])->decrypt(bytes(info[1])));
});
}
Napi::Value
identity(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
auto key = read<PrivateKey>(info[0]);
auto certificate = read<Certificate>(info[1]);
if (key->getPublicKey().getLongId() != certificate->getLongId())
throw Napi::TypeError::New(info.Env(), "Identity key and certificate do not match");
return wrap(info.Env(), Identity {std::move(key), std::move(certificate)});
});
}
Napi::Value
identityPart(const Napi::CallbackInfo& info)
{
return guarded(info, [&]() -> Napi::Value {
const auto value = readIdentity(info[0]);
if (!info[1].IsBoolean())
throw Napi::TypeError::New(info.Env(), "Expected a boolean");
return info[1].As<Napi::Boolean>().Value() ? wrap(info.Env(), value.first) : wrap(info.Env(), value.second);
});
}
} // namespace
dht::crypto::Identity
readIdentity(Napi::Value value)
{
return read<Identity>(value);
}
std::shared_ptr<dht::crypto::PublicKey>
readPublicKey(Napi::Value value)
{
return read<PublicKey>(value);
}
std::shared_ptr<dht::crypto::PublicKey>
readEncryptionKey(Napi::Value value)
{
const auto key = readPublicKey(value);
const auto algorithm = gnutls_pubkey_get_pk_algorithm(key->pk, nullptr);
if (algorithm != GNUTLS_PK_RSA
#if GNUTLS_VERSION_NUMBER >= 0x030804
&& algorithm != GNUTLS_PK_RSA_OAEP
#endif
)
throw Napi::TypeError::New(value.Env(), "Encryption requires an RSA public key");
return key;
}
void
initializeCrypto(Napi::Env env, Napi::Object exports)
{
exports.Set("cryptoGenerate", Napi::Function::New(env, generate));
exports.Set("cryptoImport", Napi::Function::New(env, import));
exports.Set("cryptoSerialize", Napi::Function::New(env, serialize));
exports.Set("cryptoPublicKey", Napi::Function::New(env, getPublicKey));
exports.Set("cryptoId", Napi::Function::New(env, getId));
exports.Set("cryptoSign", Napi::Function::New(env, sign));
exports.Set("cryptoVerify", Napi::Function::New(env, verify));
exports.Set("cryptoEncrypt", Napi::Function::New(env, encrypt));
exports.Set("cryptoDecrypt", Napi::Function::New(env, decrypt));
exports.Set("cryptoIdentity", Napi::Function::New(env, identity));
exports.Set("cryptoIdentityPart", Napi::Function::New(env, identityPart));
}
} // namespace opendht_node
+26
View File
@@ -0,0 +1,26 @@
#pragma once
#include <napi.h>
#include <opendht/crypto.h>
namespace opendht_node {
template<typename Callback>
Napi::Value
guarded(const Napi::CallbackInfo& info, Callback&& callback)
{
try {
return callback();
} catch (const Napi::Error&) {
throw;
} catch (const std::exception& error) {
throw Napi::Error::New(info.Env(), error.what());
}
}
dht::crypto::Identity readIdentity(Napi::Value value);
std::shared_ptr<dht::crypto::PublicKey> readPublicKey(Napi::Value value);
std::shared_ptr<dht::crypto::PublicKey> readEncryptionKey(Napi::Value value);
void initializeCrypto(Napi::Env env, Napi::Object exports);
} // namespace opendht_node
+56
View File
@@ -0,0 +1,56 @@
export default function createCrypto(native) {
const handles = new WeakMap();
function handle(value, Type) {
if (!(value instanceof Type) || !handles.has(value)) throw new TypeError(`Expected ${Type.name}`);
return handles.get(value);
}
function wrap(Type, nativeHandle) {
const result = Object.create(Type.prototype);
handles.set(result, nativeHandle);
return result;
}
function generate(kind, name, options = {}) {
if (!options || typeof options !== 'object' || Array.isArray(options)) throw new TypeError('Invalid crypto options');
return native.cryptoGenerate(kind, name, options.algorithm ?? 'ec', options.bits ?? 2048);
}
class PublicKey {
constructor(data) { handles.set(this, native.cryptoImport('public', data)); }
serialize() { return native.cryptoSerialize(handle(this, PublicKey)); }
getId() { return native.cryptoId(handle(this, PublicKey), false); }
getLongId() { return native.cryptoId(handle(this, PublicKey), true); }
checkSignature(data, signature) { return native.cryptoVerify(handle(this, PublicKey), data, signature); }
encrypt(data) { return native.cryptoEncrypt(handle(this, PublicKey), data); }
}
class PrivateKey {
constructor(data, password = '') { handles.set(this, native.cryptoImport('private', data, password)); }
static generate(options) { return generate('private', '', options).then(value => wrap(PrivateKey, value)); }
serialize(password = '') { return native.cryptoSerialize(handle(this, PrivateKey), password); }
getPublicKey() { return wrap(PublicKey, native.cryptoPublicKey(handle(this, PrivateKey))); }
sign(data) { return native.cryptoSign(handle(this, PrivateKey), data); }
decrypt(data) { return native.cryptoDecrypt(handle(this, PrivateKey), data); }
}
class Certificate {
constructor(data) { handles.set(this, native.cryptoImport('certificate', data)); }
serialize() { return native.cryptoSerialize(handle(this, Certificate)); }
getPublicKey() { return wrap(PublicKey, native.cryptoPublicKey(handle(this, Certificate))); }
getId() { return native.cryptoId(handle(this, Certificate), false); }
getLongId() { return native.cryptoId(handle(this, Certificate), true); }
}
class Identity {
constructor(privateKey, certificate) {
handles.set(this, native.cryptoIdentity(handle(privateKey, PrivateKey), handle(certificate, Certificate)));
}
static generate(name = 'dhtnode', options) { return generate('identity', name, options).then(value => wrap(Identity, value)); }
get privateKey() { return wrap(PrivateKey, native.cryptoIdentityPart(handle(this, Identity), true)); }
get certificate() { return wrap(Certificate, native.cryptoIdentityPart(handle(this, Identity), false)); }
}
return { PublicKey, PrivateKey, Certificate, Identity, handle };
}
+171
View File
@@ -0,0 +1,171 @@
#!/usr/bin/env node
import { clearLine, createInterface, cursorTo, type Interface } from 'node:readline';
import { parseArgs } from 'node:util';
import type { ConnectivityStatus, Listener, Value } from '../index.js';
const commands = 'get KEY | put KEY TEXT | sign KEY TEXT | listen KEY | cancel KEY | help | quit';
function portNumber(value: string, name: string, minimum = 0): number {
const port = Number(value);
if (!Number.isInteger(port) || port < minimum || port > 65535) {
throw new Error(`${name} must be an integer between ${minimum} and 65535`);
}
return port;
}
function errorMessage(error: unknown): string {
return error instanceof Error ? error.message : String(error);
}
function report(error: unknown): void {
console.error(errorMessage(error));
process.exitCode = 1;
}
async function main(): Promise<void> {
const { values } = parseArgs({
options: {
help: { type: 'boolean', short: 'h' },
port: { type: 'string', short: 'p', default: '4222' },
ipv4: { type: 'string', default: '0.0.0.0' },
ipv6: { type: 'string', default: '::' },
bootstrap: { type: 'string', short: 'b' },
'bootstrap-port': { type: 'string', default: '4222' },
},
});
if (values.help) {
console.log(`Usage: npm run dhtnode -- [options]
-p, --port PORT Local UDP port (default: 4222; 0 selects a free port)
-b, --bootstrap HOST Optional bootstrap hostname or IP address
--bootstrap-port PORT Bootstrap UDP port (default: 4222)
--ipv4 ADDRESS IPv4 bind address (default: 0.0.0.0)
--ipv6 ADDRESS IPv6 bind address (default: ::)
-h, --help Show help
Commands: ${commands}
Keys are text labels hashed with SHA-1. Values are UTF-8 text.`);
return;
}
const port = portNumber(values.port, 'port');
const bootstrapPort = portNumber(values['bootstrap-port'], 'bootstrap-port', 1);
const { DhtRunner, Identity, hash } = await import('../index.js');
const identity = await Identity.generate('dhtnode.ts');
const runner = new DhtRunner();
const listeners = new Map<string, Listener>();
const pending = new Set<Promise<void>>();
let lines: Interface | undefined;
let closing: Promise<void> | undefined;
function close(): Promise<void> {
if (!closing) {
for (const listener of listeners.values()) listener.cancel();
listeners.clear();
lines?.close();
closing = runner.close();
}
return closing;
}
function print(message: string): void {
if (process.stdout.isTTY && lines) {
clearLine(process.stdout, 0);
cursorTo(process.stdout, 0);
}
console.log(message);
if (!closing && process.stdin.isTTY) lines?.prompt(true);
}
const stop = (): void => { close().catch(report); };
const printValue = (label: string, value: Value): void => {
print(`${label} ${value.id}: ${JSON.stringify(value.data.toString('utf8'))}${value.isSigned ? ' [signed]' : ''}`);
};
const printStatus = (status: ConnectivityStatus): void => {
print(`Connectivity: IPv4 ${status.ipv4}, IPv6 ${status.ipv6}`);
};
async function execute(line: string): Promise<void> {
const [, command, key, data] = line.trim().match(/^(\S+)(?:\s+(\S+))?(?:\s+(.+))?$/) || [];
if (!command) return;
if (command === 'help') {
print(commands);
return;
}
if (!key) throw new Error('Expected a key. Type help for commands.');
const location = hash(key);
switch (command) {
case 'get': {
const found = await runner.getAll(location);
if (closing) return;
for (const value of found) printValue(key, value);
if (!found.length) print(`No values for ${key}`);
break;
}
case 'put':
case 'sign': {
if (data === undefined) throw new Error(`Usage: ${command} KEY TEXT`);
const payload = Buffer.from(data, 'utf8');
const ok = command === 'sign' ? await runner.putSigned(location, payload)
: await runner.put(location, payload);
if (closing) return;
if (!ok) throw new Error(`Put failed for ${key}`);
print(`Stored ${key}`);
break;
}
case 'listen':
if (!listeners.has(key)) {
listeners.set(key, runner.listen(location, (value, expired) => {
printValue(`${expired ? 'Expired' : 'Value'} ${key}`, value);
}));
}
print(`Listening ${key}`);
break;
case 'cancel':
listeners.get(key)?.cancel();
listeners.delete(key);
print(`Cancelled ${key}`);
break;
default:
throw new Error('Unknown command. Type help for commands.');
}
}
runner.on('error', error => { report(error); stop(); });
runner.on('statusChanged', printStatus);
process.on('SIGINT', stop);
process.on('SIGTERM', stop);
try {
runner.run({ port, ipv4: values.ipv4, ipv6: values.ipv6, identity });
if (values.bootstrap) runner.bootstrap(values.bootstrap, bootstrapPort);
lines = createInterface({ input: process.stdin, output: process.stdout });
lines.on('SIGINT', stop);
lines.setPrompt('dht> ');
print(`Node ${runner.getNodeId()} listening on UDP ${runner.getBoundPort()}`);
printStatus(runner.getStatus());
if (process.stdin.isTTY) print(commands);
for await (const line of lines) {
if (closing) break;
if (/^(quit|exit)(?:\s|$)/.test(line.trim())) {
stop();
break;
}
const operation = execute(line).catch(error => {
if (!closing) console.error(errorMessage(error));
}).finally(() => { pending.delete(operation); });
pending.add(operation);
if (process.stdin.isTTY) lines.prompt();
}
if (!closing) await Promise.all(pending);
} finally {
try {
await close();
await Promise.all(pending);
} finally {
process.removeListener('SIGINT', stop);
process.removeListener('SIGTERM', stop);
}
}
}
main().catch(report);
+120
View File
@@ -0,0 +1,120 @@
import { Buffer } from 'node:buffer';
import { EventEmitter } from 'node:events';
export const version: string;
export function hash(data: string | Uint8Array): string;
export interface KeyOptions {
algorithm?: 'ec' | 'rsa';
bits?: number;
}
export class PublicKey {
constructor(data: Uint8Array);
serialize(): Buffer;
getId(): string;
getLongId(): string;
checkSignature(data: Uint8Array, signature: Uint8Array): boolean;
encrypt(data: Uint8Array): Buffer;
}
export class PrivateKey {
constructor(data: Uint8Array, password?: string);
static generate(options?: KeyOptions): Promise<PrivateKey>;
serialize(password?: string): Buffer;
getPublicKey(): PublicKey;
sign(data: Uint8Array): Buffer;
decrypt(data: Uint8Array): Buffer;
}
export class Certificate {
constructor(data: Uint8Array);
serialize(): Buffer;
getPublicKey(): PublicKey;
getId(): string;
getLongId(): string;
}
export class Identity {
constructor(privateKey: PrivateKey, certificate: Certificate);
static generate(name?: string, options?: KeyOptions): Promise<Identity>;
readonly privateKey: PrivateKey;
readonly certificate: Certificate;
}
export interface RunOptions {
port?: number;
ipv4?: string;
ipv6?: string;
network?: number;
persistPath?: string;
identity?: Identity;
proxyServer?: string;
proxyUserAgent?: string;
}
export interface PutOptions {
id?: bigint;
type?: number;
userType?: string;
permanent?: boolean;
}
export interface Value {
id: bigint;
type: number;
seq: number;
data: Buffer;
userType: string;
owner: string | null;
isSigned: boolean;
isEncrypted: boolean;
}
export interface Listener {
cancel(): void;
}
export interface ListenEvent {
value: Value;
expired: boolean;
}
export interface ValueIterator<T> extends AsyncIterableIterator<T, undefined>, Listener, AsyncDisposable {
return(): Promise<IteratorResult<T, undefined>>;
throw(error?: unknown): Promise<IteratorResult<T, undefined>>;
}
export interface ListenerIterator extends ValueIterator<ListenEvent> { }
export type NodeStatus = 'disconnected' | 'connecting' | 'connected';
export interface ConnectivityStatus {
ipv4: NodeStatus;
ipv6: NodeStatus;
}
export interface RunnerEvents {
statusChanged: [status: ConnectivityStatus];
error: [error: Error];
}
export class DhtRunner extends EventEmitter<RunnerEvents> implements AsyncDisposable {
run(options?: RunOptions): void;
isRunning(): boolean;
getBoundPort(family?: 4 | 6): number;
getNodeId(): string;
getStatus(): ConnectivityStatus;
bootstrap(host: string, port: number): void;
get(key: string): ValueIterator<Value>;
getAll(key: string): Promise<Value[]>;
put(key: string, data: Uint8Array, options?: PutOptions): Promise<boolean>;
putSigned(key: string, data: Uint8Array, options?: PutOptions): Promise<boolean>;
putEncrypted(key: string, data: Uint8Array, recipient: PublicKey, options?: PutOptions): Promise<boolean>;
listen(key: string): ListenerIterator;
listen(key: string, callback: (value: Value, expired: boolean) => void): Listener;
close(): Promise<void>;
ref(): this;
unref(): this;
[Symbol.asyncDispose](): Promise<void>;
}
+187
View File
@@ -0,0 +1,187 @@
import { EventEmitter } from 'node:events';
import { constants } from 'node:os';
import { dlopen } from 'node:process';
import load from 'node-gyp-build';
import createCrypto from './crypto.js';
const binding = { exports: {} };
const flags = process.platform === 'linux'
? constants.dlopen.RTLD_LAZY | constants.dlopen.RTLD_DEEPBIND
: undefined;
dlopen(binding, process.env.OPENDHT_NODE_BINDING || load.path(import.meta.dirname),
flags);
const native = binding.exports;
const { PublicKey, PrivateKey, Certificate, Identity, handle } = createCrypto(native);
function eventReporter(owner, event) {
return value => { owner.deref()?.emit(event, value); };
}
class DhtRunner extends EventEmitter {
#native;
#closing;
#listeners = 0;
#pending = 0;
#iterators = new Set();
constructor() {
super();
const owner = new WeakRef(this);
this.#native = new native.NativeRunner(eventReporter(owner, 'statusChanged'));
}
#updateOwner() {
this.#native.setOwner(this.#listeners || this.#pending || this.listenerCount('statusChanged') ? this : null);
}
#track(operation) {
this.#pending++;
this.#updateOwner();
return operation.finally(() => {
this.#pending--;
this.#updateOwner();
});
}
on(event, listener) { super.on(event, listener); this.#updateOwner(); return this; }
addListener(event, listener) { return this.on(event, listener); }
prependListener(event, listener) { super.prependListener(event, listener); this.#updateOwner(); return this; }
removeListener(event, listener) { super.removeListener(event, listener); this.#updateOwner(); return this; }
off(event, listener) { return this.removeListener(event, listener); }
removeAllListeners(...args) { super.removeAllListeners(...args); this.#updateOwner(); return this; }
run(options) {
if (typeof options?.proxyServer === 'string' && options.proxyServer) {
const proxy = new URL(options.proxyServer);
if (proxy.protocol !== 'http:' && proxy.protocol !== 'https:')
throw new TypeError('proxyServer must be an HTTP(S) URL');
}
if (options?.identity !== undefined) options = { ...options, identity: handle(options.identity, Identity) };
this.#native.run(options);
}
isRunning() { return this.#native.isRunning(); }
getBoundPort(family = 4) { return this.#native.getBoundPort(family); }
getNodeId() { return this.#native.getNodeId(); }
getStatus() { return this.#native.getStatus(); }
bootstrap(host, port) { this.#native.bootstrap(host, port); }
get(key) {
return this.#iterate((push, finish) => {
const operation = this.#native.get(key, push);
this.#track(operation.done).then(() => finish(), finish);
return () => this.#native.cancelGet(operation.token);
}, true);
}
getAll(key) { return this.#track(this.#native.getAll(key)); }
put(key, data, options) { return this.#track(this.#native.put(key, data, options)); }
putSigned(key, data, options) { return this.#track(this.#native.putSigned(key, data, options)); }
putEncrypted(key, data, recipient, options) {
return this.#track(this.#native.putEncrypted(key, data, options, handle(recipient, PublicKey)));
}
listen(key, callback) {
if (callback === undefined) {
return this.#iterate(push => {
const subscription = this.listen(key, (value, expired) => push({ value, expired }));
return () => subscription.cancel();
});
}
const token = this.#native.listen(key, callback);
this.#listeners++;
this.#updateOwner();
let cancelled = false;
return {
cancel: () => {
if (cancelled) return;
this.#native.cancelListen(token);
cancelled = true;
this.#listeners--;
this.#updateOwner();
},
};
}
#iterate(subscribe, rejectOnClose = false) {
let values = [];
let offset = 0;
const pending = [];
let done = false;
let failure;
const finish = error => {
if (done) return;
done = true;
failure = error;
unsubscribe();
this.#iterators.delete(onClose);
if (failure) {
values = [];
offset = 0;
}
for (const { resolve, reject } of pending.splice(0)) {
if (failure) reject(failure);
else resolve({ value: undefined, done: true });
}
};
const cancel = () => {
values = [];
offset = 0;
failure = undefined;
finish();
};
const onClose = () => {
if (rejectOnClose) {
const error = new Error('DHT runner is closed');
error.code = 'ERR_DHT_CLOSED';
finish(error);
} else cancel();
};
const unsubscribe = subscribe(value => {
if (done) return;
const waiter = pending.shift();
if (waiter) waiter.resolve({ value, done: false });
else values.push(value);
}, finish);
this.#iterators.add(onClose);
return {
next() {
if (failure) return Promise.reject(failure);
if (offset < values.length) {
const value = values[offset];
values[offset++] = undefined;
if (offset === values.length) {
values = [];
offset = 0;
}
return Promise.resolve({ value, done: false });
}
if (done) return Promise.resolve({ value: undefined, done: true });
return new Promise((resolve, reject) => pending.push({ resolve, reject }));
},
return() {
cancel();
return Promise.resolve({ value: undefined, done: true });
},
throw(error) {
cancel();
return Promise.reject(error);
},
cancel,
[Symbol.asyncIterator]() { return this; },
[Symbol.asyncDispose]() { cancel(); return Promise.resolve(); },
};
}
close() {
if (!this.#closing) {
this.#closing = this.#native.close();
for (const cancel of this.#iterators) cancel();
}
return this.#closing;
}
ref() { this.#native.ref(); return this; }
unref() { this.#native.unref(); return this; }
[Symbol.asyncDispose]() { return this.close(); }
}
export { DhtRunner, PublicKey, PrivateKey, Certificate, Identity };
export const hash = native.hash;
export const version = native.version;
+904
View File
@@ -0,0 +1,904 @@
{
"name": "opendht",
"version": "0.1.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "opendht",
"version": "0.1.0",
"license": "MIT",
"dependencies": {
"node-gyp-build": "^4.8.4"
},
"devDependencies": {
"@types/node": "^22.20.5",
"cmake-js": "^8.0.0",
"node-addon-api": "^8.5.0",
"typescript": "^7.0.2"
},
"engines": {
"node": ">=22.9.0"
}
},
"node_modules/@isaacs/fs-minipass": {
"version": "4.0.1",
"resolved": "https://registry.npmjs.org/@isaacs/fs-minipass/-/fs-minipass-4.0.1.tgz",
"integrity": "sha512-wgm9Ehl2jpeqP3zw/7mo3kRHFp5MEDhqAdwy1fTGkHAwnkGOVsgpvQhL8B5n1qlb01jV3n/bI0ZfZp5lWA1k4w==",
"dev": true,
"license": "ISC",
"dependencies": {
"minipass": "^7.0.4"
},
"engines": {
"node": ">=18.0.0"
}
},
"node_modules/@types/node": {
"version": "22.20.5",
"resolved": "https://registry.npmjs.org/@types/node/-/node-22.20.5.tgz",
"integrity": "sha512-U2+DNr+wSjpsTS/wZGYHq7GcwfuSmKiKvoPvK22zwTlRhU91yOniN4qRR5KhIjvif7ysw/dz/hKmfDH0Ris4aA==",
"dev": true,
"license": "MIT",
"dependencies": {
"undici-types": "~6.21.0"
}
},
"node_modules/@typescript/typescript-aix-ppc64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-aix-ppc64/-/typescript-aix-ppc64-7.0.2.tgz",
"integrity": "sha512-MTKKkWB7p/0E9xi1d1tHtZ5PiLkGEMIq88pK2CubZjOsLtYTLqhgIgi6zepFa+9GHZ6h05NMCkQxGKiPXMxXtQ==",
"cpu": [
"ppc64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"aix"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-darwin-arm64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-darwin-arm64/-/typescript-darwin-arm64-7.0.2.tgz",
"integrity": "sha512-gowzar9MwS/aRWp6f3a4KUqzRjAZjOsmGNCM6LcTgXum+dBfgsBVMN+AgvOCCbguXyick6LJhpBszxMebJ8syA==",
"cpu": [
"arm64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-darwin-x64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-darwin-x64/-/typescript-darwin-x64-7.0.2.tgz",
"integrity": "sha512-SZ9xZInqApNlNGc9s0W1VSsktYSOe9cFqNOIqmN1Gs8SmkjKZYFt017G4VwPxASInODuAdbTW7sXiFUf893RgA==",
"cpu": [
"x64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-freebsd-arm64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-freebsd-arm64/-/typescript-freebsd-arm64-7.0.2.tgz",
"integrity": "sha512-W5NH4y/J0plIIS5b2xvTEkU7JFxyqdMAOgf+Ilhl0vHQXKO5dZoxd+C/jEtq56c4F3wk71RB4BMRQ2XdI+bwYQ==",
"cpu": [
"arm64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"freebsd"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-freebsd-x64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-freebsd-x64/-/typescript-freebsd-x64-7.0.2.tgz",
"integrity": "sha512-UMGDx5sTpzNw3WiPebH7l90IWfJggEd+egHt/q6p7/Cm3zqoV7VxkGXt+3DxPIw8CcmvAB0j3sVVfbhX+M4Tpw==",
"cpu": [
"x64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"freebsd"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-linux-arm": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-linux-arm/-/typescript-linux-arm-7.0.2.tgz",
"integrity": "sha512-gffT3xPz9sR7j/YJExkyPntrI0P2EP9XbOyWzth2/Gs0RstK+90RBcO0ncXoXy/beYll1SXw846Nf2zdnEz0QQ==",
"cpu": [
"arm"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-linux-arm64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-linux-arm64/-/typescript-linux-arm64-7.0.2.tgz",
"integrity": "sha512-Qh4eU4/y3yDjnfjjyPYihMj5/ODIlmt+Bzu17OI+fiSRDW57QmU5SiN63exPRNJPKUzcc1INa1NXdrJ+MqHjUQ==",
"cpu": [
"arm64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-linux-loong64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-linux-loong64/-/typescript-linux-loong64-7.0.2.tgz",
"integrity": "sha512-uEHck9i8hoAzXPiYRib1O7miOnz23SxIeVl6F4LXox+qov1K35jHcEW6VHKvZI+pyvl7fZEP4MCU5LYvIq1GuQ==",
"cpu": [
"loong64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-linux-mips64el": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-linux-mips64el/-/typescript-linux-mips64el-7.0.2.tgz",
"integrity": "sha512-R4KvAMnE43W5Qeqb0Ly56O3mWMWIAgsMyz36DCaycd5nbg/9kzm0liw3JocfRqyJY0KPmzFjbswozXyW0DnIYA==",
"cpu": [
"mips64el"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-linux-ppc64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-linux-ppc64/-/typescript-linux-ppc64-7.0.2.tgz",
"integrity": "sha512-DORx5b3sd/4S7eayxm4FQv+A7CrkUIGRaHiwI8oiHTAI1fAPWhF4J0vAlkC8biAlHSVVwxMQ3tjZ2/DVbnQiiA==",
"cpu": [
"ppc64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-linux-riscv64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-linux-riscv64/-/typescript-linux-riscv64-7.0.2.tgz",
"integrity": "sha512-wf0jqEDOjrPRnKwYRyyJDRo11KMbvMFrU+q4zqKyChODBzvlkbhNQfKvLxQCcwTpdDaXSHZTVuh0JoCrKCUMHQ==",
"cpu": [
"riscv64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-linux-s390x": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-linux-s390x/-/typescript-linux-s390x-7.0.2.tgz",
"integrity": "sha512-IkwJc3L7yhytWd/ewjyxNDfOmswCm9GWMJT/ue/dU4aZNbwZeYAetq42VyLmsmSjvoX7z74X6ZaYCtzAr0EuGw==",
"cpu": [
"s390x"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-linux-x64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-linux-x64/-/typescript-linux-x64-7.0.2.tgz",
"integrity": "sha512-EYdf2cNg7rgCWJnxCdJ+F3V39O8ihb37eHAu1LK8oAFizgTQbPOK7zHHXbPt8rX24COqODXeI3sIf0fCXG7H/A==",
"cpu": [
"x64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-netbsd-arm64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-netbsd-arm64/-/typescript-netbsd-arm64-7.0.2.tgz",
"integrity": "sha512-+polYF4MF04aPpO5FTkHran9yUQDSXqy5GiSDKpsll5jy3l3+g9QLhpf39T+ePtefhXLOGrLl0QIjkQP6VnelA==",
"cpu": [
"arm64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"netbsd"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-netbsd-x64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-netbsd-x64/-/typescript-netbsd-x64-7.0.2.tgz",
"integrity": "sha512-8YIT0EHM/3dq10ZOVF/A7pc/YSMtbcecct4rWtexrnSCHOPcpC2KTLXfTCR6vDpnSiY12heNb1GiN/wu+T/FyA==",
"cpu": [
"x64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"netbsd"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-openbsd-arm64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-openbsd-arm64/-/typescript-openbsd-arm64-7.0.2.tgz",
"integrity": "sha512-APT8+ClYnuYm1u9+kgGXoMj2VzWzcymwh2gNSQVySHfkRDGOTVkoWLjCmOQSaO+PoqQ57B0flRp9SA+7GnnkzQ==",
"cpu": [
"arm64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"openbsd"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-openbsd-x64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-openbsd-x64/-/typescript-openbsd-x64-7.0.2.tgz",
"integrity": "sha512-yX7s+Q0Dln0Dt9tEzZsAjXXR/+ytBM7AlglaqyeMPxQszJ1JhlJdZ6jLA+IzldHtflX81em7lDao1xXu+aRRkg==",
"cpu": [
"x64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"openbsd"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-sunos-x64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-sunos-x64/-/typescript-sunos-x64-7.0.2.tgz",
"integrity": "sha512-dLJDGaLZ1D4HPQn62u1n8mBDkJREwMsAkCdkwd4Ieqw+x3TUyTsqY0YiBCtE6H6OzzgGk3iuZ3vFWRS+E8/d1g==",
"cpu": [
"x64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"sunos"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-win32-arm64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-win32-arm64/-/typescript-win32-arm64-7.0.2.tgz",
"integrity": "sha512-Gyl1Vy6OsWesLzmq+EP0Fb7b4Nid5232AvcA2SFcdYreldpNtYFFofPjnt62y9hQy7VTaZp65ICJjuAQRaVcIQ==",
"cpu": [
"arm64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-win32-x64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-win32-x64/-/typescript-win32-x64-7.0.2.tgz",
"integrity": "sha512-0BQ3HkAHHlKLSp1qRvf3SUhGpGsDuhB/jgFw75guyqbxJqEaS0Cw/VFO8i2nHglJUzQCRtMMR/IBAKE3ETMC4g==",
"cpu": [
"x64"
],
"dev": true,
"license": "Apache-2.0",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/ansi-regex": {
"version": "5.0.1",
"resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-5.0.1.tgz",
"integrity": "sha512-quJQXlTSUGL2LH9SUXo8VwsY4soanhgo6LNSm84E1LBcE8s3O0wpdiRzyR9z/ZZJMlMWv37qOOb9pdJlMUEKFQ==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=8"
}
},
"node_modules/ansi-styles": {
"version": "4.3.0",
"resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-4.3.0.tgz",
"integrity": "sha512-zbB9rCJAT1rbjiVDb2hqKFHNYLxgtk8NURxZ3IZwD3F6NtxbXZQCnnSi1Lkx+IDohdPlFp222wVALIheZJQSEg==",
"dev": true,
"license": "MIT",
"dependencies": {
"color-convert": "^2.0.1"
},
"engines": {
"node": ">=8"
},
"funding": {
"url": "https://github.com/chalk/ansi-styles?sponsor=1"
}
},
"node_modules/chownr": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/chownr/-/chownr-3.0.0.tgz",
"integrity": "sha512-+IxzY9BZOQd/XuYPRmrvEVjF/nqj5kgT4kEq7VofrDoM1MxoRjEWkrCC3EtLi59TVawxTAn+orJwFQcrqEN1+g==",
"dev": true,
"license": "BlueOak-1.0.0",
"engines": {
"node": ">=18"
}
},
"node_modules/cliui": {
"version": "8.0.1",
"resolved": "https://registry.npmjs.org/cliui/-/cliui-8.0.1.tgz",
"integrity": "sha512-BSeNnyus75C4//NQ9gQt1/csTXyo/8Sb+afLAkzAptFuMsod9HFokGNudZpi/oQV73hnVK+sR+5PVRMd+Dr7YQ==",
"dev": true,
"license": "ISC",
"dependencies": {
"string-width": "^4.2.0",
"strip-ansi": "^6.0.1",
"wrap-ansi": "^7.0.0"
},
"engines": {
"node": ">=12"
}
},
"node_modules/cmake-js": {
"version": "8.0.0",
"resolved": "https://registry.npmjs.org/cmake-js/-/cmake-js-8.0.0.tgz",
"integrity": "sha512-YbUP88RDwCvoQkZhRtGURYm9RIpWdtvZuhT87fKNoLjk8kIFIFeARpKfuZQGdwfH99GZpUmqSfcDrK62X7lTgg==",
"dev": true,
"license": "MIT",
"dependencies": {
"debug": "^4.4.3",
"fs-extra": "^11.3.3",
"node-api-headers": "^1.8.0",
"rc": "1.2.8",
"semver": "^7.7.3",
"tar": "^7.5.6",
"url-join": "^4.0.1",
"which": "^6.0.0",
"yargs": "^17.7.2"
},
"bin": {
"cmake-js": "bin/cmake-js"
},
"engines": {
"node": "^20.17.0 || >=22.9.0"
}
},
"node_modules/color-convert": {
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/color-convert/-/color-convert-2.0.1.tgz",
"integrity": "sha512-RRECPsj7iu/xb5oKYcsFHSppFNnsj/52OVTRKb4zP5onXwVF3zVmmToNcOfGC+CRDpfK/U584fMg38ZHCaElKQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"color-name": "~1.1.4"
},
"engines": {
"node": ">=7.0.0"
}
},
"node_modules/color-name": {
"version": "1.1.4",
"resolved": "https://registry.npmjs.org/color-name/-/color-name-1.1.4.tgz",
"integrity": "sha512-dOy+3AuW3a2wNbZHIuMZpTcgjGuLU/uBL/ubcZF9OXbDo8ff4O8yVp5Bf0efS8uEoYo5q4Fx7dY9OgQGXgAsQA==",
"dev": true,
"license": "MIT"
},
"node_modules/debug": {
"version": "4.4.3",
"resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz",
"integrity": "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==",
"dev": true,
"license": "MIT",
"dependencies": {
"ms": "^2.1.3"
},
"engines": {
"node": ">=6.0"
},
"peerDependenciesMeta": {
"supports-color": {
"optional": true
}
}
},
"node_modules/deep-extend": {
"version": "0.6.0",
"resolved": "https://registry.npmjs.org/deep-extend/-/deep-extend-0.6.0.tgz",
"integrity": "sha512-LOHxIOaPYdHlJRtCQfDIVZtfw/ufM8+rVj649RIHzcm/vGwQRXFt6OPqIFWsm2XEMrNIEtWR64sY1LEKD2vAOA==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=4.0.0"
}
},
"node_modules/emoji-regex": {
"version": "8.0.0",
"resolved": "https://registry.npmjs.org/emoji-regex/-/emoji-regex-8.0.0.tgz",
"integrity": "sha512-MSjYzcWNOA0ewAHpz0MxpYFvwg6yjy1NG3xteoqz644VCo/RPgnr1/GGt+ic3iJTzQ8Eu3TdM14SawnVUmGE6A==",
"dev": true,
"license": "MIT"
},
"node_modules/escalade": {
"version": "3.2.0",
"resolved": "https://registry.npmjs.org/escalade/-/escalade-3.2.0.tgz",
"integrity": "sha512-WUj2qlxaQtO4g6Pq5c29GTcWGDyd8itL8zTlipgECz3JesAiiOKotd8JU6otB3PACgG6xkJUyVhboMS+bje/jA==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=6"
}
},
"node_modules/fs-extra": {
"version": "11.4.1",
"resolved": "https://registry.npmjs.org/fs-extra/-/fs-extra-11.4.1.tgz",
"integrity": "sha512-KYAb4c9BJQI6QqGKthV68OHe0badztdXJWKo0WtBA9IuCFPTKvE5ZdUBglP833aMjhaSPNO4A5j/EkzZtGlKjA==",
"dev": true,
"license": "MIT",
"dependencies": {
"graceful-fs": "^4.2.0",
"jsonfile": "^6.0.1",
"universalify": "^2.0.0"
},
"engines": {
"node": ">=14.14"
}
},
"node_modules/get-caller-file": {
"version": "2.0.5",
"resolved": "https://registry.npmjs.org/get-caller-file/-/get-caller-file-2.0.5.tgz",
"integrity": "sha512-DyFP3BM/3YHTQOCUL/w0OZHR0lpKeGrxotcHWcqNEdnltqFwXVfhEBQ94eIo34AfQpo0rGki4cyIiftY06h2Fg==",
"dev": true,
"license": "ISC",
"engines": {
"node": "6.* || 8.* || >= 10.*"
}
},
"node_modules/graceful-fs": {
"version": "4.2.11",
"resolved": "https://registry.npmjs.org/graceful-fs/-/graceful-fs-4.2.11.tgz",
"integrity": "sha512-RbJ5/jmFcNNCcDV5o9eTnBLJ/HszWV0P73bc+Ff4nS/rJj+YaS6IGyiOL0VoBYX+l1Wrl3k63h/KrH+nhJ0XvQ==",
"dev": true,
"license": "ISC"
},
"node_modules/ini": {
"version": "1.3.8",
"resolved": "https://registry.npmjs.org/ini/-/ini-1.3.8.tgz",
"integrity": "sha512-JV/yugV2uzW5iMRSiZAyDtQd+nxtUnjeLt0acNdw98kKLrvuRVyB80tsREOE7yvGVgalhZ6RNXCmEHkUKBKxew==",
"dev": true,
"license": "ISC"
},
"node_modules/is-fullwidth-code-point": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/is-fullwidth-code-point/-/is-fullwidth-code-point-3.0.0.tgz",
"integrity": "sha512-zymm5+u+sCsSWyD9qNaejV3DFvhCKclKdizYaJUuHA83RLjb7nSuGnddCHGv0hk+KY7BMAlsWeK4Ueg6EV6XQg==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=8"
}
},
"node_modules/isexe": {
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/isexe/-/isexe-4.0.0.tgz",
"integrity": "sha512-FFUtZMpoZ8RqHS3XeXEmHWLA4thH+ZxCv2lOiPIn1Xc7CxrqhWzNSDzD+/chS/zbYezmiwWLdQC09JdQKmthOw==",
"dev": true,
"license": "BlueOak-1.0.0",
"engines": {
"node": ">=20"
}
},
"node_modules/jsonfile": {
"version": "6.2.1",
"resolved": "https://registry.npmjs.org/jsonfile/-/jsonfile-6.2.1.tgz",
"integrity": "sha512-zwOTdL3rFQ/lRdBnntKVOX6k5cKJwEc1HdilT71BWEu7J41gXIB2MRp+vxduPSwZJPWBxEzv4yH1wYLJGUHX4Q==",
"dev": true,
"license": "MIT",
"dependencies": {
"universalify": "^2.0.0"
},
"optionalDependencies": {
"graceful-fs": "^4.1.6"
}
},
"node_modules/minimist": {
"version": "1.2.8",
"resolved": "https://registry.npmjs.org/minimist/-/minimist-1.2.8.tgz",
"integrity": "sha512-2yyAR8qBkN3YuheJanUpWC5U3bb5osDywNB8RzDVlDwDHbocAJveqqj1u8+SVD7jkWT4yvsHCpWqqWqAxb0zCA==",
"dev": true,
"license": "MIT",
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/minipass": {
"version": "7.1.3",
"resolved": "https://registry.npmjs.org/minipass/-/minipass-7.1.3.tgz",
"integrity": "sha512-tEBHqDnIoM/1rXME1zgka9g6Q2lcoCkxHLuc7ODJ5BxbP5d4c2Z5cGgtXAku59200Cx7diuHTOYfSBD8n6mm8A==",
"dev": true,
"license": "BlueOak-1.0.0",
"engines": {
"node": ">=16 || 14 >=14.17"
}
},
"node_modules/minizlib": {
"version": "3.1.0",
"resolved": "https://registry.npmjs.org/minizlib/-/minizlib-3.1.0.tgz",
"integrity": "sha512-KZxYo1BUkWD2TVFLr0MQoM8vUUigWD3LlD83a/75BqC+4qE0Hb1Vo5v1FgcfaNXvfXzr+5EhQ6ing/CaBijTlw==",
"dev": true,
"license": "MIT",
"dependencies": {
"minipass": "^7.1.2"
},
"engines": {
"node": ">= 18"
}
},
"node_modules/ms": {
"version": "2.1.3",
"resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz",
"integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==",
"dev": true,
"license": "MIT"
},
"node_modules/node-addon-api": {
"version": "8.9.2",
"resolved": "https://registry.npmjs.org/node-addon-api/-/node-addon-api-8.9.2.tgz",
"integrity": "sha512-VijLXbi3UACN69I0JVXJsX4tjACjNoQDgv2gTF6sx2wWEi8tkSg2eX8p5gSIFi8z2+DL3oHmY6OyKce38SDolg==",
"dev": true,
"license": "MIT",
"engines": {
"node": "^18 || ^20 || >= 21"
}
},
"node_modules/node-api-headers": {
"version": "1.9.0",
"resolved": "https://registry.npmjs.org/node-api-headers/-/node-api-headers-1.9.0.tgz",
"integrity": "sha512-2oNILP4jXwRB4ywnYKjVk1YyJ96n2D4EOVJO6S3oYZ5PtbJrw3Yt9TpAuX3nBLMuzn74rnfGQrv13pS9vC+YiA==",
"dev": true,
"license": "MIT"
},
"node_modules/node-gyp-build": {
"version": "4.8.4",
"resolved": "https://registry.npmjs.org/node-gyp-build/-/node-gyp-build-4.8.4.tgz",
"integrity": "sha512-LA4ZjwlnUblHVgq0oBF3Jl/6h/Nvs5fzBLwdEF4nuxnFdsfajde4WfxtJr3CaiH+F6ewcIB/q4jQ4UzPyid+CQ==",
"license": "MIT",
"bin": {
"node-gyp-build": "bin.js",
"node-gyp-build-optional": "optional.js",
"node-gyp-build-test": "build-test.js"
}
},
"node_modules/rc": {
"version": "1.2.8",
"resolved": "https://registry.npmjs.org/rc/-/rc-1.2.8.tgz",
"integrity": "sha512-y3bGgqKj3QBdxLbLkomlohkvsA8gdAiUQlSBJnBhfn+BPxg4bc62d8TcBW15wavDfgexCgccckhcZvywyQYPOw==",
"dev": true,
"license": "(BSD-2-Clause OR MIT OR Apache-2.0)",
"dependencies": {
"deep-extend": "^0.6.0",
"ini": "~1.3.0",
"minimist": "^1.2.0",
"strip-json-comments": "~2.0.1"
},
"bin": {
"rc": "cli.js"
}
},
"node_modules/require-directory": {
"version": "2.1.1",
"resolved": "https://registry.npmjs.org/require-directory/-/require-directory-2.1.1.tgz",
"integrity": "sha512-fGxEI7+wsG9xrvdjsrlmL22OMTTiHRwAMroiEeMgq8gzoLC/PQr7RsRDSTLUg/bZAZtF+TVIkHc6/4RIKrui+Q==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=0.10.0"
}
},
"node_modules/semver": {
"version": "7.8.5",
"resolved": "https://registry.npmjs.org/semver/-/semver-7.8.5.tgz",
"integrity": "sha512-Y7/KDsb8LjooZpwaqGyulO6DQlksgCncchHGk+sZIY4SBvUocMBEFH5Ur1fI4dV+Jvl0w6cjvucaIi40puRioA==",
"dev": true,
"license": "ISC",
"bin": {
"semver": "bin/semver.js"
},
"engines": {
"node": ">=10"
}
},
"node_modules/string-width": {
"version": "4.2.3",
"resolved": "https://registry.npmjs.org/string-width/-/string-width-4.2.3.tgz",
"integrity": "sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g==",
"dev": true,
"license": "MIT",
"dependencies": {
"emoji-regex": "^8.0.0",
"is-fullwidth-code-point": "^3.0.0",
"strip-ansi": "^6.0.1"
},
"engines": {
"node": ">=8"
}
},
"node_modules/strip-ansi": {
"version": "6.0.1",
"resolved": "https://registry.npmjs.org/strip-ansi/-/strip-ansi-6.0.1.tgz",
"integrity": "sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A==",
"dev": true,
"license": "MIT",
"dependencies": {
"ansi-regex": "^5.0.1"
},
"engines": {
"node": ">=8"
}
},
"node_modules/strip-json-comments": {
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/strip-json-comments/-/strip-json-comments-2.0.1.tgz",
"integrity": "sha512-4gB8na07fecVVkOI6Rs4e7T6NOTki5EmL7TUduTs6bu3EdnSycntVJ4re8kgZA+wx9IueI2Y11bfbgwtzuE0KQ==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=0.10.0"
}
},
"node_modules/tar": {
"version": "7.5.22",
"resolved": "https://registry.npmjs.org/tar/-/tar-7.5.22.tgz",
"integrity": "sha512-MFO/QzvtAOmJbkhOaCTvbGcFN9L9b+JunIsDwaKljSOdcLMea3NJ1k9Usz/rjdfSXTq4dfzfeS7W4p4YOAAHeA==",
"dev": true,
"license": "BlueOak-1.0.0",
"dependencies": {
"@isaacs/fs-minipass": "^4.0.0",
"chownr": "^3.0.0",
"minipass": "^7.1.2",
"minizlib": "^3.1.0",
"yallist": "^5.0.0"
},
"engines": {
"node": ">=18"
}
},
"node_modules/typescript": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/typescript/-/typescript-7.0.2.tgz",
"integrity": "sha512-8FYau96o3NKOhbjKi/qNvG/W5jhzxkbdm5sj9AbZ/5T5sWqn3hJgLfGx27sRKZWTvyzCP8dLRBTf5tBTSRVUNA==",
"dev": true,
"license": "Apache-2.0",
"bin": {
"tsc": "bin/tsc"
},
"engines": {
"node": ">=16.20.0"
},
"optionalDependencies": {
"@typescript/typescript-aix-ppc64": "7.0.2",
"@typescript/typescript-darwin-arm64": "7.0.2",
"@typescript/typescript-darwin-x64": "7.0.2",
"@typescript/typescript-freebsd-arm64": "7.0.2",
"@typescript/typescript-freebsd-x64": "7.0.2",
"@typescript/typescript-linux-arm": "7.0.2",
"@typescript/typescript-linux-arm64": "7.0.2",
"@typescript/typescript-linux-loong64": "7.0.2",
"@typescript/typescript-linux-mips64el": "7.0.2",
"@typescript/typescript-linux-ppc64": "7.0.2",
"@typescript/typescript-linux-riscv64": "7.0.2",
"@typescript/typescript-linux-s390x": "7.0.2",
"@typescript/typescript-linux-x64": "7.0.2",
"@typescript/typescript-netbsd-arm64": "7.0.2",
"@typescript/typescript-netbsd-x64": "7.0.2",
"@typescript/typescript-openbsd-arm64": "7.0.2",
"@typescript/typescript-openbsd-x64": "7.0.2",
"@typescript/typescript-sunos-x64": "7.0.2",
"@typescript/typescript-win32-arm64": "7.0.2",
"@typescript/typescript-win32-x64": "7.0.2"
}
},
"node_modules/undici-types": {
"version": "6.21.0",
"resolved": "https://registry.npmjs.org/undici-types/-/undici-types-6.21.0.tgz",
"integrity": "sha512-iwDZqg0QAGrg9Rav5H4n0M64c3mkR59cJ6wQp+7C4nI0gsmExaedaYLNO44eT4AtBBwjbTiGPMlt2Md0T9H9JQ==",
"dev": true,
"license": "MIT"
},
"node_modules/universalify": {
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/universalify/-/universalify-2.0.1.tgz",
"integrity": "sha512-gptHNQghINnc/vTGIk0SOFGFNXw7JVrlRUtConJRlvaw6DuX0wO5Jeko9sWrMBhh+PsYAZ7oXAiOnf/UKogyiw==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">= 10.0.0"
}
},
"node_modules/url-join": {
"version": "4.0.1",
"resolved": "https://registry.npmjs.org/url-join/-/url-join-4.0.1.tgz",
"integrity": "sha512-jk1+QP6ZJqyOiuEI9AEWQfju/nB2Pw466kbA0LEZljHwKeMgd9WrAEgEGxjPDD2+TNbbb37rTyhEfrCXfuKXnA==",
"dev": true,
"license": "MIT"
},
"node_modules/which": {
"version": "6.0.1",
"resolved": "https://registry.npmjs.org/which/-/which-6.0.1.tgz",
"integrity": "sha512-oGLe46MIrCRqX7ytPUf66EAYvdeMIZYn3WaocqqKZAxrBpkqHfL/qvTyJ/bTk5+AqHCjXmrv3CEWgy368zhRUg==",
"dev": true,
"license": "ISC",
"dependencies": {
"isexe": "^4.0.0"
},
"bin": {
"node-which": "bin/which.js"
},
"engines": {
"node": "^20.17.0 || >=22.9.0"
}
},
"node_modules/wrap-ansi": {
"version": "7.0.0",
"resolved": "https://registry.npmjs.org/wrap-ansi/-/wrap-ansi-7.0.0.tgz",
"integrity": "sha512-YVGIj2kamLSTxw6NsZjoBxfSwsn0ycdesmc4p+Q21c5zPuZ1pl+NfxVdxPtdHvmNVOQ6XSYG4AUtyt/Fi7D16Q==",
"dev": true,
"license": "MIT",
"dependencies": {
"ansi-styles": "^4.0.0",
"string-width": "^4.1.0",
"strip-ansi": "^6.0.0"
},
"engines": {
"node": ">=10"
},
"funding": {
"url": "https://github.com/chalk/wrap-ansi?sponsor=1"
}
},
"node_modules/y18n": {
"version": "5.0.8",
"resolved": "https://registry.npmjs.org/y18n/-/y18n-5.0.8.tgz",
"integrity": "sha512-0pfFzegeDWJHJIAmTLRP2DwHjdF5s7jo9tuztdQxAhINCdvS+3nGINqPd00AphqJR/0LhANUS6/+7SCb98YOfA==",
"dev": true,
"license": "ISC",
"engines": {
"node": ">=10"
}
},
"node_modules/yallist": {
"version": "5.0.0",
"resolved": "https://registry.npmjs.org/yallist/-/yallist-5.0.0.tgz",
"integrity": "sha512-YgvUTfwqyc7UXVMrB+SImsVYSmTS8X/tSrtdNZMImM+n7+QTriRXyXim0mBrTXNeqzVF0KWGgHPeiyViFFrNDw==",
"dev": true,
"license": "BlueOak-1.0.0",
"engines": {
"node": ">=18"
}
},
"node_modules/yargs": {
"version": "17.7.3",
"resolved": "https://registry.npmjs.org/yargs/-/yargs-17.7.3.tgz",
"integrity": "sha512-GZtjxm/J/4TSxuL3FNYjCmLktBTnIw/rVmKSIyKeYAZpmJB2ig9VauCC5xsa82GNKVKDAqpOn3KVzNt0zmrU0g==",
"dev": true,
"license": "MIT",
"dependencies": {
"cliui": "^8.0.1",
"escalade": "^3.1.1",
"get-caller-file": "^2.0.5",
"require-directory": "^2.1.1",
"string-width": "^4.2.3",
"y18n": "^5.0.5",
"yargs-parser": "^21.1.1"
},
"engines": {
"node": ">=12"
}
},
"node_modules/yargs-parser": {
"version": "21.1.1",
"resolved": "https://registry.npmjs.org/yargs-parser/-/yargs-parser-21.1.1.tgz",
"integrity": "sha512-tVpsJW7DdjecAiFpbIB1e3qxIQsE6NoPc5/eTdrbbIC4h0LVsWhnoa3g+m2HclBIujHzsxZ4VJVA+GUuc2/LBw==",
"dev": true,
"license": "ISC",
"engines": {
"node": ">=12"
}
}
}
}
+58
View File
@@ -0,0 +1,58 @@
{
"name": "opendht",
"version": "0.1.0",
"description": "Distributed hash table for Node.js",
"license": "MIT",
"repository": {
"type": "git",
"url": "git+https://github.com/savoirfairelinux/opendht.git",
"directory": "nodejs"
},
"publishConfig": {
"access": "public",
"registry": "https://registry.npmjs.org/"
},
"type": "module",
"main": "index.js",
"types": "index.d.ts",
"exports": {
".": {
"types": "./index.d.ts",
"import": "./index.js"
},
"./package.json": "./package.json"
},
"files": [
"index.js",
"index.d.ts",
"crypto.js",
"README.md",
"examples/dhtnode.js",
"examples/dhtnode.ts",
"prebuilds/"
],
"engines": {
"node": ">=22.9.0"
},
"scripts": {
"build": "cmake-js compile",
"postbuild": "npm run build:examples",
"build:examples": "tsc --strict --types node --target es2024 --module Node16 --moduleResolution Node16 --rootDir examples --outDir examples examples/dhtnode.ts",
"dhtnode": "node examples/dhtnode.js",
"prepack": "npm run build:examples",
"bundle": "node scripts/bundle.js",
"package": "npm run bundle && npm pack --pack-destination dist",
"test:package": "node scripts/test-package.js",
"test": "node --test test/*.test.js",
"typecheck": "tsc --noEmit --strict --types node --target es2024 --module Node16 --moduleResolution Node16 test/types.ts examples/dhtnode.ts"
},
"dependencies": {
"node-gyp-build": "^4.8.4"
},
"devDependencies": {
"@types/node": "^22.20.5",
"cmake-js": "^8.0.0",
"node-addon-api": "^8.8.0",
"typescript": "^7.0.2"
}
}
+105
View File
@@ -0,0 +1,105 @@
import { execFileSync } from 'node:child_process';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
const root = path.resolve(import.meta.dirname, '..');
const repository = path.dirname(root);
const input = path.resolve(process.argv[2] || path.join(root, 'build/Release/opendht.node'));
const target = `${process.platform}-${process.arch}`;
const output = path.join(root, 'prebuilds', target);
const temporary = fs.mkdtempSync(path.join(os.tmpdir(), 'opendht-bundle-'));
const run = (command, args) => execFileSync(command, args, { encoding: 'utf8', stdio: ['ignore', 'pipe', 'inherit'] });
function copyNotices(dependencies) {
const directory = path.join(output, 'licenses');
fs.mkdirSync(directory, { recursive: true });
fs.copyFileSync(path.join(repository, 'LICENSE'), path.join(directory, 'OpenDHT-LICENSE.txt'));
const copied = new Set();
for (const dependency of dependencies) {
const real = fs.realpathSync(dependency);
if (!path.relative(repository, real).startsWith('..')) continue;
let name;
let notices;
if (process.platform === 'darwin') {
const match = real.match(/^(.*\/Cellar\/([^/]+)\/[^/]+)\//);
if (!match) throw new Error(`Cannot locate license notices for ${dependency}`);
name = match[2];
notices = fs.readdirSync(match[1]).filter(file => /^(COPYING|LICENSE|COPYRIGHT)([._-]|$)/i.test(file))
.map(file => path.join(match[1], file));
} else if (process.platform === 'linux' && /^libllhttp\.so(?:\.|$)/.test(path.basename(real))) {
name = 'llhttp';
notices = [path.join(repository, 'resources', 'llhttp-LICENSE-MIT')];
} else if (process.platform === 'linux') {
let owner;
try { owner = run('dpkg-query', ['-S', real]); }
catch { owner = run('dpkg-query', ['-S', real.replace(/^\/usr\//, '/')]); }
name = owner.split('\n')[0].split(': ')[0].split(':')[0];
notices = [path.join('/usr/share/doc', name, 'copyright')];
} else {
const installed = process.env.VCPKG_INSTALLED_DIR;
const triplet = process.env.VCPKG_DEFAULT_TRIPLET || 'x64-windows-release';
if (!installed) throw new Error('VCPKG_INSTALLED_DIR is required for Windows dependency notices');
const share = path.join(installed, triplet, 'share');
for (const entry of fs.readdirSync(share)) {
const copyright = path.join(share, entry, 'copyright');
if (fs.existsSync(copyright)) fs.copyFileSync(copyright, path.join(directory, `${entry}-copyright.txt`));
}
break;
}
if (!notices.length) throw new Error(`Missing license notices for ${name}`);
if (copied.has(name)) continue;
for (const notice of notices) fs.copyFileSync(notice, path.join(directory, `${name}-${path.basename(notice)}.txt`));
copied.add(name);
}
}
function fixMachO(file) {
const commands = run('otool', ['-l', file]);
const rpaths = [...commands.matchAll(/cmd LC_RPATH\s+cmdsize \d+\s+path (.+?) \(offset \d+\)/g)];
for (const rpath of new Set(rpaths.map(match => match[1]))) run('install_name_tool', ['-delete_rpath', rpath, file]);
const dependencies = [...run('otool', ['-L', file]).matchAll(/^\s+(.+?) \(compatibility version/gm)];
for (const [, dependency] of dependencies) {
if (!/^(@loader_path\/|\/usr\/lib\/|\/System\/Library\/)/.test(dependency)) {
throw new Error(`Non-relocatable dependency in ${path.basename(file)}: ${dependency}`);
}
}
run('codesign', ['--force', '--sign', '-', file]);
}
try {
if (!['darwin', 'linux', 'win32'].includes(process.platform)) throw new Error(`Unsupported platform ${target}`);
if (!fs.existsSync(input)) throw new Error(`Build the Release addon first: ${input}`);
if (process.platform === 'linux' && !process.report.getReport().header.glibcVersionRuntime) {
throw new Error('Linux prebuilds currently require glibc');
}
const directories = [path.dirname(input), ...(process.env.OPENDHT_LIBRARY_PATH || '').split(path.delimiter).filter(Boolean)];
const dependencyList = path.join(temporary, 'dependencies.json');
run('cmake', [`-DADDON=${input}`, `-DOUTPUT=${dependencyList}`, `-DSEARCH_DIRECTORIES=${directories.join(';')}`,
'-P', path.join(import.meta.dirname, 'dependencies.cmake')]);
const dependencies = JSON.parse(fs.readFileSync(dependencyList, 'utf8'));
fs.rmSync(output, { recursive: true, force: true });
fs.mkdirSync(output, { recursive: true });
const binary = path.join(output, process.platform === 'linux' ? 'node.napi.glibc.node' : 'node.napi.node');
fs.copyFileSync(input, binary);
if (process.platform === 'darwin') {
run('dylibbundler', ['-b', '-of', '-ns', '-x', binary, '-d', `${output}/`, '-p', '@loader_path/', '-s', path.dirname(input)]);
for (const file of fs.readdirSync(output)) fixMachO(path.join(output, file));
} else {
for (const dependency of dependencies) fs.copyFileSync(dependency, path.join(output, path.basename(dependency)));
if (process.platform === 'linux') {
for (const file of fs.readdirSync(output)) run('patchelf', ['--set-rpath', '$ORIGIN', path.join(output, file)]);
}
}
copyNotices(dependencies);
fs.writeFileSync(path.join(output, 'manifest.json'), JSON.stringify({
platform: process.platform,
arch: process.arch,
napi: 8,
libraries: fs.readdirSync(output).filter(file => /\.(node|dll|dylib)$|\.so(\.|$)/.test(file)).sort(),
}, null, 2) + '\n');
fs.mkdirSync(path.join(root, 'dist'), { recursive: true });
console.log(`Bundled ${target}: ${output}`);
} finally {
fs.rmSync(temporary, { recursive: true, force: true });
}
+39
View File
@@ -0,0 +1,39 @@
cmake_minimum_required(VERSION 3.21)
if(NOT EXISTS "${ADDON}" OR NOT DEFINED OUTPUT)
message(FATAL_ERROR "ADDON and OUTPUT are required")
endif()
set(pre_exclude)
set(post_exclude)
if(CMAKE_HOST_SYSTEM_NAME STREQUAL "Darwin")
list(APPEND pre_exclude "^/usr/lib/" "^/System/Library/")
list(APPEND post_exclude "^/usr/lib/" "^/System/Library/")
elseif(CMAKE_HOST_SYSTEM_NAME STREQUAL "Linux")
list(APPEND pre_exclude "^ld-linux" "^lib(c|m|dl|rt|pthread|resolv|util|gcc_s|stdc\\+\\+)\\.so")
elseif(CMAKE_HOST_WIN32)
list(APPEND pre_exclude "[Nn][Oo][Dd][Ee]\\.exe" "[Aa][Pp][Ii]-[Mm][Ss]-.*" "[Ee][Xx][Tt]-[Mm][Ss]-.*")
list(APPEND post_exclude ".*[Ss][Yy][Ss][Tt][Ee][Mm]32[/\\\\].*")
endif()
file(GET_RUNTIME_DEPENDENCIES
MODULES "${ADDON}"
DIRECTORIES ${SEARCH_DIRECTORIES}
RESOLVED_DEPENDENCIES_VAR dependencies
UNRESOLVED_DEPENDENCIES_VAR unresolved
CONFLICTING_DEPENDENCIES_PREFIX conflicts
PRE_EXCLUDE_REGEXES ${pre_exclude}
POST_EXCLUDE_REGEXES ${post_exclude}
)
if(unresolved OR conflicts_FILENAMES)
message(FATAL_ERROR "Unresolved dependencies: ${unresolved}; conflicting dependencies: ${conflicts_FILENAMES}")
endif()
set(result "[]")
set(index 0)
foreach(dependency IN LISTS dependencies)
file(TO_CMAKE_PATH "${dependency}" dependency)
string(JSON result SET "${result}" ${index} "\"${dependency}\"")
math(EXPR index "${index} + 1")
endforeach()
file(WRITE "${OUTPUT}" "${result}\n")
+38
View File
@@ -0,0 +1,38 @@
import { execFileSync } from 'node:child_process';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
const root = path.resolve(import.meta.dirname, '..');
const npm = process.env.npm_execpath;
if (!npm) throw new Error('Run this check with npm run test:package');
fs.mkdirSync(path.join(root, 'dist'), { recursive: true });
let archive = process.argv[2] ? path.resolve(process.argv[2]) : path.join(root, 'dist',
JSON.parse(execFileSync(process.execPath, [npm, 'pack', '--json', '--pack-destination', 'dist'],
{ cwd: root, encoding: 'utf8' }))[0].filename);
if (fs.statSync(archive).isDirectory()) {
const archives = fs.readdirSync(archive).filter(file => file.endsWith('.tgz'));
if (archives.length !== 1) throw new Error('Expected exactly one npm archive');
archive = path.join(archive, archives[0]);
}
const consumer = fs.mkdtempSync(path.join(os.tmpdir(), 'opendht-consumer-'));
const env = { ...process.env, PREBUILDS_ONLY: '1' };
for (const variable of ['OPENDHT_NODE_BINDING', 'NODE_PATH', 'LD_LIBRARY_PATH', 'DYLD_LIBRARY_PATH', 'DYLD_FALLBACK_LIBRARY_PATH']) delete env[variable];
const excluded = [path.join(root, 'build'), ...(process.env.OPENDHT_LIBRARY_PATH || '').split(path.delimiter).filter(Boolean)];
const pathKey = Object.keys(env).find(key => key.toLowerCase() === 'path');
if (pathKey) env[pathKey] = env[pathKey].split(path.delimiter)
.filter(entry => !excluded.some(directory => path.resolve(entry).toLowerCase().startsWith(path.resolve(directory).toLowerCase())))
.join(path.delimiter);
try {
execFileSync(process.execPath, [npm, 'install', '--prefix', consumer, '--ignore-scripts', '--omit=dev', '--no-audit', '--no-fund', archive],
{ cwd: consumer, env, stdio: 'inherit' });
const { name } = JSON.parse(fs.readFileSync(path.join(root, 'package.json'), 'utf8'));
const installed = path.join(consumer, 'node_modules', ...name.split('/'));
execFileSync(process.execPath, ['--input-type=module', '-e', 'import load from "node-gyp-build"; console.log(load.path(process.cwd()))'],
{ cwd: installed, env, stdio: 'inherit' });
fs.cpSync(path.join(root, 'test'), path.join(installed, 'test'), { recursive: true });
execFileSync(process.execPath, [npm, 'test'], { cwd: installed, env, stdio: 'inherit' });
console.log(`Installed archive verified: ${archive}`);
} finally {
fs.rmSync(consumer, { recursive: true, force: true });
}
+549
View File
@@ -0,0 +1,549 @@
import assert from 'node:assert/strict';
import { execFile } from 'node:child_process';
import { EventEmitter, once } from 'node:events';
import { createServer } from 'node:http';
import { test } from 'node:test';
import { fileURLToPath } from 'node:url';
import { promisify } from 'node:util';
import { Worker } from 'node:worker_threads';
import { DhtRunner, hash, version } from '../index.js';
const moduleURL = new URL('../index.js', import.meta.url).href;
const exampleURL = new URL('../examples/dhtnode.js', import.meta.url);
const example = fileURLToPath(exampleURL);
function startNode() {
const runner = new DhtRunner();
runner.run({ port: 0, ipv4: '127.0.0.1', ipv6: '::1' });
return runner;
}
test('hashes strings and binary data without truncating NUL bytes', () => {
assert.match(version, /^\d+\./);
assert.equal(hash('hello'), 'aaf4c61ddcc5e8a2dabede0f3b482cd9aea9434d');
assert.equal(hash('hello'), hash(Buffer.from('hello')));
assert.notEqual(hash('hello\0world'), hash('hello'));
assert.throws(() => hash(42), TypeError);
});
test('runner lifecycle and argument validation', async () => {
const runner = new DhtRunner();
assert.equal(runner.isRunning(), false);
assert.throws(() => runner.get(hash('idle')), /running/i);
assert.throws(() => runner.getAll(hash('idle')), /running/i);
assert.throws(() => runner.run({ port: -1 }), RangeError);
assert.throws(() => runner.run({ port: 65536 }), RangeError);
assert.throws(() => runner.run({ port: 1.5 }), RangeError);
runner.run({ port: 0, ipv4: '127.0.0.1', ipv6: '::1' });
try {
assert.equal(runner.isRunning(), true);
assert.ok(runner.getBoundPort() > 0);
assert.match(runner.getNodeId(), /^[0-9a-f]{40}$/);
assert.throws(() => runner.run(), /already/i);
assert.throws(() => runner.get('short'), /hash/i);
assert.throws(() => runner.get('z'.repeat(40)), /hash/i);
assert.throws(() => runner.put(hash('bad'), 'text'), TypeError);
assert.throws(() => runner.put(hash('bad'), Buffer.alloc(0), { id: 1 }), TypeError);
assert.throws(() => runner.put(hash('bad'), Buffer.alloc(0), { id: -1n }), RangeError);
assert.throws(() => runner.put(hash('bad'), Buffer.alloc(0), { id: 1n << 64n }), RangeError);
assert.throws(() => runner.put(hash('bad'), Buffer.alloc(0), { type: 65536 }), RangeError);
} finally {
const closing = runner.close();
assert.equal(runner.close(), closing);
await closing;
}
assert.equal(runner.isRunning(), false);
assert.throws(() => runner.run(), /closed/i);
assert.throws(() => runner.get(hash('closed')), /closed/i);
});
test('closing an idle runner is supported', async () => {
const runner = new DhtRunner();
await runner.close();
await runner.close();
assert.equal(runner.isRunning(), false);
});
test('runner reports per-family connectivity changes and snapshots', { timeout: 15000 }, async () => {
const runner = new DhtRunner();
const peer = startNode();
const changes = [];
runner.on('statusChanged', status => changes.push(status));
try {
assert.deepEqual(runner.getStatus(), { ipv4: 'disconnected', ipv6: 'disconnected' });
runner.run({ port: 0, ipv4: '127.0.0.1', ipv6: '::1' });
const connected = new Promise((resolve, reject) => {
const signal = AbortSignal.timeout(10000);
signal.addEventListener('abort', () => reject(signal.reason), { once: true });
runner.on('statusChanged', status => {
if (status.ipv4 === 'connected') resolve(status);
});
});
runner.bootstrap('127.0.0.1', peer.getBoundPort());
peer.bootstrap('127.0.0.1', runner.getBoundPort());
const status = await connected;
assert.equal(status.ipv4, 'connected');
assert.ok(['disconnected', 'connecting', 'connected'].includes(status.ipv6));
assert.deepEqual(runner.getStatus(), status);
status.ipv4 = 'changed by consumer';
assert.equal(runner.getStatus().ipv4, 'connected');
await runner.close();
const count = changes.length;
await new Promise(resolve => setImmediate(resolve));
assert.equal(changes.length, count);
assert.deepEqual(runner.getStatus(), { ipv4: 'disconnected', ipv6: 'disconnected' });
} finally {
await Promise.all([runner.close(), peer.close()]);
}
});
test('invalid bind addresses produce JS exceptions', async () => {
const runner = new DhtRunner();
try {
assert.throws(() => runner.run({ ipv4: '192.0.2.1', ipv6: '2001:db8::1' }));
} finally {
await runner.close();
}
});
test('two loopback nodes exchange binary values and cancel listeners', { timeout: 20000 }, async () => {
const writer = startNode();
const reader = startNode();
const events = new EventEmitter();
const key = hash('node-addon-api integration');
const payload = Buffer.from([0, 1, 255, 0, 42]);
let subscription;
let retainedValue;
try {
writer.bootstrap('127.0.0.1', reader.getBoundPort());
reader.bootstrap('127.0.0.1', writer.getBoundPort());
subscription = reader.listen(key, (value, expired) => {
if (!expired) events.emit('value', value);
});
const received = once(events, 'value', { signal: AbortSignal.timeout(10000) });
assert.equal(await writer.put(key, payload, {
id: 18446744073709551614n,
type: 0,
userType: 'nodejs-test',
permanent: true,
}), true);
const [value] = await received;
assert.equal(value.id, 18446744073709551614n);
assert.equal(value.type, 0);
assert.ok(Buffer.isBuffer(value.data));
assert.deepEqual(value.data, payload);
assert.equal(value.userType, 'nodejs-test');
assert.equal(typeof value.seq, 'number');
assert.equal(value.owner, null);
assert.equal(value.isSigned, false);
assert.equal(value.isEncrypted, false);
value.data.fill(7);
const values = await reader.getAll(key);
retainedValue = values.find(candidate => candidate.id === value.id);
assert.ok(retainedValue);
assert.deepEqual(retainedValue.data, payload);
subscription.cancel();
subscription.cancel();
} finally {
subscription?.cancel();
await Promise.all([writer.close(), reader.close()]);
}
assert.deepEqual(retainedValue.data, payload);
});
test('close settles pending operations and suppresses queued listener deliveries', { timeout: 10000 }, async () => {
const runner = startNode();
let deliveries = 0;
const subscription = runner.listen(hash('closing'), () => deliveries++);
const pending = runner.getAll(hash('unreachable'));
const settled = assert.rejects(pending, { code: 'ERR_DHT_CLOSED' });
await runner.close();
await settled;
subscription.cancel();
assert.equal(deliveries, 0);
});
test('get iterators settle pending reads on cancellation and reject on runner close', { timeout: 10000 }, async () => {
const runner = startNode();
try {
const cancelled = runner.get(hash('cancel get'));
assert.equal(cancelled[Symbol.asyncIterator](), cancelled);
const pending = [cancelled.next(), cancelled.next()];
await cancelled.return();
cancelled.cancel();
assert.deepEqual(await Promise.all(pending), [
{ value: undefined, done: true },
{ value: undefined, done: true },
]);
assert.deepEqual(await cancelled.next(), { value: undefined, done: true });
const closing = runner.get(hash('close get'));
const rejected = assert.rejects(closing.next(), { code: 'ERR_DHT_CLOSED' });
await runner.close();
await rejected;
await assert.rejects(closing.next(), { code: 'ERR_DHT_CLOSED' });
} finally {
await runner.close();
}
});
test('async listeners deliver queued values and cancel on loop exit', { timeout: 15000 }, async () => {
const writer = startNode();
const reader = startNode();
const key = hash('async listener');
let iterator;
try {
writer.bootstrap('127.0.0.1', reader.getBoundPort());
reader.bootstrap('127.0.0.1', writer.getBoundPort());
iterator = reader.listen(key);
assert.equal(iterator[Symbol.asyncIterator](), iterator);
const pending = iterator.next();
assert.equal(await writer.put(key, Buffer.from('first'), { id: 1n }), true);
const first = await pending;
assert.equal(first.done, false);
assert.equal(first.value.expired, false);
assert.equal(first.value.value.id, 1n);
assert.equal(first.value.value.data.toString(), 'first');
assert.equal(await writer.put(key, Buffer.from('second'), { id: 2n }), true);
for await (const { value, expired } of iterator) {
assert.equal(expired, false);
assert.equal(value.id, 2n);
assert.equal(value.data.toString(), 'second');
break;
}
assert.deepEqual(await iterator.next(), { value: undefined, done: true });
iterator.cancel();
} finally {
iterator?.cancel();
await Promise.all([writer.close(), reader.close()]);
}
});
test('async listener cancellation settles concurrent reads and drops buffered values', { timeout: 10000 }, async () => {
const runner = startNode();
const peer = startNode();
const key = hash('async cancellation');
let iterator;
try {
runner.bootstrap('127.0.0.1', peer.getBoundPort());
peer.bootstrap('127.0.0.1', runner.getBoundPort());
iterator = runner.listen(key);
const pending = [iterator.next(), iterator.next()];
iterator.cancel();
iterator.cancel();
assert.deepEqual(await Promise.all(pending), [
{ value: undefined, done: true },
{ value: undefined, done: true },
]);
assert.equal(await runner.put(key, Buffer.from('cancelled'), { id: 3n }), true);
assert.deepEqual(await iterator.next(), { value: undefined, done: true });
const buffered = runner.listen(key);
await runner.getAll(key);
await buffered.return();
assert.deepEqual(await buffered.next(), { value: undefined, done: true });
} finally {
iterator?.cancel();
await Promise.all([runner.close(), peer.close()]);
}
});
test('async listener throw, disposal, and runner close settle pending reads', { timeout: 10000 }, async () => {
const runner = startNode();
try {
const thrown = runner.listen(hash('async throw'));
const pendingThrow = thrown.next();
const failure = new Error('consumer failed');
await assert.rejects(thrown.throw(failure), error => error === failure);
assert.deepEqual(await pendingThrow, { value: undefined, done: true });
assert.deepEqual(await thrown.next(), { value: undefined, done: true });
const disposed = runner.listen(hash('async dispose'));
const pendingDispose = disposed.next();
await disposed[Symbol.asyncDispose]();
assert.deepEqual(await pendingDispose, { value: undefined, done: true });
const closing = runner.listen(hash('async close'));
const pendingClose = [closing.next(), closing.next()];
const close = runner.close();
assert.equal(runner.close(), close);
assert.deepEqual(await Promise.all(pendingClose), [
{ value: undefined, done: true },
{ value: undefined, done: true },
]);
await close;
assert.deepEqual(await closing.next(), { value: undefined, done: true });
assert.throws(() => runner.listen(hash('closed iterator')), /closed/i);
} finally {
await runner.close();
}
});
test('proxy configuration validates option types before starting', async () => {
const runner = new DhtRunner();
try {
assert.throws(() => runner.run({ proxyServer: 42 }), TypeError);
assert.throws(() => runner.run({ proxyUserAgent: false }), TypeError);
assert.throws(() => runner.run({ proxyUserAgent: 'client\r\nInjected: header' }), TypeError);
assert.throws(() => runner.run({ proxyServer: 'http://127.0.0.1\0' }), TypeError);
assert.throws(() => runner.run({ proxyServer: 'ftp://127.0.0.1' }), /http/i);
assert.throws(() => runner.run({ proxyServer: 'not a URL' }), /url/i);
assert.equal(runner.isRunning(), false);
runner.run({ port: 0, ipv4: '127.0.0.1', ipv6: '::1', proxyServer: '' });
assert.ok(runner.getBoundPort() > 0);
} finally {
await runner.close();
}
});
test('proxy configuration routes operations through HTTP with a custom user agent', { timeout: 15000 }, async () => {
const runner = new DhtRunner();
const requests = [];
const values = new Map();
let streamNextGet = false;
let streamResponse;
const server = createServer(async (request, response) => {
response.setHeader('content-type', 'application/json');
if (request.url === '/') {
response.end(JSON.stringify({
node_id: hash('proxy fixture'),
ipv4: { good: 1 },
ipv6: {},
public_ip: '127.0.0.1',
}));
return;
}
requests.push({ method: request.method, url: request.url, userAgent: request.headers['user-agent'] });
if (request.url === `/key/${hash('proxy failure')}`) {
response.statusCode = 500;
response.end();
return;
}
if (request.method === 'POST') {
const chunks = [];
for await (const chunk of request) chunks.push(chunk);
const value = JSON.parse(Buffer.concat(chunks).toString());
values.set(request.url, value);
response.end(JSON.stringify(value));
} else {
const value = values.get(request.url);
const body = value ? `${JSON.stringify(value)}\n` : '';
if (streamNextGet) {
streamNextGet = false;
streamResponse = response;
response.write(body);
} else response.end(body);
}
});
server.listen(0, '127.0.0.1');
try {
await once(server, 'listening');
runner.run({
proxyServer: `http://127.0.0.1:${server.address().port}`,
proxyUserAgent: 'opendht-node-test/1.0',
persistPath: '',
});
const key = hash('proxy configuration');
const payload = Buffer.from([0, 255, 42]);
assert.equal(await runner.put(key, payload, { id: 42n }), true);
const received = await runner.getAll(key);
assert.equal(received.length, 1);
assert.equal(received[0].id, 42n);
assert.deepEqual(received[0].data, payload);
streamNextGet = true;
const iterator = runner.get(key);
const first = await iterator.next();
assert.equal(first.done, false);
assert.equal(first.value.id, 42n);
assert.deepEqual(first.value.data, payload);
assert.equal(streamResponse.writableEnded, false);
streamResponse.end();
assert.deepEqual(await iterator.next(), { value: undefined, done: true });
assert.deepEqual(await iterator.next(), { value: undefined, done: true });
assert.deepEqual(requests.map(request => request.method), ['POST', 'GET', 'GET']);
assert.ok(requests.every(request => request.url === `/key/${key}`));
assert.ok(requests.every(request => request.userAgent === 'opendht-node-test/1.0'));
const empty = runner.get(hash('proxy empty'));
assert.deepEqual(await empty.next(), { value: undefined, done: true });
const failed = runner.get(hash('proxy failure'));
await assert.rejects(failed.next(), { code: 'ERR_DHT_GET_FAILED' });
await assert.rejects(runner.getAll(hash('proxy failure')), { code: 'ERR_DHT_GET_FAILED' });
const all = [];
for await (const value of runner.get(key)) all.push(value);
assert.equal(all.length, 1);
assert.deepEqual(all[0].data, payload);
streamNextGet = true;
const cancelled = runner.get(key);
for await (const value of cancelled) {
assert.equal(value.id, 42n);
break;
}
assert.equal(streamResponse.writableEnded, false);
assert.deepEqual(await cancelled.next(), { value: undefined, done: true });
streamResponse.end();
} finally {
streamResponse?.end();
await runner.close();
await new Promise((resolve, reject) => server.close(error => error ? reject(error) : resolve()));
}
});
test('worker termination cleans up native runners', { timeout: 10000 }, async () => {
const source = `
import { parentPort } from 'node:worker_threads';
import { DhtRunner, hash } from ${JSON.stringify(moduleURL)};
const runner = new DhtRunner();
runner.run({ port: 0, ipv4: '127.0.0.1', ipv6: '::1' });
runner.listen(hash('worker'), () => {});
parentPort.postMessage('ready');
`;
const worker = new Worker(new URL(`data:text/javascript,${encodeURIComponent(source)}`));
try {
await once(worker, 'message');
} finally {
await worker.terminate();
}
});
test('listener callbacks may cancel themselves and close the runner', { timeout: 10000 }, async () => {
const runner = startNode();
const key = hash('reentrant listener');
let deliveries = 0;
let finish;
const received = new Promise(resolve => { finish = resolve; });
const subscription = runner.listen(key, () => {
deliveries++;
subscription.cancel();
finish(runner.close());
});
const puts = [];
for (let index = 0; index < 3000; index++) {
puts.push(runner.put(key, Buffer.from('value'), { id: BigInt(index + 1) }).catch(error => {
assert.equal(error.code, 'ERR_DHT_CLOSED');
}));
}
try {
await received;
await Promise.all(puts);
assert.equal(deliveries, 1);
} finally {
await runner.close();
}
});
test('unref and garbage collection do not strand native threads', { timeout: 20000 }, async () => {
for (const finish of ['runner.unref();', 'runner = null; setImmediate(() => global.gc());']) {
await promisify(execFile)(process.execPath, ['--input-type=module', '--expose-gc', '-e', `
import { DhtRunner } from ${JSON.stringify(moduleURL)};
let runner = new DhtRunner();
runner.run({ port: 0, ipv4: '127.0.0.1', ipv6: '::1' });
${finish}
`], { timeout: 8000 });
}
});
test('callback queue completes bursts beyond the former event limit', { timeout: 15000 }, async () => {
const runner = startNode();
const key = hash('unbounded callback queue');
const errors = [];
const count = 3000;
runner.on('error', error => errors.push(error));
const puts = [];
try {
for (let index = 0; index < count; index++)
puts.push(runner.put(key, Buffer.from('value'), { id: BigInt(index + 1) }));
const results = await Promise.all(puts);
assert.equal(results.length, count);
assert.ok(results.every(result => typeof result === 'boolean'));
assert.equal(runner.isRunning(), true);
assert.deepEqual(errors, []);
} finally {
await runner.close();
}
});
test('dhtnode example prints help and validates its port', async () => {
const { stdout } = await promisify(execFile)(process.execPath, [example, '--help']);
assert.match(stdout, /Usage:.*dhtnode/);
assert.match(stdout, /bootstrap/);
await assert.rejects(promisify(execFile)(process.execPath, [example, '--port', '65536']), error => {
assert.equal(error.code, 1);
assert.match(error.stderr, /port/i);
return true;
});
});
test('dhtnode example accepts piped commands and shuts down on EOF', { timeout: 20000 }, async () => {
const peer = startNode();
try {
const execution = promisify(execFile)(process.execPath, [example,
'--port', '0', '--ipv4', '127.0.0.1',
'--bootstrap', '127.0.0.1', '--bootstrap-port', String(peer.getBoundPort()),
], { timeout: 15000 });
let output = '';
let sentReads = false;
execution.child.stdout.on('data', data => {
output += data;
if (!sentReads && /Stored example/.test(output) && /Stored signed/.test(output)) {
sentReads = true;
execution.child.stdin.end('get example\nget signed\ncancel example\n');
}
});
execution.child.stdin.write('listen example\nput example hello world\nsign signed signed message\n');
const { stdout, stderr } = await execution;
assert.equal(stderr, '');
assert.match(stdout, /Node [0-9a-f]{40}/);
assert.match(stdout, /hello world/);
assert.match(stdout, /signed message.*\[signed\]/);
assert.match(stdout, /Cancelled example/);
assert.match(stdout, /Connectivity: IPv4 connected, IPv6 (disconnected|connecting|connected)/);
} finally {
await peer.close();
}
});
test('dhtnode example handles commands and quit while a get is pending', { timeout: 10000 }, async () => {
const execution = promisify(execFile)(process.execPath, ['--input-type=module', '-e', `
import { DhtRunner } from ${JSON.stringify(moduleURL)};
const close = DhtRunner.prototype.close;
let rejectGet;
DhtRunner.prototype.getAll = function () {
console.log('Get pending');
return new Promise((resolve, reject) => { rejectGet = reject; });
};
DhtRunner.prototype.close = function () {
rejectGet?.(new Error('DHT runner is closed'));
return close.call(this);
};
await import(${JSON.stringify(exampleURL.href)});
`, '--',
'--port', '0', '--ipv4', '127.0.0.1', '--ipv6', '::1',
], { timeout: 8000 });
let output = '';
execution.child.stdout.on('data', data => {
output += data;
if (/get KEY \| put KEY TEXT/.test(output)) execution.child.stdin.end('quit\n');
});
execution.child.stdin.write('get waiting\nhelp\n');
const { stdout, stderr } = await execution;
assert.match(stdout, /Get pending/);
assert.match(stdout, /get KEY \| put KEY TEXT/);
assert.equal(stderr, '');
});
test('dhtnode example shuts down on SIGTERM', { timeout: 10000 }, async () => {
if (process.platform === 'win32') return;
const execution = promisify(execFile)(process.execPath, [example, '--port', '0', '--ipv4', '127.0.0.1'], { timeout: 8000 });
let output = '';
execution.child.stdout.on('data', data => {
output += data;
if (/Node [0-9a-f]{40}/.test(output)) execution.child.kill('SIGTERM');
});
const { stderr } = await execution;
assert.equal(stderr, '');
});
+93
View File
@@ -0,0 +1,93 @@
import assert from 'node:assert/strict';
import { test } from 'node:test';
import { Certificate, DhtRunner, Identity, PrivateKey, PublicKey, hash } from '../index.js';
const payload = Buffer.from([0, 255, 1, 0, 42]);
test('RSA keys sign, encrypt, and survive password-protected serialization', async () => {
const key = await PrivateKey.generate({ algorithm: 'rsa', bits: 2048 });
const publicKey = key.getPublicKey();
assert.match(publicKey.getId(), /^[0-9a-f]{40}$/);
assert.match(publicKey.getLongId(), /^[0-9a-f]{64}$/);
const signature = key.sign(payload);
assert.equal(publicKey.checkSignature(payload, signature), true);
assert.equal(publicKey.checkSignature(Buffer.from('other'), signature), false);
assert.deepEqual(key.decrypt(publicKey.encrypt(payload)), payload);
const restored = new PrivateKey(key.serialize('test password'), 'test password');
const restoredPublic = new PublicKey(publicKey.serialize());
assert.equal(restoredPublic.getLongId(), publicKey.getLongId());
assert.equal(restoredPublic.checkSignature(payload, restored.sign(payload)), true);
assert.deepEqual(restored.decrypt(restoredPublic.encrypt(payload)), payload);
assert.throws(() => new PrivateKey(key.serialize('password'), 'wrong password'));
assert.throws(() => key.decrypt(Buffer.from('invalid ciphertext')));
assert.throws(() => key.serialize('bad\0password'), TypeError);
});
test('EC keys sign and invalid crypto inputs raise JS errors', async () => {
const key = await PrivateKey.generate();
const restored = new PrivateKey(key.serialize());
assert.equal(restored.getPublicKey().checkSignature(payload, key.sign(payload)), true);
const largePayload = Buffer.alloc(16 * 1024 * 1024 + 1, 42);
assert.equal(restored.getPublicKey().checkSignature(largePayload, key.sign(largePayload)), true);
assert.throws(() => key.getPublicKey().encrypt(payload));
for (const Type of [PrivateKey, PublicKey, Certificate]) {
assert.throws(() => new Type(Buffer.alloc(0)));
assert.throws(() => new Type(Buffer.from('not a key or certificate')));
assert.throws(() => new Type(null), TypeError);
}
assert.throws(() => PrivateKey.generate({ algorithm: 'unknown' }), TypeError);
assert.throws(() => PrivateKey.generate({ algorithm: 'rsa', bits: 1 }), RangeError);
assert.throws(() => PrivateKey.generate({ algorithm: 'rsa', bits: 2048.5 }), RangeError);
assert.throws(() => key.sign('not bytes'), TypeError);
assert.throws(() => PrivateKey.prototype.sign.call({}, payload), TypeError);
});
test('identities own matching keys and certificates and support import', async () => {
const identity = await Identity.generate('nodejs crypto');
const certificate = new Certificate(identity.certificate.serialize());
const key = new PrivateKey(identity.privateKey.serialize());
const restored = new Identity(key, certificate);
assert.equal(restored.certificate.getLongId(), key.getPublicKey().getLongId());
assert.equal(certificate.getPublicKey().checkSignature(payload, key.sign(payload)), true);
assert.throws(() => new Identity({}, certificate), TypeError);
const other = await PrivateKey.generate();
assert.throws(() => new Identity(other, certificate), /match/i);
});
test('signed and encrypted values round-trip between authenticated nodes', { timeout: 30000 }, async () => {
const writerIdentity = await Identity.generate('writer');
const readerIdentity = await Identity.generate('reader', { algorithm: 'rsa', bits: 2048 });
const writer = new DhtRunner();
const reader = new DhtRunner();
try {
writer.run({ port: 0, ipv4: '127.0.0.1', ipv6: '::1', identity: writerIdentity });
reader.run({ port: 0, ipv4: '127.0.0.1', ipv6: '::1', identity: readerIdentity });
writer.bootstrap('127.0.0.1', reader.getBoundPort());
reader.bootstrap('127.0.0.1', writer.getBoundPort());
const signedKey = hash('nodejs signed value');
assert.equal(await writer.putSigned(signedKey, payload, { id: 41n }), true);
const signed = await reader.getAll(signedKey);
assert.ok(signed.some(value => value.data.equals(payload) && value.isSigned));
assert.equal(signed[0].owner, writerIdentity.certificate.getLongId());
const encryptedKey = hash('nodejs encrypted value');
assert.equal(await writer.putEncrypted(encryptedKey, payload, readerIdentity.certificate.getPublicKey(), { id: 42n }), true);
const decrypted = await reader.getAll(encryptedKey);
assert.ok(decrypted.some(value => value.data.equals(payload)));
assert.throws(() => writer.putEncrypted(encryptedKey, payload, {}), TypeError);
assert.throws(() => writer.putEncrypted(encryptedKey, payload, writerIdentity.certificate.getPublicKey()), /RSA/i);
} finally {
await Promise.all([writer.close(), reader.close()]);
}
});
test('signed operations require an identity and reject stopped runners', async () => {
const runner = new DhtRunner();
try {
assert.throws(() => runner.run({ identity: {} }), TypeError);
runner.run({ port: 0, ipv4: '127.0.0.1', ipv6: '::1' });
assert.throws(() => runner.putSigned(hash('unsigned'), payload), /identity/i);
} finally {
await runner.close();
}
assert.throws(() => runner.putSigned(hash('closed'), payload), /closed/i);
});
+34
View File
@@ -0,0 +1,34 @@
import assert from 'node:assert/strict';
import { execFileSync } from 'node:child_process';
import { readFileSync } from 'node:fs';
import path from 'node:path';
import { test } from 'node:test';
const root = path.resolve(import.meta.dirname, '..');
test('package exposes ESM-only named exports', () => {
const manifest = JSON.parse(readFileSync(new URL('../package.json', import.meta.url), 'utf8'));
assert.equal(manifest.type, 'module');
assert.equal(manifest.exports['.'].import, './index.js');
assert.equal(manifest.exports['.'].require, undefined);
const stdout = execFileSync(process.execPath, ['--input-type=module', '-e', `
import { DhtRunner, hash } from ${JSON.stringify(manifest.name)};
console.log(hash('hello'));
const runner = new DhtRunner();
await runner.close();
`], { cwd: root, encoding: 'utf8' });
assert.equal(stdout.trim(), 'aaf4c61ddcc5e8a2dabede0f3b482cd9aea9434d');
});
test('npm archive contains runtime files and excludes development artifacts', () => {
const npm = process.env.npm_execpath;
assert.ok(npm, 'Run the package contract with npm test');
const [archive] = JSON.parse(execFileSync(process.execPath,
[npm, 'pack', '--dry-run', '--json', '--ignore-scripts'],
{ cwd: root, encoding: 'utf8' }));
const files = archive.files.map(file => file.path);
for (const file of ['index.js', 'index.d.ts', 'crypto.js', 'README.md', 'examples/dhtnode.js', 'examples/dhtnode.ts']) assert.ok(files.includes(file), file);
assert.equal(files.some(file => /^(build[^/]*|node_modules|test|scripts)\//.test(file)), false);
assert.equal(files.some(file => /\.(cpp|h|tsbuildinfo)$/.test(file)), false);
assert.ok(files.every(file => /^(index\.js|index\.d\.ts|crypto\.js|README\.md|package\.json|examples\/dhtnode\.(js|ts)|prebuilds\/)/.test(file)));
});
+56
View File
@@ -0,0 +1,56 @@
import { Certificate, DhtRunner, Identity, PrivateKey, PublicKey, hash, version } from '../index.js';
import type { ConnectivityStatus, ListenEvent, ListenerIterator, NodeStatus, Value, ValueIterator } from '../index.js';
async function exercise(): Promise<void> {
const privateKey: PrivateKey = await PrivateKey.generate({ algorithm: 'rsa', bits: 2048 });
const publicKey: PublicKey = new PublicKey(privateKey.getPublicKey().serialize());
const identity: Identity = await Identity.generate('typed identity');
const restored = new Identity(identity.privateKey, new Certificate(identity.certificate.serialize()));
const valid: boolean = publicKey.checkSignature(Buffer.alloc(0), privateKey.sign(Buffer.alloc(0)));
const plaintext: Buffer = privateKey.decrypt(publicKey.encrypt(Buffer.from('secret')));
const runner = new DhtRunner();
runner.on('error', (error: Error) => { throw error; });
runner.on('statusChanged', status => {
const ipv4: NodeStatus = status.ipv4;
const ipv6: NodeStatus = status.ipv6;
void [ipv4, ipv6];
});
const status: ConnectivityStatus = runner.getStatus();
runner.run({ port: 0, network: 1, persistPath: '', identity: restored });
runner.bootstrap('127.0.0.1', runner.getBoundPort(4));
const key: string = hash(new Uint8Array([0, 255]));
const accepted: boolean = await runner.put(key, Buffer.from(version), { id: 42n, permanent: true });
const signed: boolean = await runner.putSigned(key, plaintext);
const encrypted: boolean = await runner.putEncrypted(key, plaintext, publicKey);
const values: Value[] = await runner.getAll(key);
const get: ValueIterator<Value> = runner.get(key);
for await (const value of get) {
const payload: Buffer = value.data;
void payload;
}
const listener = runner.listen(key, (value, expired) => {
const payload: Buffer = value.data;
const id: bigint = value.id;
void [payload, id, expired];
});
runner.ref().unref();
listener.cancel();
const iterator: ListenerIterator = runner.listen(key);
for await (const event of iterator) {
const typed: ListenEvent = event;
const value: Value = typed.value;
const expired: boolean = typed.expired;
void [value, expired];
break;
}
iterator.cancel();
await iterator.return();
await iterator[Symbol.asyncDispose]();
await runner[Symbol.asyncDispose]();
const proxy = new DhtRunner();
proxy.run({ proxyServer: 'https://proxy.example.com', proxyUserAgent: 'typed-client/1.0' });
await proxy.close();
void [accepted, values, valid, signed, encrypted, status];
}
void exercise;
+1 -1
View File
@@ -8,5 +8,5 @@ Version: @VERSION@
Libs: -L${libdir} -lopendht -lfmt
Libs.private: @http_lib@ -pthread
Requires: gnutls >= 3.3@jsoncpp_lib@@openssl_lib@
Requires.private: nettle >= 2.4@argon2_lib@@llhttp_lib@@iouring_lib@@simdutf_lib@
Requires.private: nettle >= 2.4, hogweed >= 3.1@argon2_lib@@llhttp_lib@@iouring_lib@@simdutf_lib@
Cflags: -I${includedir}@opendht_public_cflags@
@@ -0,0 +1,118 @@
diff --git a/SMP/libgnutls.vcxproj.filters b/SMP/libgnutls.vcxproj.filters
index b182189..91ee160 100644
--- a/SMP/libgnutls.vcxproj.filters
+++ b/SMP/libgnutls.vcxproj.filters
@@ -103,9 +103,6 @@
<Filter Include="Header Files\lib\inih">
<UniqueIdentifier>{ae0c3eeb-53df-4c72-a85a-6b46de35e7ba}</UniqueIdentifier>
</Filter>
- <Filter Include="Source Files\libtasn1">
- <UniqueIdentifier>{32be60b7-8c65-486e-9df5-7e529443cf07}</UniqueIdentifier>
- </Filter>
<Filter Include="Source Files\lib\nettle\gost">
<UniqueIdentifier>{1f3549a8-3c3d-475f-8fd6-20451d336464}</UniqueIdentifier>
</Filter>
@@ -1553,30 +1550,6 @@
<ClCompile Include="..\lib\nettle\sysrng-bcrypt.c">
<Filter>Source Files\lib\nettle</Filter>
</ClCompile>
- <ClCompile Include="..\devel\libtasn1\lib\coding.c">
- <Filter>Source Files\libtasn1</Filter>
- </ClCompile>
- <ClCompile Include="..\devel\libtasn1\lib\decoding.c">
- <Filter>Source Files\libtasn1</Filter>
- </ClCompile>
- <ClCompile Include="..\devel\libtasn1\lib\element.c">
- <Filter>Source Files\libtasn1</Filter>
- </ClCompile>
- <ClCompile Include="..\devel\libtasn1\lib\errors.c">
- <Filter>Source Files\libtasn1</Filter>
- </ClCompile>
- <ClCompile Include="..\devel\libtasn1\lib\gstr.c">
- <Filter>Source Files\libtasn1</Filter>
- </ClCompile>
- <ClCompile Include="..\devel\libtasn1\lib\parser_aux.c">
- <Filter>Source Files\libtasn1</Filter>
- </ClCompile>
- <ClCompile Include="..\devel\libtasn1\lib\structure.c">
- <Filter>Source Files\libtasn1</Filter>
- </ClCompile>
- <ClCompile Include="..\devel\libtasn1\lib\version.c">
- <Filter>Source Files\libtasn1</Filter>
- </ClCompile>
<ClCompile Include="..\lib\nettle\gost\acpkm.c">
<Filter>Source Files\lib\nettle\gost</Filter>
</ClCompile>
diff --git a/SMP/libgnutls_files.props b/SMP/libgnutls_files.props
index adcb074..2317c2a 100644
--- a/SMP/libgnutls_files.props
+++ b/SMP/libgnutls_files.props
@@ -179,13 +179,6 @@
<ItemGroup>
<ClCompile Include="lib\gnutls_asn1_tab.c" />
<ClCompile Include="lib\pkix_asn1_tab.c" />
- <ClCompile Include="..\devel\libtasn1\lib\coding.c" />
- <ClCompile Include="..\devel\libtasn1\lib\decoding.c" />
- <ClCompile Include="..\devel\libtasn1\lib\element.c" />
- <ClCompile Include="..\devel\libtasn1\lib\gstr.c" />
- <ClCompile Include="..\devel\libtasn1\lib\parser_aux.c" />
- <ClCompile Include="..\devel\libtasn1\lib\structure.c" />
- <ClCompile Include="..\devel\libtasn1\lib\version.c" />
<ClCompile Include="..\gnulib\lib\c-strcasecmp.c" />
<ClCompile Include="..\gnulib\lib\c-strncasecmp.c" />
<ClCompile Include="..\gnulib\lib\cloexec.c" />
@@ -572,9 +565,6 @@
<ClCompile Include="..\gnulib\lib\readdir.c">
<ForcedIncludeFiles>dirent-private.h;%(ForcedIncludeFiles)</ForcedIncludeFiles>
</ClCompile>
- <ClCompile Include="..\devel\libtasn1\lib\errors.c">
- <ObjectFileName>$(IntDir)\tasn1_$(filename).obj</ObjectFileName>
- </ClCompile>
<ClCompile Include="..\lib\algorithms\ecc.c">
<ObjectFileName>$(IntDir)\alg_%(Filename).obj</ObjectFileName>
</ClCompile>
diff --git a/SMP/libgnutls_winrt.vcxproj.filters b/SMP/libgnutls_winrt.vcxproj.filters
index 014a408..ae5ecff 100644
--- a/SMP/libgnutls_winrt.vcxproj.filters
+++ b/SMP/libgnutls_winrt.vcxproj.filters
@@ -103,9 +103,6 @@
<Filter Include="Header Files\lib\inih">
<UniqueIdentifier>{ae0c3eeb-53df-4c72-a85a-6b46de35e7ba}</UniqueIdentifier>
</Filter>
- <Filter Include="Source Files\libtasn1">
- <UniqueIdentifier>{32be60b7-8c65-486e-9df5-7e529443cf07}</UniqueIdentifier>
- </Filter>
<Filter Include="Header Files\lib\nettle\gost">
<UniqueIdentifier>{4fada990-3138-4089-a6c7-ae722a0e7fe9}</UniqueIdentifier>
</Filter>
@@ -1553,30 +1550,6 @@
<ClCompile Include="..\lib\nettle\sysrng-bcrypt.c">
<Filter>Source Files\lib\nettle</Filter>
</ClCompile>
- <ClCompile Include="..\devel\libtasn1\lib\coding.c">
- <Filter>Source Files\libtasn1</Filter>
- </ClCompile>
- <ClCompile Include="..\devel\libtasn1\lib\decoding.c">
- <Filter>Source Files\libtasn1</Filter>
- </ClCompile>
- <ClCompile Include="..\devel\libtasn1\lib\element.c">
- <Filter>Source Files\libtasn1</Filter>
- </ClCompile>
- <ClCompile Include="..\devel\libtasn1\lib\errors.c">
- <Filter>Source Files\libtasn1</Filter>
- </ClCompile>
- <ClCompile Include="..\devel\libtasn1\lib\gstr.c">
- <Filter>Source Files\libtasn1</Filter>
- </ClCompile>
- <ClCompile Include="..\devel\libtasn1\lib\parser_aux.c">
- <Filter>Source Files\libtasn1</Filter>
- </ClCompile>
- <ClCompile Include="..\devel\libtasn1\lib\structure.c">
- <Filter>Source Files\libtasn1</Filter>
- </ClCompile>
- <ClCompile Include="..\devel\libtasn1\lib\version.c">
- <Filter>Source Files\libtasn1</Filter>
- </ClCompile>
<ClCompile Include="..\lib\accelerated\afalg.c">
<Filter>Source Files\lib\accelerated</Filter>
</ClCompile>
@@ -0,0 +1,12 @@
--- a/lib/gnutls.pc.in
+++ b/lib/gnutls.pc.in
@@ -18,7 +18,7 @@
Name: GnuTLS
Description: Transport Security Layer implementation for the GNU system
URL: https://www.gnutls.org/
Version: @VERSION@
-Libs: -L${libdir} -lgnutls
+Libs: -L${libdir} @GNUTLS_LIBS@
Libs.private: @LIBINTL@ @LIBSOCKET@ @INET_PTON_LIB@ @LIBPTHREAD@ @LIB_SELECT@ @TSS_LIBS@ @GMP_LIBS@ @LIBUNISTRING@ @LIBATOMIC_LIBS@ @GNUTLS_LIBS_PRIVATE@
@GNUTLS_REQUIRES_PRIVATE@
Cflags: -I${includedir}
+187
View File
@@ -0,0 +1,187 @@
set(GNULIB_REF "1cc0125a28cba4e8503d853bf99b854fe4dd454c")
vcpkg_from_github(
OUT_SOURCE_PATH SOURCE_PATH
REPO savoirfairelinux/gnutls
REF 300b0c721fc1809f3d191b7f9e289c0267eabd01
SHA512 8a74fbf0343e658f6321da0f28bcc05c580fb7b950f6edfb6a8fdaad6d814456ed2dbf94e8dd330c7740c836f01e04552e649d529ee29424c3641861c41344bf
HEAD_REF fix/ci-msvc
PATCHES
external-libtasn1.patch
pkgconfig.patch
)
file(REMOVE_RECURSE "${SOURCE_PATH}/devel/perlasm")
vcpkg_download_distfile(
GNULIB_SNAPSHOT
URLS "https://gitlab.com/libidn/gnulib-mirror/-/archive/${GNULIB_REF}/gnulib-mirror-${GNULIB_REF}.tar.gz"
FILENAME "gnulib-mirror-${GNULIB_REF}.tar.gz"
SHA512 340539bd7b8e30b1e0f81a15fb207960e9ae3c4ef515f39f1d1bdb43ed52ad94a29c6a6ce8bbca3484ba5392e1dae7447c2f6b58124209614e884345e13914f5
)
vcpkg_extract_source_archive(
GNULIB_SOURCE_PATH
ARCHIVE "${GNULIB_SNAPSHOT}"
SOURCE_BASE ${GNULIB_REF}
)
file(REMOVE_RECURSE "${SOURCE_PATH}/gnulib")
file(RENAME "${GNULIB_SOURCE_PATH}" "${SOURCE_PATH}/gnulib")
include("${CURRENT_HOST_INSTALLED_DIR}/share/yasm-tool-helper/yasm-tool-helper.cmake")
yasm_tool_helper(OUT_VAR YASM)
file(TO_NATIVE_PATH "${YASM}" YASM)
if(VCPKG_LIBRARY_LINKAGE STREQUAL "dynamic")
set(CONFIGURATION_RELEASE ReleaseDLL)
set(CONFIGURATION_DEBUG DebugDLL)
else()
set(CONFIGURATION_RELEASE Release)
set(CONFIGURATION_DEBUG Debug)
endif()
if(VCPKG_TARGET_IS_UWP)
string(APPEND CONFIGURATION_RELEASE WinRT)
string(APPEND CONFIGURATION_DEBUG WinRT)
endif()
set(_gnutlsproject "${SOURCE_PATH}/SMP/libgnutls.vcxproj")
file(READ "${_gnutlsproject}" _contents)
string(REPLACE [[<Import Project="$(VCTargetsPath)\BuildCustomizations\yasm.props" />]]
"<Import Project=\"${CURRENT_HOST_INSTALLED_DIR}/share/vs-yasm/yasm.props\" />"
_contents "${_contents}")
string(REPLACE [[<Import Project="$(VCTargetsPath)\BuildCustomizations\yasm.targets" />]]
"<Import Project=\"${CURRENT_HOST_INSTALLED_DIR}/share/vs-yasm/yasm.targets\" />"
_contents "${_contents}")
string(REGEX REPLACE "${VCPKG_ROOT_DIR}/installed/[^/]+/share" "${CURRENT_HOST_INSTALLED_DIR}/share" _contents "${_contents}") # Above already
file(WRITE "${_gnutlsproject}" "${_contents}")
if(VCPKG_CRT_LINKAGE STREQUAL "static")
set(RuntimeLibraryExt "")
else()
set(RuntimeLibraryExt "DLL")
endif()
# patch output library file path and name
foreach(PROPS IN ITEMS
"${SOURCE_PATH}/SMP/smp_deps.props"
"${SOURCE_PATH}/SMP/smp_winrt_deps.props")
vcpkg_replace_string(
"${PROPS}"
[=[_winrt</TargetName>]=]
[=[</TargetName>]=]
IGNORE_UNCHANGED
)
vcpkg_replace_string(
"${PROPS}"
[=[<TargetName>lib$(RootNamespace)]=]
[=[<TargetName>$(RootNamespace)]=]
)
endforeach()
# patch hogweed, gmp, nettle, zlib libraries file names; inject RuntimeLibrary property to control CRT linkage
foreach(VCXPROJ IN ITEMS
"${SOURCE_PATH}/SMP/libgnutls.vcxproj"
"${SOURCE_PATH}/SMP/libgnutls_winrt.vcxproj")
vcpkg_replace_string(
"${VCXPROJ}"
"_winrt.lib"
".lib"
IGNORE_UNCHANGED
)
vcpkg_replace_string(
"${VCXPROJ}"
"libhogweed"
"hogweed"
)
vcpkg_replace_string(
"${VCXPROJ}"
"hogweedd"
"hogweed"
)
vcpkg_replace_string(
"${VCXPROJ}"
"libgmp"
"gmp"
)
vcpkg_replace_string(
"${VCXPROJ}"
"gmpd"
"gmp"
)
vcpkg_replace_string(
"${VCXPROJ}"
"libnettle"
"nettle"
)
vcpkg_replace_string(
"${VCXPROJ}"
"nettled"
"nettle"
)
set(zlib_basename "zlib")
if(EXISTS "${CURRENT_INSTALLED_DIR}/lib/zs.lib")
set(zlib_basename "zs")
elseif(EXISTS "${CURRENT_INSTALLED_DIR}/lib/z.lib")
set(zlib_basename "z")
endif()
vcpkg_replace_string(
"${VCXPROJ}"
"libzlib"
"${zlib_basename}"
)
vcpkg_replace_string(
"${VCXPROJ}"
"zlib"
"${zlib_basename}"
)
vcpkg_replace_string(
"${VCXPROJ}"
[=[</DisableSpecificWarnings>]=]
[=[</DisableSpecificWarnings><ForcedIncludeFiles>gnutls_msvc_fixes.h;%(ForcedIncludeFiles)</ForcedIncludeFiles><RuntimeLibrary>$(RuntimeLibrary)</RuntimeLibrary>]=]
)
endforeach()
vcpkg_install_msbuild(
USE_VCPKG_INTEGRATION
SOURCE_PATH "${SOURCE_PATH}"
PROJECT_SUBPATH SMP/libgnutls.sln
PLATFORM ${TRIPLET_SYSTEM_ARCH}
LICENSE_SUBPATH LICENSE
RELEASE_CONFIGURATION ${CONFIGURATION_RELEASE}
DEBUG_CONFIGURATION ${CONFIGURATION_DEBUG}
SKIP_CLEAN
OPTIONS /p:YasmPath="${YASM}" /p:OutDir=..\\msvc
OPTIONS_DEBUG /p:RuntimeLibrary=MultiThreadedDebug${RuntimeLibraryExt}
OPTIONS_RELEASE /p:RuntimeLibrary=MultiThreaded${RuntimeLibraryExt}
)
get_filename_component(SOURCE_PATH_SUFFIX "${SOURCE_PATH}" NAME)
if(VCPKG_TARGET_IS_UWP)
set(WINRT_SUBFOLDER libgnutls_winrt)
endif()
file(INSTALL "${CURRENT_BUILDTREES_DIR}/${TARGET_TRIPLET}-rel/${SOURCE_PATH_SUFFIX}/msvc/${WINRT_SUBFOLDER}/include" DESTINATION "${CURRENT_PACKAGES_DIR}")
set(GNUTLS_REQUIRES_PRIVATE "Requires.private: gmp, nettle, hogweed, libtasn1, zlib")
set(GNUTLS_LIBS_PRIVATE "-lcrypt32 -lws2_32 -lkernel32 -lncrypt")
set(prefix "${CURRENT_INSTALLED_DIR}")
set(exec_prefix "\${prefix}")
set(libdir "\${prefix}/lib")
set(includedir "\${prefix}/include")
set(GNUTLS_LIBS "-lgnutls")
configure_file("${SOURCE_PATH}/lib/gnutls.pc.in" "${CURRENT_PACKAGES_DIR}/lib/pkgconfig/gnutls.pc" @ONLY)
if(NOT VCPKG_BUILD_TYPE)
set(prefix "${CURRENT_INSTALLED_DIR}/debug")
set(exec_prefix "\${prefix}")
set(libdir "\${prefix}/lib")
set(includedir "\${prefix}/../include")
set(GNUTLS_LIBS "-lgnutlsd")
configure_file("${SOURCE_PATH}/lib/gnutls.pc.in" "${CURRENT_PACKAGES_DIR}/debug/lib/pkgconfig/gnutls.pc" @ONLY)
endif()
vcpkg_fixup_pkgconfig()
vcpkg_copy_pdbs()
file(COPY "${CURRENT_PORT_DIR}/vcpkg-cmake-wrapper.cmake" DESTINATION "${CURRENT_PACKAGES_DIR}/share/gnutls")
@@ -0,0 +1,2 @@
find_library(GNUTLS_LIBRARY NAMES gnutls gnutlsd NAMES_PER_DIR)
_find_package(${ARGS})
+23
View File
@@ -0,0 +1,23 @@
{
"name": "shiftmedia-libgnutls",
"version": "3.8.13",
"port-version": 0,
"description": "GnuTLS fork maintained by Savoir-faire Linux with native Visual Studio project build tools.",
"homepage": "https://github.com/savoirfairelinux/gnutls",
"license": "LGPL-2.1-only",
"supports": "windows & !arm & !mingw & !xbox",
"dependencies": [
"gmp",
"libtasn1",
"nettle",
{
"name": "vs-yasm",
"host": true
},
{
"name": "yasm-tool-helper",
"host": true
},
"zlib"
]
}
+171
View File
@@ -78,6 +78,177 @@ async def dht_async_demo(key_str: str):
asyncio.run(dht_async_demo("unique_key"))
```
## Node State Persistence
Set the state file path before starting a node:
```python
import threading
import opendht as dht
config = dht.DhtConfig()
config.setPersistPath("dht-state.msgpack")
node = dht.DhtRunner()
node.run(config=config)
stopped = threading.Event()
node.shutdown(stopped.set)
stopped.wait()
node.join()
```
The node loads its saved DHT node ID, known peers, and stored values on startup,
and saves state during `shutdown()`. Wait for the shutdown callback before
calling `join()` or reusing the state path; `join()` alone does not save state.
Port hints are
stored in a sibling file named `<path>_port.txt`; saved ports are reused when
available. The parent directory must already exist and be writable. Use a
different state path for each node.
Persistence is disabled by default. `config.setPersistPath("")` disables it
explicitly. This state does not include private keys or certificates; persist
cryptographic identities separately as described below.
## Identity Persistence
Keys can be imported from PEM or DER bytes. Private keys export PKCS#8 PEM;
public keys export DER. Certificates export their complete PEM chain.
```python
import opendht as dht
identity = dht.Identity.generate("my-node")
private_key_pem = identity.key.serialize(password)
certificate_pem = bytes(identity.certificate)
public_key_der = bytes(identity.publickey)
restored = dht.Identity(
dht.PrivateKey(private_key_pem, password),
dht.Certificate(certificate_pem),
)
public_key = dht.PublicKey(public_key_der)
node = dht.DhtRunner()
node.run(id=restored)
```
Here `password` is an application-supplied string. Save the exported bytes using
your application's storage, protect private key files, and keep the password
separate from the key. `bytes(private_key)` and `private_key.serialize()` without
a password export **unencrypted** private keys. Empty or malformed key data,
incorrect passwords, and passwords containing NUL characters raise exceptions.
Exporting an uninitialized key raises `ValueError`.
## Value Metadata
The following properties are readable and writable on `Value`, including values
returned by `get()` and `listen()`:
| Property | Type | Meaning |
|----------|------|---------|
| `type` | unsigned 16-bit integer | Native value type identifier |
| `seq` | unsigned 16-bit integer | Sequence number for signed value updates |
| `priority` | unsigned integer | Push priority: `0` high, `1` normal |
| `pushType` | string | Push notification type, such as `audioCall` |
```python
value = dht.Value(b"some binary data")
value.type = 42
value.seq = 1
value.priority = 1
value.pushType = "audioCall"
```
Set metadata before publishing the value. Numeric values outside the native
integer range raise `OverflowError`.
`value.isSigned()` reports whether a public key and signature are present;
it does not verify the signature. `value.isEncrypted()` reports whether the
value currently contains ciphertext. Values delivered by `get()` and `listen()`
are transparently decrypted when possible, so decrypted values return `False`
from `isEncrypted()`, even if they were published using `putEncrypted()`.
An uninitialized `PublicKey()` cannot be used as an encryption recipient and
raises `ValueError`.
Register custom `ValueType` instances after `run()` and before `join()`.
`registerType()` raises `RuntimeError` when the runner is stopped, `TypeError`
for `None`, and `ValueError` for an uninitialized type.
## Observability
`getNodeInfo()` returns a read-only `NodeInfo` snapshot with node identifiers,
IPv4/IPv6 routing statistics, bound ports, pending-operation count, and local
and remote storage counters. `getNodesStats(socket.AF_INET)` and
`getNodesStats(socket.AF_INET6)` return `NodeStats` snapshots. Snapshots remain
valid after the runner stops. `getPublicAddress()` returns known public
`SockAddr` objects; pass an address family to restrict the results, or use
`getPublicAddressStr()` for strings. These lists may be empty before discovery.
`getNodeMessageStats(incoming=True)` selects incoming message counters.
```python
info = node.getNodeInfo()
print(info.node_id, info.bound4, info.ipv4.good_nodes, info.storage_size)
print(node.getPublicAddressStr())
node.setOnStatusChanged(lambda ipv4, ipv6: print(ipv4.name, ipv6.name))
node.setOnStatusChanged(None)
```
The status callback receives two `NodeStatus` values: `Disconnected`,
`Connecting`, or `Connected`. Registration replaces the previous callback and
persists across restarts; `None` disables it. Core callbacks run on the DHT
thread while native locks are held: do not call snapshot getters, type
registration, logging controls, `run()`, blocking operations, or `join()` from
them. The asyncio wrapper schedules status callbacks on its event-loop thread,
where snapshot getters are safe to use.
Call `node.connectivityChanged()` after a network-interface or IP-address
change. It queues native connectivity recovery, including recontacting peers
and renewing subscriptions. It is also available on the asyncio runner as a
synchronous, nonblocking method.
## Queries And Filtered Listeners
`query()` retrieves metadata rather than complete value payloads. Each row is
a dictionary keyed by `ValueField`. Network projections support `Id`,
`ValueType`, `OwnerPk`, and `UserType`; values are integers, an `InfoHash` owner
fingerprint, or a UTF-8 string respectively. Explicit sequence-number
projections are not supported by the native wire format.
```python
selection = dht.Select().field(dht.ValueField.Id).field(dht.ValueField.UserType)
query = dht.Query()
query.buildFrom(selection, dht.Where().userType("record"))
rows = node.query(key, query=query)
query = dht.Query("SELECT id,user_type WHERE user_type=record")
node.query(key, query_cb=lambda row: print(row) or True,
done_cb=lambda ok, nodes: print(ok), query=query)
```
Without an explicit query, all four supported metadata fields are selected.
Supplying `query_cb` makes the core call nonblocking; returning `False` stops
delivery. The blocking core API returns the collected rows, as `get()` does.
The asyncio API offers `query()` as a context-managed async iterator and
`await queryAll()` as a collector; these raise `RuntimeError` on failure.
```python
rows = await node.queryAll(key, query=query)
with node.query(key, query=query) as results:
async for row in results:
print(row[dht.ValueField.Id])
```
Both core and asyncio `listen()` accept optional `where` and `filter` arguments.
`Where` is sent to the DHT to narrow the subscription; the Python predicate
runs locally. Existing two-argument core listeners are unchanged.
```python
token = node.listen(key, lambda value, expired: True,
where=dht.Where().userType("record"),
filter=lambda value: len(value.data) < 1024)
node.cancelListen(token)
```
## Documentation
For more information, examples and documentation, or to repport issues, visit: https://opendht.net
+453 -79
View File
@@ -12,15 +12,21 @@ from libcpp.map cimport map as map
from libcpp cimport bool
from libcpp.utility cimport pair
from libcpp.string cimport string
from libcpp.memory cimport shared_ptr
from libcpp.memory cimport shared_ptr, unique_ptr
from cython.operator cimport dereference as deref, preincrement as inc, predecrement as dec
from cpython cimport ref
from datetime import timedelta
from enum import IntEnum as _IntEnum
from operator import index as _index
cimport opendht_cpp as cpp
ctypedef cpp.Pht* _PhtPtr
import threading
import socket as _socket
import weakref as _weakref
cdef inline void _initialize_gnutls():
cdef int err = cpp.gnutls_global_init()
@@ -39,6 +45,58 @@ _initialize_gnutls()
def version() -> str:
return cpp.version().decode()
class NodeStatus(_IntEnum):
Disconnected = 0
Connecting = 1
Connected = 2
class ValueField(_IntEnum):
Id = 1
ValueType = 2
OwnerPk = 3
SeqNum = 4
UserType = 5
class _StatusCallbackState:
__slots__ = ("callback", "__weakref__")
def __init__(self):
self.callback = None
cdef inline void status_callback(int ipv4, int ipv6, void* user_data) noexcept with gil:
state = (<object>user_data)()
if state is None:
return
callback = state.callback
if callback is not None:
callback(NodeStatus(ipv4), NodeStatus(ipv6))
cdef inline bool query_callback(shared_ptr[cpp.FieldValueIndex] value, void* user_data) noexcept with gil:
cdef cpp.Blob blob
row = {}
for entry in value.get().index:
field = ValueField(<int>entry.first)
if field == ValueField.OwnerPk:
owner = InfoHash()
owner._infohash = entry.second.getHash()
row[field] = owner
elif field == ValueField.UserType:
blob = entry.second.getBlob()
row[field] = (<char*>blob.data())[:blob.size()].decode()
else:
row[field] = entry.second.getInt()
return (<object>user_data)['query'](row)
cdef inline void query_done_callback(bool done, cpp.vector[shared_ptr[cpp.Node]]* nodes, void* user_data) noexcept with gil:
callback = (<object>user_data)['done']
if callback is not None:
entries = []
for node in deref(nodes):
entry = NodeEntry()
entry._v.first = node.get().getId()
entry._v.second = node
entries.append(entry)
callback(done, entries)
cdef inline void lookup_callback(cpp.vector[cpp.shared_ptr[cpp.IndexValue]]* values, cpp.Prefix* p, void *user_data) noexcept with gil:
cbs = <object>user_data
if 'lookup' in cbs and cbs['lookup']:
@@ -51,9 +109,11 @@ cdef inline void lookup_callback(cpp.vector[cpp.shared_ptr[cpp.IndexValue]]* val
cdef inline void shutdown_callback(void* user_data) noexcept with gil:
cbs = <object>user_data
if 'shutdown' in cbs and cbs['shutdown']:
cbs['shutdown']()
ref.Py_DECREF(cbs)
try:
if 'shutdown' in cbs and cbs['shutdown']:
cbs['shutdown']()
finally:
ref.Py_DECREF(cbs)
cdef inline bool get_callback(shared_ptr[cpp.Value] value, void *user_data) noexcept with gil:
cbs = <object>user_data
@@ -72,22 +132,26 @@ cdef inline bool value_callback(shared_ptr[cpp.Value] value, bool expired, void
return cb(pv, expired) if not f or f(pv) else True
cdef inline void done_callback(bool done, cpp.vector[shared_ptr[cpp.Node]]* nodes, void *user_data) noexcept with gil:
node_ids = []
for n in deref(nodes):
h = NodeEntry()
h._v.first = n.get().getId()
h._v.second = n
node_ids.append(h)
cbs = <object>user_data
if 'done' in cbs and cbs['done']:
cbs['done'](done, node_ids)
ref.Py_DECREF(cbs)
try:
node_ids = []
for n in deref(nodes):
h = NodeEntry()
h._v.first = n.get().getId()
h._v.second = n
node_ids.append(h)
if 'done' in cbs and cbs['done']:
cbs['done'](done, node_ids)
finally:
ref.Py_DECREF(cbs)
cdef inline void done_callback_simple(bool done, void *user_data) noexcept with gil:
cbs = <object>user_data
if 'done' in cbs and cbs['done']:
cbs['done'](done)
ref.Py_DECREF(cbs)
try:
if 'done' in cbs and cbs['done']:
cbs['done'](done)
finally:
ref.Py_DECREF(cbs)
cdef class _WithID(object):
def __repr__(self):
@@ -101,6 +165,12 @@ cdef class _WithPk(object):
def __str__(self):
return self.getLongId().toString().decode()
cdef inline unsigned _checked_bit_index(bit, unsigned bit_count) except *:
bit = _index(bit)
if bit < 0 or bit >= bit_count:
raise IndexError("bit index out of range")
return bit
cdef class InfoHash(_WithID):
cdef cpp.InfoHash _infohash
def __cinit__(self, bytes str=b''):
@@ -116,9 +186,9 @@ cdef class InfoHash(_WithID):
return self._infohash == other._infohash
return NotImplemented
def getBit(InfoHash self, bit):
return self._infohash.getBit(bit)
return self._infohash.getBit(_checked_bit_index(bit, 160))
def setBit(InfoHash self, bit, b):
self._infohash.setBit(bit, b)
self._infohash.setBit(_checked_bit_index(bit, 160), b)
def getId(InfoHash self):
return self
def toString(InfoHash self):
@@ -154,9 +224,9 @@ cdef class PkId(_WithPk):
return self._pkid == other._pkid
return NotImplemented
def getBit(PkId self, bit):
return self._pkid.getBit(bit)
return self._pkid.getBit(_checked_bit_index(bit, 256))
def setBit(PkId self, bit, b):
self._pkid.setBit(bit, b)
self._pkid.setBit(_checked_bit_index(bit, 256), b)
def getLongId(PkId self):
return self
def toString(PkId self):
@@ -212,14 +282,92 @@ cdef class SockAddr(object):
cdef class Node(_WithID):
cdef shared_ptr[cpp.Node] _node
def getId(self):
if not self._node:
raise ValueError("Node is not initialized")
h = InfoHash()
h._infohash = self._node.get().getId()
return h
def getAddr(self):
if not self._node:
raise ValueError("Node is not initialized")
return self._node.get().getAddrStr()
def isExpired(self):
if not self._node:
raise ValueError("Node is not initialized")
return self._node.get().isExpired()
cdef class NodeStats(object):
cdef cpp.NodeStats _stats
@property
def good_nodes(self):
return self._stats.good_nodes
@property
def dubious_nodes(self):
return self._stats.dubious_nodes
@property
def cached_nodes(self):
return self._stats.cached_nodes
@property
def incoming_nodes(self):
return self._stats.incoming_nodes
@property
def table_depth(self):
return self._stats.table_depth
@property
def searches(self):
return self._stats.searches
@property
def node_cache_size(self):
return self._stats.node_cache_size
def getKnownNodes(self):
return self._stats.getKnownNodes()
def getNetworkSizeEstimation(self):
return self._stats.getNetworkSizeEstimation()
cdef class NodeInfo(object):
cdef cpp.NodeInfo _info
@property
def id(self):
result = InfoHash()
result._infohash = self._info.id
return result
@property
def node_id(self):
result = InfoHash()
result._infohash = self._info.node_id
return result
@property
def ipv4(self):
result = NodeStats()
result._stats = self._info.ipv4
return result
@property
def ipv6(self):
result = NodeStats()
result._stats = self._info.ipv6
return result
@property
def ongoing_ops(self):
return self._info.ongoing_ops
@property
def storage_values(self):
return self._info.storage_values
@property
def storage_size(self):
return self._info.storage_size
@property
def local_storage_values(self):
return self._info.local_storage_values
@property
def local_storage_size(self):
return self._info.local_storage_size
@property
def bound4(self):
return self._info.bound4
@property
def bound6(self):
return self._info.bound6
cdef class NodeEntry(_WithID):
cdef cpp.pair[cpp.InfoHash, shared_ptr[cpp.Node]] _v
def getId(self):
@@ -253,8 +401,11 @@ cdef class Select(object):
return self._select.toString().decode()
def isSatisfiedBy(self, Select os):
return self._select.isSatisfiedBy(os._select)
def field(self, int field):
self._select.field(<cpp.Field> field)
def field(self, field):
cdef int field_id = ValueField(_index(field))
if field_id == ValueField.SeqNum:
raise ValueError("The native wire format does not support sequence-number projections")
self._select.field(<cpp.Field>field_id)
return self
cdef class Where(object):
@@ -290,6 +441,10 @@ cdef class Value(object):
self._value.reset(new cpp.Value(id, val, len(val)))
def __str__(self):
return self._value.get().toString().decode()
def isSigned(self) -> bool:
return self._value.get().isSigned()
def isEncrypted(self) -> bool:
return self._value.get().isEncrypted()
@property
def owner(self) -> PublicKey:
h = PublicKey()
@@ -324,25 +479,51 @@ cdef class Value(object):
def id(self, cpp.uint64_t value):
self._value.get().id = value
@property
def type(self) -> int:
return self._value.get().type
@type.setter
def type(self, cpp.uint16_t value):
self._value.get().type = value
@property
def seq(self) -> int:
return self._value.get().seq
@seq.setter
def seq(self, cpp.uint16_t value):
self._value.get().seq = value
@property
def priority(self) -> int:
return self._value.get().priority
@priority.setter
def priority(self, unsigned value):
self._value.get().priority = value
@property
def pushType(self) -> str:
return self._value.get().pushType.decode()
@pushType.setter
def pushType(self, str value):
self._value.get().pushType = value.encode()
@property
def size(self) -> int:
return self._value.get().size()
cdef class ValueType(object):
cdef cpp.ValueType * _value
cdef unique_ptr[cpp.ValueType] _value
def __init__(self, cpp.uint16_t id, str name, expiration: timedelta):
if not isinstance(expiration, timedelta):
raise TypeError("expiration argument must be of type timedelta")
cdef cpp.seconds duration = cpp.seconds(int(expiration.total_seconds()))
self._value = new cpp.ValueType(id, name.encode(), duration)
self._value.reset(new cpp.ValueType(id, name.encode(), duration))
cdef class NodeSetIter(object):
cdef map[cpp.InfoHash, shared_ptr[cpp.Node]]* _nodes
cdef NodeSet _owner
cdef map[cpp.InfoHash, shared_ptr[cpp.Node]].iterator _curIter
def __init__(self, NodeSet s):
self._nodes = &s._nodes
self._curIter = self._nodes.begin()
def __init__(self, NodeSet s not None):
self._owner = s
self._curIter = self._owner._nodes.begin()
def __iter__(self):
return self
def __next__(self):
if self._curIter == self._nodes.end():
if self._curIter == self._owner._nodes.end():
raise StopIteration
h = NodeEntry()
h._v = deref(self._curIter)
@@ -374,6 +555,8 @@ cdef class NodeSet(object):
s = ''
cdef map[cpp.InfoHash, shared_ptr[cpp.Node]].iterator it = self._nodes.begin()
while it != self._nodes.end():
if not deref(it).second:
raise ValueError("Node is not initialized")
s += deref(it).first.toString().decode() + ' ' + deref(it).second.get().getAddrStr().decode() + '\n'
inc(it)
return s
@@ -383,6 +566,30 @@ cdef class NodeSet(object):
cdef class PrivateKey(_WithPk):
cdef shared_ptr[cpp.PrivateKey] _key
def __init__(self, bytes data=None, str password=""):
cdef cpp.Blob encoded
if '\0' in password:
raise ValueError("Password must not contain NUL characters")
if data is not None:
if not data:
raise ValueError("Private key data must not be empty")
encoded = data
self._key = cpp.make_shared[cpp.PrivateKey](encoded, <cpp.string>password.encode())
def serialize(self, str password="") -> bytes:
"""Export PKCS#8 PEM, encrypted when a nonempty password is supplied."""
if not self._key:
raise ValueError("PrivateKey is not initialized")
if '\0' in password:
raise ValueError("Password must not contain NUL characters")
cdef cpp.Blob encoded = self._key.get().serialize(password.encode())
if encoded.empty():
raise RuntimeError("Could not export private key")
return (<char*>encoded.data())[:encoded.size()]
def __bytes__(self):
return self.serialize()
def getId(self) -> InfoHash:
if not self._key:
raise ValueError("PrivateKey is not initialized")
@@ -405,6 +612,8 @@ cdef class PrivateKey(_WithPk):
return pk
def decrypt(self, bytes dat) -> bytes:
if not self._key:
raise ValueError("PrivateKey is not initialized")
cdef size_t d_len = len(dat)
cdef cpp.uint8_t* d_ptr = <cpp.uint8_t*>dat
cdef cpp.Blob indat
@@ -439,6 +648,25 @@ cdef class PrivateKey(_WithPk):
cdef class PublicKey(_WithPk):
cdef cpp.shared_ptr[cpp.PublicKey] _key
def __init__(self, bytes data=None):
cdef cpp.Blob encoded
if data is not None:
if not data:
raise ValueError("Public key data must not be empty")
encoded = data
self._key = cpp.make_shared[cpp.PublicKey](encoded)
def serialize(self) -> bytes:
"""Export the public key as DER bytes."""
if not self._key:
raise ValueError("PublicKey is not initialized")
cdef cpp.Blob encoded
self._key.get().pack(encoded)
return (<char*>encoded.data())[:encoded.size()]
def __bytes__(self):
return self.serialize()
def getId(self) -> InfoHash:
if not self._key:
raise ValueError("PublicKey is not initialized")
@@ -500,11 +728,17 @@ cdef class Certificate(_WithPk):
return ""
return self._cert.get().toString().decode()
def getName(self) -> str:
if not self._cert:
raise ValueError("Certificate is not initialized")
return self._cert.get().getName().decode()
def revoke(self, PrivateKey k, Certificate c):
if not self._cert or c is None or not c._cert:
raise ValueError("Certificate is not initialized")
if k is None or not k._key:
raise ValueError("PrivateKey is not initialized")
self._cert.get().revoke(deref(k._key.get()), deref(c._cert.get()));
def __bytes__(self):
return self._cert.get().toString().encode() if self._cert else b''
return self._cert.get().toString() if self._cert else b''
@property
def issuer(self) -> Certificate|None:
if self._cert and self._cert.get().issuer:
@@ -514,6 +748,10 @@ cdef class Certificate(_WithPk):
return None
@staticmethod
def generate(PrivateKey k, str name, Identity i = Identity(), bool is_ca = False):
if k is None or not k._key:
raise ValueError("PrivateKey is not initialized")
if i is None:
i = Identity()
c = Certificate()
c._cert = cpp.make_shared[cpp.Certificate](cpp.Certificate.generate(deref(k._key.get()), name.encode(), i._id, is_ca))
return c
@@ -528,10 +766,16 @@ cdef class VerifyResult(object):
cdef class TrustList(object):
cdef cpp.TrustList _trust
def add(self, Certificate cert):
if cert is None or not cert._cert:
raise ValueError("Certificate is not initialized")
self._trust.add(deref(cert._cert.get()))
def remove(self, Certificate cert):
if cert is None or not cert._cert:
raise ValueError("Certificate is not initialized")
self._trust.remove(deref(cert._cert.get()))
def verify(self, Certificate cert):
if cert is None or not cert._cert:
raise ValueError("Certificate is not initialized")
r = VerifyResult()
r._result = self._trust.verify(deref(cert._cert.get()))
return r
@@ -539,7 +783,7 @@ cdef class TrustList(object):
cdef class ListenToken(object):
cdef cpp.InfoHash _h
cdef cpp.shared_future[size_t] _t
_cb = dict()
cdef bool _cancelled
cdef class Identity(object):
cdef cpp.Identity _id
@@ -550,6 +794,8 @@ cdef class Identity(object):
self._id.second = c._cert
@staticmethod
def generate(str name = "pydht", Identity ca = Identity(), unsigned bits = 4096):
if ca is None:
ca = Identity()
i = Identity()
i._id = cpp.generateIdentity(name.encode(), ca._id, bits)
return i
@@ -610,9 +856,19 @@ cdef class DhtConfig(object):
self._config.dht_config.node_config.network = netid
def setMaintainStorage(self, bool maintain_storage):
self._config.dht_config.node_config.maintain_storage = maintain_storage
def setPersistPath(self, str path):
"""Set the native state file path; an empty string disables persistence."""
if path is None:
raise TypeError("path must be a string")
if '\0' in path:
raise ValueError("path must not contain NUL characters")
self._config.dht_config.node_config.persist_path = path.encode()
def setRateLimit(self, ssize_t max_req_per_sec, ssize_t max_peer_req_per_sec):
self._config.dht_config.node_config.max_req_per_sec = max_req_per_sec
self._config.dht_config.node_config.max_peer_req_per_sec = max_peer_req_per_sec
def setCryptoRateLimit(self, ssize_t max_crypto_per_sec, ssize_t max_peer_crypto_per_sec):
self._config.dht_config.node_config.max_crypto_per_sec = max_crypto_per_sec
self._config.dht_config.node_config.max_peer_crypto_per_sec = max_peer_crypto_per_sec
def setProxyInfo(self, str proxy_server, str push_node_id="", str push_token="", str push_topic="", str push_platform="", str user_agent="", Certificate server_ca = Certificate()):
self._config.proxy_server = proxy_server.encode()
self._config.push_node_id = push_node_id.encode()
@@ -630,24 +886,65 @@ cdef class DhtConfig(object):
cdef class DhtRunner(_WithID):
cdef cpp.shared_ptr[cpp.DhtRunner] thisptr
cdef map[_PhtPtr, shared_ptr[cpp.Pht]] _indexes
cdef object _status_callback
def __cinit__(self):
self.thisptr.reset(new cpp.DhtRunner())
self._status_callback = _StatusCallbackState()
def __dealloc__(self):
with nogil:
if self.thisptr:
self.thisptr.get().join()
self._indexes.clear()
self.thisptr.reset()
def getId(self) -> InfoHash:
h = InfoHash()
if self.thisptr:
h._infohash = self.thisptr.get().getId()
with nogil:
h._infohash = self.thisptr.get().getId()
return h
def getPublicKey(self) -> PublicKey:
if not self.thisptr:
raise ValueError("DhtRunner is not initialized")
pk = PublicKey()
pk._key = self.thisptr.get().getPublicKey()
with nogil:
pk._key = self.thisptr.get().getPublicKey()
return pk
def getNodeId(self) -> InfoHash:
h = InfoHash()
if self.thisptr:
h._infohash = self.thisptr.get().getNodeId()
with nogil:
h._infohash = self.thisptr.get().getNodeId()
return h
def getNodeInfo(self) -> NodeInfo:
result = NodeInfo()
with nogil:
result._info = self.thisptr.get().getNodeInfo()
return result
def getNodesStats(self, af=_socket.AF_INET) -> NodeStats:
if af not in (_socket.AF_INET, _socket.AF_INET6):
raise ValueError("af must be AF_INET or AF_INET6")
info = self.getNodeInfo()
return info.ipv4 if af == _socket.AF_INET else info.ipv6
def getPublicAddress(self, cpp.sa_family_t af=0):
if af not in (0, _socket.AF_INET, _socket.AF_INET6):
raise ValueError("af must be AF_UNSPEC, AF_INET or AF_INET6")
cdef cpp.vector[cpp.SockAddr] addresses
with nogil:
addresses = self.thisptr.get().getPublicAddress(af)
result = []
for address in addresses:
value = SockAddr()
value._addr = address
result.append(value)
return result
def getPublicAddressStr(self, cpp.sa_family_t af=0):
return [str(address) for address in self.getPublicAddress(af)]
def setOnStatusChanged(self, callback=None):
"""Replace the DHT-thread status callback, or disable it with None."""
if callback is not None and not callable(callback):
raise TypeError("callback must be callable or None")
self._status_callback.callback = callback
def ping(self, SockAddr addr, done_cb=None):
if done_cb:
cb_obj = {'done':done_cb}
@@ -673,44 +970,80 @@ cdef class DhtRunner(_WithID):
host_bytes = host.encode()
port_bytes = port.encode() if port else b'4222'
self.thisptr.get().bootstrap(<cpp.const_char*>host_bytes, <cpp.const_char*>port_bytes)
def run(self, Identity id=None, is_bootstrap=False, cpp.in_port_t port=0, str ipv4="", str ipv6="", DhtConfig config=DhtConfig()):
if id:
config.setIdentity(id)
def connectivityChanged(self):
with nogil:
self.thisptr.get().connectivityChanged()
def run(self, Identity id=None, is_bootstrap=False, cpp.in_port_t port=0, str ipv4="", str ipv6="", DhtConfig config=None):
cdef cpp.DhtRunnerConfig run_config = config._config if config is not None else cpp.DhtRunnerConfig()
cdef cpp.string bind4, bind6, service
cdef object status_ref = _weakref.ref(self._status_callback)
if id is not None:
run_config.dht_config.id = id._id
if is_bootstrap:
run_config.dht_config.node_config.is_bootstrap = True
if ipv4 or ipv6:
bind4 = ipv4.encode() if ipv4 else b''
bind6 = ipv6.encode() if ipv6 else b''
self.thisptr.get().run(bind4, bind6, str(port).encode(), config._config)
service = str(port).encode()
with nogil:
cpp.runPythonRunnerAddress(deref(self.thisptr.get()), bind4.c_str(),
bind6.c_str(), service.c_str(), run_config,
status_callback, status_ref)
else:
self.thisptr.get().run(port, config._config)
with nogil:
cpp.runPythonRunner(deref(self.thisptr.get()), port, run_config,
status_callback, status_ref)
def join(self):
self.thisptr.get().join()
with nogil:
self.thisptr.get().join()
self._indexes.clear()
def shutdown(self, shutdown_cb=None):
cb_obj = {'shutdown':shutdown_cb}
ref.Py_INCREF(cb_obj)
self.thisptr.get().shutdown(cpp.bindShutdownCb(shutdown_callback, <void*>cb_obj))
def registerType(self, ValueType type):
self.thisptr.get().registerType(deref(type._value))
def registerType(self, ValueType type not None):
if not type._value:
raise ValueError("ValueType is not initialized")
cdef cpp.ValueType run_type = deref(type._value.get())
with nogil:
self.thisptr.get().registerType(run_type)
def enableLogging(self):
cpp.enableLogging(self.thisptr.get()[0])
with nogil:
cpp.enableLogging(self.thisptr.get()[0])
def disableLogging(self):
cpp.disableLogging(self.thisptr.get()[0])
with nogil:
cpp.disableLogging(self.thisptr.get()[0])
def enableFileLogging(self, str path):
cpp.enableFileLogging(self.thisptr.get()[0], path.encode())
cdef cpp.string log_path = path.encode()
with nogil:
cpp.enableFileLogging(self.thisptr.get()[0], log_path)
def isRunning(self):
return self.thisptr.get().isRunning()
def getBound(self, cpp.sa_family_t af = 0):
s = SockAddr()
s._addr = self.thisptr.get().getBound(af)
with nogil:
s._addr = self.thisptr.get().getBound(af)
return s
def getStorageLog(self):
return self.thisptr.get().getStorageLog().decode()
cdef cpp.string result
with nogil:
result = self.thisptr.get().getStorageLog()
return result.decode()
def getRoutingTablesLog(self, cpp.sa_family_t af):
return self.thisptr.get().getRoutingTablesLog(af).decode()
cdef cpp.string result
with nogil:
result = self.thisptr.get().getRoutingTablesLog(af)
return result.decode()
def getSearchesLog(self, cpp.sa_family_t af):
return self.thisptr.get().getSearchesLog(af).decode()
def getNodeMessageStats(self):
cdef cpp.string result
with nogil:
result = self.thisptr.get().getSearchesLog(af)
return result.decode()
def getNodeMessageStats(self, bool incoming=False):
stats = []
cdef cpp.vector[unsigned] res = self.thisptr.get().getNodeMessageStats(False)
cdef cpp.vector[unsigned] res
with nogil:
res = self.thisptr.get().getNodeMessageStats(incoming)
for n in res:
stats.append(n)
return stats
@@ -753,6 +1086,25 @@ cdef class DhtRunner(_WithID):
lock.wait()
return res
def query(self, InfoHash key not None, query_cb=None, done_cb=None, Query query=None):
"""Return projected metadata rows, or stream them through query_cb."""
if query is None:
query = Query("SELECT id,value_type,owner_pk,user_type")
if query_cb is None:
completed = threading.Event()
rows = []
def on_row(row):
rows.append(row)
return True
self.query(key, on_row, lambda ok, nodes: completed.set(), query)
completed.wait()
return rows
if not callable(query_cb) or (done_cb is not None and not callable(done_cb)):
raise TypeError("query callbacks must be callable")
callbacks = {'query': query_cb, 'done': done_cb}
cpp.queryPython(deref(self.thisptr.get()), key._infohash, query_callback,
query_done_callback, callbacks, query._query)
def put(self, InfoHash key, Value val, done_cb=None, permanent=False):
"""Publish a new value on the DHT at key.
@@ -776,7 +1128,7 @@ cdef class DhtRunner(_WithID):
lock.notify()
with lock:
pending += 1
self.put(key, val, done_cb=tmp_done)
self.put(key, val, done_cb=tmp_done, permanent=permanent)
while pending > 0:
lock.wait()
return ok
@@ -798,7 +1150,7 @@ cdef class DhtRunner(_WithID):
lock.notify()
with lock:
pending += 1
self.putSigned(key, val, done_cb=tmp_done)
self.putSigned(key, val, done_cb=tmp_done, permanent=permanent)
while pending > 0:
lock.wait()
return ok
@@ -806,6 +1158,8 @@ cdef class DhtRunner(_WithID):
def putEncrypted(self, InfoHash key, Value val, to, done_cb=None, bool permanent=False):
if to is None:
raise ValueError("'to' must be provided")
if isinstance(to, PublicKey) and not (<PublicKey>to)._key:
raise ValueError("PublicKey is not initialized")
if done_cb:
cb_obj = {'done':done_cb}
ref.Py_INCREF(cb_obj)
@@ -845,19 +1199,23 @@ cdef class DhtRunner(_WithID):
def cancelPut(self, InfoHash key, Value val):
self.thisptr.get().cancelPut(key._infohash, val._value)
def listen(self, InfoHash key, value_cb):
def listen(self, InfoHash key not None, value_cb, filter=None, Where where=None):
if not callable(value_cb) or (filter is not None and not callable(filter)):
raise TypeError("listener callbacks must be callable")
if where is None:
where = Where()
t = ListenToken()
t._h = key._infohash
cb_obj = {'valcb':value_cb}
t._cb['cb'] = cb_obj
# avoid the callback being destructed if the token is destroyed
ref.Py_INCREF(cb_obj)
t._t = self.thisptr.get().listen(t._h, cpp.bindValueCb(value_callback, <void*>cb_obj)).share()
cb_obj = {'valcb':value_cb, 'filter':filter}
t._t = self.thisptr.get().listen(t._h, cpp.bindPythonValueCb(value_callback, cb_obj),
cpp.nullptr, where._where).share()
return t
def cancelListen(self, ListenToken token):
self.thisptr.get().cancelListen(token._h, token._t)
ref.Py_DECREF(<object>token._cb['cb'])
# fixme: not thread safe
if token is None:
raise TypeError("token must be a ListenToken")
if not token._cancelled and token._t.valid():
self.thisptr.get().cancelListen(token._h, token._t)
token._cancelled = True
cdef class IndexValue(object):
cdef cpp.shared_ptr[cpp.IndexValue] _value
@@ -874,12 +1232,18 @@ cdef class IndexValue(object):
return self._value.get().second
cdef class Pht(object):
cdef cpp.Pht* thisptr
def __cinit__(self, bytes name, key_spec, DhtRunner dht):
"""PHT index; active native indexes live until the runner is joined."""
cdef shared_ptr[cpp.Pht] thisptr
cdef DhtRunner _runner
def __cinit__(self, bytes name, key_spec, DhtRunner dht not None):
cdef cpp.IndexKeySpec cpp_key_spec
for kk, size in key_spec.items():
cpp_key_spec[bytes(kk, 'utf-8')] = size
self.thisptr = new cpp.Pht(name, cpp_key_spec, dht.thisptr)
self._runner = dht
self.thisptr.reset(new cpp.Pht(name, cpp_key_spec, dht.thisptr))
def __dealloc__(self):
with nogil:
self.thisptr.reset()
@property
def MAX_NODE_ENTRY_COUNT(self):
return cpp.PHT_MAX_NODE_ENTRY_COUNT
@@ -892,16 +1256,21 @@ cdef class Pht(object):
representation of the prefix where the value was indexed in
the PHT.
"""
cb_obj = {'lookup':lookup_cb, 'done':done_cb} # TODO: donecallback is to be removed
ref.Py_INCREF(cb_obj)
cdef cpp.IndexKey cppk
for kk, v in key.items():
cppk[bytes(kk, 'utf-8')] = bytes(v)
self.thisptr.lookup(
cppk,
cpp.Pht.bindLookupCb(lookup_callback, <void*>cb_obj),
cpp.bindDoneCbSimple(done_callback_simple, <void*>cb_obj)
)
self._runner._indexes[self.thisptr.get()] = self.thisptr
cb_obj = {'lookup':lookup_cb, 'done':done_cb} # TODO: donecallback is to be removed
ref.Py_INCREF(cb_obj)
try:
self.thisptr.get().lookup(
cppk,
cpp.Pht.bindLookupCb(lookup_callback, <void*>cb_obj),
cpp.bindDoneCbSimple(done_callback_simple, <void*>cb_obj)
)
except:
ref.Py_DECREF(cb_obj)
raise
def insert(self, key, IndexValue value, done_cb=None):
"""Add an index entry to the Index.
@@ -909,16 +1278,21 @@ cdef class Pht(object):
value -- an IndexValue object describing the indexed value.
done_cb -- Called when the operation is completed.
"""
cb_obj = {'done':done_cb}
ref.Py_INCREF(cb_obj)
cdef cpp.IndexKey cppk
for kk, v in key.items():
cppk[bytes(kk, 'utf-8')] = bytes(v)
cdef cpp.IndexValue val
val.first = (<InfoHash>value.getKey())._infohash
val.second = value.getValueId()
self.thisptr.insert(
cppk,
val,
cpp.bindDoneCbSimple(done_callback_simple, <void*>cb_obj)
)
self._runner._indexes[self.thisptr.get()] = self.thisptr
cb_obj = {'done':done_cb}
ref.Py_INCREF(cb_obj)
try:
self.thisptr.get().insert(
cppk,
val,
cpp.bindDoneCbSimple(done_callback_simple, <void*>cb_obj)
)
except:
ref.Py_DECREF(cb_obj)
raise
+83 -24
View File
@@ -3,12 +3,20 @@
from __future__ import annotations
import asyncio
from typing import Any, Callable, Iterable, List, Optional, Tuple
from typing import Any, Callable, Generic, Iterable, List, Optional, Tuple, TypeVar
from . import _core as _dht
# Re-export core types for convenience
from ._core import InfoHash, Value, ValueType, Where, SockAddr, PublicKey, PkId, DhtConfig, Identity
from ._core import NodeInfo, NodeStats, NodeStatus, Query, Select, ValueField
_Result = TypeVar("_Result")
def _set_result_if_pending(future: asyncio.Future[Any], result: Any) -> None:
if not future.done():
future.set_result(result)
class _Listener:
@@ -19,7 +27,11 @@ class _Listener:
__slots__ = ("_runner", "_token", "_loop", "_queue", "_closed")
def __init__(self, runner: DhtRunner, key: InfoHash) -> None:
def __init__(
self, runner: DhtRunner, key: InfoHash,
where: Optional[Where] = None,
filter: Optional[Callable[[Value], bool]] = None,
) -> None:
self._runner = runner
self._loop = runner._loop
self._queue: asyncio.Queue[Tuple[Value, bool]] = asyncio.Queue()
@@ -31,7 +43,7 @@ class _Listener:
self._loop.call_soon_threadsafe(self._queue.put_nowait, (v, expired))
return True # continue listening
self._token = self._runner._dht.listen(key, _on_value)
self._token = self._runner._dht.listen(key, _on_value, filter, where)
def __enter__(self) -> _Listener:
return self
@@ -62,10 +74,10 @@ class _Listener:
self._queue.put_nowait(None)
class _GetStream:
"""Async iterator around DHT get callback.
class _GetStream(Generic[_Result]):
"""Async iterator around a DHT get or query callback.
Yields Value instances as they are found, then ends.
Yields results as they are found, then ends.
Raises on failure.
"""
__slots__ = ("_runner", "_loop", "_queue", "_closed", "_error")
@@ -76,14 +88,15 @@ class _GetStream:
key: InfoHash,
where: Optional[Where],
filter: Optional[Callable[[Value], bool]],
query: Optional[Query] = None,
) -> None:
self._runner = runner
self._loop = runner._loop
self._queue: asyncio.Queue[Optional[Value]] = asyncio.Queue()
self._queue: asyncio.Queue[Optional[_Result]] = asyncio.Queue()
self._closed = False
self._error: Optional[BaseException] = None
def _on_value(v: Value) -> bool:
def _on_value(v: _Result) -> bool:
# Called on DHT thread
if self._closed:
return False
@@ -95,21 +108,24 @@ class _GetStream:
if self._closed:
return
if not ok:
self._error = RuntimeError("DHT get failed")
self._error = RuntimeError("DHT query failed" if query is not None else "DHT get failed")
self._loop.call_soon_threadsafe(self._queue.put_nowait, None)
self._runner._dht.get(key, _on_value, _on_done, filter, where)
if query is None:
self._runner._dht.get(key, _on_value, _on_done, filter, where)
else:
self._runner._dht.query(key, _on_value, _on_done, query)
def __enter__(self) -> _GetStream:
def __enter__(self) -> _GetStream[_Result]:
return self
def __exit__(self, exc_type, exc, tb) -> None:
self.cancel()
def __aiter__(self) -> _GetStream:
def __aiter__(self) -> _GetStream[_Result]:
return self
async def __anext__(self) -> Value:
async def __anext__(self) -> _Result:
if self._closed:
raise StopAsyncIteration
item = await self._queue.get()
@@ -132,7 +148,7 @@ class DhtRunner:
"""Async wrapper around opendht.DhtRunner.
"""
__slots__ = ("_dht", "_loop", "_config", "_bootstrap")
__slots__ = ("_dht", "_loop", "_config", "_bootstrap", "_status_generation")
def __init__(
self,
@@ -149,6 +165,7 @@ class DhtRunner:
self._dht = core or _dht.DhtRunner()
self._config = config
self._bootstrap = list(bootstrap) if bootstrap is not None else None
self._status_generation = [0]
def run(
self,
@@ -168,9 +185,32 @@ class DhtRunner:
def bootstrap(self, host: str, port: Optional[str] = None) -> None:
self._dht.bootstrap(host, port)
def setOnStatusChanged(
self, callback: Optional[Callable[[NodeStatus, NodeStatus], None]] = None,
) -> None:
if callback is not None and not callable(callback):
raise TypeError("callback must be callable or None")
state = self._status_generation
state[0] += 1
if callback is None:
self._dht.setOnStatusChanged(None)
return
generation = state[0]
loop = self._loop
def _deliver(ipv4: NodeStatus, ipv6: NodeStatus) -> None:
if state[0] == generation:
callback(ipv4, ipv6)
def _on_status(ipv4: NodeStatus, ipv6: NodeStatus) -> None:
if not loop.is_closed():
loop.call_soon_threadsafe(_deliver, ipv4, ipv6)
self._dht.setOnStatusChanged(_on_status)
async def ping(self, addr: SockAddr) -> bool:
fut: asyncio.Future[bool] = self._loop.create_future()
self._dht.ping(addr, lambda ok: self._loop.call_soon_threadsafe(fut.set_result, bool(ok)))
self._dht.ping(addr, lambda ok: self._loop.call_soon_threadsafe(_set_result_if_pending, fut, bool(ok)))
return await fut
def get(
@@ -179,9 +219,25 @@ class DhtRunner:
*,
where: Optional[Where] = None,
filter: Optional[Callable[[Value], bool]] = None,
):
) -> _GetStream[Value]:
# Return a streaming iterator of values; it ends on completion and raises on failure.
return _GetStream(self, key, where, filter)
return _GetStream[Value](self, key, where, filter)
def query(
self, key: InfoHash, *, query: Optional[Query] = None,
) -> _GetStream[dict[ValueField, Any]]:
if query is None:
query = Query("SELECT id,value_type,owner_pk,user_type")
return _GetStream[dict[ValueField, Any]](self, key, None, None, query)
async def queryAll(
self, key: InfoHash, *, query: Optional[Query] = None,
) -> List[dict[ValueField, Any]]:
rows: List[dict[ValueField, Any]] = []
with self.query(key, query=query) as stream:
async for row in stream:
rows.append(row)
return rows
async def getAll(
self,
@@ -220,7 +276,7 @@ class DhtRunner:
fut: asyncio.Future[bool] = self._loop.create_future()
def _done(ok: bool, _nodes: Iterable[Any]) -> None:
self._loop.call_soon_threadsafe(fut.set_result, bool(ok))
self._loop.call_soon_threadsafe(_set_result_if_pending, fut, bool(ok))
self._dht.put(key, val, _done, permanent)
return await fut
@@ -229,7 +285,7 @@ class DhtRunner:
fut: asyncio.Future[bool] = self._loop.create_future()
def _done(ok: bool, _nodes: Iterable[Any]) -> None:
self._loop.call_soon_threadsafe(fut.set_result, bool(ok))
self._loop.call_soon_threadsafe(_set_result_if_pending, fut, bool(ok))
self._dht.putSigned(key, val, _done, permanent)
return await fut
@@ -245,20 +301,23 @@ class DhtRunner:
fut: asyncio.Future[bool] = self._loop.create_future()
def _done(ok: bool, _nodes: Iterable[Any]) -> None:
self._loop.call_soon_threadsafe(fut.set_result, bool(ok))
self._loop.call_soon_threadsafe(_set_result_if_pending, fut, bool(ok))
self._dht.putEncrypted(key, val, to, _done, permanent)
return await fut
def listen(self, key: InfoHash) -> _Listener:
return _Listener(self, key)
def listen(
self, key: InfoHash, *, where: Optional[Where] = None,
filter: Optional[Callable[[Value], bool]] = None,
) -> _Listener:
return _Listener(self, key, where, filter)
async def cancelListen(self, listener: _Listener) -> None:
await listener.cancel()
listener.cancel()
async def shutdown(self) -> None:
fut: asyncio.Future[None] = self._loop.create_future()
self._dht.shutdown(lambda: self._loop.call_soon_threadsafe(fut.set_result, None))
self._dht.shutdown(lambda: self._loop.call_soon_threadsafe(_set_result_if_pending, fut, None))
await fut
async def join(self) -> None:
+178 -47
View File
@@ -98,49 +98,53 @@ ctypedef vector[uint8_t] Blob
cdef extern from "opendht/crypto.h" namespace "dht::crypto":
ctypedef pair[shared_ptr[PrivateKey], shared_ptr[Certificate]] Identity
cdef Identity generateIdentity(string name, Identity ca, unsigned bits)
cdef Identity generateIdentity(string name, Identity ca, unsigned bits) except +
cdef Blob aesEncrypt(Blob data, string password) except +
cdef Blob aesDecrypt(Blob encrypted, string password) except +
cdef cppclass PrivateKey:
PrivateKey()
shared_ptr[PublicKey] getSharedPublicKey() const
Blob decrypt(Blob data) const
Blob sign(Blob data) const
PrivateKey() except +
PrivateKey(Blob data, string password) except +
shared_ptr[PublicKey] getSharedPublicKey() except +
Blob serialize(string password) except +
Blob decrypt(Blob data) except +
Blob sign(Blob data) except +
@staticmethod
PrivateKey generate()
PrivateKey generate() except +
@staticmethod
PrivateKey generateEC()
PrivateKey generateEC() except +
cdef cppclass PublicKey:
PublicKey()
InfoHash getId() const
PkId getLongId() const
Blob encrypt(Blob data) const
bool checkSignature(Blob data, Blob signature) const
PublicKey() except +
PublicKey(Blob data) except +
InfoHash getId() except +
PkId getLongId() except +
void pack(Blob& data) except +
Blob encrypt(Blob data) except +
bool checkSignature(Blob data, Blob signature) except +
bool operator==(PublicKey) const
cdef cppclass Certificate:
Certificate()
Certificate(string pem)
InfoHash getId() const
PkId getLongId() const
string toString() const
string getName() const
void revoke(PrivateKey key, Certificate cert)
Certificate() except +
Certificate(string pem) except +
InfoHash getId() except +
PkId getLongId() except +
string toString() except +
string getName() except +
void revoke(PrivateKey key, Certificate cert) except +
@staticmethod
Certificate generate(PrivateKey key, string name, Identity ca, bool is_ca)
Certificate generate(PrivateKey key, string name, Identity ca, bool is_ca) except +
shared_ptr[Certificate] issuer
cdef cppclass TrustList:
cppclass VerifyResult:
bool operator bool() const
bool isValid() const
string toString() const
TrustList()
void add(Certificate)
void remove(Certificate)
VerifyResult verify(Certificate);
string toString() except +
TrustList() except +
void add(Certificate) except +
void remove(Certificate) except +
VerifyResult verify(Certificate) except +
ctypedef TrustList.VerifyResult TrustListVerifyResult
@@ -151,6 +155,7 @@ cdef extern from "opendht/value.h" namespace "dht::Value":
cdef extern from "opendht/value.h" namespace "dht::Value::Field":
cdef Field None
cdef Field Id
cdef Field ValueTypeField "dht::Value::Field::ValueType"
cdef Field OwnerPk
cdef Field SeqNum
cdef Field UserType
@@ -163,13 +168,20 @@ cdef extern from "opendht/value.h" namespace "dht":
Value(const uint16_t t, const uint8_t* dat_ptr, size_t dat_len) except +
string toString() const
size_t size() const
bool isSigned() const
bool isEncrypted() const
uint64_t id
shared_ptr[PublicKey] owner
InfoHash recipient
vector[uint8_t] data
string user_type
uint16_t type
uint16_t seq
unsigned priority
string pushType
cdef cppclass ValueType:
ValueType() except +
ValueType(uint16_t id, string name, seconds expiration) except +
uint16_t id
string name
@@ -191,7 +203,7 @@ cdef extern from "opendht/value.h" namespace "dht":
cdef cppclass Where:
Where() except +
Where(const string& q_str)
Where(const string& q_str) except +
bool isSatisfiedBy(const Where& where) const
Where& id(uint64_t id)
Where& valueType(uint16_t type)
@@ -200,6 +212,14 @@ cdef extern from "opendht/value.h" namespace "dht":
Where& userType(string user_type)
string toString() const
cdef cppclass FieldValue:
uint64_t getInt() const
InfoHash getHash() const
Blob getBlob() const
cdef cppclass FieldValueIndex:
map[Field, FieldValue] index
cdef extern from "opendht/node.h" namespace "dht":
cdef cppclass Node:
Node() except +
@@ -208,6 +228,30 @@ cdef extern from "opendht/node.h" namespace "dht":
bool isExpired() const
cdef extern from "opendht/callbacks.h" namespace "dht":
cdef cppclass NodeStats:
unsigned good_nodes
unsigned dubious_nodes
unsigned cached_nodes
unsigned incoming_nodes
unsigned table_depth
unsigned searches
unsigned node_cache_size
unsigned getKnownNodes() const
unsigned long getNetworkSizeEstimation() const
cdef cppclass NodeInfo:
InfoHash id
InfoHash node_id
NodeStats ipv4
NodeStats ipv6
size_t ongoing_ops
size_t storage_values
size_t storage_size
size_t local_storage_values
size_t local_storage_size
in_port_t bound4
in_port_t bound6
ctypedef void (*ShutdownCallbackRaw)(void *user_data)
ctypedef bool (*GetCallbackRaw)(shared_ptr[Value] values, void *user_data)
ctypedef bool (*ValueCallbackRaw)(shared_ptr[Value] values, bool expired, void *user_data)
@@ -239,13 +283,90 @@ cdef extern from "opendht/callbacks.h" namespace "dht":
string persist_path
size_t max_req_per_sec
size_t max_peer_req_per_sec
ssize_t max_crypto_per_sec
ssize_t max_peer_crypto_per_sec
cppclass SecureDhtConfig:
Config node_config
Identity id
cdef extern from *:
"""
#include <Python.h>
#include <opendht/dhtrunner.h>
static std::shared_ptr<PyObject> opendht_python_owner(PyObject* data)
{
auto state = PyGILState_Ensure();
Py_INCREF(data);
PyGILState_Release(state);
return std::shared_ptr<PyObject>(data, [](PyObject* object) {
auto state = PyGILState_Ensure();
Py_DECREF(object);
PyGILState_Release(state);
});
}
static dht::ValueCallback opendht_bind_python_value_cb(
dht::ValueCallbackRaw callback, PyObject* data)
{
auto owner = opendht_python_owner(data);
return [callback, owner = std::move(owner)](
const std::vector<std::shared_ptr<dht::Value>>& values, bool expired) {
for (const auto& value : values)
if (!callback(value, expired, owner.get()))
return false;
return true;
};
}
static dht::DhtRunner::Context opendht_python_context(
void (*callback)(int, int, void*), PyObject* data)
{
dht::DhtRunner::Context context;
auto owner = opendht_python_owner(data);
context.statusChangedCallback = [callback, owner = std::move(owner)](
dht::NodeStatus ipv4, dht::NodeStatus ipv6) {
callback(static_cast<int>(ipv4), static_cast<int>(ipv6), owner.get());
};
return context;
}
static void opendht_python_run(dht::DhtRunner& runner, in_port_t port,
dht::DhtRunner::Config& config, void (*callback)(int, int, void*), PyObject* data)
{
runner.run(port, config, opendht_python_context(callback, data));
}
static void opendht_python_run(dht::DhtRunner& runner,
const char* ipv4, const char* ipv6, const char* port,
dht::DhtRunner::Config& config, void (*callback)(int, int, void*), PyObject* data)
{
runner.run(ipv4, ipv6, port, config, opendht_python_context(callback, data));
}
static void opendht_python_query(dht::DhtRunner& runner, const dht::InfoHash& key,
bool (*callback)(std::shared_ptr<dht::FieldValueIndex>, void*),
dht::DoneCallbackRaw done, PyObject* data, dht::Query query)
{
auto owner = opendht_python_owner(data);
runner.query(key,
[callback, owner](const std::vector<std::shared_ptr<dht::FieldValueIndex>>& rows) {
for (const auto& row : rows)
if (!callback(row, owner.get()))
return false;
return true;
},
[done, owner](bool ok, const std::vector<std::shared_ptr<dht::Node>>& nodes) {
done(ok, const_cast<std::vector<std::shared_ptr<dht::Node>>*>(&nodes), owner.get());
}, std::move(query));
}
"""
ValueCallback bindPythonValueCb "opendht_bind_python_value_cb"(ValueCallbackRaw callback, object data) except +
ctypedef future[size_t] ListenToken
ctypedef shared_future[size_t] SharedListenToken
cdef extern from "opendht/dhtrunner.h" namespace "dht":
ctypedef future[size_t] ListenToken
ctypedef shared_future[size_t] SharedListenToken
cdef cppclass DhtRunner:
DhtRunner() except +
cppclass Config:
@@ -264,37 +385,47 @@ cdef extern from "opendht/dhtrunner.h" namespace "dht":
SockAddr bind4
SockAddr bind6
InfoHash getId() const
shared_ptr[PublicKey] getPublicKey() const
InfoHash getNodeId() const
InfoHash getId() except + nogil
shared_ptr[PublicKey] getPublicKey() except + nogil
InfoHash getNodeId() except + nogil
NodeInfo getNodeInfo() except + nogil
vector[SockAddr] getPublicAddress(sa_family_t af) except + nogil
void connectivityChanged() except + nogil
void bootstrap(const_char*, const_char*)
void bootstrap(const SockAddr&, DoneCallbackSimple done_cb)
void run(in_port_t, Config config)
void run(const_char*, const_char*, const_char*, Config config)
void join()
void run(in_port_t, Config config) except +
void run(const_char*, const_char*, const_char*, Config config) except +
void join() except + nogil
void shutdown(ShutdownCallback)
void registerType(ValueType& value);
void registerType(const ValueType& value) except + nogil
bool isRunning()
SockAddr getBound(sa_family_t af) const
string getStorageLog() const
string getRoutingTablesLog(sa_family_t af) const
string getSearchesLog(sa_family_t af) const
SockAddr getBound(sa_family_t af) except + nogil
string getStorageLog() except + nogil
string getRoutingTablesLog(sa_family_t af) except + nogil
string getSearchesLog(sa_family_t af) except + nogil
void get(InfoHash key, GetCallback get_cb, DoneCallback done_cb, nullptr_t f, Where w)
void put(InfoHash key, shared_ptr[Value] val, DoneCallback done_cb, time_point created, bool permanent)
void putSigned(InfoHash key, shared_ptr[Value] val, DoneCallback done_cb, bool permanent)
void putEncrypted(InfoHash key, PkId to, shared_ptr[Value] val, DoneCallback done_cb, bool permanent)
void putEncrypted(InfoHash key, shared_ptr[PublicKey] to, shared_ptr[Value] val, DoneCallback done_cb, bool permanent)
void cancelPut(InfoHash key, shared_ptr[Value] val)
ListenToken listen(InfoHash key, ValueCallback get_cb)
void cancelListen(InfoHash key, SharedListenToken token)
vector[unsigned] getNodeMessageStats(bool i)
ListenToken listen(InfoHash key, ValueCallback get_cb, nullptr_t f, Where w) except +
void cancelListen(InfoHash key, SharedListenToken token) except +
vector[unsigned] getNodeMessageStats(bool i) except + nogil
ctypedef DhtRunner.Config DhtRunnerConfig
cdef extern from *:
ctypedef void (*StatusCallbackRaw)(int ipv4, int ipv6, void* data)
ctypedef bool (*QueryCallbackRaw)(shared_ptr[FieldValueIndex] row, void* data)
void runPythonRunner "opendht_python_run"(DhtRunner& runner, in_port_t port, DhtRunnerConfig& config, StatusCallbackRaw callback, object data) except + nogil
void runPythonRunnerAddress "opendht_python_run"(DhtRunner& runner, const_char* ipv4, const_char* ipv6, const_char* port, DhtRunnerConfig& config, StatusCallbackRaw callback, object data) except + nogil
void queryPython "opendht_python_query"(DhtRunner& runner, InfoHash key, QueryCallbackRaw callback, DoneCallbackRaw done, object data, Query query) except +
cdef extern from "opendht/log.h" namespace "dht::log":
void enableLogging(DhtRunner& dht)
void disableLogging(DhtRunner& dht)
void enableFileLogging(DhtRunner& dht, const string& path)
void enableLogging(DhtRunner& dht) except + nogil
void disableLogging(DhtRunner& dht) except + nogil
void enableFileLogging(DhtRunner& dht, const string& path) except + nogil
cdef extern from "opendht/indexation/pht.h" namespace "dht::indexation":
size_t PHT_MAX_NODE_ENTRY_COUNT "dht::indexation::Pht::MAX_NODE_ENTRY_COUNT"
@@ -310,8 +441,8 @@ cdef extern from "opendht/indexation/pht.h" namespace "dht::indexation":
cppclass LookupCallback:
LookupCallback() except +
Pht(string, IndexKeySpec, shared_ptr[DhtRunner]) except +
void lookup(IndexKey k, LookupCallback cb, DoneCallbackSimple doneCb);
void insert(IndexKey k, IndexValue v, DoneCallbackSimple cb)
void lookup(IndexKey k, LookupCallback cb, DoneCallbackSimple doneCb) except +
void insert(IndexKey k, IndexValue v, DoneCallbackSimple cb) except +
@staticmethod
LookupCallback bindLookupCb(LookupCallbackRaw cb, void *user_data)
+5 -5
View File
@@ -4,11 +4,11 @@ build-backend = "setuptools.build_meta"
[tool.cibuildwheel]
build = [
'cp3{9,10,11,12,13,14}-manylinux_x86_64',
'cp3{10,11,12,13,14}-manylinux_aarch64',
'cp3{11,12,13,14}-macosx_x86_64',
'cp3{11,12,13,14}-macosx_arm64',
'cp3{11,12,13,14}-win_amd64',
'cp3{11,12,13,14,15}-manylinux_x86_64',
'cp3{11,12,13,14,15}-manylinux_aarch64',
'cp3{11,12,13,14,15}-macosx_x86_64',
'cp3{11,12,13,14,15}-macosx_arm64',
'cp3{11,12,13,14,15}-win_amd64',
]
[tool.cibuildwheel.linux]
+35
View File
@@ -0,0 +1,35 @@
import runpy
import unittest
from pathlib import Path
from unittest.mock import Mock, patch
import netifaces
class NodeClusterTester(unittest.TestCase):
@classmethod
def setUpClass(cls):
script = Path(__file__).resolve().parents[1] / 'tools' / 'dhtcluster.py'
with patch.dict('sys.modules', {'opendht': Mock()}):
cls.NodeCluster = runpy.run_path(str(script))['NodeCluster']
def test_find_ip_interface(self):
addresses = {
netifaces.AF_INET: [{'addr': '127.0.0.1'}],
netifaces.AF_INET6: [{'addr': '::1'}],
}
with patch('netifaces.ifaddresses', return_value=addresses) as ifaddresses:
self.assertEqual(self.NodeCluster.find_ip('lo'), ('127.0.0.1', '::1'))
self.assertTrue(ifaddresses.called)
self.assertTrue(all(call.args == ('lo',) for call in ifaddresses.call_args_list))
def test_find_ip_any(self):
with patch('netifaces.ifaddresses') as ifaddresses:
for iface in (None, '', 'any'):
with self.subTest(iface=iface):
self.assertEqual(self.NodeCluster.find_ip(iface), ('0.0.0.0', '::0'))
ifaddresses.assert_not_called()
if __name__ == '__main__':
unittest.main()
File diff suppressed because it is too large. Load diff
+1
View File
@@ -9,6 +9,7 @@ import argparse
import logging
import logging.handlers
import netifaces
import opendht as dht
logger = logging.getLogger('dhtcluster')
+1
View File
@@ -11,6 +11,7 @@ from opendht import aio as dht
async def main():
config = dht.DhtConfig()
config.setRateLimit(-1, -1)
config.setCryptoRateLimit(-1, -1)
ping_node = dht.DhtRunner()
pong_node = dht.DhtRunner()
+22
View File
@@ -0,0 +1,22 @@
This software is licensed under the MIT License.
Copyright Fedor Indutny, 2018.
Permission is hereby granted, free of charge, to any person obtaining a
copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to permit
persons to whom the Software is furnished to do so, subject to the
following conditions:
The above copyright notice and this permission notice shall be included
in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN
NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
USE OR OTHER DEALINGS IN THE SOFTWARE.
+70
View File
@@ -0,0 +1,70 @@
# Building and Testing the Rust Crate
The `opendht` crate links to the native OpenDHT C library. Cargo does not build
or bundle that library. Install Rust and the native dependencies described in
the [OpenDHT build guide](../BUILD.md).
## Install the Native Library
From the root of an OpenDHT source checkout, enable the C bindings:
```sh
cmake -S . -B build-bindings -DOPENDHT_C=ON -DBUILD_TESTING=ON
cmake --build build-bindings
ctest --test-dir build-bindings -R test_c --output-on-failure
cmake --install build-bindings
```
Installation may require administrator privileges, depending on the install
prefix. Set `CMAKE_INSTALL_PREFIX` when configuring to choose another location.
The linker and runtime loader must be able to find the installed libraries.
Applications can then depend on `opendht` from crates.io as shown in the
[README](README.md).
## Use an Uninstalled Native Build
You can use the native build directly by setting its library search paths.
For example, from the repository root on macOS:
```sh
export LIBRARY_PATH="$PWD/build-bindings${LIBRARY_PATH:+:$LIBRARY_PATH}"
export DYLD_LIBRARY_PATH="$PWD/build-bindings${DYLD_LIBRARY_PATH:+:$DYLD_LIBRARY_PATH}"
export RUSTFLAGS="-L native=$PWD/build-bindings"
cargo test --manifest-path rust/Cargo.toml --all-targets
```
On Linux use `LD_LIBRARY_PATH` instead of `DYLD_LIBRARY_PATH`; on Windows add
the native DLL directory to `PATH`. For an application outside this checkout,
use absolute paths to the native build and run Cargo in the application directory.
When linking an AddressSanitizer native build, use the same compiler runtime.
For example, with Clang:
```sh
export RUSTFLAGS="-L native=$PWD/build-bindings -C linker=clang -C link-arg=-fsanitize=address"
```
## Build and Test the Crate
With the native library available, run from the `rust` directory:
```sh
cargo build --all-targets
cargo test --all-targets
cargo test --doc
cargo test --all-features --all-targets
cargo test --all-features --doc
```
## Run the Examples
From the `rust` directory:
```sh
cargo run --example dhtnode
cargo run --features async --example dhtnode_async
cargo run --features serde --example dhtnode_async
```
The callback example contacts a public bootstrap server. The async example uses
two loopback nodes; enabling `serde` also demonstrates typed MessagePack records.
+23 -4
View File
@@ -1,11 +1,30 @@
[package]
name = "opendht"
version = "0.1.0"
authors = ["Sébastien Blin <sebastien.blin@savoirfairelinux.com>"]
edition = "2021"
version = "0.6.0"
authors = ["Sébastien Blin <sebastien.blin@savoirfairelinux.com>", "Adrien Beraud <adrien.beraud@savoirfairelinux.com>"]
edition = "2024"
description = "Rust bindings for OpenDHT"
license = "MIT"
license-file = "../LICENSE"
repository = "https://github.com/savoirfairelinux/opendht"
[features]
async = ["dep:futures-channel", "dep:futures-core"]
serde = ["async", "dep:serde", "dep:rmp-serde"]
[[example]]
name = "dhtnode_async"
required-features = ["async"]
[dependencies]
libc="0.2"
os_socketaddr="0.2.3"
futures-channel = { version = "0.3", optional = true }
futures-core = { version = "0.3", optional = true }
serde = { version = "1", optional = true }
rmp-serde = { version = "1", optional = true }
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dev-dependencies]
futures = "0.3"
serde = { version = "1", features = ["derive"] }
serde_bytes = "0.11"
+142 -11
View File
@@ -1,21 +1,152 @@
# Rust bindings for opendht
# OpenDHT: a Distributed Hash Table
## Build
OpenDHT is a Kademlia-based distributed hash table for storing and retrieving
small values over a peer-to-peer network without a central server. Common uses
include peer discovery, service announcements, and decentralized applications.
Install rust on your system (https://rustup.rs/ is the recommanded way). Once done:
The `opendht` crate provides Rust bindings to [OpenDHT](https://opendht.net), with
callback-based and runtime-independent async APIs. Capabilities include signed
and encrypted values, filtered reads, metadata queries, and typed MessagePack
records through Serde.
```
cargo build
## Get Started
Add the crate to your dependencies:
```toml
[dependencies]
opendht = "0.6"
```
## Run tests
The native OpenDHT library with C bindings must be installed separately; it is
not bundled with the crate. See the
[build guide](https://github.com/savoirfairelinux/opendht/blob/master/rust/BUILD.md)
for native installation, library paths, and source builds.
```
cargo test
`DhtRunner` manages a node, `InfoHash` identifies a DHT key, and `Value` contains
the payload and metadata stored under that key. `put` publishes values, `get`
retrieves them, and `listen` subscribes to value updates and expiration events.
This example starts a node on an OS-assigned port, bootstraps into the public
network, and listens for a value it publishes.
```rust
use opendht::{DhtRunner, InfoHash, Value};
use std::{sync::mpsc, time::Duration};
fn main() -> Result<(), Box<dyn std::error::Error>> {
let mut runner = DhtRunner::new();
runner.run(0)?;
runner.bootstrap("bootstrap.jami.net", 4222)?;
let key = InfoHash::get("my-app/hello");
let (sender, receiver) = mpsc::channel();
let listener = runner.listen(&key, move |value, expired| {
if !expired {
let _ = sender.send(value);
}
true
})?;
runner.put(&key, Value::new("Hello from Rust!"), |_| {}, false);
let value = receiver.recv_timeout(Duration::from_secs(5))?;
println!("{}", String::from_utf8_lossy(value.as_slice()));
runner.cancel_listen(&key, listener);
runner.join()?;
Ok(())
}
```
## Run dht node
A bootstrap server supplies initial peers. Applications can use public
bootstrap servers or known peers on a private network. Values expire, and their
availability depends on participating nodes; the DHT does not provide durable
storage guarantees.
## Optional Features
| Feature | Capabilities |
|---------|--------------|
| Default | Callback-based `DhtRunner`, binary values, keys, certificates, and signed/encrypted operations. |
| `async` | `aio::AsyncDhtRunner` with publication futures, bounded read/update streams, and automatic listener cancellation on drop. |
| `serde` | Serde-backed MessagePack records with `put_typed`, `get_typed`, and `listen_typed`. Enables `async`. |
Enable `async` in the dependency configuration:
```toml
[dependencies]
opendht = { version = "0.6", features = ["async"] }
futures = "0.3"
```
cargo run --example dhtnode
```
Create the adapter with `AsyncDhtRunner::new(runner)` after starting a
`DhtRunner`. The adapter is runtime-independent and thread-bound; use a local
executor or Tokio's `LocalSet`. Dropping a listener stream cancels its
subscription. Streams use bounded buffers and report `WouldBlock` on overflow.
This example waits for a bootstrap peer, publishes a value with `.await`, and
reads matching values as a stream:
```rust,no_run
use futures::{executor::block_on, TryStreamExt};
use opendht::{aio::AsyncDhtRunner, DhtRunner, InfoHash, Value};
use std::{io, net::ToSocketAddrs};
fn main() -> io::Result<()> {
let mut runner = DhtRunner::new();
runner.run(0)?;
let runner = AsyncDhtRunner::new(runner);
let peers: Vec<_> = ("bootstrap.jami.net", 4222).to_socket_addrs()?.collect();
block_on(async {
runner.bootstrap(&peers).await?;
let key = InfoHash::get("my-app/hello-async");
runner.put(&key, Value::new("Hello from async Rust!"), false).await?;
let mut values = runner.get(&key);
while let Some(value) = values.try_next().await? {
println!("{}", String::from_utf8_lossy(value.as_slice()));
}
runner.shutdown().await?;
runner.join()
})
}
```
### Typed Records
The `serde` feature adds typed async operations and also enables `async`.
`put_typed` serializes records implementing `Serialize`; `get_typed` and
`listen_typed` decode records implementing `DeserializeOwned`. Typed reads retain
the decoded record alongside its original payload and native metadata.
Structs are serialized as compact MessagePack arrays. Field order is part of
the wire format and must remain compatible across schema changes. `serde_bytes`
encodes binary fields using MessagePack's binary representation.
## Working With Values
GET and LISTEN callbacks and async streams deliver owned `Box<Value>` instances.
Received values implement `Send` but not `Sync`; they can be retained after the
runner stops, moved to another thread, modified, and republished. `as_slice()`
borrows the payload without copying; `as_bytes()` allocates a separate `Vec<u8>`.
Values expose metadata including IDs, types, sequence numbers, and owner keys.
Filters select values by metadata, and queries return selected metadata fields.
`DhtRunner::listen` returns a token that must be passed to `cancel_listen` to
unsubscribe. Dropping the token does not cancel the registration; dropping an
async listener stream does.
`join()` stops the runner and waits for its threads to exit. It must not be
called from a callback. For graceful shutdown and configured state persistence,
wait for `shutdown` to complete before calling `join()`.
## Learn More
- [API reference](https://docs.rs/opendht)
- [Callback example](https://github.com/savoirfairelinux/opendht/blob/master/rust/examples/dhtnode.rs)
- [Async and typed two-node example](https://github.com/savoirfairelinux/opendht/blob/master/rust/examples/dhtnode_async.rs)
- [Build and test guide](https://github.com/savoirfairelinux/opendht/blob/master/rust/BUILD.md)
- [OpenDHT project and issue tracker](https://github.com/savoirfairelinux/opendht)
+19 -16
View File
@@ -2,19 +2,18 @@
// SPDX-License-Identifier: MIT
extern crate opendht;
use std::{ thread, time };
use std::{thread, time};
use opendht::{ InfoHash, DhtRunner, DhtRunnerConfig, Value };
use opendht::{DhtRunner, DhtRunnerConfig, InfoHash, Value};
// use opendht::crypto::*;
fn main() {
fn main() -> std::io::Result<()> {
println!("{}", InfoHash::random());
println!("{}", InfoHash::new());
println!("{}", InfoHash::new().is_zero());
println!("{}", InfoHash::get("alice"));
println!("{}", InfoHash::get("alice").is_zero());
let mut dht = DhtRunner::new();
let /*mut*/ config = DhtRunnerConfig::new();
//// If you want to inject a certificate, uncomment the following lines and previous mut.
@@ -23,36 +22,40 @@ fn main() {
//let cert = DhtCertificate::import("example.crt").ok().expect("Invalid cert file");
//let pk = PrivateKey::import("example.key", "");
//config.set_identity(cert, pk);
dht.run_config(1412, config);
dht.bootstrap("bootstrap.jami.net", 4222);
dht.run_config(1412, config)?;
dht.bootstrap("bootstrap.jami.net", 4222)?;
println!("Current node id: {}", dht.node_id());
let /* mut */ data = 42;
let mut get_cb = |v: Box<Value>| {
let get_cb = move |v: Box<Value>| {
//data += 1;
println!("GET: VALUE CB - data: {} - v: {}", data, v);
true
};
let mut done_cb = |ok: bool| {
let done_cb = move |ok: bool| {
println!("GET: DONE CB - data: {} - ok: {}", data, ok);
};
dht.get(&InfoHash::get("alice"), &mut get_cb, &mut done_cb);
dht.get(&InfoHash::get("alice"), get_cb, done_cb);
let mut put_done_cb = |ok: bool| {
let put_done_cb = move |ok: bool| {
println!("PUT: DONE CB - data: {} - ok: {}", data, ok);
};
dht.put(&InfoHash::get("bob"), Value::new("hi!"), &mut put_done_cb, false);
dht.put(&InfoHash::get("bob"), Value::new("hi!"), put_done_cb, false);
println!("Start listening /foo");
let mut value_cb = |v, expired| {
println!("LISTEN: DONE CB - data: {} - v: {} - expired: {}", data, v, expired);
let value_cb = move |v, expired| {
println!(
"LISTEN: DONE CB - data: {} - v: {} - expired: {}",
data, v, expired
);
true
};
let token = dht.listen(&InfoHash::get("foo"), &mut value_cb);
let token = dht.listen(&InfoHash::get("foo"), value_cb)?;
let one_min = time::Duration::from_secs(10);
thread::sleep(one_min);
dht.cancel_listen(&InfoHash::get("foo"), token);
println!("Public ips: {:#?}", dht.public_addresses());
}
dht.join()?;
Ok(())
}
+80
View File
@@ -0,0 +1,80 @@
use futures::{executor::block_on, StreamExt, TryStreamExt};
use opendht::{aio::AsyncDhtRunner, DhtRunner, DhtRunnerConfig, InfoHash, Value};
use std::error::Error;
use std::io;
use std::net::SocketAddr;
fn main() -> Result<(), Box<dyn Error>> {
let mut source = DhtRunner::new();
let mut destination = DhtRunner::new();
source.run_bind("127.0.0.1", "::1", 0, DhtRunnerConfig::new())?;
destination.run_bind("127.0.0.1", "::1", 0, DhtRunnerConfig::new())?;
let address = SocketAddr::from(([127, 0, 0, 1], destination.bound_port(libc::AF_INET as _)));
let source = AsyncDhtRunner::new(source);
let destination = AsyncDhtRunner::new(destination);
block_on(async {
source.bootstrap(&[address]).await?;
let hash = InfoHash::get("async example");
let mut updates = destination.listen(&hash)?;
source
.put(&hash, Value::with_id(b"hello from Rust async", 42), false)
.await?;
let event = updates
.next()
.await
.ok_or_else(|| io::Error::other("listener ended before receiving a value"))??;
println!(
"LISTEN {}: {} (expired: {})",
event.value.id(),
String::from_utf8_lossy(event.value.as_slice()),
event.expired,
);
drop(updates);
let values: Vec<_> = destination.get(&hash).try_collect().await?;
println!("GET: {} values", values.len());
#[cfg(feature = "serde")]
{
#[derive(Debug, serde::Serialize, serde::Deserialize)]
struct Service {
version: u32,
name: String,
port: u16,
#[serde(with = "serde_bytes")]
payload: Vec<u8>,
}
let hash = InfoHash::get("typed async example");
let mut updates = destination.listen_typed::<Service>(&hash)?;
let record = Service {
version: 1,
name: "chat".into(),
port: 4222,
payload: vec![0, 0x80, 0xff],
};
source.put_typed(&hash, &record, false).await?;
let event = updates.next().await.ok_or_else(|| {
io::Error::other("typed listener ended before receiving a value")
})??;
println!(
"TYPED LISTEN {}: {:?}",
event.value.raw.id(),
event.value.data
);
drop(updates);
let values: Vec<_> = destination
.get_typed::<Service>(&hash)
.try_collect()
.await?;
println!("TYPED GET: {} records", values.len());
}
source.shutdown().await?;
destination.shutdown().await?;
source.join()?;
destination.join()?;
Ok(())
})
}
+429
View File
@@ -0,0 +1,429 @@
//! Runtime-independent futures and bounded streams, enabled by the `async` feature.
//!
//! Runners and listeners stay on their creating thread. Use a local executor or
//! Tokio `LocalSet`; received values own their native storage and can be
//! transferred between threads without another payload copy.
use crate::{DhtRunner, InfoHash, OpToken, PkId, QueryResult, Value};
use futures_channel::{mpsc, oneshot};
use futures_core::Stream;
use std::cell::RefCell;
use std::future::Future;
use std::io;
use std::net::SocketAddr;
use std::pin::Pin;
use std::rc::Rc;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Arc, Mutex};
use std::task::{Context, Poll};
/// A listener update; expiration events retain the expired value's metadata.
pub struct ValueEvent {
pub value: Box<Value>,
pub expired: bool,
}
/// A native completion mapped to an I/O result.
///
/// Operations start immediately, before this future is polled. Dropping it
/// abandons the result but does not undo a put, ping, bootstrap, or shutdown.
#[must_use = "operations run immediately; await their result or explicitly drop it"]
pub struct Operation {
receiver: oneshot::Receiver<io::Result<()>>,
}
impl Future for Operation {
type Output = io::Result<()>;
fn poll(mut self: Pin<&mut Self>, context: &mut Context<'_>) -> Poll<Self::Output> {
Pin::new(&mut self.receiver).poll(context).map(|outcome| {
outcome.unwrap_or_else(|_| Err(stopped("DHT operation ended without completion")))
})
}
}
fn operation() -> (oneshot::Sender<io::Result<()>>, Operation) {
let (sender, receiver) = oneshot::channel();
(sender, Operation { receiver })
}
fn outcome(success: bool) -> io::Result<()> {
if success {
Ok(())
} else {
Err(io::Error::other(
"DHT operation failed or the runner stopped",
))
}
}
fn stopped(message: &'static str) -> io::Error {
io::Error::new(io::ErrorKind::Interrupted, message)
}
#[derive(Clone)]
struct StreamCompletion(Arc<Mutex<Option<oneshot::Sender<io::Result<()>>>>>);
impl StreamCompletion {
fn finish(&self, result: io::Result<()>) {
let sender = self
.0
.lock()
.unwrap_or_else(|error| error.into_inner())
.take();
if let Some(sender) = sender {
let _ = sender.send(result);
}
}
}
struct StreamSender<T> {
values: mpsc::Sender<T>,
completion: StreamCompletion,
}
impl<T> StreamSender<T> {
fn send(&mut self, value: io::Result<T>) -> bool {
let value = match value {
Ok(value) => value,
Err(error) => {
self.completion.finish(Err(error));
return false;
}
};
match self.values.try_send(value) {
Ok(()) => true,
Err(error) => {
if error.is_full() {
self.completion.finish(Err(io::Error::new(
io::ErrorKind::WouldBlock,
"DHT stream capacity exceeded; restart the operation to resynchronize",
)));
}
false
}
}
}
}
/// A bounded get or query stream.
///
/// Buffered items precede a terminal error. Native failure, runner shutdown,
/// and buffer overflow are never reported as successful EOF.
/// Dropping the stream stops delivery; native work stops at its next callback
/// or when the runner is joined.
#[must_use = "streams must be polled to receive values and completion errors"]
pub struct OperationStream<T> {
values: mpsc::Receiver<T>,
completion: oneshot::Receiver<io::Result<()>>,
completed: bool,
outcome: Option<io::Result<()>>,
}
impl<T: Unpin> Stream for OperationStream<T> {
type Item = io::Result<T>;
fn poll_next(self: Pin<&mut Self>, context: &mut Context<'_>) -> Poll<Option<Self::Item>> {
let stream = self.get_mut();
if !stream.completed
&& let Poll::Ready(result) = Pin::new(&mut stream.completion).poll(context)
{
stream.completed = true;
stream.outcome = Some(
result.unwrap_or_else(|_| Err(stopped("DHT stream ended without completion"))),
);
stream.values.close();
}
match Pin::new(&mut stream.values).poll_next(context) {
Poll::Ready(Some(value)) => Poll::Ready(Some(Ok(value))),
Poll::Ready(None) if stream.completed => {
Poll::Ready(stream.outcome.take().and_then(Result::err).map(Err))
}
_ => Poll::Pending,
}
}
}
fn stream_channel<T>(capacity: usize) -> (StreamSender<T>, OperationStream<T>) {
let (values, receiver) = mpsc::channel(capacity - 1);
let (completed, completion) = oneshot::channel();
(
StreamSender {
values,
completion: StreamCompletion(Arc::new(Mutex::new(Some(completed)))),
},
OperationStream {
values: receiver,
completion,
completed: false,
outcome: None,
},
)
}
struct RunnerState {
runner: RefCell<Box<DhtRunner>>,
stopping: Arc<AtomicBool>,
}
impl Drop for RunnerState {
fn drop(&mut self) {
self.stopping.store(true, Ordering::SeqCst);
}
}
/// A listener registration that cancels itself when dropped or terminated.
///
/// The runner is retained until this registration is dropped. Calling `join()`
/// stops all registrations and makes pending listeners report an error.
#[must_use = "dropping a listener cancels its native registration"]
pub struct Listener {
state: Rc<RunnerState>,
hash: InfoHash,
token: Option<Box<OpToken>>,
stream: OperationStream<ValueEvent>,
}
impl Listener {
fn cancel(&mut self) {
if let Some(token) = self.token.take() {
self.state
.runner
.borrow_mut()
.cancel_listen(&self.hash, token);
}
}
}
impl Stream for Listener {
type Item = io::Result<ValueEvent>;
fn poll_next(mut self: Pin<&mut Self>, context: &mut Context<'_>) -> Poll<Option<Self::Item>> {
let result = Pin::new(&mut self.stream).poll_next(context);
if matches!(result, Poll::Ready(None) | Poll::Ready(Some(Err(_)))) {
self.cancel();
}
result
}
}
impl Drop for Listener {
fn drop(&mut self) {
self.cancel();
}
}
/// A shared, thread-bound async adapter for an already configured runner.
///
/// Clones share a runner without making native handles `Send` or `Sync`.
/// Native calls are short; futures and streams never hold a mutable runner
/// borrow while waiting. The default stream capacity is 64 items.
///
/// ```compile_fail
/// use opendht::{aio::AsyncDhtRunner, DhtRunner};
/// let runner = AsyncDhtRunner::new(DhtRunner::new());
/// std::thread::spawn(move || runner.join().unwrap());
/// ```
#[derive(Clone)]
pub struct AsyncDhtRunner {
state: Rc<RunnerState>,
capacity: usize,
}
impl AsyncDhtRunner {
pub fn new(runner: Box<DhtRunner>) -> Self {
Self::with_capacity(runner, 64).expect("default stream capacity is valid")
}
/// Set the maximum number of queued values per get, query, or listener.
/// A slow consumer receives `WouldBlock` and the stream terminates.
pub fn with_capacity(runner: Box<DhtRunner>, capacity: usize) -> io::Result<Self> {
if capacity == 0 {
return Err(io::Error::new(
io::ErrorKind::InvalidInput,
"stream capacity must be positive",
));
}
Ok(Self {
state: Rc::new(RunnerState {
runner: RefCell::new(runner),
stopping: Arc::new(AtomicBool::new(false)),
}),
capacity,
})
}
pub fn put(&self, hash: &InfoHash, value: Box<Value>, permanent: bool) -> Operation {
let (sender, pending) = operation();
self.state.runner.borrow_mut().put(
hash,
value,
move |success| {
let _ = sender.send(outcome(success));
},
permanent,
);
pending
}
pub fn put_signed(&self, hash: &InfoHash, value: Box<Value>, permanent: bool) -> Operation {
let (sender, pending) = operation();
self.state.runner.borrow_mut().put_signed(
hash,
value,
move |success| {
let _ = sender.send(outcome(success));
},
permanent,
);
pending
}
pub fn put_encrypted(
&self,
hash: &InfoHash,
recipient: &PkId,
value: Box<Value>,
permanent: bool,
) -> Operation {
let (sender, pending) = operation();
self.state.runner.borrow_mut().put_encrypted(
hash,
recipient,
value,
move |success| {
let _ = sender.send(outcome(success));
},
permanent,
);
pending
}
pub fn ping(&self, address: SocketAddr) -> Operation {
let (sender, pending) = operation();
self.state
.runner
.borrow_mut()
.ping(address, move |success| {
let _ = sender.send(outcome(success));
});
pending
}
/// Bootstrap pre-resolved addresses concurrently; one successful peer suffices.
/// DNS resolution is left to the application's executor.
pub fn bootstrap(&self, addresses: &[SocketAddr]) -> Operation {
let (sender, pending) = operation();
if addresses.is_empty() {
let _ = sender.send(Err(io::Error::new(
io::ErrorKind::InvalidInput,
"at least one bootstrap address is required",
)));
return pending;
}
let completion = StreamCompletion(Arc::new(Mutex::new(Some(sender))));
let remaining = Arc::new(std::sync::atomic::AtomicUsize::new(addresses.len()));
for &address in addresses {
let completion = completion.clone();
let remaining = remaining.clone();
self.state
.runner
.borrow_mut()
.ping(address, move |success| {
if success {
completion.finish(Ok(()));
}
if remaining.fetch_sub(1, Ordering::SeqCst) == 1 {
completion.finish(outcome(false));
}
});
}
pending
}
pub fn get(&self, hash: &InfoHash) -> OperationStream<Box<Value>> {
let (mut sender, stream) = stream_channel(self.capacity);
let completion = sender.completion.clone();
self.state.runner.borrow_mut().get(
hash,
move |value| sender.send(Ok(value)),
move |success| completion.finish(outcome(success)),
);
stream
}
pub fn get_filtered(
&self,
hash: &InfoHash,
filter: &str,
) -> io::Result<OperationStream<Box<Value>>> {
let (mut sender, stream) = stream_channel(self.capacity);
let completion = sender.completion.clone();
self.state.runner.borrow_mut().get_filtered(
hash,
filter,
move |value| sender.send(Ok(value)),
move |success| completion.finish(outcome(success)),
)?;
Ok(stream)
}
pub fn query(&self, hash: &InfoHash, query: &str) -> io::Result<OperationStream<QueryResult>> {
let (mut sender, stream) = stream_channel(self.capacity);
let completion = sender.completion.clone();
self.state.runner.borrow_mut().query(
hash,
query,
move |row| sender.send(Ok(row)),
move |success| completion.finish(outcome(success)),
)?;
Ok(stream)
}
pub fn listen(&self, hash: &InfoHash) -> io::Result<Listener> {
self.listen_filtered(hash, "")
}
pub fn listen_filtered(&self, hash: &InfoHash, filter: &str) -> io::Result<Listener> {
let (mut sender, stream) = stream_channel(self.capacity);
let token = self.state.runner.borrow_mut().listen_filtered(
hash,
filter,
move |value, expired| sender.send(Ok(ValueEvent { value, expired })),
)?;
Ok(Listener {
state: self.state.clone(),
hash: InfoHash { d: hash.d },
token: Some(token),
stream,
})
}
/// Request graceful shutdown. Call `join()` after this future succeeds when
/// persistence is required. Joining first reports an interrupted shutdown.
pub fn shutdown(&self) -> Operation {
let (sender, pending) = operation();
let stopping = self.state.stopping.clone();
self.state.runner.borrow_mut().shutdown(move || {
let result = if stopping.load(Ordering::SeqCst) {
Err(stopped("runner joined before graceful shutdown completed"))
} else {
Ok(())
};
let _ = sender.send(result);
});
pending
}
/// Drive a non-threaded runner without waiting for any async operation.
pub fn loop_once(&self) -> io::Result<()> {
self.state.runner.borrow_mut().loop_once()
}
/// Stop the runner and wait for native callbacks. This is a blocking call,
/// not an async wait, and must not be invoked from a native DHT callback.
pub fn join(&self) -> io::Result<()> {
self.state.stopping.store(true, Ordering::SeqCst);
self.state.runner.borrow_mut().join()
}
}
+9 -7
View File
@@ -1,18 +1,20 @@
// Copyright (c) 2014-2026 Savoir-faire Linux Inc.
// SPDX-License-Identifier: MIT
use crate::ffi::*;
use crate::ffi::{self, DataView};
pub use crate::ffi::Blob;
native_handle!(Blob, dht_blob_delete);
impl Blob {
pub fn data(&self) -> DataView {
unsafe { dht_blob_get_data(self) }
unsafe { ffi::dht_blob_get_data(self.as_ptr()) }
}
}
impl Drop for Blob {
fn drop(&mut self) {
unsafe { dht_blob_delete(&mut *self) }
pub fn as_bytes(&self) -> Vec<u8> {
let data = self.data();
if data.size == 0 {
return Vec::new();
}
unsafe { std::slice::from_raw_parts(data.data, data.size).to_vec() }
}
}
+137 -74
View File
@@ -1,150 +1,213 @@
// Copyright (c) 2014-2026 Savoir-faire Linux Inc.
// SPDX-License-Identifier: MIT
#![allow(dead_code)]
pub use crate::ffi::*;
use crate::{ffi, Blob, InfoHash, PkId};
use std::ffi::CString;
use std::fs::File;
use std::fs;
use std::io;
use std::io::prelude::*;
use std::ptr;
native_handle!(PublicKey, dht_publickey_delete);
native_handle!(PrivateKey, dht_privatekey_delete);
native_handle!(DhtCertificate, dht_certificate_delete);
impl PublicKey {
pub fn new(data: &str) -> Box<PublicKey> {
unsafe { Box::from_raw(dht_publickey_import(data.as_ptr(), data.len())) }
pub fn new(data: impl AsRef<[u8]>) -> io::Result<Box<PublicKey>> {
let data = data.as_ref();
unsafe { Self::from_raw(ffi::dht_publickey_import(data.as_ptr(), data.len())) }
}
pub fn unpack(&mut self, data: Vec<u8>) -> i32 {
unsafe { dht_publickey_unpack(&mut *self, data.as_ptr(), data.len()) }
pub fn unpack(&mut self, data: impl AsRef<[u8]>) -> io::Result<()> {
let data = data.as_ref();
let result = unsafe { ffi::dht_publickey_unpack(self.as_ptr(), data.as_ptr(), data.len()) };
if result == 0 {
Ok(())
} else {
Err(io::Error::other("invalid public key"))
}
}
pub fn pack(&mut self, data: &str) -> i32 {
let data = CString::new(data).unwrap();
unsafe { dht_publickey_pack(&mut *self, data.as_ptr(), data.as_bytes().len()) }
pub fn pack(&self) -> io::Result<Vec<u8>> {
unsafe { Ok(Blob::from_raw(ffi::dht_publickey_export_blob(self.as_ptr()))?.as_bytes()) }
}
pub fn id(&self) -> InfoHash {
unsafe { dht_publickey_get_id(self) }
unsafe { ffi::dht_publickey_get_id(self.as_ptr()) }
}
pub fn long_id(&self) -> PkId {
unsafe { dht_publickey_get_long_id(self) }
unsafe { ffi::dht_publickey_get_long_id(self.as_ptr()) }
}
pub fn check_signature(&self, data: &str, signature: &str) -> bool {
let data = CString::new(data).unwrap();
let signature = CString::new(signature).unwrap();
pub fn check_signature(&self, data: impl AsRef<[u8]>, signature: impl AsRef<[u8]>) -> bool {
let data = data.as_ref();
let signature = signature.as_ref();
unsafe {
dht_publickey_check_signature(
self,
data.as_ptr(),
data.as_bytes().len(),
signature.as_ptr(),
signature.as_bytes().len(),
ffi::dht_publickey_check_signature(
self.as_ptr(),
data.as_ptr().cast(),
data.len(),
signature.as_ptr().cast(),
signature.len(),
)
}
}
pub fn encrypt(&self, data: &str) -> Box<Blob> {
let data = CString::new(data).unwrap();
pub fn encrypt(&self, data: impl AsRef<[u8]>) -> io::Result<Box<Blob>> {
let data = data.as_ref();
unsafe {
Box::from_raw(dht_publickey_encrypt(
self,
data.as_ptr(),
data.as_bytes().len(),
Blob::from_raw(ffi::dht_publickey_encrypt(
self.as_ptr(),
data.as_ptr().cast(),
data.len(),
))
}
}
}
impl Drop for PublicKey {
fn drop(&mut self) {
unsafe { dht_publickey_delete(&mut *self) }
}
}
impl PrivateKey {
pub fn new(key_length_bits: u32) -> Box<PrivateKey> {
unsafe { Box::from_raw(dht_privatekey_generate(key_length_bits)) }
pub fn new(key_length_bits: u32) -> io::Result<Box<PrivateKey>> {
unsafe { Self::from_raw(ffi::dht_privatekey_generate(key_length_bits)) }
}
pub fn new_ec() -> io::Result<Box<PrivateKey>> {
unsafe { Self::from_raw(ffi::dht_privatekey_generate_ec()) }
}
pub fn import(file: &str, password: &str) -> io::Result<Box<PrivateKey>> {
let mut f = File::open(file)?;
let mut buffer = Vec::new();
f.read_to_end(&mut buffer)?;
Ok(PrivateKey::from_bytes(&buffer, password))
Self::from_bytes(&fs::read(file)?, password)
}
pub fn from_bytes(buffer: &Vec<u8>, password: &str) -> Box<PrivateKey> {
pub fn from_bytes(buffer: &[u8], password: &str) -> io::Result<Box<PrivateKey>> {
let password = CString::new(password)?;
unsafe {
Box::from_raw(dht_privatekey_import(
(&*buffer).as_ptr(),
Self::from_raw(ffi::dht_privatekey_import(
buffer.as_ptr(),
buffer.len(),
password.as_ptr() as *const i8,
password.as_ptr(),
))
}
}
pub fn public_key(&self) -> Box<PublicKey> {
unsafe { Box::from_raw(dht_privatekey_get_publickey(self)) }
unsafe {
PublicKey::from_raw(ffi::dht_privatekey_get_publickey(self.as_ptr()))
.expect("private key has a public key")
}
}
}
impl Drop for PrivateKey {
fn drop(&mut self) {
unsafe { dht_privatekey_delete(&mut *self) }
pub fn serialize(&self, password: &str) -> io::Result<Vec<u8>> {
let password = CString::new(password)?;
unsafe {
Ok(Blob::from_raw(ffi::dht_privatekey_export_blob(
self.as_ptr(),
password.as_ptr(),
))?
.as_bytes())
}
}
pub fn sign(&self, data: impl AsRef<[u8]>) -> io::Result<Vec<u8>> {
let data = data.as_ref();
unsafe {
Ok(Blob::from_raw(ffi::dht_privatekey_sign(
self.as_ptr(),
data.as_ptr(),
data.len(),
))?
.as_bytes())
}
}
pub fn decrypt(&self, data: &[u8]) -> io::Result<Box<Blob>> {
unsafe {
Blob::from_raw(ffi::dht_privatekey_decrypt(
self.as_ptr(),
data.as_ptr().cast(),
data.len(),
))
}
}
}
impl DhtCertificate {
pub fn import(file: &str) -> io::Result<Box<DhtCertificate>> {
let mut f = File::open(file)?;
let mut buffer = Vec::new();
f.read_to_end(&mut buffer)?;
Ok(DhtCertificate::from_bytes(&buffer))
Self::from_bytes(&fs::read(file)?)
}
pub fn from_bytes(buffer: &Vec<u8>) -> Box<DhtCertificate> {
unsafe { Box::from_raw(dht_certificate_import((&*buffer).as_ptr(), buffer.len())) }
pub fn from_bytes(buffer: &[u8]) -> io::Result<Box<DhtCertificate>> {
unsafe { Self::from_raw(ffi::dht_certificate_import(buffer.as_ptr(), buffer.len())) }
}
pub fn from_slice(buffer: &str) -> Box<DhtCertificate> {
unsafe { Box::from_raw(dht_certificate_import((&*buffer).as_ptr(), buffer.len())) }
pub fn from_slice(buffer: &str) -> io::Result<Box<DhtCertificate>> {
Self::from_bytes(buffer.as_bytes())
}
pub fn serialize(&self) -> io::Result<Vec<u8>> {
unsafe { Ok(Blob::from_raw(ffi::dht_certificate_export(self.as_ptr()))?.as_bytes()) }
}
pub fn id(&self) -> InfoHash {
unsafe { dht_certificate_get_id(&*self) }
unsafe { ffi::dht_certificate_get_id(self.as_ptr()) }
}
pub fn long_id(&self) -> PkId {
unsafe { dht_certificate_get_long_id(&*self) }
unsafe { ffi::dht_certificate_get_long_id(self.as_ptr()) }
}
pub fn publickey(&self) -> Box<PublicKey> {
unsafe { Box::from_raw(dht_certificate_get_publickey(&*self)) }
unsafe {
PublicKey::from_raw(ffi::dht_certificate_get_publickey(self.as_ptr()))
.expect("certificate has a public key")
}
}
}
impl Drop for DhtCertificate {
fn drop(&mut self) {
unsafe { dht_certificate_delete(&mut *self) }
}
pub struct DhtIdentity {
pub privatekey: Box<PrivateKey>,
pub certificate: Box<DhtCertificate>,
}
impl DhtIdentity {
pub fn new(common_name: &str) -> DhtIdentity {
unsafe { DhtIdentity::generate(common_name, Box::from_raw(ptr::null_mut())) }
pub fn new(common_name: &str) -> io::Result<Self> {
Self::generate_inner(common_name, None, false)
}
pub fn generate(common_name: &str, ca: Box<DhtIdentity>) -> DhtIdentity {
let common_name = CString::new(common_name).unwrap();
unsafe { dht_identity_generate(common_name.as_ptr(), &*ca) }
pub fn new_ec(common_name: &str) -> io::Result<Self> {
Self::generate_inner(common_name, None, true)
}
}
impl Drop for DhtIdentity {
fn drop(&mut self) {
unsafe { dht_identity_delete(&mut *self) }
pub fn generate(common_name: &str, ca: &DhtIdentity) -> io::Result<Self> {
Self::generate_inner(common_name, Some(ca), false)
}
pub(crate) fn raw(&self) -> ffi::DhtIdentity {
ffi::DhtIdentity {
privatekey: self.privatekey.as_ptr(),
certificate: self.certificate.as_ptr(),
}
}
fn generate_inner(common_name: &str, ca: Option<&Self>, ec: bool) -> io::Result<Self> {
let name = CString::new(common_name)?;
let ca = ca.map(Self::raw);
let ca_ptr = ca
.as_ref()
.map_or(ptr::null(), |identity| identity as *const _);
unsafe {
let mut raw = if ec {
ffi::dht_identity_generate_ec(name.as_ptr(), ca_ptr)
} else {
ffi::dht_identity_generate(name.as_ptr(), ca_ptr)
};
if raw.privatekey.is_null() || raw.certificate.is_null() {
ffi::dht_identity_delete(&mut raw);
return Err(io::Error::other("identity generation failed"));
}
Ok(Self {
privatekey: PrivateKey::from_raw(raw.privatekey)?,
certificate: DhtCertificate::from_raw(raw.certificate)?,
})
}
}
}
+717 -167
View File
File diff suppressed because it is too large. Load diff
+245 -80
View File
@@ -3,71 +3,66 @@
#![allow(dead_code)]
use libc::{c_char, c_int, c_uint, c_void, in_port_t, size_t};
use os_socketaddr::OsSocketAddr;
use libc::{c_char, c_int, c_uint, c_void, in_port_t, sa_family_t, size_t, sockaddr, socklen_t};
const HASH_LEN: usize = 20;
const PKID_LEN: usize = 32;
#[repr(C)]
pub struct DataView
{
pub struct DataView {
pub data: *const u8,
pub size: size_t
pub size: size_t,
}
#[repr(C)]
pub struct Value
{
_opaque: [u8; 0]
pub struct Value {
_opaque: [u8; 0],
_thread_bound: std::marker::PhantomData<*mut ()>,
}
#[repr(C)]
pub struct Blob
{
_opaque: [u8; 0]
pub struct Blob {
_opaque: [u8; 0],
_thread_bound: std::marker::PhantomData<*mut ()>,
}
#[repr(C)]
#[repr(C, align(4))]
#[derive(PartialEq)]
pub struct InfoHash
{
pub struct InfoHash {
pub d: [u8; HASH_LEN],
}
#[repr(C)]
pub struct PkId
{
#[repr(C, align(4))]
pub struct PkId {
pub d: [u8; PKID_LEN],
}
#[repr(C)]
pub struct PublicKey
{
_opaque: [u8; 0]
pub struct PublicKey {
_opaque: [u8; 0],
_thread_bound: std::marker::PhantomData<*mut ()>,
}
#[repr(C)]
pub struct PrivateKey
{
_opaque: [u8; 0]
pub struct PrivateKey {
_opaque: [u8; 0],
_thread_bound: std::marker::PhantomData<*mut ()>,
}
#[repr(C)]
pub struct OpToken
{
_opaque: [u8; 0]
pub struct OpToken {
_opaque: [u8; 0],
_thread_bound: std::marker::PhantomData<*mut ()>,
}
#[repr(C)]
pub struct DhtRunner
{
_opaque: [u8; 0]
pub struct DhtRunner {
_opaque: [u8; 0],
_thread_bound: std::marker::PhantomData<*mut ()>,
}
#[repr(C)]
pub struct DhtNodeConfig
{
pub struct DhtNodeConfig {
pub node_id: InfoHash,
pub network: u32,
pub is_bootstrap: bool,
@@ -76,28 +71,25 @@ pub struct DhtNodeConfig
}
#[repr(C)]
pub struct DhtCertificate
{
_opaque: [u8; 0]
pub struct DhtCertificate {
_opaque: [u8; 0],
_thread_bound: std::marker::PhantomData<*mut ()>,
}
#[repr(C)]
pub struct DhtIdentity
{
pub struct DhtIdentity {
pub privatekey: *mut PrivateKey,
pub certificate: *mut DhtCertificate,
}
#[repr(C)]
pub struct DhtSecureConfig
{
pub struct DhtSecureConfig {
pub node_config: DhtNodeConfig,
pub id: DhtIdentity,
}
#[repr(C)]
pub struct DhtRunnerConfig
{
pub struct DhtRunnerConfig {
pub dht_config: DhtSecureConfig,
pub threaded: bool,
pub proxy_server: *const c_char,
@@ -113,17 +105,55 @@ pub struct DhtRunnerConfig
pub log: bool,
}
pub type CertificateCallback = extern "C" fn(*const DhtCertificate, *mut c_void);
#[repr(C)]
pub struct DhtRunnerContext {
pub status_cb: Option<extern "C" fn(c_int, c_int, *mut c_void)>,
pub certificate_store_cb:
Option<extern "C" fn(*const PkId, CertificateCallback, *mut c_void, *mut c_void)>,
pub released_cb: Option<extern "C" fn(*mut c_void)>,
pub user_data: *mut c_void,
}
#[repr(C)]
pub struct QueryResult {
pub fields: u32,
pub id: u64,
pub value_type: u16,
pub seq: u16,
pub owner: InfoHash,
pub user_type: DataView,
}
#[repr(C)]
pub struct PushToken {
_opaque: [u8; 0],
_thread_bound: std::marker::PhantomData<*mut ()>,
}
#[link(name = "opendht-c")]
extern {
unsafe extern "C" {
// dht::Value
pub fn dht_value_new(data: *const u8, size: size_t) -> *mut Value;
pub fn dht_value_with_id_new(data: *const u8, size: size_t, id: u64) -> *mut Value;
pub fn dht_value_ref(data: *const Value) -> *mut Value;
pub fn dht_value_copy(data: *const Value) -> *mut Value;
pub fn dht_value_unref(data: *mut Value);
pub fn dht_value_get_data(data: *const Value) -> DataView;
pub fn dht_value_get_id(data: *const Value) -> u64;
pub fn dht_value_get_owner(data: *const Value) -> *mut PublicKey;
pub fn dht_value_get_recipient(data: *const Value) -> InfoHash;
pub fn dht_value_get_user_type(data: *const Value) -> *const c_char;
pub fn dht_value_set_user_type(data: *mut Value, user_type: *const c_char);
pub fn dht_value_get_type(data: *const Value) -> u16;
pub fn dht_value_set_type(data: *mut Value, value_type: u16);
pub fn dht_value_get_seq(data: *const Value) -> u16;
pub fn dht_value_set_seq(data: *mut Value, seq: u16);
pub fn dht_value_get_priority(data: *const Value) -> c_uint;
pub fn dht_value_set_priority(data: *mut Value, priority: c_uint);
pub fn dht_value_get_push_type(data: *const Value) -> *const c_char;
pub fn dht_value_set_push_type(data: *mut Value, push_type: *const c_char);
// dht::Blob
pub fn dht_blob_get_data(data: *const Blob) -> DataView;
@@ -132,8 +162,9 @@ extern {
// dht::InfoHash
pub fn dht_infohash_print(h: *const InfoHash) -> *const c_char;
pub fn dht_infohash_random(h: *mut InfoHash);
pub fn dht_infohash_get(h: *mut InfoHash, dat: *mut u8, dat_size: size_t);
pub fn dht_infohash_get(h: *mut InfoHash, dat: *const u8, dat_size: size_t);
pub fn dht_infohash_from_hex(h: *mut InfoHash, dat: *const c_char);
pub fn dht_infohash_from_hex_null(h: *mut InfoHash, dat: *const c_char);
pub fn dht_infohash_is_zero(j: *const InfoHash) -> bool;
// dht::PkId
@@ -142,27 +173,56 @@ extern {
// dht::crypto::PublicKey
pub fn dht_publickey_import(dat: *const u8, dat_size: size_t) -> *mut PublicKey;
pub fn dht_publickey_delete(pk: *mut PublicKey);
pub fn dht_publickey_unpack(pk: *mut PublicKey, dat: *const u8, dat_size: size_t) -> c_int;
pub fn dht_publickey_pack(pk: *mut PublicKey, out: *const c_char, out_size: size_t) -> c_int;
pub fn dht_publickey_export_blob(pk: *const PublicKey) -> *mut Blob;
pub fn dht_publickey_unpack(pk: *mut PublicKey, data: *const u8, size: size_t) -> c_int;
pub fn dht_publickey_get_id(pk: *const PublicKey) -> InfoHash;
pub fn dht_publickey_get_long_id(pk: *const PublicKey) -> PkId;
pub fn dht_publickey_check_signature(pk: *const PublicKey, data: *const c_char, data_size: size_t, signature: *const c_char, signature_size: size_t) -> bool;
pub fn dht_publickey_encrypt(pk: *const PublicKey, data: *const c_char, data_size: size_t) -> *mut Blob;
pub fn dht_publickey_check_signature(
pk: *const PublicKey,
data: *const c_char,
data_size: size_t,
signature: *const c_char,
signature_size: size_t,
) -> bool;
pub fn dht_publickey_encrypt(
pk: *const PublicKey,
data: *const c_char,
data_size: size_t,
) -> *mut Blob;
// dht::crypto::PrivateKey
pub fn dht_privatekey_generate(key_length_bits: c_uint) -> *mut PrivateKey;
pub fn dht_privatekey_import(dat: *const u8, data_size: size_t, password: *const c_char) -> *mut PrivateKey;
pub fn dht_privatekey_generate_ec() -> *mut PrivateKey;
pub fn dht_privatekey_import(
dat: *const u8,
data_size: size_t,
password: *const c_char,
) -> *mut PrivateKey;
pub fn dht_privatekey_get_publickey(pk: *const PrivateKey) -> *mut PublicKey;
pub fn dht_privatekey_export_blob(pk: *const PrivateKey, password: *const c_char) -> *mut Blob;
pub fn dht_privatekey_sign(pk: *const PrivateKey, data: *const u8, size: size_t) -> *mut Blob;
pub fn dht_privatekey_decrypt(
pk: *const PrivateKey,
data: *const c_char,
size: size_t,
) -> *mut Blob;
pub fn dht_privatekey_delete(pk: *mut PrivateKey);
// dht::crypto::Certificate
pub fn dht_certificate_import(dat: *const u8, dat_size: size_t) -> *mut DhtCertificate;
pub fn dht_certificate_export(cert: *const DhtCertificate) -> *mut Blob;
pub fn dht_certificate_ref(cert: *const DhtCertificate) -> *mut DhtCertificate;
pub fn dht_certificate_get_id(cert: *const DhtCertificate) -> InfoHash;
pub fn dht_certificate_get_long_id(cert: *const DhtCertificate) -> PkId;
pub fn dht_certificate_get_publickey(cert: *const DhtCertificate) -> *mut PublicKey;
pub fn dht_certificate_delete(cert: *mut DhtCertificate);
pub fn dht_identity_generate(common_name: *const c_char, ca: *const DhtIdentity) -> DhtIdentity;
pub fn dht_identity_generate(common_name: *const c_char, ca: *const DhtIdentity)
-> DhtIdentity;
pub fn dht_identity_generate_ec(
common_name: *const c_char,
ca: *const DhtIdentity,
) -> DhtIdentity;
pub fn dht_identity_delete(ca: *mut DhtIdentity);
// dht::OpToken
@@ -174,37 +234,142 @@ extern {
pub fn dht_runner_get_id(dht: *const DhtRunner) -> InfoHash;
pub fn dht_runner_get_node_id(dht: *const DhtRunner) -> InfoHash;
pub fn dht_runner_delete(dht: *mut DhtRunner);
pub fn dht_runner_run(dht: *mut DhtRunner, port: in_port_t);
pub fn dht_runner_run_config(dht: *mut DhtRunner, port: in_port_t, config: *const DhtRunnerConfig);
pub fn dht_runner_run(dht: *mut DhtRunner, port: in_port_t) -> c_int;
pub fn dht_runner_run_config(
dht: *mut DhtRunner,
port: in_port_t,
config: *const DhtRunnerConfig,
) -> c_int;
pub fn dht_runner_run_bind(
dht: *mut DhtRunner,
ipv4: *const c_char,
ipv6: *const c_char,
port: in_port_t,
config: *const DhtRunnerConfig,
context: *const DhtRunnerContext,
) -> c_int;
pub fn dht_runner_connectivity_changed(dht: *mut DhtRunner) -> c_int;
pub fn dht_runner_register_type(
dht: *mut DhtRunner,
id: u16,
name: *const c_char,
expiration_seconds: u64,
insecure: bool,
) -> c_int;
pub fn dht_runner_get_publickey(dht: *const DhtRunner) -> *mut PublicKey;
pub fn dht_runner_register_certificate(
dht: *mut DhtRunner,
cert: *const DhtCertificate,
) -> c_int;
pub fn dht_runner_find_certificate(
dht: *mut DhtRunner,
id: *const PkId,
callback: CertificateCallback,
released: extern "C" fn(*mut c_void),
data: *mut c_void,
);
pub fn dht_runner_bootstrap(dht: *mut DhtRunner, host: *const c_char, service: *const c_char);
pub fn dht_runner_get(dht: *mut DhtRunner, h: *const InfoHash,
get_cb: extern fn(*mut Value, *mut c_void) -> bool,
done_cb: extern fn(bool, *mut c_void),
cb_user_data: *mut c_void);
pub fn dht_runner_put(dht: *mut DhtRunner, h: *const InfoHash, v: *const Value,
done_cb: extern fn(bool, *mut c_void),
cb_user_data: *mut c_void,
permanent: bool);
pub fn dht_runner_put_signed(dht: *mut DhtRunner, h: *const InfoHash, v: *const Value,
done_cb: extern fn(bool, *mut c_void),
cb_user_data: *mut c_void,
permanent: bool);
pub fn dht_runner_put_encrypted(dht: *mut DhtRunner, h: *const InfoHash,
to: *const PkId, v: *const Value,
done_cb: extern fn(bool, *mut c_void),
cb_user_data: *mut c_void,
permanent: bool);
pub fn dht_runner_put_permanent(dht: *mut DhtRunner, h: *const InfoHash, v: *const Value,
done_cb: extern fn(bool, *mut c_void),
cb_user_data: *mut c_void);
pub fn dht_runner_get(
dht: *mut DhtRunner,
h: *const InfoHash,
get_cb: extern "C" fn(*const Value, *mut c_void) -> bool,
done_cb: extern "C" fn(bool, *mut c_void),
cb_user_data: *mut c_void,
);
pub fn dht_runner_get_filtered(
dht: *mut DhtRunner,
h: *const InfoHash,
filter: *const c_char,
callback: extern "C" fn(*const Value, *mut c_void) -> bool,
done: extern "C" fn(bool, *mut c_void),
data: *mut c_void,
);
pub fn dht_runner_query(
dht: *mut DhtRunner,
h: *const InfoHash,
query: *const c_char,
callback: extern "C" fn(*const QueryResult, *mut c_void) -> bool,
done: extern "C" fn(bool, *mut c_void),
data: *mut c_void,
);
pub fn dht_runner_put(
dht: *mut DhtRunner,
h: *const InfoHash,
v: *const Value,
done_cb: extern "C" fn(bool, *mut c_void),
cb_user_data: *mut c_void,
permanent: bool,
);
pub fn dht_runner_put_signed(
dht: *mut DhtRunner,
h: *const InfoHash,
v: *const Value,
done_cb: extern "C" fn(bool, *mut c_void),
cb_user_data: *mut c_void,
permanent: bool,
);
pub fn dht_runner_put_encrypted(
dht: *mut DhtRunner,
h: *const InfoHash,
to: *const PkId,
v: *const Value,
done_cb: extern "C" fn(bool, *mut c_void),
cb_user_data: *mut c_void,
permanent: bool,
);
pub fn dht_runner_cancel_put(dht: *mut DhtRunner, h: *const InfoHash, vid: u64);
pub fn dht_runner_listen(dht: *mut DhtRunner, h: *const InfoHash,
cb: extern fn(*mut Value, bool, *mut c_void) -> bool,
done_cb: extern fn(*mut c_void),
cb_user_data: *mut c_void) -> *mut OpToken;
pub fn dht_runner_cancel_listen(dht: *mut DhtRunner, h: *const InfoHash,
token: *const OpToken);
pub fn dht_runner_shutdown(dht: *mut DhtRunner, done_cb: extern fn(bool, *mut c_void),
cb_user_data: *mut c_void);
pub fn dht_runner_get_public_address(dht: *const DhtRunner) -> *mut *mut OsSocketAddr;
}
pub fn dht_runner_listen(
dht: *mut DhtRunner,
h: *const InfoHash,
cb: extern "C" fn(*const Value, bool, *mut c_void) -> bool,
done_cb: extern "C" fn(*mut c_void),
cb_user_data: *mut c_void,
) -> *mut OpToken;
pub fn dht_runner_listen_filtered(
dht: *mut DhtRunner,
h: *const InfoHash,
filter: *const c_char,
callback: extern "C" fn(*const Value, bool, *mut c_void) -> bool,
released: extern "C" fn(*mut c_void),
data: *mut c_void,
) -> *mut OpToken;
pub fn dht_runner_cancel_listen(dht: *mut DhtRunner, h: *const InfoHash, token: *mut OpToken);
pub fn dht_runner_shutdown(
dht: *mut DhtRunner,
done_cb: extern "C" fn(*mut c_void),
cb_user_data: *mut c_void,
);
pub fn dht_runner_get_public_address(dht: *const DhtRunner) -> *mut *mut sockaddr;
pub fn dht_sockaddr_array_delete(addresses: *mut *mut sockaddr);
pub fn dht_runner_loop(dht: *mut DhtRunner) -> c_int;
pub fn dht_runner_join(dht: *mut DhtRunner) -> c_int;
pub fn dht_runner_is_running(dht: *const DhtRunner) -> bool;
pub fn dht_runner_get_bound_port(dht: *const DhtRunner, family: sa_family_t) -> in_port_t;
pub fn dht_runner_ping(
dht: *mut DhtRunner,
addr: *mut sockaddr,
len: socklen_t,
callback: extern "C" fn(bool, *mut c_void),
data: *mut c_void,
);
pub fn dht_runner_set_push_notification_token(
dht: *mut DhtRunner,
value: *const c_char,
) -> c_int;
pub fn dht_runner_set_push_notification_topic(
dht: *mut DhtRunner,
value: *const c_char,
) -> c_int;
pub fn dht_runner_set_push_notification_platform(
dht: *mut DhtRunner,
value: *const c_char,
) -> c_int;
pub fn dht_runner_push_notification_received(
dht: *mut DhtRunner,
keys: *const *const c_char,
values: *const *const c_char,
count: size_t,
) -> *mut PushToken;
pub fn dht_push_token_poll(token: *mut PushToken, result: *mut c_int) -> c_int;
pub fn dht_push_token_delete(token: *mut PushToken);
}
+1 -1
View File
@@ -42,7 +42,7 @@ impl InfoHash {
let mut h = InfoHash::new();
unsafe {
let c_str = CString::new(data).unwrap();
dht_infohash_from_hex(&mut h, c_str.as_ptr());
dht_infohash_from_hex_null(&mut h, c_str.as_ptr());
}
h
}
+39 -2
View File
@@ -4,6 +4,34 @@
extern crate libc;
extern crate os_socketaddr;
macro_rules! native_handle {
($name:ident, $delete:ident) => {
pub struct $name {
raw: std::ptr::NonNull<crate::ffi::$name>,
}
impl $name {
pub(crate) unsafe fn from_raw(
raw: *mut crate::ffi::$name,
) -> std::io::Result<Box<Self>> {
let raw = std::ptr::NonNull::new(raw)
.ok_or_else(|| std::io::Error::other("native operation failed"))?;
Ok(Box::new(Self { raw }))
}
pub(crate) fn as_ptr(&self) -> *mut crate::ffi::$name {
self.raw.as_ptr()
}
}
impl Drop for $name {
fn drop(&mut self) {
unsafe { crate::ffi::$delete(self.as_ptr()) }
}
}
};
}
mod blob;
pub mod crypto;
mod dhtrunner;
@@ -12,8 +40,17 @@ mod infohash;
mod pkid;
mod value;
#[cfg(feature = "async")]
pub mod aio;
#[cfg(feature = "serde")]
pub mod typed;
pub use blob::Blob;
pub use dhtrunner::{ DhtRunner, DhtRunnerConfig, OpToken };
pub use dhtrunner::{
DhtNodeConfig, DhtRunner, DhtRunnerConfig, DhtSecureConfig, NodeStatus, OpToken,
PushNotification, PushNotificationResult, QueryResult,
};
pub use infohash::InfoHash;
pub use pkid::PkId;
pub use value::{ DataView, Value };
pub use value::{DataView, Value};
+201
View File
@@ -0,0 +1,201 @@
//! MessagePack-encoded typed operations, enabled by the `serde` feature.
//!
//! This feature enables `async` and adds methods to [`AsyncDhtRunner`]. Values
//! use compact MessagePack payloads, with no binding-specific envelope or
//! implicit schema version. Structs are encoded as arrays in field order.
//! Applications can include a version field in their records and use
//! `serde_bytes` for binary fields rather than arrays of integers.
//!
//! ```no_run
//! use futures::TryStreamExt;
//! use opendht::{aio::AsyncDhtRunner, InfoHash};
//! use serde::{Deserialize, Serialize};
//!
//! #[derive(Debug, Serialize, Deserialize)]
//! struct Service {
//! version: u32,
//! name: String,
//! port: u16,
//! #[serde(with = "serde_bytes")]
//! payload: Vec<u8>,
//! }
//!
//! async fn exchange(
//! runner: &AsyncDhtRunner,
//! key: &InfoHash,
//! ) -> Result<(), Box<dyn std::error::Error>> {
//! let mut updates = runner.listen_typed::<Service>(key)?;
//! let record = Service {
//! version: 1, name: "chat".into(), port: 4222, payload: vec![0, 0x80, 0xff],
//! };
//! runner.put_typed(key, &record, false).await?;
//! if let Some(event) = updates.try_next().await? {
//! println!("{}: {:?}", event.value.raw.id(), event.value.data);
//! }
//! drop(updates);
//! let records: Vec<_> = runner.get_typed::<Service>(key).try_collect().await?;
//! Ok(())
//! }
//! ```
use crate::aio::{AsyncDhtRunner, ValueEvent};
use crate::{InfoHash, Value};
use futures_core::Stream;
use serde::{de::DeserializeOwned, Serialize};
use std::error::Error;
use std::fmt;
use std::io;
use std::pin::Pin;
use std::task::{Context, Poll};
/// A decoded record together with its original payload and native metadata.
pub struct TypedValue<T> {
pub data: T,
pub raw: Box<Value>,
}
impl<T: fmt::Debug> fmt::Debug for TypedValue<T> {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
formatter
.debug_struct("TypedValue")
.field("data", &self.data)
.field("id", &self.raw.id())
.field("seq", &self.raw.seq())
.finish_non_exhaustive()
}
}
/// A decoded listener update, including whether the record expired.
#[derive(Debug)]
pub struct TypedEvent<T> {
pub value: TypedValue<T>,
pub expired: bool,
}
/// Typed operations distinguish native failures from payload format errors.
#[derive(Debug)]
#[non_exhaustive]
pub enum TypedError {
Operation(io::Error),
Serialize(rmp_serde::encode::Error),
Deserialize {
id: u64,
source: rmp_serde::decode::Error,
},
}
impl fmt::Display for TypedError {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Self::Operation(error) => write!(formatter, "DHT operation failed: {error}"),
Self::Serialize(error) => write!(formatter, "could not serialize typed value: {error}"),
Self::Deserialize { id, source } => {
write!(formatter, "could not deserialize DHT value {id}: {source}")
}
}
}
}
impl Error for TypedError {
fn source(&self) -> Option<&(dyn Error + 'static)> {
match self {
Self::Operation(error) => Some(error),
Self::Serialize(error) => Some(error),
Self::Deserialize { source, .. } => Some(source),
}
}
}
impl<T: DeserializeOwned> TryFrom<Box<Value>> for TypedValue<T> {
type Error = TypedError;
fn try_from(raw: Box<Value>) -> Result<Self, Self::Error> {
let data =
rmp_serde::from_slice(raw.as_slice()).map_err(|source| TypedError::Deserialize {
id: raw.id(),
source,
})?;
Ok(Self { data, raw })
}
}
impl<T: DeserializeOwned> TryFrom<ValueEvent> for TypedEvent<T> {
type Error = TypedError;
fn try_from(event: ValueEvent) -> Result<Self, Self::Error> {
Ok(Self {
value: event.value.try_into()?,
expired: event.expired,
})
}
}
struct DecodeStream<Source: Stream, Output> {
source: Source,
decode: fn(Source::Item) -> Result<Output, TypedError>,
}
impl<Source: Stream + Unpin, Output> Stream for DecodeStream<Source, Output> {
type Item = Result<Output, TypedError>;
fn poll_next(self: Pin<&mut Self>, context: &mut Context<'_>) -> Poll<Option<Self::Item>> {
let stream = self.get_mut();
Pin::new(&mut stream.source)
.poll_next(context)
.map(|item| item.map(stream.decode))
}
}
fn decode<Input, Output: TryFrom<Input, Error = TypedError>>(
input: io::Result<Input>,
) -> Result<Output, TypedError> {
input.map_err(TypedError::Operation)?.try_into()
}
impl AsyncDhtRunner {
/// Serialize a record as MessagePack and publish it when this future is polled.
///
/// Unlike untyped `put`, this operation is lazy. Serialization failure does
/// not start a native put. Dropping it after publication starts does not
/// undo the put. Use the untyped API to customize value IDs and metadata.
pub async fn put_typed<T: Serialize + ?Sized>(
&self,
hash: &InfoHash,
value: &T,
permanent: bool,
) -> Result<(), TypedError> {
let bytes = rmp_serde::to_vec(value).map_err(TypedError::Serialize)?;
self.put(hash, Value::from_bytes(&bytes), permanent)
.await
.map_err(TypedError::Operation)
}
/// Start a get and decode each MessagePack payload on the consumer's thread.
///
/// A malformed or incompatible record yields a per-value error; polling can
/// continue to subsequent records. Native errors remain terminal. The stream
/// does not borrow the runner or hash, and records need not be `Unpin`.
pub fn get_typed<T: DeserializeOwned>(
&self,
hash: &InfoHash,
) -> impl Stream<Item = Result<TypedValue<T>, TypedError>> + Unpin + use<T> {
DecodeStream {
source: self.get(hash),
decode: decode::<_, TypedValue<T>>,
}
}
/// Listen for MessagePack records, preserving metadata and expiration events.
///
/// Decode errors do not cancel the registration. Dropping this stream does
/// cancel it, including when a short-circuiting consumer stops on an error.
pub fn listen_typed<T: DeserializeOwned>(
&self,
hash: &InfoHash,
) -> io::Result<impl Stream<Item = Result<TypedEvent<T>, TypedError>> + Unpin + use<T>> {
Ok(DecodeStream {
source: self.listen(hash)?,
decode: decode::<_, TypedEvent<T>>,
})
}
}
+141 -42
View File
@@ -2,91 +2,190 @@
// SPDX-License-Identifier: MIT
#![allow(dead_code)]
use crate::ffi::*;
use crate::crypto::PublicKey;
use crate::{ffi, InfoHash};
use std::ffi::{CStr, CString};
use std::fmt;
use std::ffi::{ CString, IntoStringError };
use std::str;
use std::io;
use std::slice;
use std::str;
pub use crate::ffi::{DataView, Value};
pub use crate::ffi::DataView;
native_handle!(Value, dht_value_unref);
/// Rust owns an unaliased native value: constructors allocate it, received
/// values are deep-copied once, and boxed() deep-copies explicitly. Puts consume
/// the handle before native code retains it. Shared owner keys are const and
/// their lazy fingerprints are initialized before received values cross threads.
/// Mutation requires exclusive access; native storage may be freed on any thread.
/// Value is Send, not Sync.
unsafe impl Send for Value {}
impl AsRef<[u8]> for Value {
fn as_ref(&self) -> &[u8] {
self.as_slice()
}
}
impl fmt::Debug for Value {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
formatter
.debug_struct("Value")
.field("id", &self.id())
.field("seq", &self.seq())
.finish_non_exhaustive()
}
}
impl Value {
pub fn new(data: &str) -> Box<Value> {
pub(crate) unsafe fn copy_from_received(value: *const ffi::Value) -> io::Result<Box<Self>> {
let value = unsafe { Self::from_raw(ffi::dht_value_copy(value))? };
if let Some(owner) = value.owner() {
owner.id();
owner.long_id();
}
Ok(value)
}
pub fn with_id(data: &[u8], id: u64) -> Box<Value> {
unsafe {
Box::from_raw(dht_value_new(data.as_bytes().as_ptr(),
data.as_bytes().len()))
Self::from_raw(ffi::dht_value_with_id_new(data.as_ptr(), data.len(), id))
.expect("value allocation failed")
}
}
pub fn from_bytes(data: &Vec<u8>) -> Box<Value> {
pub fn new(data: &str) -> Box<Value> {
Self::from_bytes(data.as_bytes())
}
pub fn from_bytes(data: &[u8]) -> Box<Value> {
unsafe {
Box::from_raw(dht_value_new(data.as_ptr(),
data.len()))
Self::from_raw(ffi::dht_value_new(data.as_ptr(), data.len()))
.expect("value allocation failed")
}
}
pub fn id(&self) -> u64 {
unsafe {
dht_value_get_id(self)
}
unsafe { ffi::dht_value_get_id(self.as_ptr()) }
}
fn dataview(&self) -> DataView {
unsafe {
dht_value_get_data(self)
}
unsafe { ffi::dht_value_get_data(self.as_ptr()) }
}
pub fn as_bytes(&self) -> Vec<u8> {
unsafe {
let dv = dht_value_get_data(self);
let slice = slice::from_raw_parts_mut(dv.data as *mut _, dv.size);
slice.to_vec()
self.as_slice().to_vec()
}
/// Borrow the payload without allocating or copying it.
pub fn as_slice(&self) -> &[u8] {
let view = self.dataview();
if view.size == 0 {
&[]
} else {
unsafe { slice::from_raw_parts(view.data, view.size) }
}
}
pub fn boxed(&mut self) -> Box<Value> {
unsafe {
Box::from_raw(dht_value_ref(self))
}
pub fn boxed(&self) -> Box<Value> {
unsafe { Self::from_raw(ffi::dht_value_copy(self.as_ptr())).expect("value copy failed") }
}
pub fn recipient(&self) -> InfoHash {
unsafe {
dht_value_get_recipient(self)
}
unsafe { ffi::dht_value_get_recipient(self.as_ptr()) }
}
pub fn owner(&self) -> Option<Box<PublicKey>> {
unsafe {
let ptr = dht_value_get_owner(self);
if ptr.is_null() {
return None;
}
Some(Box::from_raw(ptr))
}
unsafe { PublicKey::from_raw(ffi::dht_value_get_owner(self.as_ptr())).ok() }
}
pub fn user_type(&self) -> Result<String, IntoStringError> {
pub fn user_type(&self) -> Result<&str, str::Utf8Error> {
unsafe { CStr::from_ptr(ffi::dht_value_get_user_type(self.as_ptr())).to_str() }
}
pub fn set_user_type(&mut self, value: &str) -> io::Result<()> {
let value = CString::new(value)?;
unsafe {
CString::from_raw(dht_value_get_user_type(self) as *mut _).into_string()
ffi::dht_value_set_user_type(self.as_ptr(), value.as_ptr());
}
Ok(())
}
pub fn value_type(&self) -> u16 {
unsafe { ffi::dht_value_get_type(self.as_ptr()) }
}
pub fn set_value_type(&mut self, value: u16) {
unsafe {
ffi::dht_value_set_type(self.as_ptr(), value);
}
}
}
impl Drop for Value {
fn drop(&mut self) {
pub fn seq(&self) -> u16 {
unsafe { ffi::dht_value_get_seq(self.as_ptr()) }
}
pub fn set_seq(&mut self, value: u16) {
unsafe {
dht_value_unref(&mut *self)
ffi::dht_value_set_seq(self.as_ptr(), value);
}
}
pub fn priority(&self) -> u32 {
unsafe { ffi::dht_value_get_priority(self.as_ptr()) }
}
pub fn set_priority(&mut self, value: u32) {
unsafe {
ffi::dht_value_set_priority(self.as_ptr(), value);
}
}
pub fn push_type(&self) -> Result<&str, str::Utf8Error> {
unsafe { CStr::from_ptr(ffi::dht_value_get_push_type(self.as_ptr())).to_str() }
}
pub fn set_push_type(&mut self, value: &str) -> io::Result<()> {
let value = CString::new(value)?;
unsafe {
ffi::dht_value_set_push_type(self.as_ptr(), value.as_ptr());
}
Ok(())
}
}
impl fmt::Display for Value {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
unsafe {
let dataview = self.dataview();
if dataview.size == 0 {
return Ok(());
}
let slice = slice::from_raw_parts(dataview.data, dataview.size);
write!(f, "{}", str::from_utf8(slice).unwrap_or(""))
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn value_moves_between_threads_without_copying_payload() {
let mut value = Value::from_bytes(b"native\0payload\xff");
value.set_user_type("borrowed metadata").unwrap();
let view = unsafe { ffi::dht_value_get_data(value.as_ptr()) };
let value = std::thread::spawn(move || {
value.set_seq(7);
value
})
.join()
.unwrap();
assert_eq!(value.as_slice().as_ptr(), view.data);
assert_eq!(value.as_slice(), b"native\0payload\xff");
assert_eq!(value.seq(), 7);
let user_type: &str = value.user_type().unwrap();
assert_eq!(user_type, "borrowed metadata");
}
#[test]
fn empty_value_payload_is_borrowed_safely() {
let value = Value::from_bytes(&[]);
assert!(value.as_slice().is_empty());
assert_eq!(value.as_slice(), &[]);
}
}
File diff suppressed because it is too large. Load diff
+15
View File
@@ -36,4 +36,19 @@ base64_decode(std::string_view str)
return buffer;
}
std::vector<unsigned char>
base64_default_or_url_decode(std::string_view str)
{
std::vector<unsigned char> buffer(simdutf::maximal_binary_length_from_base64(str.data(), str.size()));
simdutf::result r = simdutf::base64_to_binary(str.data(),
str.size(),
(char*) buffer.data(),
simdutf::base64_options::base64_default_or_url);
if (r.error) {
throw std::invalid_argument(concat("Invalid base64 input: "sv, simdutf::error_to_string(r.error)));
} else {
buffer.resize(r.count);
}
return buffer;
}
} // namespace dht
+8 -1
View File
@@ -21,5 +21,12 @@ std::string base64_encode(const std::vector<unsigned char>& str);
* @return a base64-decoded buffer
*/
std::vector<unsigned char> base64_decode(std::string_view str);
/**
* Decode a buffer in base64 or url style base64 (including '-' '_' ).
*
* @param str the input buffer
* @return a base64-decoded buffer
*/
std::vector<unsigned char> base64_default_or_url_decode(std::string_view str);
} // namespace dht
} // namespace dht
+41 -76
View File
@@ -571,6 +571,12 @@ PublicKey::toString() const
}
if (err != GNUTLS_E_SUCCESS)
throw CryptoException(std::string("Could not print public key: ") + gnutls_strerror(err));
// GnuTLS reports the PEM size including the terminating null byte, which
// does not belong to the text and would end up embedded in whatever the
// string is copied into, such as the "owner" field of a serialized value.
ret.resize(sz);
if (not ret.empty() and ret.back() == '\0')
ret.pop_back();
return ret;
}
@@ -592,7 +598,9 @@ PublicKey::checkSignature(const uint8_t* data, size_t data_len, const uint8_t* s
return false;
const gnutls_datum_t sig {(uint8_t*) signature, (unsigned) signature_len};
const gnutls_datum_t dat {(uint8_t*) data, (unsigned) data_len};
int rc = gnutls_pubkey_verify_data2(pk, GNUTLS_SIGN_RSA_SHA512, 0, &dat, &sig);
auto algorithm = gnutls_pk_to_sign(static_cast<gnutls_pk_algorithm_t>(gnutls_pubkey_get_pk_algorithm(pk, nullptr)),
GNUTLS_DIG_SHA512);
int rc = gnutls_pubkey_verify_data2(pk, algorithm, 0, &dat, &sig);
return rc >= 0;
}
@@ -1288,14 +1296,16 @@ PrivateKey::generate(unsigned key_length, gnutls_pk_algorithm_t algo)
}
PrivateKey
PrivateKey::generateEC()
PrivateKey::generateEC(gnutls_ecc_curve_t curve)
{
gnutls_x509_privkey_t key;
if (gnutls_x509_privkey_init(&key) != GNUTLS_E_SUCCESS)
throw CryptoException("Unable to initialize private key.");
int err = gnutls_x509_privkey_generate(key,
GNUTLS_PK_EC,
gnutls_sec_param_to_pk_bits(GNUTLS_PK_EC, GNUTLS_SEC_PARAM_ULTRA),
curve == GNUTLS_ECC_CURVE_INVALID
? gnutls_sec_param_to_pk_bits(GNUTLS_PK_EC, GNUTLS_SEC_PARAM_ULTRA)
: GNUTLS_CURVE_TO_BITS(curve),
0);
if (err != GNUTLS_E_SUCCESS) {
gnutls_x509_privkey_deinit(key);
@@ -1396,8 +1406,11 @@ setRandomSerial(gnutls_x509_crt_t cert)
{
std::random_device rdev;
std::uniform_int_distribution<int64_t> dist {1};
int64_t cert_serial = dist(rdev);
gnutls_x509_crt_set_serial(cert, &cert_serial, sizeof(cert_serial));
const auto cert_serial = dist(rdev);
std::array<uint8_t, sizeof(cert_serial)> packed_serial;
for (size_t i = 0; i < packed_serial.size(); i++)
packed_serial[i] = cert_serial >> (8 * (packed_serial.size() - 1 - i));
gnutls_x509_crt_set_serial(cert, packed_serial.data(), packed_serial.size());
}
Certificate
@@ -1648,24 +1661,8 @@ OcspResponse::toString(const bool compact) const
gnutls_ocsp_cert_status_t
OcspResponse::getCertificateStatus() const
{
int ret;
unsigned int status;
// MSVC/vcpkg GnuTLS cast workaround
#if _WIN32 && (GNUTLS_VERSION_NUMBER == 0x030807)
ret = gnutls_ocsp_resp_get_single(response,
0,
NULL,
NULL,
NULL,
NULL,
reinterpret_cast<gnutls_ocsp_cert_status_t*>(&status),
NULL,
NULL,
NULL,
NULL);
#else
ret = gnutls_ocsp_resp_get_single(response, 0, NULL, NULL, NULL, NULL, &status, NULL, NULL, NULL, NULL);
#endif
int ret = gnutls_ocsp_resp_get_single(response, 0, NULL, NULL, NULL, NULL, &status, NULL, NULL, NULL, NULL);
if (ret < 0)
throw CryptoException(gnutls_strerror(ret));
return (gnutls_ocsp_cert_status_t) status;
@@ -1724,22 +1721,7 @@ OcspResponse::verifyDirect(const Certificate& crt, const Blob& nonce)
// Check certificate revocation status
unsigned int status_ocsp;
// MSVC/vcpkg GnuTLS cast workaround
#if _WIN32 && (GNUTLS_VERSION_NUMBER == 0x030807)
ret = gnutls_ocsp_resp_get_single(response,
0,
NULL,
NULL,
NULL,
NULL,
reinterpret_cast<gnutls_ocsp_cert_status_t*>(&status_ocsp),
NULL,
NULL,
NULL,
NULL);
#else
ret = gnutls_ocsp_resp_get_single(response, 0, NULL, NULL, NULL, NULL, &status_ocsp, NULL, NULL, NULL, NULL);
#endif
if (ret < 0)
throw CryptoException(gnutls_strerror(ret));
return (gnutls_ocsp_cert_status_t) status_ocsp;
@@ -1873,34 +1855,8 @@ RevocationList::getUpdateTime() const
return std::chrono::system_clock::from_time_t(t);
}
enum class Endian : uint32_t { LITTLE = 0, BIG = 1 };
template<typename T>
T
endian(T w, Endian endian = Endian::BIG)
{
// this gets optimized out into if (endian == host_endian) return w;
union {
uint64_t quad;
uint32_t islittle;
} t;
t.quad = 1;
if (t.islittle ^ (uint32_t) endian)
return w;
T r = 0;
// decent compilers will unroll this (gcc)
// or even convert straight into single bswap (clang)
for (size_t i = 0; i < sizeof(r); i++) {
r <<= 8;
r |= w & 0xff;
w >>= 8;
}
return r;
}
void
RevocationList::sign(const PrivateKey& key, const Certificate& ca, duration validity)
RevocationList::sign(const PrivateKey& key, const Certificate& ca, duration validity, uint64_t crlNumber)
{
if (auto err = gnutls_x509_crl_set_version(crl, 2))
throw CryptoException(std::string("Unable to set CRL version: ") + gnutls_strerror(err));
@@ -1910,19 +1866,28 @@ RevocationList::sign(const PrivateKey& key, const Certificate& ca, duration vali
throw CryptoException(std::string("Unable to set CRL update time: ") + gnutls_strerror(err));
if (auto err = gnutls_x509_crl_set_next_update(crl, std::chrono::system_clock::to_time_t(next_update)))
throw CryptoException(std::string("Unable to set CRL next update time: ") + gnutls_strerror(err));
uint64_t number {0};
size_t number_sz {sizeof(number)};
unsigned critical {0};
gnutls_x509_crl_get_number(crl, &number, &number_sz, &critical);
uint64_t number {crlNumber};
if (number == 0) {
// initialize to a random number
number_sz = sizeof(number);
std::random_device rdev;
std::generate_n((uint8_t*) &number, sizeof(number), std::bind(rand_byte, std::ref(rdev)));
} else
number = endian(endian(number) + 1);
if (auto err = gnutls_x509_crl_set_number(crl, &number, sizeof(number)))
throw CryptoException(std::string("Unable to set CRL update time: ") + gnutls_strerror(err));
// The CRL Number extension holds a DER integer of variable length, so it must be
// read as a big-endian byte string rather than straight into an uint64_t.
auto current = getNumber();
for (uint8_t byte : current)
number = (number << 8) | byte;
if (number == 0) {
// Initialize to a random number.
std::random_device rdev;
do {
std::generate_n((uint8_t*) &number, sizeof(number), std::bind(rand_byte, std::ref(rdev)));
} while (number == 0);
} else
number++;
}
// Store the number in network byte order.
uint8_t packedNumber[sizeof(number)];
for (size_t i = 0; i < sizeof(packedNumber); i++)
packedNumber[i] = (uint8_t) ((number >> (8 * (sizeof(packedNumber) - 1 - i))) & 0xff);
if (auto err = gnutls_x509_crl_set_number(crl, packedNumber, sizeof(packedNumber)))
throw CryptoException(std::string("Unable to set CRL number: ") + gnutls_strerror(err));
if (auto err = gnutls_x509_crl_sign2(crl, ca.cert, key.x509_key, GNUTLS_DIG_SHA512, 0))
throw CryptoException(std::string("Unable to sign certificate revocation list: ") + gnutls_strerror(err));
// to be able to actually use the CRL we need to serialize/deserialize it
+162 -126
View File
@@ -10,6 +10,7 @@
#include <msgpack.hpp>
#include <algorithm>
#include <filesystem>
#include <random>
#include <sstream>
#include <fstream>
@@ -27,6 +28,7 @@ constexpr duration Dht::LISTEN_EXPIRE_TIME;
constexpr duration Dht::LISTEN_EXPIRE_TIME_PUBLIC;
constexpr duration Dht::REANNOUNCE_MARGIN;
static constexpr size_t MAX_REQUESTS_PER_SEC {8 * 1024};
static constexpr size_t MAX_CRYPTO_PER_SEC {512};
static constexpr duration BOOTSTRAP_PERIOD_MAX {std::chrono::hours(24)};
NodeStatus
@@ -304,7 +306,7 @@ Dht::searchNodeGetDone(const net::Request& req, net::RequestAnswer&& answer, std
should not be sent anymore */
for (auto& g : sr->callbacks) {
auto& q = g.second.query;
if (q->isSatisfiedBy(*query) and q != query) {
if (q != query and g.second.isSatisfiedBy(*query)) {
auto dummy_req = std::make_shared<net::Request>();
dummy_req->cancel();
srn->getStatus[q] = std::move(dummy_req);
@@ -333,6 +335,30 @@ Dht::searchNodeGetExpired(const net::Request& status, bool over, std::weak_ptr<S
}
}
void
Dht::searchNodeRoutingDone(const net::Request& req, net::RequestAnswer&& answer, std::weak_ptr<Search> ws)
{
if (auto sr = ws.lock()) {
if (auto sn = sr->getNode(req.node); sn and sn->routingProbe.get() == &req) {
sr->insertNode(req.node, scheduler.time(), answer.ntoken);
scheduler.edit(sr->nextSearchStep, scheduler.time());
}
}
}
void
Dht::searchNodeRoutingExpired(const net::Request& req, bool over, std::weak_ptr<Search> ws)
{
if (auto sr = ws.lock()) {
if (auto sn = sr->getNode(req.node); sn and sn->routingProbe.get() == &req) {
sn->candidate = not over;
if (over)
sn->routingProbe.reset();
scheduler.edit(sr->nextSearchStep, scheduler.time());
}
}
}
void
Dht::paginate(std::weak_ptr<Search> ws, Sp<Query> query, SearchNode* n)
{
@@ -402,17 +428,18 @@ Dht::searchSendGetValues(Sp<Search> sr, SearchNode* pn, bool update)
std::weak_ptr<Search> ws = sr;
auto cb = sr->callbacks.begin();
static const auto ANY_QUERY = std::make_shared<Query>(Select {}, Where {}, true);
static const Get ANY_GET {time_point::min(), {}, std::make_shared<Query>(Select {}, Where {}, true), {}, {}, {}};
do { /* for all requests to send */
SearchNode* n = nullptr;
auto& query = sr->callbacks.empty() ? ANY_QUERY : cb->second.query;
const auto& get = sr->callbacks.empty() ? ANY_GET : cb->second;
const auto& query = get.query;
const time_point up = (not sr->callbacks.empty() and update) ? sr->getLastGetTime(*query) : time_point::min();
if (pn and pn->canGet(now, up, query)) {
if (pn and pn->canGet(now, up, get)) {
n = pn;
} else {
for (auto& sn : sr->nodes) {
if (sn->canGet(now, up, query)) {
if (sn->canGet(now, up, get)) {
n = sn.get();
break;
}
@@ -437,6 +464,16 @@ Dht::searchSendGetValues(Sp<Search> sr, SearchNode* pn, bool update)
if (not n)
continue;
if (network_engine.needsSession(*n->node)) {
n->routingProbe = network_engine
.sendFindNode(n->node,
sr->id,
-1,
std::bind(&Dht::searchNodeRoutingDone, this, _1, _2, ws),
std::bind(&Dht::searchNodeRoutingExpired, this, _1, _2, ws));
return n;
}
if (query and not query->select.empty()) {
/* The request contains a select. No need to paginate… */
/* if (logger_)
@@ -529,8 +566,8 @@ Dht::searchSendAnnounceValue(const Sp<Search>& sr, unsigned syncLevel)
sn->node->toString(),
a.value->id);
auto created = a.permanent ? time_point::max() : a.created;
acked = {network_engine
.sendAnnounceValue(sn->node, sr->id, a.value, created, sn->token, onDone, onExpired),
acked = {network_engine.sendAnnounceValue(
sn->node, sr->id, a.value, created, sn->token.value_or(Blob {}), onDone, onExpired),
next_refresh_time};
} else if (hasValue and a.permanent) {
if (logger_)
@@ -542,7 +579,7 @@ Dht::searchSendAnnounceValue(const Sp<Search>& sr, unsigned syncLevel)
sn->node,
sr->id,
a.value->id,
sn->token,
sn->token.value_or(Blob {}),
onDone,
[this,
ws,
@@ -562,7 +599,8 @@ Dht::searchSendAnnounceValue(const Sp<Search>& sr, unsigned syncLevel)
sr->id,
v,
created,
sn->token,
sn->token.value_or(
Blob {}),
onDone,
onExpired),
next_refresh_time};
@@ -621,8 +659,13 @@ Dht::searchSendAnnounceValue(const Sp<Search>& sr, unsigned syncLevel)
sr->id.toString(),
n.node->toString(),
a.value->id);
n.acked[a.value->id] = {network_engine.sendAnnounceValue(
n.node, sr->id, a.value, a.created, n.token, onDone, onExpired),
n.acked[a.value->id] = {network_engine.sendAnnounceValue(n.node,
sr->id,
a.value,
a.created,
n.token.value_or(Blob {}),
onDone,
onExpired),
now + getType(a.value->type).expiration};
}
}
@@ -662,8 +705,8 @@ Dht::searchSynchedNodeListen(const Sp<Search>& sr, SearchNode& n)
if (auto sr = ws.lock()) {
scheduler.edit(sr->nextSearchStep, scheduler.time());
if (auto sn = sr->getNode(node)) {
if (not answer.ntoken.empty())
sn->refreshToken(answer.ntoken);
if (answer.ntoken)
sn->refreshToken(*answer.ntoken);
sn->onValues(query, std::move(answer), types, scheduler);
}
}
@@ -697,7 +740,7 @@ Dht::searchSynchedNodeListen(const Sp<Search>& sr, SearchNode& n)
n.node,
sr->id,
*query,
n.token,
n.token.value_or(Blob {}),
r->second.socketId,
[this, ws, query](const net::Request& req, net::RequestAnswer&& answer) mutable {
/* on done */
@@ -1404,12 +1447,10 @@ Dht::storageChanged(const InfoHash& id, Storage& st, const Sp<Value>& v, bool ne
node_listeners.first->toString());
std::vector<Sp<Value>> vals;
vals.push_back(v);
Blob ntoken = makeToken(node_listeners.first->getAddr(), false);
network_engine.tellListener(node_listeners.first,
l.first,
id,
0,
ntoken,
{},
{},
std::move(vals),
@@ -1506,7 +1547,6 @@ Dht::storageAddListener(const InfoHash& id, const Sp<Node>& node, size_t socket_
socket_id,
id,
WANT4 | WANT6,
makeToken(node->getAddr(), false),
dht4.buckets.findClosestNodes(id, now, TARGET_NODES),
dht6.buckets.findClosestNodes(id, now, TARGET_NODES),
std::move(vals),
@@ -1561,8 +1601,7 @@ Dht::storageRemoved(const InfoHash& id, Storage& st, const std::vector<Sp<Value>
logger_->warn("[store {}] [node {}] Sending expired",
id.to_view(),
node_listeners.first->toString());
Blob ntoken = makeToken(node_listeners.first->getAddr(), false);
network_engine.tellListenerExpired(node_listeners.first, l.first, id, ntoken, ids, l.second.version);
network_engine.tellListenerExpired(node_listeners.first, l.first, id, ids, l.second.version);
}
}
}
@@ -1644,59 +1683,6 @@ Dht::connectivityChanged(sa_family_t af)
startBootstrap(); // will only happen if disconnected
}
void
Dht::rotateSecrets()
{
oldsecret = secret;
secret = std::uniform_int_distribution<uint64_t> {}(rd);
uniform_duration_distribution<> time_dist(std::chrono::minutes(15), std::chrono::minutes(45));
auto rotate_secrets_time = scheduler.time() + time_dist(rd);
scheduler.add(rotate_secrets_time, std::bind(&Dht::rotateSecrets, this));
}
Blob
Dht::makeToken(const SockAddr& addr, bool old) const
{
const void* ip;
size_t iplen;
in_port_t port;
auto family = addr.getFamily();
if (family == AF_INET) {
const auto& sin = addr.getIPv4();
ip = &sin.sin_addr;
iplen = 4;
port = sin.sin_port;
} else if (family == AF_INET6) {
const auto& sin6 = addr.getIPv6();
ip = &sin6.sin6_addr;
iplen = 16;
port = sin6.sin6_port;
} else {
return {};
}
const auto& c1 = old ? oldsecret : secret;
Blob data;
data.reserve(sizeof(secret) + sizeof(in_port_t) + iplen);
data.insert(data.end(), (uint8_t*) &c1, ((uint8_t*) &c1) + sizeof(c1));
data.insert(data.end(), (uint8_t*) ip, (uint8_t*) ip + iplen);
data.insert(data.end(), (uint8_t*) &port, ((uint8_t*) &port) + sizeof(in_port_t));
return crypto::hash(data, TOKEN_SIZE);
}
bool
Dht::tokenMatch(const Blob& token, const SockAddr& addr) const
{
if (not addr or token.size() != TOKEN_SIZE)
return false;
if (token == makeToken(addr, false))
return true;
if (token == makeToken(addr, true))
return true;
return false;
}
NodeStats
Dht::getNodesStats(sa_family_t af) const
{
@@ -1982,10 +1968,33 @@ fromDhtConfig(const Config& config)
{
net::NetworkConfig netConf;
netConf.network = config.network;
// global limiting should be triggered by at least 8 different IPs
auto peerLimit = [](ssize_t global, ssize_t peer) -> ssize_t {
return peer ? peer : (global < 0 ? -1 : global / 8);
};
netConf.max_req_per_sec = config.max_req_per_sec ? config.max_req_per_sec : MAX_REQUESTS_PER_SEC;
netConf.max_peer_req_per_sec = config.max_peer_req_per_sec ? config.max_peer_req_per_sec
: netConf.max_req_per_sec / 8;
netConf.max_peer_req_per_sec = peerLimit(netConf.max_req_per_sec, config.max_peer_req_per_sec);
// left at 0, the engine uses the limits above
netConf.max_unauth_req_per_sec = config.max_unauth_req_per_sec;
netConf.max_peer_unauth_req_per_sec = config.max_unauth_req_per_sec
? peerLimit(config.max_unauth_req_per_sec,
config.max_peer_unauth_req_per_sec)
: config.max_peer_unauth_req_per_sec;
netConf.max_crypto_per_sec = config.max_crypto_per_sec ? config.max_crypto_per_sec : MAX_CRYPTO_PER_SEC;
netConf.max_peer_crypto_per_sec = peerLimit(netConf.max_crypto_per_sec, config.max_peer_crypto_per_sec);
netConf.is_client = config.client_mode;
netConf.transport = config.transport;
auto& transport = netConf.transport;
if (transport.isPrivate() and not transport.enabled)
throw DhtException("Private networks require transport encryption");
if (transport.enabled and not transport.key)
transport.key = transport::KeyPair::generate();
if (transport.isPrivate()) {
if (transport.credential.key != transport.key.pub)
throw DhtException("Transport credential doesn't match the transport key");
if (not transport.credential.verify(transport.authorities))
throw DhtException("Invalid or expired transport credential");
}
return netConf;
}
@@ -2015,14 +2024,17 @@ Dht::Dht(std::unique_ptr<net::DatagramSocket>&& sock,
std::bind(&Dht::onPing, this, _1),
std::bind(&Dht::onFindNode, this, _1, _2, _3),
std::bind(&Dht::onGetValues, this, _1, _2, _3, _4),
std::bind(&Dht::onListen, this, _1, _2, _3, _4, _5, _6),
std::bind(&Dht::onAnnounce, this, _1, _2, _3, _4, _5),
std::bind(&Dht::onRefresh, this, _1, _2, _3, _4))
std::bind(&Dht::onListen, this, _1, _2, _3, _4, _5),
std::bind(&Dht::onAnnounce, this, _1, _2, _3, _4),
std::bind(&Dht::onRefresh, this, _1, _2, _3))
, persistPath(config.persist_path)
, is_bootstrap(config.is_bootstrap)
, maintain_storage(config.maintain_storage)
, public_stable(config.public_stable)
{
if (auto key = network_engine.getTransportKey())
myid = transport::nodeId(key->pub);
persistTransportKey = config.transport.enabled and not config.transport.key and not persistPath.empty();
scheduler.syncTime();
auto s = network_engine.getSocket();
if (not s or (not s->hasIPv4() and not s->hasIPv6()))
@@ -2041,10 +2053,6 @@ Dht::Dht(std::unique_ptr<net::DatagramSocket>&& sock,
uniform_duration_distribution<> time_dis {std::chrono::seconds(3), std::chrono::seconds(5)};
nextNodesConfirmation = scheduler.add(scheduler.time() + time_dis(rd), std::bind(&Dht::confirmNodes, this));
// Fill old secret
secret = std::uniform_int_distribution<uint64_t> {}(rd);
rotateSecrets();
if (not persistPath.empty())
loadState(persistPath);
@@ -2270,8 +2278,16 @@ Dht::onDisconnected()
void
Dht::bootstrap()
{
if (dht4.status != NodeStatus::Disconnected || dht6.status != NodeStatus::Disconnected)
if (dht4.status != NodeStatus::Disconnected || dht6.status != NodeStatus::Disconnected) {
// The job that led here has already run, and the scheduler does not
// clear the handle to a job it has executed. onDisconnected() reads a
// set handle as "a bootstrap is already scheduled" and does nothing,
// so leaving it set here means the node never tries to rejoin the
// network once it has lost it. Clearing it is what says that no
// bootstrap is pending any more.
bootstrapJob.reset();
return;
}
if (logger_)
logger_->debug("[{}] Bootstraping", myid.to_view());
bootstrap_pending = false;
@@ -2355,6 +2371,9 @@ Dht::confirmNodes()
std::vector<ValuesExport>
Dht::exportValues() const
{
const auto& now = scheduler.time();
const auto system_now = system_clock::now();
std::vector<ValuesExport> e {};
e.reserve(store.size());
for (const auto& h : store) {
@@ -2367,7 +2386,7 @@ Dht::exportValues() const
pk.pack_array(vals.size());
for (const auto& v : vals) {
pk.pack_array(4);
pk.pack(v.created.time_since_epoch().count());
pk.pack(to_system_time(v.created, now, system_now));
v.data->msgpack_pack(pk);
if (const auto& store_addr = v.store_bucket ? v.store_bucket->getAddr() : SockAddr {}) {
pk.pack_bin(store_addr.getLength());
@@ -2375,7 +2394,7 @@ Dht::exportValues() const
} else {
pk.pack_bin(0);
}
pk.pack(v.expiration.time_since_epoch().count());
pk.pack(to_system_time(v.expiration, now, system_now));
}
ve.second = {buffer.data(), buffer.data() + buffer.size()};
e.push_back(std::move(ve));
@@ -2387,6 +2406,7 @@ void
Dht::importValues(const std::vector<ValuesExport>& import)
{
const auto& now = scheduler.time();
const auto system_now = system_clock::now();
unsigned imported = 0, ignored = 0;
size_t imported_size = 0;
@@ -2411,8 +2431,15 @@ Dht::importValues(const std::vector<ValuesExport>& import)
Value tmp_val;
SockAddr store_addr;
try {
val_time = time_point {
time_point::duration {valel.via.array.ptr[0].as<time_point::duration::rep>()}};
const bool uses_system_time = valel.via.array.ptr[0].type == msgpack::type::EXT;
if (uses_system_time) {
val_time = from_system_time(valel.via.array.ptr[0].as<system_clock::time_point>(),
now,
system_now);
} else {
val_time = time_point {
time_point::duration {valel.via.array.ptr[0].as<time_point::duration::rep>()}};
}
tmp_val.msgpack_unpack(valel.via.array.ptr[1]);
if (valel.via.array.size >= 3) {
auto addr_blob = valel.via.array.ptr[2].as<Blob>();
@@ -2422,8 +2449,14 @@ Dht::importValues(const std::vector<ValuesExport>& import)
}
}
if (valel.via.array.size >= 4) {
expiration = time_point {
time_point::duration {valel.via.array.ptr[3].as<time_point::duration::rep>()}};
if (uses_system_time) {
expiration = from_system_time(valel.via.array.ptr[3].as<system_clock::time_point>(),
now,
system_now);
} else {
expiration = time_point {
time_point::duration {valel.via.array.ptr[3].as<time_point::duration::rep>()}};
}
}
} catch (const std::exception&) {
if (logger_)
@@ -2536,7 +2569,7 @@ Dht::onError(Sp<net::Request> req, net::DhtProtocolException e)
for (auto& n : sr->nodes) {
if (n->node != node)
continue;
n->token.clear();
n->token.reset();
n->last_get_reply = time_point::min();
for (auto& listen : n->listenStatus) {
if (listen.second.req and listen.second.req->pending())
@@ -2575,11 +2608,10 @@ Dht::onPing(Sp<Node>)
}
net::RequestAnswer
Dht::onFindNode(Sp<Node> node, const InfoHash& target, want_t want)
Dht::onFindNode(Sp<Node>, const InfoHash& target, want_t want)
{
const auto& now = scheduler.time();
net::RequestAnswer answer;
answer.ntoken = makeToken(node->getAddr(), false);
if (want & WANT4)
answer.nodes4 = dht4.buckets.findClosestNodes(target, now, TARGET_NODES);
if (want & WANT6)
@@ -2599,7 +2631,6 @@ Dht::onGetValues(Sp<Node> node, const InfoHash& hash, want_t, const Query& query
const auto& now = scheduler.time();
net::RequestAnswer answer {};
auto st = store.find(hash);
answer.ntoken = makeToken(node->getAddr(), false);
answer.nodes4 = dht4.buckets.findClosestNodes(hash, now, TARGET_NODES);
answer.nodes6 = dht6.buckets.findClosestNodes(hash, now, TARGET_NODES);
if (st != store.end() && not st->second.empty()) {
@@ -2623,7 +2654,7 @@ Dht::onGetValuesDone(const Sp<Node>& node, net::RequestAnswer& a, Sp<Search>& sr
logger_->d(sr->id, "[search %s] [node %s] Got reply to 'get' with %u nodes",
sr->id.toString().c_str(), node->toString().c_str(), a.nodes4.size()+a.nodes6.size());*/
if (not a.ntoken.empty()) {
if (a.ntoken) {
if (not a.values.empty() or not a.fields.empty()) {
if (logger_)
logger_->debug("[search {}] [node {}] Found {} values",
@@ -2633,7 +2664,7 @@ Dht::onGetValuesDone(const Sp<Node>& node, net::RequestAnswer& a, Sp<Search>& sr
for (auto& getp : sr->callbacks) { /* call all callbacks for this search */
auto& get = getp.second;
if (not(get.get_cb or get.query_cb)
or (orig_query and get.query and not get.query->isSatisfiedBy(*orig_query)))
or (orig_query and get.query and not get.isSatisfiedBy(*orig_query)))
continue;
if (get.query_cb) { /* in case of a request with query */
@@ -2689,7 +2720,7 @@ Dht::onGetValuesDone(const Sp<Node>& node, net::RequestAnswer& a, Sp<Search>& sr
}
net::RequestAnswer
Dht::onListen(Sp<Node> node, const InfoHash& hash, const Blob& token, size_t socket_id, const Query& query, int version)
Dht::onListen(Sp<Node> node, const InfoHash& hash, size_t socket_id, const Query& query, int version)
{
if (not hash) {
if (logger_)
@@ -2697,17 +2728,9 @@ Dht::onListen(Sp<Node> node, const InfoHash& hash, const Blob& token, size_t soc
throw net::DhtProtocolException {net::DhtProtocolException::NON_AUTHORITATIVE_INFORMATION,
net::DhtProtocolException::LISTEN_NO_INFOHASH};
}
if (not tokenMatch(token, node->getAddr())) {
if (logger_)
logger_->warn("[node {}] Incorrect token {} for 'listen'", node->toString(), hash.toString());
throw net::DhtProtocolException {net::DhtProtocolException::UNAUTHORIZED,
net::DhtProtocolException::LISTEN_WRONG_TOKEN};
}
Query q = query;
storageAddListener(hash, node, socket_id, std::move(q), version);
net::RequestAnswer answer {};
answer.ntoken = makeToken(node->getAddr(), false);
return answer;
return {};
}
void
@@ -2719,9 +2742,9 @@ Dht::onListenDone(const Sp<Node>& node, net::RequestAnswer& answer, Sp<Search>&
if (not sr->done) {
const auto& now = scheduler.time();
if (not answer.ntoken.empty())
if (answer.ntoken)
if (auto sn = sr->getNode(node))
sn->refreshToken(answer.ntoken);
sn->refreshToken(*answer.ntoken);
searchSendGetValues(sr);
scheduler.edit(sr->nextSearchStep, now);
}
@@ -2730,7 +2753,6 @@ Dht::onListenDone(const Sp<Node>& node, net::RequestAnswer& answer, Sp<Search>&
net::RequestAnswer
Dht::onAnnounce(Sp<Node> n,
const InfoHash& hash,
const Blob& token,
const std::vector<Sp<Value>>& values,
const time_point& creation_date)
{
@@ -2741,12 +2763,6 @@ Dht::onAnnounce(Sp<Node> n,
throw net::DhtProtocolException {net::DhtProtocolException::NON_AUTHORITATIVE_INFORMATION,
net::DhtProtocolException::PUT_NO_INFOHASH};
}
if (!tokenMatch(token, node.getAddr())) {
if (logger_)
logger_->warn("[node {}] Incorrect token {} for 'put'", node.toString(), hash.toString());
throw net::DhtProtocolException {net::DhtProtocolException::UNAUTHORIZED,
net::DhtProtocolException::PUT_WRONG_TOKEN};
}
{
// We store a value only if we think we're part of the
// SEARCH_NODES nodes around the target id.
@@ -2807,15 +2823,10 @@ Dht::onAnnounce(Sp<Node> n,
}
net::RequestAnswer
Dht::onRefresh(Sp<Node> node, const InfoHash& hash, const Blob& token, const Value::Id& vid)
Dht::onRefresh(Sp<Node> node, const InfoHash& hash, const Value::Id& vid)
{
using namespace net;
if (not tokenMatch(token, node->getAddr())) {
if (logger_)
logger_->warn("[node {}] Incorrect token {} for 'put'", node->toString(), hash.toString());
throw DhtProtocolException {DhtProtocolException::UNAUTHORIZED, DhtProtocolException::PUT_WRONG_TOKEN};
}
if (storageRefresh(hash, vid)) {
if (logger_)
logger_->debug("[store {}] [node {}] Refreshed value {:016x}", hash.toString(), node->toString(), vid);
@@ -2846,9 +2857,7 @@ Dht::storageRefresh(const InfoHash& id, Value::Id vid)
logger_->warn("[store {}] [node {}] Sending refresh",
id.toString(),
node_listeners.first->toString());
Blob ntoken = makeToken(node_listeners.first->getAddr(), false);
network_engine
.tellListenerRefreshed(node_listeners.first, l.first, id, ntoken, ids, l.second.version);
network_engine.tellListenerRefreshed(node_listeners.first, l.first, id, ids, l.second.version);
}
}
}
@@ -2881,8 +2890,10 @@ struct DhtState
InfoHash id;
std::vector<NodeExport> nodes;
std::vector<ValuesExport> values;
/* secret transport key, if generated by the node */
Blob transport_key;
MSGPACK_DEFINE_MAP(v, id, nodes, values)
MSGPACK_DEFINE_MAP(v, id, nodes, values, transport_key)
};
void
@@ -2892,8 +2903,26 @@ Dht::saveState(const std::string& path) const
state.id = myid;
state.nodes = exportNodes();
state.values = exportValues();
std::ofstream file(path);
msgpack::pack(file, state);
auto key = persistTransportKey ? network_engine.getTransportKey() : nullptr;
if (key)
state.transport_key.assign(key->secret.begin(), key->secret.end());
auto temporary = path + "." + InfoHash::getRandom().toString() + ".tmp";
try {
std::ofstream file;
file.exceptions(std::ios::failbit | std::ios::badbit);
file.open(temporary, std::ios::binary | std::ios::trunc);
if (key)
std::filesystem::permissions(temporary,
std::filesystem::perms::owner_read | std::filesystem::perms::owner_write);
msgpack::pack(file, state);
file.close();
std::filesystem::rename(temporary, path);
} catch (const std::exception& e) {
std::error_code ec;
std::filesystem::remove(temporary, ec);
if (logger_)
logger_->warn("Error saving state to {}: {}", path, e.what());
}
}
void
@@ -2922,8 +2951,15 @@ Dht::loadState(const std::string& path)
auto state = oh.get().as<DhtState>();
if (logger_)
logger_->debug("Importing {} nodes", state.nodes.size());
if (state.id)
if (persistTransportKey and state.transport_key.size() == transport::KEY_SIZE) {
transport::Key secret;
std::copy(state.transport_key.begin(), state.transport_key.end(), secret.begin());
auto key = transport::KeyPair::fromSecret(secret);
network_engine.setTransportKey(key);
myid = transport::nodeId(key.pub);
} else if (state.id and not network_engine.getTransportKey()) {
myid = state.id;
}
std::vector<Sp<Node>> tmpNodes;
tmpNodes.reserve(state.nodes.size());
for (const auto& node : state.nodes)
Loaded 100 of 148 files, more files were not shown because too many files have changed in this diff. Show more