Compare commits

..
340 Commits
Author SHA1 Message Date
Adrien Béraud cf378c1a0e update version to 3.2.0 2024-05-25 23:59:20 -04:00
Adrien Béraud 9a987a0586 ci: build proxy client for Autotools build 2024-05-25 23:59:20 -04:00
Adrien Béraud 8806960d2b http: migrate from http-parser to llhttp 2024-05-25 23:59:20 -04:00
Lanius-collaris 318d02c55a update http, dht_proxy_client
http: simplify Connection::async_connect()
dht_proxy_client: add listenKeepIdle setting
2024-05-25 23:57:52 -04:00
Lanius-collaris a69da52f89 update http.h, http.cpp 2024-05-25 23:57:52 -04:00
Lanius-collaris ac0ea6ac57 proxy client: decrease the frequency of keepalive for listen op 2024-05-25 23:57:52 -04:00
Lanius-collaris 88d65a55a7 http: allow to modify keepalive option 2024-05-25 23:57:52 -04:00
Lanius-collaris dffe7d91ee return to 386943de 2024-05-25 23:57:52 -04:00
Lanius-collaris da638b3ae6 cleanup 2024-05-25 23:57:52 -04:00
Lanius-collaris 379568ae0d Update http.cpp 2024-05-25 23:57:52 -04:00
Lanius-collaris 24adfbf2c4 Adjust TCP keepalive options for listen op 2024-05-25 23:57:52 -04:00
François-Simon Fauteux-Chapleau 89179cad4f network_utils: use poll instead of select
select is limited to file descriptor numbers below FD_SETSIZE (1024),
and calling FD_SET with a higher value can cause a buffer overflow. This
can be problematic for applications using OpenDHT as a library.
2024-05-22 11:57:38 -04:00
François-Simon Fauteux-Chapleau f2cee8e9ce thread pool: add function to detach threads 2024-04-23 14:02:32 -04:00
Adrien Béraud 386943de39 tests/proxy: use random port 2024-04-06 23:29:38 -04:00
Adrien Béraud 60b810af7e meson: disable long tests by default 2024-04-06 23:29:35 -04:00
Adrien Béraud aa2c0c2da9 meson: add test for PeerDiscovery 2024-04-06 23:01:41 -04:00
Adrien Béraud 62debfa58a meson: add test prefix for unit tests 2024-04-06 22:59:12 -04:00
Adrien Béraud 9741dba974 update version to 3.1.11 2024-04-06 22:26:56 -04:00
Adrien Béraud 93ac28ead2 meson: configure pkgconfig files 2024-04-06 20:10:33 -04:00
Adrien Beraud a0a3cd3028 proxy: add long test 2024-04-06 19:42:42 -04:00
Adrien Béraud 611ecbd06a meson: add ability to build static or shared 2024-04-06 19:18:27 -04:00
Adrien Béraud 70c465b006 meson: link tests with msgpack 2024-04-06 18:43:36 -04:00
Adrien Beraud 59c8d9b9be op cache: return cache result in onValue 2024-04-06 18:24:16 -04:00
Adrien Beraud 7f70fcd316 proxy client: cancel http request on listen cancel 2024-04-06 18:24:16 -04:00
Adrien Beraud 17e9ad5093 meson: fix build with proxy server 2024-04-06 15:51:11 -04:00
Adrien Beraud 42435bdd4f meson: allow to build unit tests 2024-04-06 15:51:11 -04:00
Adrien Beraud c125a2af1f meson: build thread pool 2024-04-06 15:51:11 -04:00
Andreas Traczyk c0a1356049 dht_proxy_server: use lambda instead of std::bind for listen callback
Avoids issues compiling with MSC.

Also move PushSessionContext definition outside the OPENDHT_PUSH_NOTIFICATIONS condition.
2024-04-04 14:20:01 -04:00
Adrien Béraud 5e6679139b update version to 3.1.10 2024-04-03 14:35:42 -04:00
Andreas Traczyk c118f85886 Merge pull request #694 from savoirfairelinux/push-value-IDs
provide value IDs for push notifications
2024-04-03 11:52:01 -04:00
Andreas Traczyk b40d604ef7 provide value IDs for push notifications 2024-04-02 17:34:16 -04:00
SASANO Takayoshi f630b013ee replace CPPFLAGS += ... (2) 2024-04-02 13:04:52 -04:00
SASANO Takayoshi 36167c3e45 replace CPPFLAGS += ... 2024-04-02 13:04:52 -04:00
Adrien Béraud 4eaa53a4a6 update version to 3.1.9 2024-03-24 23:42:44 -04:00
Adrien Béraud 4859e9c4e7 meson: support building with push notification support 2024-03-24 23:42:01 -04:00
Adrien Béraud c82c393102 ci: use Cython from pip 2024-03-24 23:31:18 -04:00
Adrien Béraud 5dc162a4ec meson: support building python 2024-03-24 23:26:05 -04:00
Adrien Béraud 5a4c599139 meson: support building with indexation 2024-03-24 23:25:40 -04:00
Adrien Béraud 07ca10a451 Revert "ci: build proxy client for Autotools build"
This reverts commit 2dc9f25ff4.
2024-03-24 22:14:32 -04:00
Adrien Béraud 0e556b88be docker: use cython, meson from pip 2024-03-24 22:08:03 -04:00
Adrien Béraud 0b242e62c5 update version to 3.1.8 2024-03-24 21:46:15 -04:00
Adrien Béraud acde8fa6ec crypto: remove legacy decrypt 2024-03-24 18:12:27 -04:00
Adrien Béraud c8b5611386 dhtnode: fix small typo in put encrypted log 2024-03-24 15:45:55 -04:00
Adrien Béraud f7e4533282 crypto: always zero-initialize gnutls_datum_t 2024-03-24 15:45:52 -04:00
Adrien Beraud 5abbef547b use new log format 2024-03-22 13:49:44 -04:00
Adrien Beraud 3c27b2c777 dhtrunner: avoid blocking when stopping
A deadlock was possible when if shutdown operation occured when the dht was in "connecting" state.
2024-03-22 13:49:44 -04:00
Adrien Beraud 903eba6a31 proxy client: cancel resolver on stop 2024-03-22 13:49:44 -04:00
Adrien Beraud a37e42af37 Request: cancel resolver on cancel 2024-03-22 13:49:44 -04:00
Adrien Beraud cc6cce6157 http: allow to cancel resolver 2024-03-22 13:49:44 -04:00
Adrien Béraud 2dc9f25ff4 ci: build proxy client for Autotools build 2024-03-17 16:38:35 -04:00
Adrien Béraud 7ca39b4a20 http: use OCSP_check_validity to check OCSP validity 2024-03-17 15:56:48 -04:00
Adrien Béraud fcffcee383 build/cmake: require cmake 3.16 2024-03-17 13:38:17 -04:00
Adrien Béraud e371bddd48 ci/github: use Cython from pip on Ubuntu 22.04 2024-03-17 13:16:44 -04:00
Adrien Béraud 26d23cf94f ci/github: use recent Meson for build 2024-03-17 12:48:10 -04:00
Adrien Béraud b608349e94 ci/github: use BUILD_TESTING instead of OPENDHT_TESTS 2024-03-17 12:19:02 -04:00
Lanius-collaris f9b46168d0 dht_proxy_server: Allow user to specify non-standard ports in UnifiedPush endpoint 2024-01-17 15:54:04 -05:00
Adrien Béraud 88d15be4ea build: update version to 3.1.7 2024-01-02 16:25:16 -05:00
Adrien Béraud 0c4942439b move secure_vector to its own file 2024-01-02 16:22:02 -05:00
Adrien Béraud b03df3f7a6 aesEncrypt: allow to provide salt 2024-01-02 15:18:44 -05:00
Adrien Béraud 6a6816ada7 build/meson: allow to build c binding 2023-12-11 22:16:58 -05:00
Adrien Béraud 372c87ac12 build: remove rng.cpp 2023-12-11 21:18:15 -05:00
Adrien Béraud 728c99999f getDerivedRandomEngine: use 32 bit input for std::seed_seq 2023-12-11 21:12:50 -05:00
Adrien Béraud 2017473156 build: update version to 3.1.6 2023-12-11 09:57:51 -05:00
Adrien Béraud 08d6367fdc pht: move to std::random_device 2023-12-10 16:02:56 -05:00
Adrien Béraud 23d07dae38 rng: drop custom random_device implementation for MinGW/Windows
MinGW now has std::random_device support on Windows:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=85494
2023-12-10 15:18:59 -05:00
Adrien Béraud 29c83d9c97 rng: improve getSeededRandomEngine
* Make sure to use 32 bit input for std::seed_seq
* Retry using random device every use
* Adjust delays and retry count to keep approx. the same max timeout.
 In practice the actual average timeout should be reduced, because
 the algo doesn't lose existing data and can move forward in case
 of occasional failure/retry.
2023-12-10 15:07:39 -05:00
Adrien Béraud abbc77744e dhtrunner: allow to configure rng used by the dht 2023-12-10 15:07:39 -05:00
Adrien Béraud 7a61fa371f dht: allow to provide rng to constructor 2023-12-10 15:06:51 -05:00
Adrien Béraud ce7f592b1c search: return if added node is actually removed 2023-12-08 11:02:21 -05:00
Adrien Béraud 111bfea6f3 build: update version to 3.1.4 2023-12-03 22:09:34 -05:00
Adrien Béraud e35a11a495 tests: add test for aesBuildEncrypted 2023-12-03 22:03:22 -05:00
Adrien Béraud 0592b9ff51 crypto: add aesBuildEncrypted 2023-12-03 22:03:22 -05:00
Adrien Béraud 297e182576 crypto: fix typo in comment 2023-11-28 19:57:01 -05:00
Adrien Béraud b285e6fd2b ci/github: use macOS 12 2023-11-28 18:34:37 -05:00
Adrien Béraud 8ed72e204b build: update version to 3.1.3 2023-11-28 18:18:05 -05:00
Amin Bandali 4592663fd2 build/cmake: Link libatomic where needed for 64-bit atomic ops.
Link against libatomic also on architectures that need it for 64-bit
atomic operations.  ARM EABI (armel) and little-endian MIPS (mipsel)
are two such architectures.
2023-11-28 10:52:01 -05:00
Adrien Béraud 4c1c6cf8e9 python: add aesEncrypt, aesDecrypt with password 2023-11-28 10:49:41 -05:00
Adrien Béraud 09046ec698 tests: add tests for AES encryption and key streching 2023-11-28 10:49:41 -05:00
Adrien Béraud 24e7b00e4e crypto: add aesGetKey, aesGetSalt, aesGetEncrypted
Allows to retrieve the encryption key and salt from
an encrypted blob, leading the way for biometry support.
2023-11-28 10:49:41 -05:00
Adrien Béraud 95fdadf16b build: update version to 3.1.2 2023-11-07 13:15:56 -05:00
Adrien Béraud a271ab1680 python: add putSigned, putEncrypted to DhtRunner 2023-11-07 13:15:56 -05:00
Moreno Giussani ac9d7942ce Add loadIdentity 2023-11-07 09:29:46 -05:00
Adrien Béraud 5ca7b8d6dc python: add cancelPut 2023-11-07 09:15:11 -05:00
Adrien Béraud 805cdbed1d python: add biding for missing DhtConfig fields 2023-11-06 22:57:50 -05:00
Adrien Béraud 29efd42de3 python: add binding for SockAddr.resolve 2023-11-06 22:57:50 -05:00
Adrien Béraud ae2a1d83cf python: use shared_ptr, make_shared from Cython 2023-11-06 21:59:22 -05:00
Adrien Béraud bad8bdfe13 python: support 'permanent' argument for put 2023-11-06 21:55:16 -05:00
Adrien Béraud 13194b66d3 proxy client: retry permanent put after reconnect 2023-11-02 15:58:11 -04:00
Adrien Béraud 601758f5fd proxy server: use isMember for timeout 2023-09-27 13:56:43 -04:00
Adrien Béraud 3191498855 update version to 3.0.0 2023-09-24 18:13:22 -04:00
Adrien Béraud 4882826f58 ci, docker: update restinio 2023-09-24 18:13:22 -04:00
Kateryna Kostiuk 61f43c6b05 iOS: use high priority for resubscribe notifications 2023-09-18 15:27:41 -04:00
Adrien Béraud 8b91c24414 python: make callbacks noexcept 2023-09-11 11:52:55 -04:00
Adrien Béraud dbebaae1f3 callbacks: make raw callbacks except 2023-09-11 11:42:05 -04:00
Adrien Béraud 7cc743ff5e callbacks: make raw callbacks noexcept 2023-09-11 11:30:22 -04:00
Adrien Béraud 3b02cee495 python: update classifiers 2023-09-11 11:30:04 -04:00
Adrien Béraud 2e520bea4f unit tests: make testListen run 256 times
Makes the test pass in many more cases.
2023-09-09 12:51:15 -04:00
Adrien Béraud f9f57b50b1 update README.md 2023-09-09 12:41:16 -04:00
Adrien Béraud fb9e6eee48 Revert "build: do not define ASN1_time_parse when using openssl"
This reverts commit 3a8adb4e17.
2023-08-30 17:59:07 -04:00
Adrien Béraud b53af75a7f meson, autotools: link with fmt 2023-08-30 17:58:45 -04:00
Adrien Béraud 2ed40aeefe proxy client: use [[maybe_unused]] 2023-08-30 17:58:45 -04:00
Adrien Béraud bb61b8468b proxy client: implement OnPublicAddressChanged 2023-08-30 17:58:45 -04:00
Adrien Béraud 55688a25aa http: don't encode whole url
url encoding should be used for url components
by users of the library,
otherwise characters like '/' get escaped.
2023-08-30 17:58:45 -04:00
Adrien Béraud 21fd58a39c tools: show log on PublicAddressChanged 2023-08-30 17:58:45 -04:00
Adrien Béraud ae56d4e9be dht: implement OnPublicAddressChanged 2023-08-30 17:58:45 -04:00
Adrien Béraud 982896a718 dht interface: add OnPublicAddressChanged 2023-08-30 17:58:45 -04:00
Adrien Béraud 2ae092e266 storage: remove unused variable 2023-08-30 17:58:45 -04:00
Adrien Béraud 27ce8b00ed add syncLevel to send announce/listen early 2023-08-30 16:31:47 -04:00
Kateryna Kostiuk 3a8adb4e17 build: do not define ASN1_time_parse when using openssl 2023-08-27 16:54:16 -04:00
Adrien Béraud 899d18968e thread pool: add missing header 2023-08-20 23:44:49 -04:00
Adrien Béraud 285105c8d1 thread pool: simplify cleanup logic 2023-08-20 23:02:41 -04:00
Adrien Béraud 9e0ad9d926 thread pool: wait for tasks to complete on stop 2023-08-19 18:48:53 -04:00
Adrien Béraud 0fdded0606 thread pool: allow pool to reduce in size after some time 2023-08-19 18:48:53 -04:00
Adrien Béraud 9947172abb tests/http: wait for the server to start 2023-08-19 18:48:53 -04:00
Adrien Béraud b034a823a9 dhtcnode: reduce exposed stack storage 2023-08-19 18:48:53 -04:00
Jakob Haufe 7baee925f6 doc: fix dhtnode(1) man page formatting
This fixes the following lintian issue:
    22: warning: macro 'mF' not defined [usr/share/man/man1/dhtnode.1.gz:1]
2023-08-16 22:44:06 -04:00
Adrien Béraud ed6c5381d8 http: remove incorrect static usage 2023-07-29 00:02:48 -04:00
Adrien Béraud d58e4c8a89 build/cmake: remove unsupported attribute 2023-07-28 23:58:46 -04:00
Léo Banno-Cloutier 2f2ed9bca7 url_encode: escape url query
For routes such as /api/auth/directory/search, when searching for names such as
"Léo", jams would not respond to the request
since the URL contained invalid characters defined in RFC3986. `url_encode`
turns such characters into their equivalent escaped characters.
2023-07-25 10:28:22 -04:00
Amna 3e888d8e2d network engine: remove link-local condition from isMartian() 2023-07-05 19:38:24 -04:00
Adrien Béraud f461eaa191 log: cleanup, avoid using default constructor 2023-06-28 11:27:03 -04:00
Adrien Béraud 411d8dbe62 network_engine: add missing headers 2023-06-28 11:27:03 -04:00
Lanius-collaris 825cf98bf4 dhtnode.cpp: fix void print_usage()
Print pushserver parameter instead of openpush
2023-06-28 09:08:52 -04:00
Adrien Béraud aff1ed4575 network engine: make UA private, constexpr 2023-06-27 20:07:03 -04:00
Adrien Béraud 6b1e7513b3 logger: make class public 2023-06-26 18:25:43 -04:00
Adrien Béraud 8c94dc8f89 dht: update User-Agent (UA) to "o2"
The update reflects the slow evolution of the protocol.
Also, saves two bytes in every packet.
2023-06-26 18:25:31 -04:00
Adrien Béraud a7f459eab9 log: fix build on Windows 2023-06-26 16:10:11 -04:00
Adrien Béraud 7383e3ea29 node cache: increase cleanup period 2023-06-19 18:36:58 -04:00
Adrien Béraud a416d472fb node: properly skip id 0 2023-06-19 18:36:17 -04:00
Adrien Béraud 8fb707824e dockerfile: use deps from ghcr 2023-06-13 20:40:00 -04:00
Adrien Béraud e646636bde docker/alpine: use Alpine 3.18 2023-06-13 20:21:34 -04:00
Adrien Béraud fafd2f0233 update copyright year 2023-06-02 23:16:04 -04:00
Adrien Béraud 9f031303a2 securedht: destroy dht before other fields 2023-06-02 21:50:25 -04:00
Adrien Béraud 9a2e69ea9c log: simplify logger 2023-06-02 21:50:25 -04:00
Sébastien Blin 54eff3b0aa proxyServer: no need for a pushServer_ for type==UnifiedPush 2023-06-02 13:18:58 -04:00
Adrien Béraud 817a5baef2 build/cmake: apply OPENDHT_SANITIZE to the C binding 2023-06-01 14:26:22 -04:00
Adrien Béraud 79dfc0403a dht: remove deprecated constructor 2023-06-01 14:25:45 -04:00
Adrien Béraud 675d892fc1 update version to 2.6.0 2023-05-18 13:13:25 -04:00
Adrien Béraud 2654970499 logger: remove OPENDHT_LOG, rename to logger.h 2023-05-18 12:31:42 -04:00
Adrien Béraud b3c5c2fcfa hash: add fmt formatter 2023-05-18 12:31:42 -04:00
Adrien Béraud 6919df592f sockaddr: add fmt formatter 2023-05-18 12:31:42 -04:00
Adrien Béraud 6cfede0868 logs: use fmt 2023-05-18 12:31:42 -04:00
Adrien Béraud e176e4baa6 logs: use fmt 2023-05-17 23:43:55 -04:00
Adrien Béraud 4041b69800 DhtRunner, dht_interface: remove setLoggers 2023-05-17 23:43:55 -04:00
Adrien Béraud da6579132c log: move Color, printLog as private 2023-05-17 23:43:55 -04:00
Adrien Béraud 24628ff13d meson: build dhtchat, dhtscanner 2023-05-17 23:43:55 -04:00
Adrien Beraud d2f29b72c3 update version to 2.5.5 2023-04-24 18:37:47 -04:00
Adrien Beraud 546a960951 cmake: use full path for pkgconfig 2023-04-24 18:36:24 -04:00
Adrien Béraud 27978f3b9f update version to 2.5.4 2023-04-21 18:34:24 -04:00
Adrien Béraud a831afdc1e docker: build using current repo state 2023-04-21 18:33:29 -04:00
Adrien Béraud 13bb50b863 CMake: handle missing SYSTEMD_UNIT_INSTALL_DIR 2023-04-21 18:32:49 -04:00
Adrien Béraud d16d30c483 Hash: add Hash<N>::get(Hash<M>) to compute an hash from another 2023-04-21 12:00:15 -04:00
Adrien Béraud 4480436cb3 securedht: allow certificate to be stored on longId hash 2023-04-20 22:41:38 -04:00
Adrien Béraud d4114830be update version to 2.5.3 2023-04-20 22:41:38 -04:00
Adrien Béraud 9fbac2707d build/cmake: handle empty SYSTEMD_UNIT_FILE_LOCATION 2023-04-20 22:41:38 -04:00
Adrien Béraud 7cdd6948bf misc: use bootstrap.jami.net instead of ring.cx 2023-04-20 22:41:38 -04:00
Adrien Beraud d9c5ff9624 build/cmake: don't use testing on MSVC 2023-04-20 22:41:35 -04:00
Adrien Beraud 47691b8242 cmake: fix msvc build 2023-04-10 11:45:50 -04:00
Adrien Beraud adf57ce2f0 update version to 2.5.2 2023-04-10 11:39:37 -04:00
Adrien Beraud aea84977fd msgpack: support more package names 2023-04-10 11:27:38 -04:00
Adrien Beraud 2f1aef272a msgpack: support more package names 2023-04-10 11:12:51 -04:00
Adrien Beraud 76089d09a4 msgpack: support more package names 2023-04-10 10:51:14 -04:00
Adrien Beraud f88f99959b update version to 2.5.1 2023-04-05 15:57:47 -04:00
Sébastien Blin dc6aca83c6 Makefile: add node_export 2023-04-05 09:41:22 -04:00
Adrien Beraud b0e18e9ca4 docker: use dep alpine base image 2023-03-25 18:51:41 -04:00
Adrien Beraud 8516e35e5f ci/github: cleanup 2023-03-25 18:40:22 -04:00
Adrien Beraud 41eabc7261 thread pool: handle exceptions starting threads 2023-03-25 18:22:58 -04:00
Adrien Beraud 424fe61a49 dht: send large amounts of values in multiple messages 2023-03-25 17:49:16 -04:00
Adrien Beraud 02e03771d4 docker: add Alpine deps 2023-03-25 17:49:08 -04:00
Adrien Beraud e35f8c29ee ci/github: don't run ci with autotools 2023-03-25 17:17:40 -04:00
Adrien Beraud 64e3bcf0dc ci/github: use ninja with meson 2023-03-25 15:08:31 -04:00
Adrien Beraud a9f3883bfa meson: install, include public dir from tools 2023-03-25 15:04:11 -04:00
Adrien Beraud 3c3b3dc303 ci/github: add meson build 2023-03-25 14:42:46 -04:00
Adrien Beraud 970e71d215 ci/github: add meson build 2023-03-25 14:38:48 -04:00
Adrien Beraud b6552ec300 NodeExport: add missing include 2023-03-25 10:46:12 -04:00
Adrien Béraud 58bc772256 NodeExport: use SockAddr, move to separate header 2023-03-23 16:50:19 -04:00
Adrien Béraud 31af08feaa build/meson: build durl with http and tools 2023-03-23 16:50:19 -04:00
Adrien Béraud 9e60a681f3 use ostringstream instead of stringstream where possible 2023-03-23 16:50:19 -04:00
Adrien Beraud 51525d144b certificate: add public key cache 2023-03-15 20:44:29 -04:00
Adrien Beraud 965fbb43c7 more attempts to find msgpackc-cxx in a universal way 2023-03-15 12:50:06 -04:00
Adrien Béraud 133d77d985 thread pool: increase max io pool size to 512 2023-03-14 13:11:29 -04:00
Adrien Béraud 9451861f51 update version to 2.5.0 2023-03-13 13:24:21 -04:00
Adrien Béraud 4b489fb7d7 build/meson: make tools an option 2023-03-13 13:24:05 -04:00
Adrien Béraud 7fcaf19ca3 meson: add initial support 2023-03-12 23:57:32 -04:00
Adrien Béraud c51d59a3a3 Revert "cmake: update msgpack package name"
This reverts commit b9d7957bf6.
2023-03-11 22:30:19 -05:00
Adrien Béraud bcc02837d3 use proper image name for github action 2023-03-11 22:25:22 -05:00
Adrien Béraud 611017e187 add Alpine image 2023-03-11 22:21:50 -05:00
Adrien Béraud 7f7bff6a28 readme: cleanup 2023-03-11 22:18:52 -05:00
Adrien Béraud b9d7957bf6 cmake: update msgpack package name 2023-03-11 22:18:27 -05:00
Adrien Béraud fa31cec73e pkgconfig: gnutls jsoncpp, openssl are public dependencies 2023-03-09 00:37:46 -05:00
Adrien Béraud 550a8cf362 cmake: use CTest 2023-02-28 19:45:05 -05:00
Adrien Béraud 50996eaf7e build/cmake: unify static and shared build 2023-02-28 12:58:13 -05:00
Adrien Béraud 813c3b43d2 move filter 2023-02-28 12:58:13 -05:00
Adrien Béraud 4ce9292b28 proxy server: allow to ping push 2023-02-23 15:43:16 -05:00
Adrien Béraud 655aa074bf update version to 2.4.11 2023-01-31 23:47:01 -05:00
Adrien Béraud 2e9c119dca build/cmake: add SOVERSION for c binding 2023-01-31 23:45:48 -05:00
Adrien Béraud 7bf2d2a7a9 update version to 2.4.11 2023-01-30 12:05:02 -05:00
Adrien Béraud d9a97e2cec use pkg-config for gnutls, fix msgpack 2023-01-29 18:30:28 -05:00
Adrien Béraud 6395399c2d cmake: use IMPORTED_TARGET with pkg_search_module 2023-01-29 18:30:28 -05:00
Adrien Béraud e6971c196b dht: fix value edition, add unit test 2023-01-27 17:31:59 -05:00
Adrien Béraud b1009a54e3 crypto: compare public keys with longId 2023-01-27 17:31:59 -05:00
Adrien Béraud 33d73d5d55 securedht: compare public key content instead of pointer 2023-01-27 17:31:59 -05:00
Amin Bandali f42d94da59 build/cmake: Set variables needed by pkg-config files.
Up until now, these were only set by the GNU Build System in
configure.ac, and so the pkg-config files generated from CMake builds
did not have the right Libs.private and Requires.private, especially
useful when linking statically.
2023-01-26 21:31:59 -05:00
Adrien Béraud 2418b355d7 ci: build macOS with push notifications 2023-01-25 19:53:53 -05:00
Adrien Béraud 2cb57251a1 proxy client: protect resolver 2023-01-25 19:53:53 -05:00
Adrien Béraud 446da4b792 value: use move in where 2022-12-30 20:42:16 -05:00
Adrien Béraud aa958b0f1d value: inline filter 2022-12-30 20:42:14 -05:00
Adrien Béraud 58be07a8b0 infohash: add to_view() 2022-12-30 20:42:11 -05:00
Adrien Béraud 6c61ac53c4 proxy client: cleanup 2022-12-30 20:42:08 -05:00
Sébastien Blin 6e73a78a8b unifiedpush: remove unifiedpush parameter
Because there is multiple unifiedpush endpoint (ntfy.sh is one),
we should not pass it as an option to dhtnode, but use the one
provided by the user in the token (from unified push library).
2022-12-30 20:36:49 -05:00
Amin Bandali f2623bd51d build/cmake: Set opendht-c-static's OUTPUT_NAME to opendht-c. 2022-12-30 20:35:17 -05:00
Sébastien Blin d6a99d0edd proxyclient: add possibility to set the platform
This allow android to change the platform to "unifiedpush".
2022-11-17 11:46:09 -05:00
Sébastien Blin b9ccc968b2 proxyserver: add support for UnifiedPush standard
Make the proxy able to support the UnifiedPush platform, so that
applications using the proxy server can use any UnifiedPush provider.

Documentation: https://unifiedpush.org/

How to test:
+ Install a UnifiedPush provider on a device (e.g. https://ntfy.sh)
+ Run dhtnode with a proxyserver and --unifiedpush https://ntfy.sh
+ Subscribe to a random topic on ntfy (e.g. "dhtproxytest")
+ Subscribe to a hash on the DHT:
    curl -X SUBSCRIBE http://my.dht.proxy:8000/key/foo \
         -d '{"key":"dhtproxytest", "platform":"unifiedpush"}'
+ Send data on the subscribed hash

Note: the topic should be completely random.
2022-10-11 15:14:51 -04:00
Sébastien Blin 7e33c4fc76 dht_proxy_server: fix push notifications if tls is enabled
If the proxy got TLS enabled, this doesn't mean that the REST
endpoint for push notifications is also using TLS.

Also, fix an incorrect log.
2022-10-11 11:54:37 -04:00
Adrien Béraud 39bb520339 proxy server: don't send refresh notification for non-push operations 2022-09-22 12:46:54 -04:00
Sébastien Blin 18e53b9ec8 proxyserver: send correct value id to client
when searching for a value to put, the proxy server returned
a json with value->id = 0, causing the client to not refreshing
correctly the values later (when timeout-ing, the value to refresh
is not found).
2022-09-15 11:45:11 -04:00
Sébastien Blin 0b5379380a rust: update os_socketaddr to latest version 2022-09-08 11:47:02 -04:00
Sébastien Blin b800012c57 proxyserver: add topic to msgpack 2022-09-08 11:15:28 -04:00
Sébastien Blin ebeff20215 dht_proxy_server: fix permanent put
When a second permanent put request was received, we didn't call
async_wait after changing the expiration, causing push to not be
sent the second day and presence to expire on some devices after
~3 days.
2022-09-07 16:47:37 -04:00
Adrien Béraud 16ff16bd39 docker, ci: update restinio 2022-09-02 17:15:29 -04:00
Adrien Béraud 0d28a7bdab ci/github: update restinio 2022-09-02 17:14:58 -04:00
Adrien Béraud 69a1bdfc45 update version to 2.4.10 2022-09-02 17:01:23 -04:00
Adrien Béraud af7d88b057 crypto: allow to build all from std::string_view 2022-08-11 14:52:14 -04:00
Sébastien Blin abe2e018b6 Merge pull request #629 from AmarOk1412/fix/request_timeout
http: modify timeout behaviour
2022-08-11 14:41:10 -04:00
Sébastien Blin 2e79ee4f11 dht_proxy_client: fix not contextually convertible to 'bool' error
The current master is broken with clang, this fix the issue.
2022-08-11 14:30:59 -04:00
Sébastien Blin 8b022d0022 Merge branch 'master' into fix/request_timeout 2022-08-11 13:49:44 -04:00
Sébastien Blin 5399dbdfbc proxy_client: reconnect account if local ip changes
When switching between two network interfaces, local address will
change but not the public ip. This generally cause the client to
not correctly updates its candidates.
In this patch, we inject in the proxy client the local candidates
and use it to check if the network changed.
2022-08-11 12:32:44 -04:00
Sébastien Blin 0e3bca1dde http: modify timeout behaviour
In the previous version, the timeout for the connection could be
set only after async_connect while the request is sending.
This means if async_connect is slow (it can happen as if there is
an ipv6 endpoint, it can try to resolve it during dozens of seconds
before falling back on ipv4).

In this patch, it moves the timeout callback in the request. This
callback will be set just before async_connect, to be sure that we
are using the callback while connecting.
2022-08-11 11:17:30 -04:00
Adrien Béraud a992aa1f05 crypto: allow to build all from std::string_view 2022-07-28 17:11:25 -04:00
Adrien Béraud 1a267418dd thread pool: release tasks on stop 2022-07-28 17:08:08 -04:00
kkostiuk 0f8d891476 iOS: drop support for voip notifications 2022-07-25 15:26:57 -04:00
Adrien Béraud b9c79a1de5 thread pool: release tasks on stop 2022-07-20 16:35:47 -04:00
Adrien Béraud 2c7edb695f update version to 2.4.9 2022-07-18 11:43:35 -04:00
Adrien Béraud c3ad33687e misc: fix various warnings 2022-07-17 14:41:41 -04:00
Adrien Béraud 9c5e5cd7b7 proxy server: export symbols for ProxyServerConfig 2022-07-17 14:41:02 -04:00
Adrien Béraud 9048e4a31c peer discovery: cleanup, reduce memory usage 2022-07-17 14:29:54 -04:00
Adrien Béraud ba1b3c26c5 procy client: use new routes 2022-07-17 13:00:05 -04:00
Adrien Béraud c6b4e317df dhtrunner: only start peer discovery if proxy is disabled 2022-07-17 13:00:05 -04:00
Adrien Béraud 44267a72de ci/github: cleanup 2022-07-17 13:00:05 -04:00
Adrien Béraud 8c1c5d09c7 build: use Wno-deprecated 2022-07-17 13:00:05 -04:00
Adrien Béraud acadeb220a docker: add dhtnode image 2022-07-05 20:02:45 -04:00
Adrien Béraud ff7c80d416 build/dev: build in sanitize mode 2022-07-05 19:39:36 -04:00
Adrien Béraud 577f2a0645 proxy server: allow to configure address to bind 2022-07-05 19:32:29 -04:00
Adrien Béraud 48e718ffd8 ci/github: update release action 2022-07-05 16:49:54 -04:00
Adrien Béraud 07d25a4140 docker: install systemd 2022-07-05 16:48:23 -04:00
Adrien Béraud 2f7fabf1c7 ci/github: use secrets.AUTH_TOKEN 2022-07-05 16:25:42 -04:00
Adrien Béraud 60274655dd ci/github: release on tag 2022-07-05 16:20:20 -04:00
Adrien Béraud eace3d91b2 docker: add python3-virtualenv 2022-07-05 15:21:32 -04:00
Adrien Béraud 86cc41f446 docker: use Ubuntu 22.04 for LLVM build 2022-07-05 15:12:27 -04:00
Adrien Béraud 3ac5b9466a ci/github: don't build static lib 2022-07-05 15:10:02 -04:00
Adrien Béraud c95c3de136 docker: use Ubuntu 22.04 2022-07-05 15:09:07 -04:00
Adrien Béraud a865d6854c ci/github: use unique names for steps 2022-07-05 14:57:31 -04:00
Adrien Béraud 52b735b1b9 ci/github: build Python Wheel 2022-07-05 14:27:02 -04:00
Adrien Béraud f862305f78 ci/github: build LLVM Docker images 2022-07-05 14:18:24 -04:00
Adrien Béraud ce7ffbf2cd build/python: add dist target 2022-07-05 14:18:24 -04:00
Adrien Béraud 687f643059 build/python: add pyproject.toml 2022-07-05 14:18:24 -04:00
Adrien Béraud f6e3fc99e6 build/cmake: refactor library linking 2022-07-05 14:18:24 -04:00
Adrien Béraud 293662a710 proxyserver: cleanup, avoid ref-to-temporary 2022-07-05 12:16:41 -04:00
Adrien Béraud 437252bfcc proxyserver: add bundleId at the end of ProxyServerConfig 2022-07-04 18:20:32 -04:00
kkostiuk ead21d24a0 proxyserver: add application bundle id option 2022-07-04 18:17:12 -04:00
Adrien Béraud 53de63e829 update version to 2.4.8 2022-06-30 16:10:25 -04:00
Adrien Béraud 633a03d76a ios: support multiple notification types
- VOIP notifications for iOS < 14.5
- alert and background notifications for iOS >= 14.5
2022-06-30 15:44:16 -04:00
Adrien Béraud 3e5fbb0b5c Update codeql-analysis.yml 2022-06-30 10:48:25 -04:00
Adrien Béraud 6d13e301f9 Create codeql-analysis.yml 2022-06-30 10:43:18 -04:00
Adrien Béraud 4d356b6dca ci/github: add experimental docker build action 2022-06-29 16:39:49 -04:00
Adrien Béraud 44a251de56 build/autotools: don't export internal symbols 2022-06-28 16:23:40 -04:00
Adrien Béraud 1c536a4c51 update version to 2.4.7 2022-06-28 14:23:23 -04:00
Adrien Béraud 502a649f42 build/cmake: use max version for cmake_minimum_required 2022-06-28 14:22:49 -04:00
Adrien Béraud 12a52c78ac x509: make PrivateKey::getPublicKey() thread-safe 2022-06-28 14:12:45 -04:00
Adrien Béraud df2a775927 build/autotols: allow to build with header-only msgpack-cxx installation 2022-06-28 14:12:45 -04:00
Adrien Béraud 2c86ccbe15 build/pkgconfig: update project description 2022-06-28 14:12:45 -04:00
Adrien Béraud b9321a338b buiild/autotools: use -fvisibility=hidden in release mode 2022-06-28 14:12:45 -04:00
Adrien Béraud dc9a211fb0 crypto: protect Id caching with atomic_bool 2022-06-27 15:40:45 -04:00
Adrien Béraud d49b394eae crypto: cache public key id 2022-06-27 15:40:45 -04:00
Adrien Béraud f82c68a871 crypto: remove support for GnuTLS < 3.4 2022-06-27 15:40:45 -04:00
Sébastien Blin 3f4235da69 infohash: fix alignment warning 2022-06-27 15:32:50 -04:00
Sébastien Blin e259cb0847 infohash: remove optimization due to weird warnings
Current operator== creates a very weird warning about using an
uninitialized value (no warning without the reinterpret_cast).
2022-06-27 14:37:25 -04:00
Adrien Béraud 08f9b424de cmake: allow to build with header-only msgpack-cxx installation 2022-06-23 14:26:26 -04:00
Adrien Béraud d33c3515f8 update version to 2.4.6 2022-06-23 10:05:28 -04:00
Adrien Béraud 0a70b0f80f dhtrunner: don't bind port if proxy is used 2022-06-22 22:20:15 -04:00
Adrien Béraud 949cbb9fc7 sockaddr: allow to release ownership 2022-06-22 22:20:15 -04:00
Adrien Béraud 37cfbffdca dhtcnode: cleanup 2022-06-22 22:20:15 -04:00
Adrien Béraud efad2ada89 dhtrunner: make getPublicAddress const 2022-06-22 22:20:15 -04:00
Adrien Béraud d825f4e5ae misc: use std::make_unique 2022-06-08 13:14:09 -04:00
Adrien Béraud baa2a5d21c dhtrunner: remove support for running both dht and proxy client
If configured to run as a proxy client, no port will be bound and the local dht node
won't be instantiated.
2022-06-08 13:14:09 -04:00
Sébastien Blin b3f9683018 securedht: avoid to update cache if not necessary 2022-06-08 13:07:06 -04:00
Adrien Béraud 6d1819acb9 securedht: fix warning 2022-06-07 15:58:16 -04:00
Adrien Béraud d6ee44b640 dhtrunner: use consistent path for state 2022-06-07 15:38:44 -04:00
Peter Saveliev 383c53d71f python/tools: network builder v2
* `-n`: interfaces number check (1 <= ifnum <= 245)
* `-b`: turn on NDB debug logging, very chatty, dump topology DB files
* `-v`: verbose mode, print main operations
* fix code formatting (isort, flake8, black)
* code refactoring
2022-06-07 15:29:07 -04:00
Peter Saveliev 833c647524 python/tools: update network builder
Use updated pyroute2 API:

* NSPopen import from the root
* NDB instead of IPDB

No logic changes, only translated obsoleted API calls.
No code formatting.
2022-06-07 15:29:07 -04:00
Adrien Béraud 7f6f50411c autotools: fix check for CppUnit 2022-05-30 10:07:04 -04:00
Adrien Béraud 1c7089c055 github: add minimal build 2022-05-25 11:27:15 -04:00
Adrien Béraud 4641780fe6 update version to 2.4.5 2022-05-25 11:05:34 -04:00
Adrien Béraud c67df2ead3 tasks: generate compile commands 2022-05-25 11:04:59 -04:00
Adrien Béraud 5f743344e5 value: cleanup, move sign & encrypt to cpp 2022-05-25 11:04:46 -04:00
Michael Cho 56442acc84 value: fix build without jsoncpp 2022-05-25 09:49:30 -04:00
Adrien Béraud 41c9104b8a value: add inline specifier to inline methods 2022-05-05 11:33:53 -04:00
Adrien Béraud 126f0ca966 update version to 2.4.4 2022-05-03 14:46:07 -04:00
Adrien Béraud adeca7fa1f tests/dhtrunner: split count per key 2022-04-25 13:48:57 -04:00
Adrien Béraud 5527b6553d c: use void for dht_version 2022-04-25 13:43:59 -04:00
Adrien Béraud 5a6b9db60c securedht: move crypto ops to Value 2022-04-25 13:43:57 -04:00
Adrien Béraud 8f1c075897 update version to 2.4.3 2022-04-22 21:12:54 -04:00
Adrien Béraud fca11e621c tools/dhtcnode: use version arg, cleanup 2022-04-22 21:10:53 -04:00
Adrien Béraud 38a7234ae3 dht: increase max hashes to 1G 2022-04-22 21:01:36 -04:00
Adrien Béraud 115a0c683f c: add dht_infohash_from_hex_null 2022-04-22 21:00:18 -04:00
Adrien Béraud 61b7bed0f8 network engine: minor refactoring 2022-04-22 20:25:18 -04:00
Adrien Béraud dcd8ec7394 tests/dhtrunner: use unlimited store size 2022-04-22 20:25:18 -04:00
Adrien Béraud 51880afcce tests/dhtrunner: consider value expiration for counting 2022-04-22 20:25:18 -04:00
Adrien Béraud db3a4e52b2 dht: add config max_store_keys 2022-04-22 20:24:59 -04:00
Adrien Béraud af8e66b813 tools/dhtnode: show basic storage stats on "ll" 2022-04-20 16:11:21 -04:00
Adrien Béraud 9646eb9374 NodeInfo: add storage stats 2022-04-20 16:10:25 -04:00
Adrien Béraud 0df2fbe5de tools/dhtcnode: print version on startup 2022-04-20 16:09:24 -04:00
Adrien Béraud c39eceeddf tools/dhtcnode: include stdio.h before readline.h 2022-04-20 16:09:08 -04:00
Adrien Béraud 11396c315e c: add dht_version() 2022-04-20 16:08:12 -04:00
Adrien Béraud 955af06e29 dht: add getStorageLimit 2022-04-20 15:56:14 -04:00
Adrien Béraud e207e59226 tools/dhtcnode: count values in listen, add "ll" 2022-04-20 12:23:25 -04:00
Adrien Béraud 0302cee8d5 tools/dhtcnode: use memset instead of bzero 2022-04-19 17:45:14 -04:00
Adrien Béraud 8f4031c8cb update version to 2.4.2
Fix typo in 2.4.1 release
2022-04-19 15:37:46 -04:00
Adrien Béraud f72d1ffa64 update version to 2.4.1 2022-04-19 15:07:55 -04:00
Adrien Béraud 90c65fb0be tests/dhtrunner: increase put count 2022-04-19 15:07:13 -04:00
Adrien Béraud c8eb704a92 dhtrunner: prevent integer underflow in UDP rx callback 2022-04-19 14:51:24 -04:00
Adrien Béraud 4c5e26922c net: increase max queue size
Queue is also time-bounded.
Allow for larger bursts of packets.
2022-04-18 17:24:27 -04:00
Adrien Béraud 724c3e431c dht: notify listeners when value is removed because of limits 2022-04-18 17:19:32 -04:00
Adrien Béraud a9ddf49ecc storage: check size before returning oldest 2022-04-18 17:18:08 -04:00
Adrien Béraud de3dba7ce4 cmake/c: respect OPENDHT_STATIC/SHARED flags 2022-04-15 21:07:56 -04:00
Adrien Béraud cd9eda6511 tools/dhtcnode: increase shutdown pooling period 2022-04-15 21:06:45 -04:00
Adrien Béraud ed972fd365 dht: allow to store up to 64k values per hash 2022-04-15 20:12:48 -04:00
Adrien Béraud 75834fb08f dht: fix potential of-by-one largest computation, cleanup 2022-04-15 20:12:48 -04:00
Adrien Béraud 9991c976ff dht: prevent potential use-after-free in storageChanged 2022-04-15 20:12:48 -04:00
Adrien Béraud 92b08bbbc9 dht: pass value directly to storageChanged 2022-04-15 20:12:48 -04:00
Adrien Béraud 4f9b50dd60 print_addr: use ostringstream insted of stringstream 2022-04-15 19:18:24 -04:00
Adrien Béraud d6d73efa37 dht: update storage bucket on refresh 2022-04-15 19:14:12 -04:00
Adrien Béraud 19a4729bf2 test/dhtrunner: use random port, add more assertions 2022-04-14 20:12:57 -04:00
Adrien Béraud 405b8739ab dhtrunner: always call opEnded once per op 2022-04-14 20:12:55 -04:00
Adrien Béraud 7ed11a02ae c: cleanup dhtcnode 2022-04-12 17:04:55 -04:00
Adrien Béraud 3442e0acd9 build/dev: cleanup launch.json 2022-04-12 15:38:48 -04:00
Adrien Béraud 59f86861d2 docker: install libtool 2022-04-12 15:38:48 -04:00
Adrien Béraud 091d01a799 c: update dhtcnode 2022-04-12 15:38:48 -04:00
Adrien Béraud 1ed45cdcf9 c: support log to stdout/stderr 2022-04-12 15:38:48 -04:00
Adrien Béraud c40ba83cb3 c: support building with autotools 2022-04-12 15:38:48 -04:00
151 changed files with 4415 additions and 2652 deletions

No files matched your search

+99 -40
View File
@@ -4,53 +4,112 @@ on: [push, pull_request]
jobs:
build-ubuntu:
name: Ubuntu/GCC build
name: Ubuntu/GCC Autotools build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: deps
- name: Dependencies
run: |
sudo apt install libncurses5-dev libreadline-dev nettle-dev \
libgnutls28-dev libuv1-dev cython3 python3-dev python3-setuptools libcppunit-dev libjsoncpp-dev \
autotools-dev autoconf libfmt-dev libhttp-parser-dev libmsgpack-dev libargon2-0-dev
- name: autogen
run: ./autogen.sh
- name: asio
sudo apt install libncurses5-dev libreadline-dev nettle-dev libasio-dev \
libgnutls28-dev libuv1-dev python3-pip python3-dev python3-setuptools libcppunit-dev libjsoncpp-dev \
autotools-dev autoconf libfmt-dev libmsgpack-dev libargon2-0-dev
- name: Cython
run: sudo pip3 install Cython
- name: llhttp
run: |
wget https://github.com/aberaud/asio/archive/b2b7a1c166390459e1c169c8ae9ef3234b361e3f.tar.gz \
&& tar -xvf b2b7a1c166390459e1c169c8ae9ef3234b361e3f.tar.gz && cd asio-b2b7a1c166390459e1c169c8ae9ef3234b361e3f/asio \
&& ./autogen.sh && ./configure --prefix=/usr --without-boost --disable-examples --disable-tests \
&& sudo make install \
&& cd ../../ && rm -rf asio*
- name: configure
run: ./configure
- name: make
run: make
- name: make check
run: make check
build-macos:
name: macOS/Clang build
runs-on: macos-11
steps:
- uses: actions/checkout@v3
- name: deps
mkdir llhttp
wget https://github.com/nodejs/llhttp/archive/refs/tags/release/v9.2.0.tar.gz -O llhttp.tar.gz
tar -xzf llhttp.tar.gz -C llhttp --strip-components=1
cd llhttp && cmake -DCMAKE_INSTALL_PREFIX=/usr . && make -j2 && sudo make install
cd ../ && rm -rf llhttp*
- name: expected-lite
run: |
brew install msgpack-cxx asio gnutls nettle readline fmt jsoncpp argon2 openssl http-parser cppunit
sudo mkdir /usr/local/include/nonstd && \
sudo wget https://raw.githubusercontent.com/martinmoene/expected-lite/master/include/nonstd/expected.hpp \
-O /usr/local/include/nonstd/expected.hpp
- name: restinio
run: |
mkdir restinio && cd restinio
wget https://github.com/aberaud/restinio/archive/e0a261dd8488246a3cb8bbb3ea781ea5139c3c94.tar.gz
ls -l && tar -xzf e0a261dd8488246a3cb8bbb3ea781ea5139c3c94.tar.gz
cd restinio-e0a261dd8488246a3cb8bbb3ea781ea5139c3c94/dev
cmake -DCMAKE_INSTALL_PREFIX=/usr/local -DRESTINIO_TEST=OFF -DRESTINIO_SAMPLE=OFF \
-DRESTINIO_INSTALL_SAMPLES=OFF -DRESTINIO_BENCH=OFF -DRESTINIO_INSTALL_BENCHES=OFF \
-DRESTINIO_FIND_DEPS=ON -DRESTINIO_ALLOW_SOBJECTIZER=Off -DRESTINIO_USE_BOOST_ASIO=none .
make -j8 && sudo make install
cd ../../.. && rm -rf restinio
wget https://github.com/Stiffstream/restinio/releases/download/v.0.7.2/restinio-0.7.2.tar.bz2
ls -l && tar -xjf restinio-0.7.2.tar.bz2
cd restinio-0.7.2/dev
cmake -DCMAKE_INSTALL_PREFIX=/usr/local -DRESTINIO_TEST=Off -DRESTINIO_SAMPLE=Off -DRESTINIO_BENCHMARK=Off \
-DRESTINIO_WITH_SOBJECTIZER=Off -DRESTINIO_DEP_STANDALONE_ASIO=system -DRESTINIO_DEP_LLHTTP=system \
-DRESTINIO_DEP_FMT=system -DRESTINIO_DEP_EXPECTED_LITE=system .
sudo make install
cd ../../ && rm -rf restinio*
- name: Configure
run: |
./autogen.sh
./configure --enable-proxy-client
- name: Build
run: make
- name: cmake
build-ubuntu-meson:
name: Ubuntu/GCC Meson build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Dependencies
run: |
sudo apt install ninja-build libncurses5-dev libreadline-dev nettle-dev libasio-dev \
libgnutls28-dev libuv1-dev python3-dev python3-setuptools python3-pip \
libcppunit-dev libjsoncpp-dev libfmt-dev libmsgpack-dev libargon2-0-dev
- name: Python dependencies
run: sudo pip3 install meson Cython
- name: Configure
run: meson setup build .
- name: Build
run: cd build && ninja
build-ubuntu-minimal:
name: Ubuntu/GCC minimal build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Dependencies
run: |
sudo apt install libncurses5-dev libreadline-dev nettle-dev libfmt-dev \
libgnutls28-dev libcppunit-dev libmsgpack-dev libargon2-0-dev
- name: Configure
run: |
mkdir build && cd build
cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local -DCMAKE_BUILD_TYPE=Debug \
-DOPENDHT_C=Off -DBUILD_TESTING=On -DOPENDHT_PEER_DISCOVERY=Off -DOPENDHT_PYTHON=Off \
-DOPENDHT_TOOLS=On -DOPENDHT_PROXY_SERVER=Off -DOPENDHT_PROXY_CLIENT=Off
- name: Build
run: cd build && make
- name: Unit tests
run: cd build && ./opendht_unit_tests
build-macos:
name: macOS/Clang build
runs-on: macos-12
steps:
- uses: actions/checkout@v3
- name: Dependencies
run: |
brew install msgpack-cxx asio gnutls nettle readline fmt jsoncpp argon2 openssl cppunit
- name: expected-lite
run: |
sudo mkdir /usr/local/include/nonstd && \
sudo wget https://raw.githubusercontent.com/martinmoene/expected-lite/master/include/nonstd/expected.hpp \
-O /usr/local/include/nonstd/expected.hpp
- name: restinio
run: |
mkdir restinio && cd restinio
wget https://github.com/Stiffstream/restinio/releases/download/v.0.7.2/restinio-0.7.2.tar.bz2
ls -l && tar -xjf restinio-0.7.2.tar.bz2
cd restinio-0.7.2/dev
cmake -DCMAKE_INSTALL_PREFIX=/usr/local -DRESTINIO_TEST=Off -DRESTINIO_SAMPLE=Off -DRESTINIO_BENCHMARK=Off \
-DRESTINIO_WITH_SOBJECTIZER=Off -DRESTINIO_DEP_STANDALONE_ASIO=system -DRESTINIO_DEP_LLHTTP=system \
-DRESTINIO_DEP_FMT=system -DRESTINIO_DEP_EXPECTED_LITE=system .
make -j2 && sudo make install
cd ../../ && rm -rf restinio*
- name: Configure
run: |
mkdir build && cd build
export PATH="/opt/homebrew/opt/openssl@3/bin:$PATH"
@@ -58,9 +117,9 @@ jobs:
export CPPFLAGS="-I/usr/local/opt/openssl@3/include"
export PKG_CONFIG_PATH="/usr/local/opt/openssl@3/lib/pkgconfig"
cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local -DCMAKE_BUILD_TYPE=Debug \
-DOPENDHT_C=On -DOPENDHT_TESTS=On -DOPENDHT_PEER_DISCOVERY=On -DOPENDHT_PYTHON=Off \
-DOPENDHT_TOOLS=On -DOPENDHT_PROXY_SERVER=On -DOPENDHT_PROXY_CLIENT=On
-DOPENDHT_C=On -DBUILD_TESTING=On -DOPENDHT_PEER_DISCOVERY=On -DOPENDHT_PYTHON=Off \
-DOPENDHT_TOOLS=On -DOPENDHT_PROXY_SERVER=On -DOPENDHT_PROXY_CLIENT=On -DOPENDHT_PUSH_NOTIFICATIONS=On
- name: make
- name: Build
run: cd build && make
+20 -14
View File
@@ -1,6 +1,6 @@
name: Clang Static Analysis
on:
push
on: [push, pull_request]
jobs:
clang-analyzer:
name: Clang Static Analysis
@@ -10,26 +10,32 @@ jobs:
- name: deps
run: |
sudo apt install libncurses5-dev libreadline-dev nettle-dev \
libgnutls28-dev libuv1-dev cython3 python3-dev python3-setuptools libcppunit-dev libjsoncpp-dev \
autotools-dev autoconf libfmt-dev libhttp-parser-dev libmsgpack-dev libargon2-0-dev libasio-dev \
libgnutls28-dev libuv1-dev python3-dev python3-setuptools libcppunit-dev libjsoncpp-dev \
autotools-dev autoconf libfmt-dev libmsgpack-dev libargon2-0-dev libasio-dev \
llvm llvm-dev clang clang-tools && \
sudo apt remove gcc g++
- name: expected-lite
run: |
sudo mkdir /usr/include/nonstd && \
sudo wget https://raw.githubusercontent.com/martinmoene/expected-lite/master/include/nonstd/expected.hpp \
-O /usr/include/nonstd/expected.hpp
- name: restinio
run: |
mkdir restinio && cd restinio \
&& wget https://github.com/aberaud/restinio/archive/e0a261dd8488246a3cb8bbb3ea781ea5139c3c94.tar.gz \
&& ls -l && tar -xzf e0a261dd8488246a3cb8bbb3ea781ea5139c3c94.tar.gz \
&& cd restinio-e0a261dd8488246a3cb8bbb3ea781ea5139c3c94/dev \
&& cmake -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=OFF -DRESTINIO_SAMPLE=OFF \
-DRESTINIO_INSTALL_SAMPLES=OFF -DRESTINIO_BENCH=OFF -DRESTINIO_INSTALL_BENCHES=OFF \
-DRESTINIO_FIND_DEPS=ON -DRESTINIO_ALLOW_SOBJECTIZER=Off -DRESTINIO_USE_BOOST_ASIO=none . \
&& make -j8 && sudo make install \
&& cd ../../.. && rm -rf restinio
mkdir restinio && cd restinio
wget https://github.com/Stiffstream/restinio/releases/download/v.0.7.2/restinio-0.7.2.tar.bz2
ls -l && tar -xjf restinio-0.7.2.tar.bz2
cd restinio-0.7.2/dev
cmake -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=Off -DRESTINIO_SAMPLE=Off -DRESTINIO_BENCHMARK=Off \
-DRESTINIO_WITH_SOBJECTIZER=Off -DRESTINIO_DEP_STANDALONE_ASIO=system -DRESTINIO_DEP_LLHTTP=system \
-DRESTINIO_DEP_FMT=system -DRESTINIO_DEP_EXPECTED_LITE=system .
make -j2 && sudo make install
cd ../../ && rm -rf restinio*
- name: cmake
run: |
mkdir build && cd build && \
cmake .. -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_BUILD_TYPE=Debug -DOPENDHT_C=On -DOPENDHT_PEER_DISCOVERY=On -DOPENDHT_PYTHON=Off -DOPENDHT_TOOLS=On -DOPENDHT_PROXY_SERVER=On -DOPENDHT_PROXY_CLIENT=On
cmake .. -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_BUILD_TYPE=Debug -DOPENDHT_C=On -DOPENDHT_PEER_DISCOVERY=On -DOPENDHT_PYTHON=Off -DOPENDHT_TOOLS=On -DOPENDHT_PROXY_SERVER=On -DOPENDHT_PROXY_CLIENT=On -DOPENDHT_PUSH_NOTIFICATIONS=On
- name: scan-build
run: cd build && scan-build --status-bugs make
+53
View File
@@ -0,0 +1,53 @@
name: "CodeQL"
on:
push:
branches: [ "master" ]
pull_request:
# The branches below must be a subset of the branches above
branches: [ "master" ]
jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest
container:
image: ghcr.io/savoirfairelinux/opendht/opendht-deps:latest
permissions:
actions: read
contents: read
security-events: write
strategy:
fail-fast: false
matrix:
language: [ 'cpp' ]
steps:
- name: Checkout repository
uses: actions/checkout@v3
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v2
with:
languages: ${{ matrix.language }}
# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v2
# ℹ️ Command-line programs to run using the OS shell.
# 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun
# If the Autobuild fails above, remove it and uncomment the following three lines.
# modify them (or add more) to build your code if your project, please refer to the EXAMPLE below for guidance.
# - run: |
# echo "Run, Build Application using script"
# ./location_of_script_within_repo/buildscript.sh
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v2
+293
View File
@@ -0,0 +1,293 @@
name: Release actions
on:
push:
tags:
- 'v*'
env:
REGISTRY: ghcr.io
IMAGE_NAME_DEPS: ${{ github.repository }}/opendht-deps
IMAGE_NAME: ${{ github.repository }}/opendht
IMAGE_NAME_DEPS_LLVM: ${{ github.repository }}/opendht-deps-llvm
IMAGE_NAME_LLVM: ${{ github.repository }}/opendht-llvm
IMAGE_NAME_DHTNODE: ${{ github.repository }}/dhtnode
IMAGE_NAME_ALPINE: ${{ github.repository }}/opendht-alpine
IMAGE_NAME_ALPINE_DEPS: ${{ github.repository }}/opendht-deps-alpine
jobs:
build-and-push-deps-image:
name: Dependency Docker image
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Checkout repository
uses: actions/checkout@v3
- name: Log in to the Container registry
uses: docker/login-action@v2
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v4
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME_DEPS }}
- name: Build and push Docker image
uses: docker/build-push-action@v3
with:
context: .
file: docker/DockerfileDeps
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
build-and-push-image:
name: OpenDHT Docker image
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Checkout repository
uses: actions/checkout@v3
- name: Log in to the Container registry
uses: docker/login-action@v2
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v4
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
- name: Build and push Docker image
uses: docker/build-push-action@v3
with:
context: .
file: docker/Dockerfile
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
build-and-push-deps-image-llvm:
name: Dependency Docker image (LLVM)
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Checkout repository
uses: actions/checkout@v3
- name: Log in to the Container registry
uses: docker/login-action@v2
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v4
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME_DEPS_LLVM }}
- name: Build and push Docker image
uses: docker/build-push-action@v3
with:
context: .
file: docker/DockerfileDepsLlvm
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
build-and-push-image-llvm:
name: OpenDHT Docker image (LLVM)
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Checkout repository
uses: actions/checkout@v3
- name: Log in to the Container registry
uses: docker/login-action@v2
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v4
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME_LLVM }}
- name: Build and push Docker image
uses: docker/build-push-action@v3
with:
context: .
file: docker/DockerfileLlvm
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
build-and-push-image-dhtnode:
name: dhtnode Docker image
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Checkout repository
uses: actions/checkout@v3
- name: Log in to the Container registry
uses: docker/login-action@v2
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v4
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME_DHTNODE }}
- name: Build and push Docker image
uses: docker/build-push-action@v3
with:
context: .
file: docker/DockerfileDhtnode
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
build-and-push-image-alpine-deps:
name: Alpine Deps Docker image
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Checkout repository
uses: actions/checkout@v3
- name: Log in to the Container registry
uses: docker/login-action@v2
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v4
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME_ALPINE_DEPS }}
- name: Build and push Docker image
uses: docker/build-push-action@v3
with:
context: .
file: docker/DockerfileDepsAlpine
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
build-and-push-image-alpine:
name: Alpine Docker image
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Checkout repository
uses: actions/checkout@v3
- name: Log in to the Container registry
uses: docker/login-action@v2
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v4
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME_ALPINE }}
- name: Build and push Docker image
uses: docker/build-push-action@v3
with:
context: .
file: docker/DockerfileAlpine
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
build-python-wheel:
name: Release and build Python Wheel package
runs-on: ubuntu-latest
container: ghcr.io/savoirfairelinux/opendht/opendht-deps:latest
permissions:
contents: write
steps:
- name: Checkout repository
uses: actions/checkout@v3
- name: cmake
run: |
mkdir build && cd build && \
cmake .. -DCMAKE_INSTALL_PREFIX=/usr \
-DCMAKE_BUILD_TYPE=Release \
-DOPENDHT_STATIC=Off \
-DOPENDHT_PYTHON=On \
-DOPENDHT_C=Off \
-DOPENDHT_PEER_DISCOVERY=On \
-DOPENDHT_TOOLS=Off \
-DOPENDHT_PROXY_SERVER=On \
-DOPENDHT_PROXY_CLIENT=On
- name: build
run: cd build && make dist
- uses: actions/upload-artifact@v3
with:
name: opendht-wheels-linux
path: build/python/dist/*.whl
- name: Create Release
id: create_release
uses: softprops/action-gh-release@v1
with:
draft: true
files: build/python/dist/*.whl
+3 -1
View File
@@ -20,10 +20,12 @@
"command": "cmake",
"args": [
"${workspaceRoot}",
"-DCMAKE_EXPORT_COMPILE_COMMANDS=On",
"-DCMAKE_BUILD_TYPE=Debug",
"-DBUILD_TESTING=On",
"-DOPENDHT_SANITIZE=On",
"-DOPENDHT_PROXY_CLIENT=On",
"-DOPENDHT_PROXY_SERVER=On",
"-DOPENDHT_TESTS=On",
"-DOPENDHT_C=On"
],
"dependsOn": "builddir"
+123 -130
View File
@@ -1,33 +1,26 @@
cmake_minimum_required (VERSION 3.10)
if(POLICY CMP0065)
cmake_policy(SET CMP0065 NEW)
endif()
if(POLICY CMP0069)
cmake_policy(SET CMP0069 NEW)
endif()
if(POLICY CMP0073)
cmake_policy(SET CMP0073 NEW)
endif()
if(POLICY CMP0074)
cmake_policy(SET CMP0074 NEW)
cmake_minimum_required (VERSION 3.16..3.28)
if(POLICY CMP0135)
cmake_policy(SET CMP0135 NEW)
endif()
project (opendht)
include(CMakePackageConfigHelpers)
include(CMakeDependentOption)
include(CheckIncludeFileCXX)
include(FindPkgConfig)
include(cmake/CheckAtomic.cmake)
include(CTest)
include(FetchContent)
set (opendht_VERSION_MAJOR 2)
set (opendht_VERSION_MINOR 4.0)
set (opendht_VERSION_MAJOR 3)
set (opendht_VERSION_MINOR 2.0)
set (opendht_VERSION ${opendht_VERSION_MAJOR}.${opendht_VERSION_MINOR})
set (PACKAGE_VERSION ${opendht_VERSION})
set (VERSION "${opendht_VERSION}")
# Options
option (OPENDHT_STATIC "Build static library" ON)
option (OPENDHT_SHARED "Build shared library" ON)
option (OPENDHT_LOG "Build with logs" ON)
option (BUILD_SHARED_LIBS "Build shared library" ON)
CMAKE_DEPENDENT_OPTION (OPENDHT_STATIC "Build static library" OFF BUILD_SHARED_LIBS ON)
option (OPENDHT_PYTHON "Build Python bindings" OFF)
option (OPENDHT_TOOLS "Build DHT tools" ON)
option (OPENDHT_SYSTEMD "Install systemd module" OFF)
@@ -41,15 +34,20 @@ option (OPENDHT_PROXY_OPENSSL "Build DHT proxy with OpenSSL" ON)
CMAKE_DEPENDENT_OPTION(OPENDHT_HTTP "Build embedded http(s) client" OFF "NOT OPENDHT_PROXY_SERVER;NOT OPENDHT_PROXY_CLIENT" ON)
option (OPENDHT_PEER_DISCOVERY "Enable multicast peer discovery" ON)
option (OPENDHT_INDEX "Build DHT indexation feature" OFF)
option (OPENDHT_TESTS "Add unit tests executable" OFF)
option (OPENDHT_TESTS_NETWORK "Enable unit tests that require network access" ON)
option (OPENDHT_C "Build C bindings" OFF)
find_package(Doxygen)
option (OPENDHT_DOCUMENTATION "Create and install the HTML based API documentation (requires Doxygen)" ${DOXYGEN_FOUND})
# Build flags
set (CMAKE_CXX_STANDARD 17)
set (CMAKE_CXX_STANDARD_REQUIRED on)
# Dependencies
if (NOT HAVE_CXX_ATOMICS_WITHOUT_LIB)
if (NOT HAVE_CXX_ATOMICS_WITHOUT_LIB
# For ARM EABI (armel), little-endian MIPS (mipsel), etc.
OR NOT HAVE_CXX_ATOMICS64_WITHOUT_LIB)
link_libraries (atomic)
endif ()
@@ -58,16 +56,27 @@ if (NOT MSVC)
set (THREADS_PREFER_PTHREAD_FLAG TRUE)
find_package (Threads)
find_package (PkgConfig REQUIRED)
find_package (GnuTLS 3.3 REQUIRED)
pkg_search_module (Nettle REQUIRED nettle)
find_package (msgpack REQUIRED)
pkg_search_module (GnuTLS REQUIRED IMPORTED_TARGET gnutls)
pkg_search_module (Nettle REQUIRED IMPORTED_TARGET nettle)
check_include_file_cxx(msgpack.hpp HAVE_MSGPACKCXX)
if (NOT HAVE_MSGPACKCXX)
find_package(msgpack QUIET CONFIG NAMES msgpack msgpackc-cxx)
if (NOT msgpack_FOUND)
find_package(msgpack REQUIRED CONFIG NAMES msgpack-cxx)
set(MSGPACK_TARGET "msgpack-cxx")
else()
set(MSGPACK_TARGET "msgpackc-cxx")
endif()
endif()
if (OPENDHT_TOOLS)
find_package (Readline 6 REQUIRED)
endif ()
pkg_search_module(argon2 REQUIRED libargon2)
pkg_search_module(Jsoncpp jsoncpp)
pkg_search_module(argon2 REQUIRED IMPORTED_TARGET libargon2)
set(argon2_lib ", libargon2")
pkg_search_module(Jsoncpp IMPORTED_TARGET jsoncpp)
if (Jsoncpp_FOUND)
add_definitions(-DOPENDHT_JSONCPP)
set(jsoncpp_lib ", jsoncpp")
list (APPEND opendht_SOURCES
src/base64.h
src/base64.cpp
@@ -78,22 +87,48 @@ if (NOT MSVC)
find_path(ASIO_INCLUDE_DIR asio.hpp REQUIRED)
endif ()
find_package(fmt)
if (OPENDHT_HTTP)
find_package(Restinio REQUIRED)
find_library(FMT_LIBRARY fmt)
add_library(fmt SHARED IMPORTED)
find_library(HTTP_PARSER_LIBRARY http_parser)
add_library(http_parser SHARED IMPORTED)
# llhttp
find_path(LLHTTP_INCLUDE_DIR llhttp.h)
find_library(LLHTTP_LIBRARY libllhttp.a)
if (LLHTTP_INCLUDE_DIR AND LLHTTP_LIBRARY)
message(STATUS "Found llhttp ${LLHTTP_INCLUDE_DIR} ${LLHTTP_LIBRARY}")
add_library(llhttp_static STATIC IMPORTED)
set_target_properties(llhttp_static PROPERTIES
IMPORTED_LOCATION ${LLHTTP_LIBRARY}
INTERFACE_INCLUDE_DIRECTORIES ${LLHTTP_INCLUDE_DIR}
)
set(llhttp_target llhttp_static)
else()
FetchContent_Declare(llhttp-local URL "https://github.com/nodejs/llhttp/archive/refs/tags/release/v9.2.0.tar.gz")
if (BUILD_SHARED_LIBS)
set(BUILD_SHARED_LIBS ON CACHE INTERNAL "")
else()
set(BUILD_SHARED_LIBS OFF CACHE INTERNAL "")
set(BUILD_STATIC_LIBS ON CACHE INTERNAL "")
endif()
FetchContent_MakeAvailable(llhttp-local)
if (BUILD_SHARED_LIBS)
set(llhttp_target llhttp_shared)
else()
set(llhttp_target llhttp_static)
endif()
endif()
set(http_lib "-lllhttp")
if (NOT Jsoncpp_FOUND)
message(SEND_ERROR "Jsoncpp is required for DHT proxy support")
endif()
if (OPENDHT_PROXY_OPENSSL)
# https://cmake.org/cmake/help/latest/module/FindOpenSSL.html
pkg_search_module(OPENSSL REQUIRED openssl)
pkg_search_module(OPENSSL REQUIRED IMPORTED_TARGET openssl)
if (OPENSSL_FOUND)
message(STATUS "Found OpenSSL ${OPENSSL_VERSION} ${OPENSSL_INCLUDE_DIRS}")
include_directories(SYSTEM ${OPENSSL_INCLUDE_DIRS})
link_directories (${OPENSSL_LIBRARY_DIRS})
set(openssl_lib ", openssl")
else ()
message(SEND_ERROR "OpenSSL is required for DHT proxy as specified")
endif()
@@ -127,14 +162,11 @@ if (OPENDHT_HTTP OR OPENDHT_PEER_DISCOVERY)
add_definitions(-DASIO_STANDALONE)
endif()
# Build flags
set (CMAKE_CXX_STANDARD 17)
set (CMAKE_CXX_STANDARD_REQUIRED on)
if (NOT MSVC)
set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-return-type -Wall -Wextra -Wnon-virtual-dtor -pedantic-errors -fvisibility=hidden")
set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-return-type -Wno-deprecated -Wall -Wextra -Wnon-virtual-dtor -pedantic-errors -fvisibility=hidden")
if (OPENDHT_SANITIZE)
set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fsanitize=address -fstack-protector-strong")
set (CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -fsanitize=address -fstack-protector-strong")
endif ()
else ()
add_definitions(-D_WINSOCK_DEPRECATED_NO_WARNINGS
@@ -148,37 +180,14 @@ else ()
endif ()
set (CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -DMSGPACK_NO_BOOST -DMSGPACK_DISABLE_LEGACY_NIL -DMSGPACK_DISABLE_LEGACY_CONVERT")
if (NOT CMAKE_BUILD_TYPE)
set(CMAKE_BUILD_TYPE Release)
endif ()
add_definitions(-DPACKAGE_VERSION="${opendht_VERSION}")
if (OPENDHT_LOG)
add_definitions(-DOPENDHT_LOG=true)
else ()
add_definitions(-DOPENDHT_LOG=false)
endif()
if (MSGPACK_INCLUDE_DIRS)
include_directories (SYSTEM "${MSGPACK_INCLUDE_DIRS}")
endif ()
if (GNUTLS_INCLUDE_DIRS)
include_directories (SYSTEM "${GNUTLS_INCLUDE_DIRS}")
endif ()
if (Nettle_INCLUDE_DIRS)
include_directories (SYSTEM "${Nettle_INCLUDE_DIRS}")
endif ()
if (ASIO_INCLUDE_DIR)
include_directories (SYSTEM "${ASIO_INCLUDE_DIR}")
endif ()
if (Restinio_INCLUDE_DIR)
include_directories (SYSTEM "${Restinio_INCLUDE_DIR}")
endif ()
if (Jsoncpp_INCLUDE_DIRS)
include_directories (SYSTEM "${Jsoncpp_INCLUDE_DIRS}")
endif ()
link_directories (${Nettle_LIBRARY_DIRS})
link_directories (${Jsoncpp_LIBRARY_DIRS})
link_directories (${argon2_LIBRARY_DIRS})
include_directories (
./
include/
@@ -190,8 +199,8 @@ include_directories (
include (GNUInstallDirs)
set (prefix ${CMAKE_INSTALL_PREFIX})
set (exec_prefix "\${prefix}")
set (libdir "\${exec_prefix}/${CMAKE_INSTALL_LIBDIR}")
set (includedir "\${prefix}/${CMAKE_INSTALL_INCLUDEDIR}")
set (libdir "${CMAKE_INSTALL_FULL_LIBDIR}")
set (includedir "${CMAKE_INSTALL_FULL_INCLUDEDIR}")
set (bindir "${CMAKE_INSTALL_FULL_BINDIR}")
set (sysconfdir "${CMAKE_INSTALL_FULL_SYSCONFDIR}")
set (top_srcdir "${CMAKE_CURRENT_SOURCE_DIR}")
@@ -199,7 +208,6 @@ set (top_srcdir "${CMAKE_CURRENT_SOURCE_DIR}")
# Sources
list (APPEND opendht_SOURCES
src/utils.cpp
src/infohash.cpp
src/crypto.cpp
src/default_types.cpp
src/node.cpp
@@ -226,6 +234,7 @@ list (APPEND opendht_SOURCES
)
list (APPEND opendht_HEADERS
include/opendht.h
include/opendht/def.h
include/opendht/utils.h
include/opendht/sockaddr.h
@@ -245,7 +254,7 @@ list (APPEND opendht_HEADERS
include/opendht/rate_limiter.h
include/opendht/securedht.h
include/opendht/log.h
include/opendht/log_enable.h
include/opendht/logger.h
include/opendht/thread_pool.h
include/opendht/network_utils.h
include/opendht.h
@@ -311,20 +320,8 @@ if (MSVC)
endif ()
# Targets
if (OPENDHT_STATIC)
if (NOT MSVC)
add_library (opendht-static STATIC
${opendht_SOURCES}
${opendht_HEADERS}
)
set_target_properties (opendht-static PROPERTIES OUTPUT_NAME "opendht")
target_include_directories(opendht-static SYSTEM PRIVATE ${argon2_INCLUDE_DIRS})
target_link_libraries(opendht-static
PRIVATE ${argon2_LIBRARIES}
PUBLIC ${CMAKE_THREAD_LIBS_INIT} ${GNUTLS_LIBRARIES} ${Nettle_STATIC_LIBRARIES}
${Jsoncpp_STATIC_LIBRARIES} ${FMT_LIBRARY} ${HTTP_PARSER_LIBRARY}
${OPENSSL_STATIC_LIBRARIES})
else ()
if (MSVC)
if (OPENDHT_STATIC)
if (OPENDHT_TOOLS)
function (add_obj_lib name libfile)
add_library(${name} OBJECT IMPORTED)
@@ -363,56 +360,65 @@ if (OPENDHT_STATIC)
${PROJECT_SOURCE_DIR}/../openssl/libcrypto.lib
)
endif ()
add_library (opendht-static STATIC
${opendht_SOURCES}
${opendht_HEADERS}
${obj_libs}
)
target_link_libraries(opendht-static PUBLIC ${Win32_STATIC_LIBRARIES} ${Win32_IMPORT_LIBRARIES})
set(CMAKE_STATIC_LINKER_FLAGS "${CMAKE_STATIC_LINKER_FLAGS} /ignore:4006")
set_target_properties (opendht-static PROPERTIES OUTPUT_NAME "libopendht")
endif()
install (TARGETS opendht-static DESTINATION ${CMAKE_INSTALL_LIBDIR} EXPORT opendht)
endif ()
if (OPENDHT_SHARED)
add_library (opendht SHARED
${opendht_SOURCES}
${opendht_HEADERS}
add_library (opendht
${opendht_SOURCES}
${opendht_HEADERS}
${obj_libs}
)
set_target_properties (opendht PROPERTIES OUTPUT_NAME "opendht")
if (NOT HAVE_MSGPACKCXX)
target_link_libraries(opendht PUBLIC ${MSGPACK_TARGET})
endif()
if (APPLE)
target_link_libraries(opendht PRIVATE "-framework CoreFoundation" "-framework Security")
endif()
if (MSVC)
if (OPENDHT_STATIC)
target_link_libraries(opendht PUBLIC ${Win32_STATIC_LIBRARIES} ${Win32_IMPORT_LIBRARIES})
set_target_properties (opendht PROPERTIES OUTPUT_NAME "libopendht")
endif()
else()
target_link_libraries(opendht
PRIVATE
PkgConfig::argon2
PkgConfig::Nettle
PUBLIC
${CMAKE_THREAD_LIBS_INIT}
PkgConfig::GnuTLS
fmt::fmt
)
if (OPENDHT_HTTP)
target_link_libraries(opendht PUBLIC ${llhttp_target})
endif()
if (Jsoncpp_FOUND)
target_link_libraries(opendht PUBLIC PkgConfig::Jsoncpp)
endif()
if (OPENDHT_PROXY_OPENSSL)
target_link_libraries(opendht PUBLIC PkgConfig::OPENSSL)
endif()
endif()
if (BUILD_SHARED_LIBS)
set_target_properties (opendht PROPERTIES IMPORT_SUFFIX "_import.lib")
set_target_properties (opendht PROPERTIES SOVERSION ${opendht_VERSION_MAJOR} VERSION ${opendht_VERSION})
target_compile_definitions(opendht PRIVATE OPENDHT_BUILD)
target_include_directories(opendht SYSTEM PRIVATE ${argon2_INCLUDE_DIRS})
if (APPLE)
target_link_libraries(opendht
PUBLIC ${CMAKE_THREAD_LIBS_INIT} ${OPENSSL_LIBRARIES}
PRIVATE ${GNUTLS_LIBRARIES} ${Nettle_LIBRARIES}
${Jsoncpp_LIBRARIES}
${FMT_LIBRARY} ${HTTP_PARSER_LIBRARY} ${argon2_LIBRARIES}
SYSTEM "-framework CoreFoundation" "-framework Security")
else ()
target_link_libraries(opendht
PUBLIC ${CMAKE_THREAD_LIBS_INIT} ${OPENSSL_LIBRARIES}
PRIVATE ${GNUTLS_LIBRARIES} ${Nettle_LIBRARIES}
${Jsoncpp_LIBRARIES}
${FMT_LIBRARY} ${HTTP_PARSER_LIBRARY} ${argon2_LIBRARIES})
endif ()
install (TARGETS opendht DESTINATION ${CMAKE_INSTALL_LIBDIR} EXPORT opendht)
endif ()
install (TARGETS opendht DESTINATION ${CMAKE_INSTALL_LIBDIR} EXPORT opendht)
if (OPENDHT_C)
add_library (opendht-c SHARED
add_library (opendht-c
c/opendht.cpp
c/opendht_c.h
)
target_compile_definitions(opendht-c PRIVATE OPENDHT_C_BUILD)
if (OPENDHT_SHARED)
target_link_libraries(opendht-c opendht)
else ()
target_link_libraries(opendht-c opendht-static)
endif ()
target_link_libraries(opendht-c PRIVATE opendht)
set_target_properties (opendht-c PROPERTIES SOVERSION ${opendht_VERSION_MAJOR} VERSION ${opendht_VERSION})
install (TARGETS opendht-c DESTINATION ${CMAKE_INSTALL_LIBDIR} EXPORT opendht-c)
# PkgConfig module
configure_file (
@@ -420,7 +426,6 @@ if (OPENDHT_C)
opendht-c.pc
@ONLY
)
install (TARGETS opendht-c DESTINATION ${CMAKE_INSTALL_LIBDIR} EXPORT opendht-c)
install (FILES ${CMAKE_CURRENT_BINARY_DIR}/opendht-c.pc DESTINATION ${CMAKE_INSTALL_LIBDIR}/pkgconfig)
install (FILES c/opendht_c.h DESTINATION ${CMAKE_INSTALL_PREFIX}/include/opendht)
endif ()
@@ -453,8 +458,8 @@ install (EXPORT opendht DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake/opendht FILE o
install (FILES ${CMAKE_CURRENT_BINARY_DIR}/opendhtConfigVersion.cmake DESTINATION ${CMAKE_INSTALL_LIBDIR}/cmake/opendht)
# Unit tests
if (OPENDHT_TESTS)
pkg_search_module(Cppunit REQUIRED cppunit)
if (BUILD_TESTING AND NOT MSVC)
pkg_search_module(Cppunit REQUIRED IMPORTED_TARGET cppunit)
# unit testing
list (APPEND test_FILES
tests/infohashtester.h
@@ -488,22 +493,10 @@ if (OPENDHT_TESTS)
tests/tests_runner.cpp
${test_FILES}
)
target_include_directories(opendht_unit_tests SYSTEM PRIVATE ${Cppunit_INCLUDE_DIR})
target_link_directories(opendht_unit_tests PRIVATE ${Cppunit_LIBRARY_DIRS})
if (OPENDHT_SHARED)
target_link_libraries(opendht_unit_tests opendht)
else ()
target_link_libraries(opendht_unit_tests opendht-static)
endif ()
target_link_libraries(opendht_unit_tests
target_link_libraries(opendht_unit_tests PRIVATE
opendht
${CMAKE_THREAD_LIBS_INIT}
${Cppunit_LIBRARIES}
${GNUTLS_LIBRARIES}
${Jsoncpp_LIBRARIES}
PkgConfig::Cppunit
)
if (OPENDHT_PROXY_OPENSSL)
target_link_libraries(opendht_unit_tests ${OPENSSL_LIBRARIES})
endif()
enable_testing()
add_test(TEST opendht_unit_tests)
endif()
+8 -10
View File
@@ -21,15 +21,15 @@ Values are distributed over the network, with redundancy.
## Documentation
See the wiki: <https://github.com/savoirfairelinux/opendht/wiki>
#### How to run a node
You can help contributing to the public network by running a stable node with a public IP address.
https://github.com/savoirfairelinux/opendht/wiki/Running-a-node-with-dhtnode
#### How-to build and install
Build instructions: <https://github.com/savoirfairelinux/opendht/wiki/Build-the-library>
#### How-to build a simple client app
```bash
g++ main.cpp -std=c++17 -lopendht
```
## Examples
### C++ example
The `tools` directory includes simple example programs :
@@ -96,17 +96,18 @@ for r in results:
- msgpack-c 1.2+, used for data serialization.
- GnuTLS 3.3+, used for cryptographic operations.
- Nettle 2.4+, a GnuTLS dependency for crypto.
- {fmt} 9.0+, for log formatting.
- (optional) restinio used for the REST API.
- (optional) jsoncpp 1.7.4-3+, used for the REST API.
- Build tested with GCC 7+ (GNU/Linux, Windows with MinGW), Clang/LLVM (GNU/Linux, Android, macOS, iOS).
- Build tested with Microsoft Visual Studio 2017, 2019, 2022
- Build tested with Microsoft Visual Studio 2019, 2022
## Contact
IRC: join us on Libera.chat at [`#opendht`](https://web.libera.chat/#opendht).
## License
Copyright (C) 2014-2022 Savoir-faire Linux Inc.
Copyright (C) 2014-2023 Savoir-faire Linux Inc.
OpenDHT is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version.
@@ -117,6 +118,3 @@ See COPYING or https://www.gnu.org/licenses/gpl-3.0.en.html for the full GPLv3 l
## Acknowledgements
This project was originally based on https://github.com/jech/dht by Juliusz Chroboczek.
It is independent from another project called OpenDHT (Sean Rhea. Ph.D. Thesis, 2005), now extinct.
## Donations
We gratefully accept Bitcoin donations to support OpenDHT development at: `bitcoin:3EykSd1An888efq4Bq3KaV3hJ3JQ4FPnwm`.
+4 -1
View File
@@ -1,7 +1,10 @@
lib_LTLIBRARIES = libopendht-c.la
noinst_HEADERS = opendht_c.h
AM_CPPFLAGS = -isystem @top_srcdir@/include @JsonCpp_CFLAGS@ @MsgPack_CFLAGS@
AM_CPPFLAGS = -DOPENDHT_C_BUILD -isystem @top_srcdir@/include @JsonCpp_CFLAGS@ @MsgPack_CFLAGS@
if OPENDHT_SHARED
AM_CPPFLAGS += -Dopendht_c_EXPORTS
endif
libopendht_c_la_LIBADD = $(LIBOBJS) ../src/.libs/libopendht.la
+16 -11
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author : Adrien Béraud <adrien.beraud@savoirfairelinux.com>
*
* This program is free software; you can redistribute it and/or modify
@@ -32,6 +32,11 @@ extern "C" {
#include <errno.h>
const char* dht_version()
{
return dht::version();
}
// dht::InfoHash
inline dht_infohash dht_infohash_to_c(const dht::InfoHash& h) {
@@ -71,7 +76,11 @@ bool dht_infohash_is_zero(const dht_infohash* h) {
}
void dht_infohash_from_hex(dht_infohash* h, const char* dat) {
*h = dht_infohash_to_c(dht::InfoHash(std::string(dat, HASH_LEN*2)));
*h = dht_infohash_to_c(dht::InfoHash(std::string_view(dat, HASH_LEN*2)));
}
void dht_infohash_from_hex_null(dht_infohash* h, const char* dat) {
*h = dht_infohash_to_c(dht::InfoHash(std::string_view(dat)));
}
const char* dht_pkid_print(const dht_pkid* h) {
@@ -252,7 +261,7 @@ dht_pkid dht_certificate_get_long_id(const dht_certificate* c) {
dht_publickey* dht_certificate_get_publickey(const dht_certificate* c) {
const auto& cert = *reinterpret_cast<const CertSp*>(c);
return reinterpret_cast<dht_publickey*>(new PubkeySp(std::make_shared<dht::crypto::PublicKey>(cert->getPublicKey())));
return reinterpret_cast<dht_publickey*>(new PubkeySp(cert->getSharedPublicKey()));
}
// dht::crypto::Identity
@@ -333,6 +342,8 @@ int dht_runner_run_config(dht_runner* r, in_port_t port, const dht_runner_config
config.proxy_server = conf->proxy_server ? std::string(conf->proxy_server) : std::string{};
config.push_node_id = conf->push_node_id ? std::string(conf->push_node_id) : std::string{};
config.push_token = conf->push_token ? std::string(conf->push_token) : std::string{};
config.push_topic = conf->push_topic ? std::string(conf->push_topic) : std::string{};
config.push_platform = conf->push_platform ? std::string(conf->push_platform) : std::string{};
config.peer_discovery = conf->peer_discovery;
config.peer_publish = conf->peer_publish;
@@ -495,14 +506,8 @@ struct sockaddr** dht_runner_get_public_address(const dht_runner* r) {
if (addrs.empty())
return nullptr;
auto ret = (struct sockaddr**)malloc(sizeof(struct sockaddr*) * (addrs.size() + 1));
for (size_t i=0; i<addrs.size(); i++) {
if (const auto& addr = addrs[i]) {
ret[i] = (struct sockaddr*)malloc(addr.getLength());
memcpy((struct sockaddr*)ret[i], addr.get(), addr.getLength());
} else {
ret[i] = nullptr;
}
}
for (size_t i=0; i<addrs.size(); i++)
ret[i] = addrs[i].release();
ret[addrs.size()] = nullptr;
return ret;
}
+6 -1
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author : Adrien Béraud <adrien.beraud@savoirfairelinux.com>
*
* This program is free software; you can redistribute it and/or modify
@@ -36,6 +36,8 @@ struct OPENDHT_C_PUBLIC dht_data_view {
};
typedef struct dht_data_view dht_data_view;
OPENDHT_C_PUBLIC const char* dht_version(void);
// dht::Blob
struct OPENDHT_C_PUBLIC dht_blob;
typedef struct dht_blob dht_blob;
@@ -48,6 +50,7 @@ typedef struct dht_infohash dht_infohash;
OPENDHT_C_PUBLIC void dht_infohash_zero(dht_infohash* h);
OPENDHT_C_PUBLIC void dht_infohash_random(dht_infohash* h);
OPENDHT_C_PUBLIC void dht_infohash_from_hex(dht_infohash* h, const char* dat);
OPENDHT_C_PUBLIC void dht_infohash_from_hex_null(dht_infohash* h, const char* dat);
OPENDHT_C_PUBLIC void dht_infohash_get(dht_infohash* h, const uint8_t* dat, size_t dat_size);
OPENDHT_C_PUBLIC void dht_infohash_get_from_string(dht_infohash* h, const char* str);
OPENDHT_C_PUBLIC const char* dht_infohash_print(const dht_infohash* h);
@@ -143,6 +146,8 @@ struct OPENDHT_PUBLIC dht_runner_config {
const char* proxy_server;
const char* push_node_id;
const char* push_token;
const char* push_topic;
const char* push_platform;
bool peer_discovery;
bool peer_publish;
dht_certificate* server_ca;
+36 -36
View File
@@ -1,6 +1,6 @@
dnl define macros
m4_define([opendht_major_version], 2)
m4_define([opendht_minor_version], 4)
m4_define([opendht_major_version], 3)
m4_define([opendht_minor_version], 2)
m4_define([opendht_patch_version], 0)
m4_define([opendht_version],
[opendht_major_version.opendht_minor_version.opendht_patch_version])
@@ -18,20 +18,22 @@ AC_SUBST(OPENDHT_PATCH_VERSION, opendht_patch_version)
AC_ARG_ENABLE([debug], AS_HELP_STRING([--enable-debug], [Build in debug mode, adds stricter warnings, disables optimization]))
AS_IF([test "x$enable_debug" = "xyes"],
[CXXFLAGS="${CXXFLAGS} -g -Wno-return-type -Wall -Wextra -Wnon-virtual-dtor -O0 -pedantic-errors"],
[CXXFLAGS="${CXXFLAGS} -O3 -pedantic-errors"])
[CXXFLAGS="${CXXFLAGS} -g -Og -Wno-return-type -Wall -Wextra -Wnon-virtual-dtor -Wno-deprecated -pedantic-errors"],
[CXXFLAGS="${CXXFLAGS} -O3 -Wno-deprecated -pedantic-errors -fvisibility=hidden"])
AC_PROG_CXX
AM_PROG_AR
dnl Check for logs
AC_ARG_ENABLE([logs], [AS_HELP_STRING([--disable-logs], [Disable DHT logs])])
AS_IF([test "x$enable_logs" != "xno"], [
AC_DEFINE([OPENDHT_LOG], [true], [Define if DHT logs are enabled])
], [
AC_DEFINE([OPENDHT_LOG], [false], [Define if DHT logs are enabled])
CPPFLAGS="${CPPFLAGS} -DOPENDHT_BUILD"
AM_CONDITIONAL([OPENDHT_SHARED], [test "x$enable_shared" != xno])
AM_COND_IF(OPENDHT_SHARED, [
CPPFLAGS="${CPPFLAGS} -Dopendht_EXPORTS"
])
AM_PROG_AR
LT_INIT()
LT_LANG(C++)
AC_LANG(C++)
AC_PROG_CXX
AX_CXX_COMPILE_STDCXX(17,[noext],[mandatory])
dnl Check for C binding
AC_ARG_ENABLE([c], [AS_HELP_STRING([--disable-c], [Disable DHT C binding])])
AM_CONDITIONAL(ENABLE_C, test x$enable_c != "xno")
@@ -106,11 +108,6 @@ AM_CONDITIONAL(WIN32, [test "x$SYS" = "xmingw32"])
AS_IF([test "x$SYS" = "xandroid"],
[], [LDFLAGS="${LDFLAGS} -lpthread"])
LT_INIT()
LT_LANG(C++)
AX_CXX_COMPILE_STDCXX(17,[noext],[mandatory])
PKG_PROG_PKG_CONFIG()
AC_ARG_ENABLE([proxy_server], AS_HELP_STRING([--enable-proxy-server], [Enable proxy server ability]), proxy_server=yes, proxy_server=no)
@@ -129,8 +126,8 @@ AC_ARG_ENABLE([proxy_client], AS_HELP_STRING([--enable-proxy-client], [Enable pr
AM_CONDITIONAL(ENABLE_PROXY_CLIENT, test x$proxy_client == xyes)
AC_ARG_ENABLE([tests], AS_HELP_STRING([--enable-tests], [Enable tests]), build_tests=yes, build_tests=no)
AM_CONDITIONAL(ENABLE_TESTS, test x$build_tests == xyes)
AM_COND_IF([ENABLE_TESTS], [
AM_CONDITIONAL(ENABLE_TESTS, test "x$build_tests" == xyes)
AS_IF([test "x$build_tests" == xyes], [
PKG_CHECK_MODULES([CppUnit], [cppunit >= 1.12])
])
@@ -138,7 +135,9 @@ AM_CONDITIONAL(PROXY_CLIENT_OR_SERVER, test x$proxy_client == xyes || test x$pro
PKG_CHECK_MODULES([Nettle], [nettle >= 2.4])
PKG_CHECK_MODULES([GnuTLS], [gnutls >= 3.3])
PKG_CHECK_MODULES([MsgPack], [msgpack >= 1.2])
AC_CHECK_HEADERS([msgpack.hpp], [], [
PKG_CHECK_MODULES([MsgPack], [msgpack >= 1.2])
])
PKG_CHECK_MODULES([Argon2], [libargon2])
AC_SUBST(argon2_lib, [", libargon2"])
@@ -148,7 +147,7 @@ AS_IF([test "x$with_jsoncpp" != "xno"],
[have_jsoncpp=no])
AS_IF([test "x$have_jsoncpp" = "xyes"], [
AC_MSG_NOTICE([Using JsonCpp])
CPPFLAGS+=" -DOPENDHT_JSONCPP"
CPPFLAGS="${CPPFLAGS} -DOPENDHT_JSONCPP"
AC_SUBST(jsoncpp_lib, [", jsoncpp"])
], [
AC_MSG_NOTICE([Not using JsonCpp])
@@ -166,20 +165,21 @@ AS_IF([test "x$have_openssl" = "xyes"], [
AC_MSG_NOTICE([Not using OpenSSL])
])
PKG_CHECK_MODULES([Fmt], [fmt >= 5.3.0], [have_fmt=yes], [have_fmt=no])
AS_IF([test "x$have_fmt" = "xyes"], [
AC_MSG_NOTICE([Using libfmt])
], [
AC_MSG_NOTICE([Missing libfmt files])
])
AM_COND_IF([PROXY_CLIENT_OR_SERVER], [
AC_CHECK_HEADERS([asio.hpp], exit,, AC_MSG_ERROR([Missing Asio headers files]))
AC_CHECK_HEADERS([asio.hpp],, AC_MSG_ERROR([Missing Asio headers files]))
CXXFLAGS="${CXXFLAGS} -DASIO_STANDALONE"
PKG_CHECK_MODULES([Fmt], [fmt >= 5.3.0], [have_fmt=yes], [have_fmt=no])
AS_IF([test "x$have_fmt" = "xyes"], [
AC_MSG_NOTICE([Using libfmt])
], [
AC_MSG_NOTICE([Missing libfmt files])
])
# http_parser has no pkgconfig, instead we check with:
AC_CHECK_LIB(http_parser, exit,, AC_MSG_ERROR([Missing HttpParser library files]))
AC_CHECK_HEADERS([http_parser.h], [http_parser_headers=yes; break;])
AC_SUBST(http_parser_lib, ["-lhttp_parser"])
AS_IF([test "x$http_parser_headers" != "xyes"], AC_MSG_ERROR([Missing HttpParser headers files]))
# llhttp has no pkgconfig, instead we check with:
AC_CHECK_LIB(llhttp, exit,, AC_MSG_ERROR([Missing llhttp library files]))
AC_CHECK_HEADERS([llhttp.h], [llhttp_headers=yes; break;])
AC_SUBST(http_lib, ["-lllhttp"])
AS_IF([test "x$llhttp_headers" != "xyes"], AC_MSG_ERROR([Missing llhttp headers files]))
])
CXXFLAGS="${CXXFLAGS} -DMSGPACK_NO_BOOST -DMSGPACK_DISABLE_LEGACY_NIL -DMSGPACK_DISABLE_LEGACY_CONVERT"
@@ -194,8 +194,8 @@ AM_COND_IF([ENABLE_TOOLS], [
AM_COND_IF(ENABLE_PROXY_SERVER, AC_DEFINE([OPENDHT_PROXY_SERVER], [], [Building with proxy server]))
AM_COND_IF(ENABLE_PROXY_CLIENT, AC_DEFINE([OPENDHT_PROXY_CLIENT], [], [Building with proxy client]))
AM_COND_IF(ENABLE_PUSH_NOTIFICATIONS, [CPPFLAGS+=" -DOPENDHT_PUSH_NOTIFICATIONS"], [])
AM_COND_IF(ENABLE_PROXY_SERVER_IDENTITY, [CPPFLAGS+=" -DOPENDHT_PROXY_SERVER_IDENTITY"], [])
AM_COND_IF(ENABLE_PUSH_NOTIFICATIONS, [CPPFLAGS="${CPPFLAGS} -DOPENDHT_PUSH_NOTIFICATIONS"], [])
AM_COND_IF(ENABLE_PROXY_SERVER_IDENTITY, [CPPFLAGS="${CPPFLAGS} -DOPENDHT_PROXY_SERVER_IDENTITY"], [])
dnl Configure setup.py if we build the python module
AC_SUBST(CURRENT_SOURCE_DIR, ".")
+2 -1
View File
@@ -19,7 +19,7 @@
.SH DESCRIPTION
Runs an OpenDHT node, with a CLI (default) or as a daemon (with \fB'-d'\fP or \fB'-s'\fP).
Commands available in the interactive shell are:
.EE
.EX
h, help Print this help message.
q, quit Quit the program.
log Start/stop printing DHT logs.
@@ -39,6 +39,7 @@ Commands available in the interactive shell are:
private key.
e [key] [dest] [str] Put string value at [key], encrypted for [dest] with
its public key (if found).
.EE
.SH OPTIONS
.TP
\fB\-h\fP
+4 -2
View File
@@ -2,8 +2,9 @@ FROM ghcr.io/savoirfairelinux/opendht/opendht-deps:latest
LABEL maintainer="Adrien Béraud <adrien.beraud@savoirfairelinux.com>"
LABEL org.opencontainers.image.source https://github.com/savoirfairelinux/opendht
RUN git clone https://github.com/savoirfairelinux/opendht.git \
&& cd opendht && mkdir build && cd build \
COPY . opendht
RUN cd opendht && mkdir build && cd build \
&& cmake .. -DCMAKE_INSTALL_PREFIX=/usr \
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=On \
-DOPENDHT_C=On \
@@ -12,5 +13,6 @@ RUN git clone https://github.com/savoirfairelinux/opendht.git \
-DOPENDHT_TOOLS=On \
-DOPENDHT_PROXY_SERVER=On \
-DOPENDHT_PROXY_CLIENT=On \
-DOPENDHT_SYSTEMD=On \
&& make -j8 && make install \
&& cd ../.. && rm -rf opendht
+37
View File
@@ -0,0 +1,37 @@
FROM ghcr.io/savoirfairelinux/opendht/opendht-deps-alpine:latest AS build
LABEL maintainer="Adrien Béraud <adrien.beraud@savoirfairelinux.com>"
LABEL org.opencontainers.image.source https://github.com/savoirfairelinux/opendht
COPY . opendht
RUN mkdir /install
ENV DESTDIR /install
RUN cd opendht && mkdir build && cd build \
&& cmake .. -DCMAKE_INSTALL_PREFIX=/usr \
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=On \
-DOPENDHT_C=On \
-DOPENDHT_PEER_DISCOVERY=On \
-DOPENDHT_PYTHON=On \
-DOPENDHT_TOOLS=On \
-DOPENDHT_PROXY_SERVER=On \
-DOPENDHT_PROXY_CLIENT=On \
-DOPENDHT_SYSTEMD=On \
&& make -j8 && make install
FROM alpine:3.19 AS install
COPY --from=build /install /
RUN apk add --no-cache \
libstdc++ \
gnutls \
nettle \
openssl \
argon2-dev \
jsoncpp \
llhttp \
fmt \
readline \
ncurses
CMD ["dhtnode", "-b", "bootstrap.jami.net", "-p", "4222", "--proxyserver", "8080"]
EXPOSE 4222/udp
EXPOSE 8080/tcp
+3 -2
View File
@@ -2,8 +2,9 @@ FROM ghcr.io/savoirfairelinux/opendht/opendht-deps-bionic:latest
LABEL maintainer="Adrien Béraud <adrien.beraud@savoirfairelinux.com>"
LABEL org.opencontainers.image.source https://github.com/savoirfairelinux/opendht
RUN git clone https://github.com/savoirfairelinux/opendht.git \
&& cd opendht && mkdir build && cd build \
COPY . opendht
RUN cd opendht && mkdir build && cd build \
&& cmake .. -DCMAKE_INSTALL_PREFIX=/usr \
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=On \
-DOPENDHT_C=On \
+17 -11
View File
@@ -1,4 +1,4 @@
FROM ubuntu:20.04
FROM ubuntu:22.04
LABEL maintainer="Adrien Béraud <adrien.beraud@savoirfairelinux.com>"
LABEL org.opencontainers.image.source https://github.com/savoirfairelinux/opendht
@@ -10,19 +10,25 @@ RUN apt-get update && apt-get install -y \
RUN apt-get update && apt-get install -y \
build-essential pkg-config cmake git wget \
libtool autotools-dev autoconf \
cython3 python3-dev python3-setuptools \
python3-pip python3-dev python3-setuptools python3-build python3-virtualenv \
libncurses5-dev libreadline-dev nettle-dev libcppunit-dev \
libgnutls28-dev libuv1-dev libjsoncpp-dev libargon2-dev \
libssl-dev libfmt-dev libhttp-parser-dev libasio-dev libmsgpack-dev \
libssl-dev libfmt-dev libasio-dev libmsgpack-dev \
&& apt-get clean && rm -rf /var/lib/apt/lists/* /var/cache/apt/*
RUN pip3 install meson Cython
RUN mkdir /usr/include/nonstd \
&& wget https://raw.githubusercontent.com/martinmoene/expected-lite/master/include/nonstd/expected.hpp \
-O /usr/include/nonstd/expected.hpp
RUN echo "*** Downloading RESTinio ***" \
&& mkdir restinio && cd restinio \
&& wget https://github.com/aberaud/restinio/archive/e0a261dd8488246a3cb8bbb3ea781ea5139c3c94.tar.gz \
&& ls -l && tar -xzf e0a261dd8488246a3cb8bbb3ea781ea5139c3c94.tar.gz \
&& cd restinio-e0a261dd8488246a3cb8bbb3ea781ea5139c3c94/dev \
&& cmake -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=OFF -DRESTINIO_SAMPLE=OFF \
-DRESTINIO_INSTALL_SAMPLES=OFF -DRESTINIO_BENCH=OFF -DRESTINIO_INSTALL_BENCHES=OFF \
-DRESTINIO_FIND_DEPS=ON -DRESTINIO_ALLOW_SOBJECTIZER=Off -DRESTINIO_USE_BOOST_ASIO=none . \
&& make -j8 && make install \
&& cd ../../.. && rm -rf restinio
&& wget https://github.com/Stiffstream/restinio/releases/download/v.0.7.2/restinio-0.7.2.tar.bz2 \
&& ls -l && tar -xjf restinio-0.7.2.tar.bz2 \
&& cd restinio-0.7.2/dev \
&& cmake -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=Off -DRESTINIO_SAMPLE=Off -DRESTINIO_BENCHMARK=Off \
-DRESTINIO_WITH_SOBJECTIZER=Off -DRESTINIO_DEP_STANDALONE_ASIO=system -DRESTINIO_DEP_LLHTTP=system \
-DRESTINIO_DEP_FMT=system -DRESTINIO_DEP_EXPECTED_LITE=system . \
&& make -j2 && make install \
&& cd ../../ && rm -rf restinio*
+27
View File
@@ -0,0 +1,27 @@
FROM alpine:3.19
LABEL maintainer="Adrien Béraud <adrien.beraud@savoirfairelinux.com>"
LABEL org.opencontainers.image.source https://github.com/savoirfairelinux/opendht
RUN apk add --no-cache \
build-base cmake ninja git wget \
cython python3-dev py3-setuptools \
ncurses-dev readline-dev nettle-dev \
cppunit-dev gnutls-dev jsoncpp-dev \
argon2-dev openssl-dev fmt-dev \
llhttp-dev asio-dev msgpack-cxx-dev \
&& rm -rf /var/cache/apk/*
RUN mkdir /usr/include/nonstd \
&& wget https://raw.githubusercontent.com/martinmoene/expected-lite/master/include/nonstd/expected.hpp \
-O /usr/include/nonstd/expected.hpp
RUN echo "*** Downloading RESTinio ***" \
&& mkdir restinio && cd restinio \
&& wget https://github.com/Stiffstream/restinio/releases/download/v.0.7.2/restinio-0.7.2.tar.bz2 \
&& ls -l && tar -xjf restinio-0.7.2.tar.bz2 \
&& cd restinio-0.7.2/dev \
&& cmake -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=Off -DRESTINIO_SAMPLE=Off -DRESTINIO_BENCHMARK=Off \
-DRESTINIO_WITH_SOBJECTIZER=Off -DRESTINIO_DEP_STANDALONE_ASIO=system -DRESTINIO_DEP_LLHTTP=system \
-DRESTINIO_DEP_FMT=system -DRESTINIO_DEP_EXPECTED_LITE=system . \
&& make -j2 && make install \
&& cd ../../ && rm -rf restinio*
+14 -10
View File
@@ -5,12 +5,12 @@ LABEL org.opencontainers.image.source https://github.com/savoirfairelinux/opendh
RUN echo "APT::Acquire::Retries \"3\";" > /etc/apt/apt.conf.d/80-retries
RUN apt-get update && apt-get install -y \
apt-transport-https build-essential pkg-config git wget libncurses5-dev libreadline-dev nettle-dev \
libgnutls28-dev libuv1-dev cython3 python3-dev python3-setuptools libcppunit-dev libjsoncpp-dev \
libgnutls28-dev libuv1-dev python3-dev python3-setuptools libcppunit-dev libjsoncpp-dev \
libargon2-0-dev \
autotools-dev autoconf libfmt-dev libhttp-parser-dev libmsgpack-dev libssl-dev python3-pip \
autotools-dev autoconf libfmt-dev libmsgpack-dev libssl-dev python3-pip \
&& apt-get clean && rm -rf /var/lib/apt/lists/* /var/cache/apt/*
RUN pip3 install --upgrade cmake
RUN pip3 install --upgrade cmake meson Cython
# libasio-dev (1.10) is too old
RUN echo "** Building a recent version of asio ***" \
@@ -20,13 +20,17 @@ RUN echo "** Building a recent version of asio ***" \
&& make install \
&& cd ../../ && rm -rf asio*
RUN mkdir /usr/include/nonstd \
&& wget https://raw.githubusercontent.com/martinmoene/expected-lite/master/include/nonstd/expected.hpp \
-O /usr/include/nonstd/expected.hpp
RUN echo "*** Downloading RESTinio ***" \
&& mkdir restinio && cd restinio \
&& wget https://github.com/aberaud/restinio/archive/8d5d3e8237e0947adb9ba1ffc8281f4ad7cb2a59.tar.gz \
&& ls -l && tar -xzf 8d5d3e8237e0947adb9ba1ffc8281f4ad7cb2a59.tar.gz \
&& cd restinio-8d5d3e8237e0947adb9ba1ffc8281f4ad7cb2a59/dev \
&& cmake -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=OFF -DRESTINIO_SAMPLE=OFF \
-DRESTINIO_INSTALL_SAMPLES=OFF -DRESTINIO_BENCH=OFF -DRESTINIO_INSTALL_BENCHES=OFF \
-DRESTINIO_FIND_DEPS=ON -DRESTINIO_ALLOW_SOBJECTIZER=Off -DRESTINIO_USE_BOOST_ASIO=none . \
&& make -j8 && make install \
&& wget https://github.com/Stiffstream/restinio/releases/download/v.0.7.2/restinio-0.7.2.tar.bz2 \
&& ls -l && tar -xjf restinio-0.7.2.tar.bz2 \
&& cd restinio-0.7.2/dev \
&& cmake -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=Off -DRESTINIO_SAMPLE=Off -DRESTINIO_BENCHMARK=Off \
-DRESTINIO_WITH_SOBJECTIZER=Off -DRESTINIO_DEP_STANDALONE_ASIO=system -DRESTINIO_DEP_LLHTTP=system \
-DRESTINIO_DEP_FMT=system -DRESTINIO_DEP_EXPECTED_LITE=system . \
&& make -j2 && make install \
&& cd ../../ && rm -rf restinio*
+34
View File
@@ -0,0 +1,34 @@
FROM ubuntu:20.04
LABEL maintainer="Adrien Béraud <adrien.beraud@savoirfairelinux.com>"
LABEL org.opencontainers.image.source https://github.com/savoirfairelinux/opendht
RUN apt-get update && apt-get install -y \
dialog apt-utils \
&& apt-get clean \
&& echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections
RUN apt-get update && apt-get install -y \
build-essential pkg-config cmake git wget \
libtool autotools-dev autoconf \
python3-dev python3-setuptools python3-pip \
libncurses5-dev libreadline-dev nettle-dev libcppunit-dev \
libgnutls28-dev libuv1-dev libjsoncpp-dev libargon2-dev \
libssl-dev libfmt-dev libasio-dev libmsgpack-dev \
&& apt-get clean && rm -rf /var/lib/apt/lists/* /var/cache/apt/*
RUN pip3 install meson Cython
RUN mkdir /usr/include/nonstd \
&& wget https://raw.githubusercontent.com/martinmoene/expected-lite/master/include/nonstd/expected.hpp \
-O /usr/include/nonstd/expected.hpp
RUN echo "*** Downloading RESTinio ***" \
&& mkdir restinio && cd restinio \
&& wget https://github.com/Stiffstream/restinio/releases/download/v.0.7.2/restinio-0.7.2.tar.bz2 \
&& ls -l && tar -xjf restinio-0.7.2.tar.bz2 \
&& cd restinio-0.7.2/dev \
&& cmake -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=Off -DRESTINIO_SAMPLE=Off -DRESTINIO_BENCHMARK=Off \
-DRESTINIO_WITH_SOBJECTIZER=Off -DRESTINIO_DEP_STANDALONE_ASIO=system -DRESTINIO_DEP_LLHTTP=system \
-DRESTINIO_DEP_FMT=system -DRESTINIO_DEP_EXPECTED_LITE=system . \
&& make -j2 && make install \
&& cd ../../ && rm -rf restinio*
+17 -11
View File
@@ -1,4 +1,4 @@
FROM ubuntu:20.04
FROM ubuntu:22.04
LABEL maintainer="Adrien Béraud <adrien.beraud@savoirfairelinux.com>"
LABEL org.opencontainers.image.source https://github.com/savoirfairelinux/opendht
@@ -10,23 +10,29 @@ RUN apt-get update && apt-get install -y \
RUN apt-get update \
&& apt-get install -y llvm llvm-dev lldb clang gdb make cmake pkg-config \
libtool git wget libncurses5-dev libreadline-dev \
nettle-dev libgnutls28-dev libuv1-dev libmsgpack-dev libjsoncpp-dev cython3 python3-dev \
python3-setuptools libcppunit-dev python3-pip \
nettle-dev libgnutls28-dev libuv1-dev libmsgpack-dev libjsoncpp-dev python3-dev \
python3-setuptools libcppunit-dev python3-pip python3-build python3-virtualenv \
autotools-dev autoconf libssl-dev libargon2-dev \
libfmt-dev libhttp-parser-dev libasio-dev \
libfmt-dev libasio-dev \
&& apt-get remove -y gcc g++ && apt-get autoremove -y \
&& apt-get clean && rm -rf /var/lib/apt/lists/* /var/cache/apt/*
RUN pip3 install meson Cython
ENV CC cc
ENV CXX c++
RUN mkdir /usr/include/nonstd \
&& wget https://raw.githubusercontent.com/martinmoene/expected-lite/master/include/nonstd/expected.hpp \
-O /usr/include/nonstd/expected.hpp
RUN echo "*** Downloading RESTinio ***" \
&& mkdir restinio && cd restinio \
&& wget https://github.com/aberaud/restinio/archive/e0a261dd8488246a3cb8bbb3ea781ea5139c3c94.tar.gz \
&& ls -l && tar -xzf e0a261dd8488246a3cb8bbb3ea781ea5139c3c94.tar.gz \
&& cd restinio-e0a261dd8488246a3cb8bbb3ea781ea5139c3c94/dev \
&& cmake -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=OFF -DRESTINIO_SAMPLE=OFF \
-DRESTINIO_INSTALL_SAMPLES=OFF -DRESTINIO_BENCH=OFF -DRESTINIO_INSTALL_BENCHES=OFF \
-DRESTINIO_FIND_DEPS=ON -DRESTINIO_ALLOW_SOBJECTIZER=Off -DRESTINIO_USE_BOOST_ASIO=none . \
&& make -j8 && make install \
&& wget https://github.com/Stiffstream/restinio/releases/download/v.0.7.2/restinio-0.7.2.tar.bz2 \
&& ls -l && tar -xjf restinio-0.7.2.tar.bz2 \
&& cd restinio-0.7.2/dev \
&& cmake -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=Off -DRESTINIO_SAMPLE=Off -DRESTINIO_BENCHMARK=Off \
-DRESTINIO_WITH_SOBJECTIZER=Off -DRESTINIO_DEP_STANDALONE_ASIO=system -DRESTINIO_DEP_LLHTTP=system \
-DRESTINIO_DEP_FMT=system -DRESTINIO_DEP_EXPECTED_LITE=system . \
&& make -j2 && make install \
&& cd ../../ && rm -rf restinio*
+4
View File
@@ -0,0 +1,4 @@
FROM ghcr.io/savoirfairelinux/opendht/opendht:latest
CMD ["dhtnode", "-b", "bootstrap.jami.net", "-p", "4222", "--proxyserver", "8080"]
EXPOSE 4222/udp
EXPOSE 8080/tcp
+17
View File
@@ -0,0 +1,17 @@
FROM ghcr.io/savoirfairelinux/opendht/opendht-deps-bionic:latest
LABEL maintainer="Adrien Béraud <adrien.beraud@savoirfairelinux.com>"
LABEL org.opencontainers.image.source https://github.com/savoirfairelinux/opendht
COPY . opendht
RUN cd opendht && mkdir build && cd build \
&& cmake .. -DCMAKE_INSTALL_PREFIX=/usr \
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=On \
-DOPENDHT_C=On \
-DOPENDHT_PEER_DISCOVERY=On \
-DOPENDHT_PYTHON=On \
-DOPENDHT_TOOLS=On \
-DOPENDHT_PROXY_SERVER=On \
-DOPENDHT_PROXY_CLIENT=On \
&& make -j8 && make install \
&& cd ../.. && rm -rf opendht
+4 -2
View File
@@ -2,8 +2,9 @@ FROM ghcr.io/savoirfairelinux/opendht/opendht-deps-llvm:latest
LABEL maintainer="Adrien Béraud <adrien.beraud@savoirfairelinux.com>"
LABEL org.opencontainers.image.source https://github.com/savoirfairelinux/opendht
RUN git clone https://github.com/savoirfairelinux/opendht.git \
&& cd opendht && mkdir build && cd build \
COPY . opendht
RUN cd opendht && mkdir build && cd build \
&& cmake .. -DCMAKE_INSTALL_PREFIX=/usr \
-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=On \
-DOPENDHT_C=On \
@@ -12,5 +13,6 @@ RUN git clone https://github.com/savoirfairelinux/opendht.git \
-DOPENDHT_TOOLS=On \
-DOPENDHT_PROXY_SERVER=On \
-DOPENDHT_PROXY_CLIENT=On \
-DOPENDHT_SYSTEMD=On \
&& make -j8 && make install \
&& cd ../.. && rm -rf opendht
+1 -1
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author : Adrien Béraud <adrien.beraud@savoirfairelinux.com>
*
* This program is free software; you can redistribute it and/or modify
+9 -4
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Authors: Adrien Béraud <adrien.beraud@savoirfairelinux.com>
* Simon Désaulniers <simon.desaulniers@savoirfairelinux.com>
* Sébastien Blin <sebastien.blin@savoirfairelinux.com>
@@ -82,6 +82,8 @@ struct OPENDHT_PUBLIC NodeInfo {
NodeStats ipv4 {};
NodeStats ipv6 {};
size_t ongoing_ops {0};
size_t storage_values {0};
size_t storage_size {0};
in_port_t bound4 {0};
in_port_t bound6 {0};
@@ -132,6 +134,9 @@ struct OPENDHT_PUBLIC Config {
/* If non-0, overrides the default maximum store size. -1 means no limit. */
ssize_t max_store_size {0};
/* If non-0, overrides the default maximum store key count. -1 means no limit. */
ssize_t max_store_keys {0};
/**
* Use appropriate bahavior for a public IP, stable node:
* - No connectivity change triggered when a search fails
@@ -165,19 +170,19 @@ using ValueCallback = std::function<bool(const std::vector<std::shared_ptr<Value
using GetCallbackSimple = std::function<bool(std::shared_ptr<Value> value)>;
using ShutdownCallback = std::function<void()>;
using IdentityAnnouncedCb = std::function<void(bool)>;
using PublicAddressChangedCb = std::function<void(std::vector<SockAddr>)>;
using CertificateStoreQuery = std::function<std::vector<std::shared_ptr<crypto::Certificate>>(const InfoHash& pk_id)>;
using DoneCallback = std::function<void(bool success, const std::vector<std::shared_ptr<Node>>& nodes)>;
using DoneCallbackSimple = std::function<void(bool success)>;
typedef bool (*GetCallbackRaw)(std::shared_ptr<Value>, void *user_data);
typedef bool (*ValueCallbackRaw)(std::shared_ptr<Value>, bool expired, void *user_data);
using DoneCallback = std::function<void(bool success, const std::vector<std::shared_ptr<Node>>& nodes)>;
typedef void (*DoneCallbackRaw)(bool, std::vector<std::shared_ptr<Node>>*, void *user_data);
typedef void (*ShutdownCallbackRaw)(void *user_data);
typedef void (*DoneCallbackSimpleRaw)(bool, void *user_data);
typedef bool (*FilterRaw)(const Value&, void *user_data);
using DoneCallbackSimple = std::function<void(bool success)>;
OPENDHT_PUBLIC GetCallbackSimple bindGetCb(GetCallbackRaw raw_cb, void* user_data);
OPENDHT_PUBLIC GetCallback bindGetCb(GetCallbackSimple cb);
+89 -108
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author : Adrien Béraud <adrien.beraud@savoirfairelinux.com>
* Vsevolod Ivanov <vsevolod.ivanov@savoirfairelinux.com>
*
@@ -32,6 +32,9 @@ extern "C" {
#include <vector>
#include <memory>
#include <atomic>
#include <mutex>
#include <string_view>
#ifdef _WIN32
#include <iso646.h>
@@ -84,12 +87,13 @@ struct OPENDHT_PUBLIC PublicKey
/** Import public key from serialized data */
PublicKey(const uint8_t* dat, size_t dat_size);
PublicKey(const Blob& pk) : PublicKey(pk.data(), pk.size()) {}
PublicKey(PublicKey&& o) noexcept : pk(o.pk) { o.pk = nullptr; };
PublicKey(std::string_view pk) : PublicKey((const uint8_t*)pk.data(), pk.size()) {}
PublicKey(PublicKey&& o) noexcept : pk(o.pk) { o.pk = nullptr; }
~PublicKey();
explicit operator bool() const { return pk; }
bool operator ==(const PublicKey& o) const {
return pk == o.pk || getId() == o.getId();
return pk == o.pk || getLongId() == o.getLongId();
}
bool operator !=(const PublicKey& o) const {
return !(*this == o);
@@ -100,12 +104,12 @@ struct OPENDHT_PUBLIC PublicKey
/**
* Get public key fingerprint
*/
InfoHash getId() const;
const InfoHash& getId() const;
/**
* Get public key long fingerprint
*/
PkId getLongId() const;
const PkId& getLongId() const;
bool checkSignature(const uint8_t* data, size_t data_len, const uint8_t* signature, size_t signature_len) const;
inline bool checkSignature(const Blob& data, const Blob& signature) const {
@@ -116,7 +120,9 @@ struct OPENDHT_PUBLIC PublicKey
inline Blob encrypt(const Blob& data) const {
return encrypt(data.data(), data.size());
}
inline Blob encrypt(std::string_view data) const {
return encrypt((const uint8_t*)data.data(), data.size());
}
void pack(Blob& b) const;
int pack(uint8_t* out, size_t* out_len) const;
void unpack(const uint8_t* dat, size_t dat_size);
@@ -138,6 +144,11 @@ struct OPENDHT_PUBLIC PublicKey
gnutls_pubkey_t pk {nullptr};
private:
mutable InfoHash cachedId_ {};
mutable PkId cachedLongId_ {};
mutable std::atomic_bool idCached_ {false};
mutable std::atomic_bool longIdCached_ {false};
PublicKey(const PublicKey&) = delete;
PublicKey& operator=(const PublicKey&) = delete;
void encryptBloc(const uint8_t* src, size_t src_size, uint8_t* dst, size_t dst_size) const;
@@ -160,7 +171,9 @@ struct OPENDHT_PUBLIC PrivateKey
PrivateKey& operator=(PrivateKey&& o) noexcept;
PrivateKey(const uint8_t* src, size_t src_size, const char* password = nullptr);
PrivateKey(const Blob& src, const std::string& password = {}) : PrivateKey(src.data(), src.size(), password.data()) {}
PrivateKey(const Blob& src, const std::string& password = {}) : PrivateKey(src.data(), src.size(), password.c_str()) {}
PrivateKey(std::string_view src, const std::string& password = {}) : PrivateKey((const uint8_t*)src.data(), src.size(), password.c_str()) {}
~PrivateKey();
explicit operator bool() const { return key; }
@@ -175,6 +188,7 @@ struct OPENDHT_PUBLIC PrivateKey
* @returns the signature data.
*/
Blob sign(const uint8_t* data, size_t data_len) const;
inline Blob sign(std::string_view dat) const { return sign((const uint8_t*)dat.data(), dat.size()); }
inline Blob sign(const Blob& dat) const { return sign(dat.data(), dat.size()); }
/**
@@ -201,6 +215,7 @@ private:
PrivateKey& operator=(const PrivateKey&) = delete;
Blob decryptBloc(const uint8_t* src, size_t src_size) const;
mutable std::mutex publicKeyMutex_ {};
mutable std::shared_ptr<PublicKey> publicKey_ {};
};
@@ -286,6 +301,7 @@ class OPENDHT_PUBLIC CertificateRequest {
public:
CertificateRequest();
CertificateRequest(const uint8_t* data, size_t size);
CertificateRequest(std::string_view src) : CertificateRequest((const uint8_t*)src.data(), src.size()) {}
CertificateRequest(const Blob& data) : CertificateRequest(data.data(), data.size()) {}
CertificateRequest(CertificateRequest&& o) noexcept : request(std::move(o.request)) {
@@ -319,8 +335,9 @@ private:
class OPENDHT_PUBLIC OcspRequest
{
public:
OcspRequest(gnutls_ocsp_req_t r) : request(r) {};
OcspRequest(gnutls_ocsp_req_t r) : request(r) {}
OcspRequest(const uint8_t* dat_ptr, size_t dat_size);
OcspRequest(std::string_view dat): OcspRequest((const uint8_t*)dat.data(), dat.size()) {}
~OcspRequest();
/*
@@ -338,7 +355,7 @@ class OPENDHT_PUBLIC OcspResponse
{
public:
OcspResponse(const uint8_t* dat_ptr, size_t dat_size);
OcspResponse(const std::string& response) : OcspResponse((const uint8_t*)response.data(), response.size()) {};
OcspResponse(std::string_view response) : OcspResponse((const uint8_t*)response.data(), response.size()) {}
~OcspResponse();
Blob pack() const;
@@ -373,19 +390,21 @@ struct OPENDHT_PUBLIC Certificate {
*/
Certificate(gnutls_x509_crt_t crt) noexcept : cert(crt) {}
Certificate(Certificate&& o) noexcept : cert(o.cert), issuer(std::move(o.issuer)) { o.cert = nullptr; };
Certificate(Certificate&& o) noexcept
: cert(o.cert)
, issuer(std::move(o.issuer))
, publicKey_(std::move(o.publicKey_))
{ o.cert = nullptr; };
/**
* Import certificate (PEM or DER) or certificate chain (PEM),
* ordered from subject to issuer
*/
Certificate(const Blob& crt);
Certificate(const std::string& pem) : cert(nullptr) {
unpack((const uint8_t*)pem.data(), pem.size());
}
Certificate(const uint8_t* dat, size_t dat_size) : cert(nullptr) {
unpack(dat, dat_size);
}
Certificate(std::string_view pem) : Certificate((const uint8_t*)pem.data(), pem.size()) {}
/**
* Import certificate chain (PEM or DER),
@@ -483,12 +502,13 @@ struct OPENDHT_PUBLIC Certificate {
void msgpack_unpack(const msgpack::object& o);
explicit operator bool() const { return cert; }
PublicKey getPublicKey() const;
const PublicKey& getPublicKey() const;
const std::shared_ptr<PublicKey>& getSharedPublicKey() const;
/** Same as getPublicKey().getId() */
InfoHash getId() const;
const InfoHash& getId() const;
/** Same as getPublicKey().getLongId() */
PkId getLongId() const;
const PkId& getLongId() const;
Blob getSerialNumber() const;
@@ -606,6 +626,8 @@ private:
Certificate& operator=(const Certificate&) = delete;
mutable InfoHash cachedId_ {};
mutable PkId cachedLongId_ {};
mutable std::atomic_bool idCached_ {false};
mutable std::atomic_bool longIdCached_ {false};
struct crlNumberCmp {
bool operator() (const std::shared_ptr<RevocationList>& lhs, const std::shared_ptr<RevocationList>& rhs) const {
@@ -614,6 +636,9 @@ private:
};
std::set<std::shared_ptr<RevocationList>, crlNumberCmp> revocation_lists;
mutable std::mutex publicKeyMutex_ {};
mutable std::shared_ptr<PublicKey> publicKey_ {};
};
struct OPENDHT_PUBLIC TrustList
@@ -645,94 +670,6 @@ private:
gnutls_x509_trust_list_t trust {nullptr};
};
template <class T>
class OPENDHT_PUBLIC secure_vector
{
public:
secure_vector() {}
secure_vector(secure_vector<T> const&) = default;
secure_vector(secure_vector<T> &&) = default;
explicit secure_vector(unsigned size): data_(size) {}
explicit secure_vector(unsigned size, T _item): data_(size, _item) {}
explicit secure_vector(const std::vector<T>& c): data_(c) {}
secure_vector(std::vector<T>&& c): data_(std::move(c)) {}
~secure_vector() { clean(); }
static secure_vector<T> getRandom(size_t size) {
secure_vector<T> ret(size);
crypto::random_device rdev;
#ifdef _WIN32
std::uniform_int_distribution<int> rand_byte{ 0, std::numeric_limits<uint8_t>::max() };
#else
std::uniform_int_distribution<uint8_t> rand_byte;
#endif
std::generate_n((uint8_t*)ret.data_.data(), ret.size()*sizeof(T), std::bind(rand_byte, std::ref(rdev)));
return ret;
}
secure_vector<T>& operator=(const secure_vector<T>& c) {
if (&c == this)
return *this;
clean();
data_ = c.data_;
return *this;
}
secure_vector<T>& operator=(secure_vector<T>&& c) {
if (&c == this)
return *this;
clean();
data_ = std::move(c.data_);
return *this;
}
secure_vector<T>& operator=(std::vector<T>&& c) {
clean();
data_ = std::move(c);
return *this;
}
std::vector<T>& writable() { clean(); return data_; }
const std::vector<T>& makeInsecure() const { return data_; }
const uint8_t* data() const { return data_.data(); }
void clean() {
clean(data_.begin(), data_.end());
}
void clear() { clean(); data_.clear(); }
size_t size() const { return data_.size(); }
bool empty() const { return data_.empty(); }
void swap(secure_vector<T>& other) { data_.swap(other.data_); }
void resize(size_t s) {
if (s == data_.size()) return;
if (s < data_.size()) {
//shrink
clean(data_.begin()+s, data_.end());
data_.resize(s);
} else {
//grow
auto data = std::move(data_); // move protected data
clear();
data_.resize(s);
std::copy(data.begin(), data.end(), data_.begin());
clean(data.begin(), data.end());
}
}
private:
/**
* Securely wipe memory
*/
static void clean(const typename std::vector<T>::iterator& i, const typename std::vector<T>::iterator& j) {
volatile uint8_t* b = reinterpret_cast<uint8_t*>(&*i);
volatile uint8_t* e = reinterpret_cast<uint8_t*>(&*j);
std::fill(b, e, 0);
}
std::vector<T> data_;
};
using SecureBlob = secure_vector<uint8_t>;
/**
* Generate an RSA key pair (4096 bits) and a certificate.
* @param name the name used in the generated certificate
@@ -747,6 +684,7 @@ OPENDHT_PUBLIC Identity generateEcIdentity(const std::string& name, const Identi
OPENDHT_PUBLIC Identity generateEcIdentity(const std::string& name = "dhtnode", const Identity& ca = {});
OPENDHT_PUBLIC void saveIdentity(const Identity& id, const std::string& path, const std::string& privkey_password = {});
OPENDHT_PUBLIC Identity loadIdentity(const std::string &path,const std::string &privkey_password = {});
/**
* Performs SHA512, SHA256 or SHA1, depending on hash_length.
@@ -767,7 +705,7 @@ OPENDHT_PUBLIC void hash(const uint8_t* data, size_t data_length, uint8_t* hash,
* that can be transmitted in clear, and will be generated if
* not provided (32 bytes).
*/
OPENDHT_PUBLIC Blob stretchKey(const std::string& password, Blob& salt, size_t key_length = 512/8);
OPENDHT_PUBLIC Blob stretchKey(std::string_view password, Blob& salt, size_t key_length = 512/8);
/**
* AES-GCM encryption. Key must be 128, 192 or 256 bits long (16, 24 or 32 bytes).
@@ -776,15 +714,58 @@ OPENDHT_PUBLIC Blob aesEncrypt(const uint8_t* data, size_t data_length, const Bl
OPENDHT_PUBLIC inline Blob aesEncrypt(const Blob& data, const Blob& key) {
return aesEncrypt(data.data(), data.size(), key);
}
OPENDHT_PUBLIC Blob aesEncrypt(const Blob& data, const std::string& password);
/**
* AES-GCM encryption with argon2 key derivation.
* This function uses `stretchKey` to generate an AES key from the password and a random salt.
* The result is a bundle including the salt that can be decrypted with `aesDecrypt(data, password)`.
* If needed, the salt or encrypted data can be individually extracted from the bundle with `aesGetSalt` and `aesGetEncrypted`.
* @param data: data to encrypt
* @param password: password to encrypt the data with
* @param salt: optional salt to use for key derivation. If not provided, a random salt will be generated.
*/
OPENDHT_PUBLIC Blob aesEncrypt(const Blob& data, std::string_view password, const Blob& salt = {});
/**
* AES-GCM decryption.
*/
OPENDHT_PUBLIC Blob aesDecrypt(const uint8_t* data, size_t data_length, const Blob& key);
OPENDHT_PUBLIC inline Blob aesDecrypt(const Blob& data, const Blob& key) { return aesDecrypt(data.data(), data.size(), key); }
OPENDHT_PUBLIC Blob aesDecrypt(const uint8_t* data, size_t data_length, const std::string& password);
OPENDHT_PUBLIC inline Blob aesDecrypt(const Blob& data, const std::string& password) { return aesDecrypt(data.data(), data.size(), password); }
OPENDHT_PUBLIC inline Blob aesDecrypt(std::string_view data, const Blob& key) { return aesDecrypt((uint8_t*)data.data(), data.size(), key); }
OPENDHT_PUBLIC Blob aesDecrypt(const uint8_t* data, size_t data_length, std::string_view password);
OPENDHT_PUBLIC inline Blob aesDecrypt(const Blob& data, std::string_view password) { return aesDecrypt(data.data(), data.size(), password); }
OPENDHT_PUBLIC inline Blob aesDecrypt(std::string_view data, std::string_view password) { return aesDecrypt((uint8_t*)data.data(), data.size(), password); }
/**
* Get raw AES key from password and salt stored with the encrypted data.
*/
OPENDHT_PUBLIC Blob aesGetKey(const uint8_t* data, size_t data_length, std::string_view password);
OPENDHT_PUBLIC Blob inline aesGetKey(const Blob& data, std::string_view password) {
return aesGetKey(data.data(), data.size(), password);
}
/** Get the salt part of data password-encrypted with `aesEncrypt(data, password)` */
OPENDHT_PUBLIC Blob aesGetSalt(const uint8_t* data, size_t data_length);
OPENDHT_PUBLIC Blob inline aesGetSalt(const Blob& data) {
return aesGetSalt(data.data(), data.size());
}
/** Get the encrypted data part of data password-encrypted with `aesEncrypt(data, password)` */
OPENDHT_PUBLIC std::string_view aesGetEncrypted(const uint8_t* data, size_t data_length);
OPENDHT_PUBLIC std::string_view inline aesGetEncrypted(const Blob& data) {
return aesGetEncrypted(data.data(), data.size());
}
/** Build an encrypted bundle that can be decrypted with aesDecrypt(data, password).
* @param encryptedData: result of `aesEncrypt(data, key)` or `aesGetEncrypted`
* @param salt: should match the encryption key and password so that `stretchKey(password, salk) == key`.
* Can be obtained from an existing bundle with `aesGetSalt`.
**/
OPENDHT_PUBLIC Blob aesBuildEncrypted(const uint8_t* encryptedData, size_t data_length, const Blob& salt);
OPENDHT_PUBLIC Blob inline aesBuildEncrypted(const Blob& encryptedData, const Blob& salt) {
return aesBuildEncrypted(encryptedData.data(), encryptedData.size(), salt);
}
OPENDHT_PUBLIC Blob inline aesBuildEncrypted(std::string_view encryptedData, const Blob& salt) {
return aesBuildEncrypted((const uint8_t*)encryptedData.data(), encryptedData.size(), salt);
}
}
}
+114
View File
@@ -0,0 +1,114 @@
/*
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
#pragma once
#include <vector>
#include <random>
namespace dht {
namespace crypto {
template <class T>
class secure_vector
{
public:
secure_vector() {}
secure_vector(secure_vector<T> const&) = default;
secure_vector(secure_vector<T> &&) = default;
explicit secure_vector(unsigned size): data_(size) {}
explicit secure_vector(unsigned size, T _item): data_(size, _item) {}
explicit secure_vector(const std::vector<T>& c): data_(c) {}
secure_vector(std::vector<T>&& c): data_(std::move(c)) {}
~secure_vector() { clean(); }
static secure_vector<T> getRandom(size_t size) {
secure_vector<T> ret(size);
std::random_device rdev;
#ifdef _WIN32
std::uniform_int_distribution<int> rand_byte{ 0, std::numeric_limits<uint8_t>::max() };
#else
std::uniform_int_distribution<uint8_t> rand_byte;
#endif
std::generate_n((uint8_t*)ret.data_.data(), ret.size()*sizeof(T), std::bind(rand_byte, std::ref(rdev)));
return ret;
}
secure_vector<T>& operator=(const secure_vector<T>& c) {
if (&c == this)
return *this;
clean();
data_ = c.data_;
return *this;
}
secure_vector<T>& operator=(secure_vector<T>&& c) {
if (&c == this)
return *this;
clean();
data_ = std::move(c.data_);
return *this;
}
secure_vector<T>& operator=(std::vector<T>&& c) {
clean();
data_ = std::move(c);
return *this;
}
std::vector<T>& writable() { clean(); return data_; }
const std::vector<T>& makeInsecure() const { return data_; }
const uint8_t* data() const { return data_.data(); }
void clean() {
clean(data_.begin(), data_.end());
}
void clear() { clean(); data_.clear(); }
size_t size() const { return data_.size(); }
bool empty() const { return data_.empty(); }
void swap(secure_vector<T>& other) { data_.swap(other.data_); }
void resize(size_t s) {
if (s == data_.size()) return;
if (s < data_.size()) {
//shrink
clean(data_.begin()+s, data_.end());
data_.resize(s);
} else {
//grow
auto data = std::move(data_); // move protected data
clear();
data_.resize(s);
std::copy(data.begin(), data.end(), data_.begin());
clean(data.begin(), data.end());
}
}
private:
/**
* Securely wipe memory
*/
static void clean(const typename std::vector<T>::iterator& i, const typename std::vector<T>::iterator& j) {
volatile uint8_t* b = reinterpret_cast<uint8_t*>(&*i);
volatile uint8_t* e = reinterpret_cast<uint8_t*>(&*j);
std::fill(b, e, 0);
}
std::vector<T> data_;
};
using SecureBlob = secure_vector<uint8_t>;
}
}
+1 -1
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author : Adrien Béraud <adrien.beraud@savoirfairelinux.com>
*
* This program is free software; you can redistribute it and/or modify
+18 -14
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Authors: Adrien Béraud <adrien.beraud@savoirfairelinux.com>
* Simon Désaulniers <simon.desaulniers@savoirfairelinux.com>
* Sébastien Blin <sebastien.blin@savoirfairelinux.com>
@@ -65,10 +65,7 @@ public:
* Initialise the Dht with two open sockets (for IPv4 and IP6)
* and an ID for the node.
*/
Dht(std::unique_ptr<net::DatagramSocket>&& sock, const Config& config, const Sp<Logger>& l = {});
Dht(std::unique_ptr<net::DatagramSocket>&& sock, const Config& config, const Logger& l = {})
: Dht(std::move(sock), config, std::make_shared<Logger>(l)) {}
Dht(std::unique_ptr<net::DatagramSocket>&& sock, const Config& config, const Sp<Logger>& l = {}, std::unique_ptr<std::mt19937_64>&& rd = {});
virtual ~Dht();
@@ -76,6 +73,9 @@ public:
* Get the ID of the node.
*/
inline const InfoHash& getNodeId() const override { return myid; }
void setOnPublicAddressChanged(PublicAddressChangedCb cb) override {
publicAddressChangedCb_ = std::move(cb);
}
NodeStatus updateStatus(sa_family_t af) override;
@@ -128,7 +128,7 @@ public:
*/
void insertNode(const InfoHash& id, const SockAddr&) override;
void insertNode(const NodeExport& n) override {
insertNode(n.id, SockAddr(n.ss, n.sslen));
insertNode(n.id, n.addr);
}
void pingNode(SockAddr, DoneCallbackSimple&& cb={}) override;
@@ -265,7 +265,6 @@ public:
*/
void connectivityChanged(sa_family_t) override;
void connectivityChanged() override {
reported_addr.clear();
connectivityChanged(AF_INET);
connectivityChanged(AF_INET6);
}
@@ -302,6 +301,9 @@ public:
void setStorageLimit(size_t limit = DEFAULT_STORAGE_LIMIT) override {
max_store_size = limit;
}
size_t getStorageLimit() const override {
return max_store_size;
}
/**
* Returns the total memory usage of stored values and the number
@@ -340,7 +342,7 @@ private:
static constexpr unsigned LISTEN_NODES {4};
/* The maximum number of hashes we're willing to track. */
static constexpr unsigned MAX_HASHES {1024 * 1024};
static constexpr unsigned MAX_HASHES {1024 * 1024 * 1024};
/* The maximum number of searches we keep data about. */
static constexpr unsigned MAX_SEARCHES {1024 * 1024};
@@ -370,7 +372,7 @@ private:
Dht(const Dht&) = delete;
Dht& operator=(const Dht&) = delete;
std::mt19937_64 rd {crypto::getSeededRandomEngine<std::mt19937_64>()};
std::mt19937_64 rd;
InfoHash myid {};
@@ -381,11 +383,14 @@ private:
TypeStore types;
using SearchMap = std::map<InfoHash, Sp<Search>>;
using ReportedAddr = std::pair<unsigned, SockAddr>;
struct Kad {
RoutingTable buckets {};
SearchMap searches {};
unsigned pending_pings {0};
NodeStatus status;
std::vector<ReportedAddr> reported_addr;
NodeStatus getStatus(time_point now) const;
NodeStats getNodesStats(time_point now, const InfoHash& myid) const;
@@ -393,6 +398,7 @@ private:
Kad dht4 {};
Kad dht6 {};
PublicAddressChangedCb publicAddressChangedCb_ {};
std::vector<std::pair<std::string,std::string>> bootstrap_nodes {};
std::chrono::steady_clock::duration bootstrap_period {BOOTSTRAP_PERIOD};
@@ -420,8 +426,6 @@ private:
Sp<Scheduler::Job> nextStorageMaintenance {};
net::NetworkEngine network_engine;
using ReportedAddr = std::pair<unsigned, SockAddr>;
std::vector<ReportedAddr> reported_addr;
std::string persistPath;
@@ -447,13 +451,13 @@ private:
// Storage
void storageAddListener(const InfoHash& id, const Sp<Node>& node, size_t tid, Query&& = {}, int version = 0);
bool storageStore(const InfoHash& id, const Sp<Value>& value, time_point created, const SockAddr& sa = {}, bool permanent = false);
bool storageErase(const InfoHash& id, Value::Id vid);
bool storageRefresh(const InfoHash& id, Value::Id vid);
void expireStore();
void expireStorage(InfoHash h);
void expireStore(decltype(store)::iterator);
void storageChanged(const InfoHash& id, Storage& st, ValueStorage&, bool newValue);
void storageRemoved(const InfoHash& id, Storage& st, const std::vector<Sp<Value>>& values, size_t totalSize);
void storageChanged(const InfoHash& id, Storage& st, const Sp<Value>&, bool newValue);
std::string printStorageLog(const decltype(store)::value_type&) const;
/**
@@ -564,7 +568,7 @@ private:
* @param sr The search for which we want to announce a value.
* @param announce The 'announce' structure.
*/
void searchSendAnnounceValue(const Sp<Search>& sr);
void searchSendAnnounceValue(const Sp<Search>& sr, unsigned syncLevel = TARGET_NODES);
/**
* Main process of a Search's operations. This function will demand the
+12 -18
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author: Sébastien Blin <sebastien.blin@savoirfairelinux.com>
*
* This program is free software; you can redistribute it and/or modify
@@ -19,7 +19,8 @@
#pragma once
#include "infohash.h"
#include "log_enable.h"
#include "logger.h"
#include "node_export.h"
#include <queue>
@@ -32,8 +33,8 @@ namespace net {
class OPENDHT_PUBLIC DhtInterface {
public:
DhtInterface() = default;
DhtInterface(const Logger& l) : logger_(std::make_shared<Logger>(l)) {};
DhtInterface(const std::shared_ptr<Logger>& l) : logger_(l) {};
DhtInterface(const Logger& l) : logger_(std::make_shared<Logger>(l)) {}
DhtInterface(const std::shared_ptr<Logger>& l) : logger_(l) {}
virtual ~DhtInterface() = default;
// [[deprecated]]
@@ -44,15 +45,16 @@ public:
/**
* Get the current status of the node for the given family.
*/
virtual NodeStatus updateStatus(sa_family_t af) { return getStatus(af); };
virtual NodeStatus updateStatus(sa_family_t af) { return getStatus(af); }
virtual NodeStatus getStatus(sa_family_t af) const = 0;
virtual NodeStatus getStatus() const = 0;
void addOnConnectedCallback(std::function<void()> cb) {
onConnectCallbacks_.emplace(std::move(cb));
}
virtual void setOnPublicAddressChanged(PublicAddressChangedCb) {}
virtual net::DatagramSocket* getSocket() const { return {}; };
virtual net::DatagramSocket* getSocket() const { return {}; }
/**
* Get the ID of the DHT node.
@@ -221,6 +223,7 @@ public:
* Set the in-memory storage limit in bytes
*/
virtual void setStorageLimit(size_t limit = DEFAULT_STORAGE_LIMIT) = 0;
virtual size_t getStorageLimit() const = 0;
/**
* Returns the total memory usage of stored values and the number
@@ -230,18 +233,6 @@ public:
virtual std::vector<SockAddr> getPublicAddress(sa_family_t family = 0) = 0;
/**
* Enable or disable logging of DHT internal messages
*/
virtual void setLoggers(LogMethod error = {}, LogMethod warn = {}, LogMethod debug = {}) {
if (logger_) {
logger_->DBG = std::move(debug);
logger_->WARN = std::move(warn);
logger_->ERR = std::move(error);
} else
logger_= std::make_shared<Logger>(std::move(error), std::move(warn), std::move(debug));
}
virtual void setLogger(const Logger& l) {
if (logger_)
*logger_ = l;
@@ -264,6 +255,9 @@ public:
virtual void setPushNotificationToken(const std::string&) {};
virtual void setPushNotificationTopic(const std::string&) {};
virtual void setPushNotificationPlatform(const std::string&) {};
/**
* Call linked callback with a push notification
* @param notification to process
+54 -1
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author: Sébastien Blin <sebastien.blin@savoirfairelinux.com>
* Adrien Béraud <adrien.beraud@savoirfairelinux.com>
* Vsevolod Ivanov <vsevolod.ivanov@savoirfairelinux.com>
@@ -67,12 +67,31 @@ public:
#endif
}
virtual void setPushNotificationTopic(const std::string& topic) override {
#ifdef OPENDHT_PUSH_NOTIFICATIONS
notificationTopic_ = topic;
#else
(void) topic;
#endif
}
virtual void setPushNotificationPlatform(const std::string& platform) override {
#ifdef OPENDHT_PUSH_NOTIFICATIONS
platform_ = platform;
#else
(void) platform;
#endif
}
virtual ~DhtProxyClient();
/**
* Get the ID of the node.
*/
inline const InfoHash& getNodeId() const override { return myid; }
void setOnPublicAddressChanged(PublicAddressChangedCb cb) override {
publicAddressChangedCb_ = std::move(cb);
}
/**
* Get the current status of the node for the given family.
@@ -263,6 +282,7 @@ public:
void dumpTables() const override {}
std::vector<unsigned> getNodeMessageStats(bool) override { return {}; }
void setStorageLimit(size_t) override {}
virtual size_t getStorageLimit() const override { return 0; }
void connectivityChanged(sa_family_t) override {
getProxyInfos();
}
@@ -271,6 +291,11 @@ public:
loopSignal_();
}
void listenKeepIdle(uint32_t seconds) {
listenKeepIdle_ = seconds;
}
inline uint32_t listenKeepIdle() { return listenKeepIdle_; }
private:
/**
* Start the connection with a server.
@@ -335,9 +360,12 @@ private:
NodeStatus statusIpv6_ {NodeStatus::Disconnected};
NodeStats stats4_ {};
NodeStats stats6_ {};
SockAddr localAddrv4_;
SockAddr localAddrv6_;
SockAddr publicAddressV4_;
SockAddr publicAddressV6_;
std::atomic_bool launchConnectedCbs_ {false};
PublicAddressChangedCb publicAddressChangedCb_ {};
InfoHash myid {};
@@ -349,6 +377,7 @@ private:
* Note: Each context is used in one thread only
*/
asio::io_context httpContext_;
mutable std::mutex resolverLock_;
std::shared_ptr<http::Resolver> resolver_;
mutable std::mutex requestLock_;
@@ -367,6 +396,11 @@ private:
size_t listenerToken_ {0};
std::map<InfoHash, ProxySearch> searches_;
/**
* Keepalive idle time for listen
*/
uint32_t listenKeepIdle_ {120};
/**
* Callbacks should be executed in the main thread.
*/
@@ -399,6 +433,25 @@ private:
*/
std::string deviceKey_ {};
/**
* Notification topic for ios notifications.
*/
std::string notificationTopic_ {};
/**
* Platform for push notifications (supported android, ios, unifiedpush)
*/
std::string platform_
#ifdef __ANDROID__
{"android"};
#else
#ifdef __APPLE__
{"ios"};
#else
{};
#endif
#endif
const std::function<void()> loopSignal_;
#ifdef OPENDHT_PUSH_NOTIFICATIONS
+56 -21
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author: Sébastien Blin <sebastien.blin@savoirfairelinux.com>
* Adrien Béraud <adrien.beraud@savoirfairelinux.com>
* Vsevolod Ivanov <vsevolod.ivanov@savoirfairelinux.com>
@@ -40,32 +40,35 @@ namespace dht {
enum class PushType {
None = 0,
Android,
iOS
iOS,
UnifiedPush
};
}
MSGPACK_ADD_ENUM(dht::PushType)
namespace http {
class Request;
struct ListenerSession;
}
namespace Json {
class Value;
}
namespace dht {
namespace http {
class Request;
struct ListenerSession;
}
class DhtRunner;
using RestRouter = restinio::router::express_router_t<>;
using RequestStatus = restinio::request_handling_status_t;
struct ProxyServerConfig {
struct OPENDHT_PUBLIC ProxyServerConfig {
std::string address {};
in_port_t port {8000};
std::string pushServer {};
std::string persistStatePath {};
dht::crypto::Identity identity {};
std::string bundleId {};
};
/**
@@ -84,7 +87,7 @@ public:
*/
DhtProxyServer(const std::shared_ptr<DhtRunner>& dht,
const ProxyServerConfig& config = {},
const std::shared_ptr<dht::Logger>& logger = {});
const std::shared_ptr<log::Logger>& logger = {});
virtual ~DhtProxyServer();
@@ -272,7 +275,18 @@ private:
RequestStatus options(restinio::request_handle_t request,
restinio::router::route_params_t params);
struct PushSessionContext {
std::mutex lock;
std::string sessionId;
PushSessionContext(const std::string& id) : sessionId(id) {}
};
#ifdef OPENDHT_PUSH_NOTIFICATIONS
PushType getTypeFromString(const std::string& type);
std::string getDefaultTopic(PushType type);
RequestStatus pingPush(restinio::request_handle_t request,
restinio::router::route_params_t /*params*/);
/**
* Subscribe to push notifications for an iOS or Android device.
* Method: SUBSCRIBE "/{InfoHash: .*}"
@@ -300,7 +314,7 @@ private:
* @param key of the device
* @param json, the content to send
*/
void sendPushNotification(const std::string& key, Json::Value&& json, PushType type, bool highPriority);
void sendPushNotification(const std::string& key, Json::Value&& json, PushType type, bool highPriority, const std::string& topic);
/**
* Send push notification with an expire timeout.
@@ -308,9 +322,10 @@ private:
* @param pushToken
* @param json
* @param type
* @param topic
*/
void handleNotifyPushListenExpire(const asio::error_code &ec, const std::string pushToken,
std::function<Json::Value()> json, PushType type);
std::function<Json::Value()> json, PushType type, const std::string& topic);
/**
* Remove a push listener between a client and a hash
@@ -322,6 +337,24 @@ private:
void handleCancelPushListen(const asio::error_code &ec, const std::string pushToken,
const InfoHash key, const std::string clientId);
/**
* Handles a push listen request.
*
* @param infoHash The information hash associated with the push listen request.
* @param pushToken The push token associated with the push listen request.
* @param type The type of the push listen request.
* @param clientId The client ID associated with the push listen request.
* @param sessionCtx The shared pointer to the push session context associated with the push listen request.
* @param topic The topic associated with the push listen request.
* @param values The vector of shared pointers to values associated with the push listen request.
* @param expired A boolean indicating whether the push listen request has expired.
* @return true.
*/
bool handlePushListen(const InfoHash& infoHash, const std::string& pushToken,
PushType type, const std::string& clientId,
const std::shared_ptr<DhtProxyServer::PushSessionContext>& sessionCtx, const std::string& topic,
const std::vector<std::shared_ptr<Value>>& values, bool expired);
#endif //OPENDHT_PUSH_NOTIFICATIONS
void handlePrintStats(const asio::error_code &ec);
@@ -355,7 +388,7 @@ private:
mutable std::mutex requestLock_;
std::map<unsigned int /*id*/, std::shared_ptr<http::Request>> requests_;
std::shared_ptr<dht::Logger> logger_;
std::shared_ptr<log::Logger> logger_;
std::shared_ptr<ServerStats> stats_;
std::shared_ptr<NodeInfo> nodeInfo_ {};
@@ -367,12 +400,6 @@ private:
std::map<restinio::connection_id_t, http::ListenerSession> listeners_;
// Connection Listener observing conn state changes.
std::shared_ptr<ConnectionListener> connListener_;
struct PushSessionContext {
std::mutex lock;
std::string sessionId;
PushSessionContext(const std::string& id) : sessionId(id) {}
};
struct PermanentPut {
time_point expiration;
std::string pushToken;
@@ -382,11 +409,12 @@ private:
std::unique_ptr<asio::steady_timer> expireNotifyTimer;
Sp<Value> value;
PushType type;
std::string topic;
template <typename Packer>
void msgpack_pack(Packer& p) const
{
p.pack_map(2 + (sessionCtx ? 1 : 0) + (clientId.empty() ? 0 : 1) + (type == PushType::None ? 0 : 2));
p.pack_map(2 + (sessionCtx ? 1 : 0) + (clientId.empty() ? 0 : 1) + (type == PushType::None ? 0 : 2) + (topic.empty() ? 0 : 1));
p.pack("value"); p.pack(value);
p.pack("exp"); p.pack(to_time_t(expiration));
if (not clientId.empty()) {
@@ -400,6 +428,9 @@ private:
p.pack("t"); p.pack(type);
p.pack("token"); p.pack(pushToken);
}
if (not topic.empty()) {
p.pack("top"); p.pack(topic);
}
}
void msgpack_unpack(const msgpack::object& o);
@@ -415,6 +446,7 @@ private:
mutable std::atomic<time_point> lastStatsReset_ {time_point::min()};
std::string pushServer_;
std::string bundleId_;
#ifdef OPENDHT_PUSH_NOTIFICATIONS
struct Listener {
@@ -425,11 +457,12 @@ private:
std::unique_ptr<asio::steady_timer> expireTimer;
std::unique_ptr<asio::steady_timer> expireNotifyTimer;
PushType type;
std::string topic;
template <typename Packer>
void msgpack_pack(Packer& p) const
{
p.pack_map(sessionCtx ? 4 : 3);
p.pack_map(3 + (sessionCtx ? 1 : 0) + (topic.empty() ? 0 : 1));
p.pack("cid"); p.pack(clientId);
p.pack("exp"); p.pack(to_time_t(expiration));
if (sessionCtx) {
@@ -437,6 +470,9 @@ private:
p.pack("sid"); p.pack(sessionCtx->sessionId);
}
p.pack("t"); p.pack(type);
if (!topic.empty()) {
p.pack("top"); p.pack(topic);
}
}
void msgpack_unpack(const msgpack::object& o);
@@ -446,7 +482,6 @@ private:
MSGPACK_DEFINE_ARRAY(listeners)
};
std::map<std::string, PushListener> pushListeners_;
proxy::ListenToken tokenPushNotif_ {0};
#endif //OPENDHT_PUSH_NOTIFICATIONS
};
+21 -21
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Authors: Adrien Béraud <adrien.beraud@savoirfairelinux.com>
* Simon Désaulniers <simon.desaulniers@savoirfairelinux.com>
* Sébastien Blin <sebastien.blin@savoirfairelinux.com>
@@ -25,8 +25,9 @@
#include "value.h"
#include "callbacks.h"
#include "sockaddr.h"
#include "log_enable.h"
#include "logger.h"
#include "network_utils.h"
#include "node_export.h"
#include <thread>
#include <mutex>
@@ -61,6 +62,8 @@ public:
std::string proxy_server {};
std::string push_node_id {};
std::string push_token {};
std::string push_topic {};
std::string push_platform {};
bool peer_discovery {false};
bool peer_publish {false};
std::shared_ptr<dht::crypto::Certificate> server_ca;
@@ -75,6 +78,8 @@ public:
StatusCallback statusChangedCallback {};
CertificateStoreQuery certificateStore {};
IdentityAnnouncedCb identityAnnouncedCb {};
PublicAddressChangedCb publicAddressChangedCb {};
std::unique_ptr<std::mt19937_64> rng {};
Context() {}
};
@@ -335,6 +340,7 @@ public:
std::pair<size_t, size_t> getStoreSize() const;
void getStorageLimit() const;
void setStorageLimit(size_t limit = DEFAULT_STORAGE_LIMIT);
std::vector<NodeExport> exportNodes() const;
@@ -345,7 +351,6 @@ public:
void setLogger(const Logger& logger) {
setLogger(std::make_shared<Logger>(logger));
}
void setLoggers(LogMethod err = {}, LogMethod warn = {}, LogMethod debug = {});
/**
* Only print logs related to the given InfoHash (if given), or disable filter (if zeroes).
@@ -371,8 +376,8 @@ public:
std::string getRoutingTablesLog(sa_family_t af) const;
std::string getSearchesLog(sa_family_t af = AF_UNSPEC) const;
std::string getSearchLog(const InfoHash&, sa_family_t af = AF_UNSPEC) const;
std::vector<SockAddr> getPublicAddress(sa_family_t af = AF_UNSPEC);
std::vector<std::string> getPublicAddressStr(sa_family_t af = AF_UNSPEC);
std::vector<SockAddr> getPublicAddress(sa_family_t af = AF_UNSPEC) const;
std::vector<std::string> getPublicAddressStr(sa_family_t af = AF_UNSPEC) const;
void getPublicAddress(std::function<void(std::vector<SockAddr>&&)>, sa_family_t af = AF_UNSPEC);
// securedht methods
@@ -436,7 +441,6 @@ public:
/**
* Start or stop the proxy
* @param proxify if we want to use the proxy
* @param deviceKey non empty to enable push notifications
*/
void enableProxy(bool proxify);
@@ -447,6 +451,16 @@ public:
*/
void setPushNotificationToken(const std::string& token);
/**
* Sets the push notification topic
*/
void setPushNotificationTopic(const std::string& topic);
/**
* Sets the push notification platform
*/
void setPushNotificationPlatform(const std::string& platform);
/**
* Insert a push notification to process for OpenDHT
*/
@@ -473,12 +487,9 @@ private:
DoneCallback bindOpDoneCallback(DoneCallback&& cb);
DoneCallbackSimple bindOpDoneCallback(DoneCallbackSimple&& cb);
/** Local DHT instance */
/** DHT instance */
std::unique_ptr<SecureDht> dht_;
/** Proxy client instance */
std::unique_ptr<SecureDht> dht_via_proxy_;
/** true if we are currently using a proxy */
std::atomic_bool use_proxy {false};
@@ -490,17 +501,6 @@ private:
* reset dht clients
*/
void resetDht();
/**
* @return the current active DHT
*/
SecureDht* activeDht() const;
/**
* Store current listeners and translates global tokens for each client.
*/
struct Listener;
std::map<size_t, Listener> listeners_;
size_t listener_token_ {1};
mutable std::mutex dht_mtx {};
std::thread dht_thread {};
+48 -41
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author: Vsevolod Ivanov <vsevolod.ivanov@savoirfairelinux.com>
*
* This program is free software; you can redistribute it and/or modify
@@ -29,23 +29,20 @@
# define snprintf snprintf
#endif
#include <asio/ip/tcp.hpp>
#include <asio/streambuf.hpp>
#include <asio/ssl/context.hpp>
#include <restinio/http_headers.hpp>
#include <restinio/message_builders.hpp>
#include <memory>
#include <queue>
#include <mutex>
#include <future>
namespace Json {
class Value;
}
extern "C" {
struct http_parser;
struct http_parser_settings;
}
namespace restinio {
namespace impl {
class tls_socket_t;
@@ -53,7 +50,9 @@ class tls_socket_t;
}
namespace dht {
namespace log {
struct Logger;
}
namespace crypto {
struct Certificate;
@@ -88,9 +87,9 @@ public:
class OPENDHT_PUBLIC Connection : public std::enable_shared_from_this<Connection>
{
public:
Connection(asio::io_context& ctx, const bool ssl = true, std::shared_ptr<dht::Logger> l = {});
Connection(asio::io_context& ctx, const bool ssl = true, std::shared_ptr<log::Logger> l = {});
Connection(asio::io_context& ctx, std::shared_ptr<dht::crypto::Certificate> server_ca,
const dht::crypto::Identity& identity, std::shared_ptr<dht::Logger> l = {});
const dht::crypto::Identity& identity, std::shared_ptr<log::Logger> l = {});
~Connection();
inline unsigned int id() const { return id_; };
@@ -114,13 +113,18 @@ public:
void async_read(size_t bytes, BytesHandlerCb cb);
void async_read_some(size_t bytes, BytesHandlerCb cb);
void timeout(const std::chrono::seconds timeout, HandlerCb cb = {});
void set_keepalive(uint32_t seconds);
const asio::ip::address& local_address() const;
void timeout(const std::chrono::seconds& timeout, HandlerCb cb = {});
void close();
private:
template<typename T>
T wrapCallabck(T cb) const {
T wrapCallback(T cb) const {
return [t=shared_from_this(),cb=std::move(cb)](auto ...params) {
cb(params...);
};
@@ -142,20 +146,11 @@ private:
asio::streambuf read_buf_;
std::istream istream_;
std::unique_ptr<asio::steady_timer> timeout_timer_;
std::shared_ptr<dht::Logger> logger_;
bool checkOcsp_ {false};
};
asio::ip::address local_address_;
/**
* Session value associated with a connection_id_t key.
*/
struct ListenerSession
{
ListenerSession() = default;
dht::InfoHash hash;
std::future<size_t> token;
std::shared_ptr<restinio::response_builder_t<restinio::chunked_output_t>> response;
std::unique_ptr<asio::steady_timer> timeout_timer_;
std::shared_ptr<log::Logger> logger_;
bool checkOcsp_ {false};
};
/* @class Resolver
@@ -167,15 +162,15 @@ public:
using ResolverCb = std::function<void(const asio::error_code& ec,
const std::vector<asio::ip::tcp::endpoint>& endpoints)>;
Resolver(asio::io_context& ctx, const std::string& url, std::shared_ptr<dht::Logger> logger = {});
Resolver(asio::io_context& ctx, const std::string& url, std::shared_ptr<log::Logger> logger = {});
Resolver(asio::io_context& ctx, const std::string& host, const std::string& service,
const bool ssl = false, std::shared_ptr<dht::Logger> logger = {});
const bool ssl = false, std::shared_ptr<log::Logger> logger = {});
// use already resolved endpoints with classes using this resolver
Resolver(asio::io_context& ctx, std::vector<asio::ip::tcp::endpoint> endpoints,
const bool ssl = false, std::shared_ptr<dht::Logger> logger = {});
const bool ssl = false, std::shared_ptr<log::Logger> logger = {});
Resolver(asio::io_context& ctx, const std::string& url, std::vector<asio::ip::tcp::endpoint> endpoints,
std::shared_ptr<dht::Logger> logger = {});
std::shared_ptr<log::Logger> logger = {});
~Resolver();
@@ -185,10 +180,12 @@ public:
void add_callback(ResolverCb cb, sa_family_t family = AF_UNSPEC);
std::shared_ptr<Logger> getLogger() const {
std::shared_ptr<log::Logger> getLogger() const {
return logger_;
}
void cancel();
private:
void resolve(const std::string& host, const std::string& service);
@@ -203,7 +200,7 @@ private:
bool completed_ {false};
std::queue<ResolverCb> cbs_;
std::shared_ptr<dht::Logger> logger_;
std::shared_ptr<log::Logger> logger_;
};
class Request;
@@ -235,14 +232,14 @@ public:
// resolves implicitly
Request(asio::io_context& ctx, const std::string& url, const Json::Value& json, OnJsonCb jsoncb,
std::shared_ptr<dht::Logger> logger = {});
std::shared_ptr<log::Logger> logger = {});
Request(asio::io_context& ctx, const std::string& url, OnJsonCb jsoncb,
std::shared_ptr<dht::Logger> logger = {});
std::shared_ptr<log::Logger> logger = {});
Request(asio::io_context& ctx, const std::string& url, std::shared_ptr<dht::Logger> logger = {});
Request(asio::io_context& ctx, const std::string& url, std::shared_ptr<log::Logger> logger = {});
Request(asio::io_context& ctx, const std::string& host, const std::string& service,
const bool ssl = false, std::shared_ptr<dht::Logger> logger = {});
Request(asio::io_context& ctx, const std::string& url, OnDoneCb onDone, std::shared_ptr<dht::Logger> logger = {});
const bool ssl = false, std::shared_ptr<log::Logger> logger = {});
Request(asio::io_context& ctx, const std::string& url, OnDoneCb onDone, std::shared_ptr<log::Logger> logger = {});
// user defined resolver
Request(asio::io_context& ctx, std::shared_ptr<Resolver> resolver, sa_family_t family = AF_UNSPEC);
@@ -250,7 +247,7 @@ public:
// user defined resolved endpoints
Request(asio::io_context& ctx, std::vector<asio::ip::tcp::endpoint>&& endpoints,
const bool ssl = false, std::shared_ptr<dht::Logger> logger = {});
const bool ssl = false, std::shared_ptr<log::Logger> logger = {});
~Request();
@@ -261,6 +258,11 @@ public:
return resolver_->get_url();
};
void timeout(const std::chrono::seconds& timeout, HandlerCb cb = {}) {
timeout_ = timeout;
timeoutCb_ = cb;
}
/** The previous request in case of redirect following */
std::shared_ptr<Request> getPrevious() const {
return prev_.lock();
@@ -272,7 +274,7 @@ public:
void set_certificate_authority(std::shared_ptr<dht::crypto::Certificate> certificate);
void set_identity(const dht::crypto::Identity& identity);
void set_logger(std::shared_ptr<dht::Logger> logger);
void set_logger(std::shared_ptr<log::Logger> logger);
/**
* Define the HTTP header/body as per https://tools.ietf.org/html/rfc7230.
@@ -318,9 +320,11 @@ private:
void build();
static std::string url_encode(std::string_view value);
void init_default_headers();
/**
* Initialized and wraps the http_parser callbacks with our user defined callbacks.
* Initialized and wraps the llhttp callbacks with our user defined callbacks.
*/
void init_parser();
@@ -337,7 +341,7 @@ private:
mutable std::mutex mutex_;
std::shared_ptr<dht::Logger> logger_;
std::shared_ptr<log::Logger> logger_;
restinio::http_request_header_t header_;
std::map<restinio::http_field_t, std::string> headers_;
@@ -362,14 +366,17 @@ private:
Response response_ {};
std::string request_;
std::atomic<bool> finishing_ {false};
std::unique_ptr<http_parser> parser_;
std::unique_ptr<http_parser_settings> parser_s_;
std::unique_ptr<llhttp_t> parser_;
std::unique_ptr<llhttp_settings_t> parser_s_;
// Next request in case of redirect following
std::shared_ptr<Request> next_;
std::weak_ptr<Request> prev_;
unsigned num_redirect {0};
bool follow_redirect {true};
HandlerCb timeoutCb_ {};
std::chrono::seconds timeout_ {0};
};
} // namespace http
+1 -1
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author(s) : Adrien Béraud <adrien.beraud@savoirfairelinux.com>
* Simon Désaulniers <simon.desaulniers@savoirfairelinux.com>
* Nicolas Reynaud <nicolas.reynaud@savoirfairelinux.com>
+19 -30
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author : Adrien Béraud <adrien.beraud@savoirfairelinux.com>
*
* This program is free software; you can redistribute it and/or modify
@@ -22,6 +22,7 @@
#include "rng.h"
#include <msgpack.hpp>
#include <fmt/core.h>
#ifndef _WIN32
#include <netinet/in.h>
@@ -44,6 +45,7 @@ typedef uint16_t in_port_t;
#include <algorithm>
#include <stdexcept>
#include <sstream>
#include <cstring>
#include <cstddef>
@@ -103,13 +105,7 @@ public:
static constexpr inline Hash zero() noexcept { return Hash{}; }
bool operator==(const Hash& h) const {
auto a = reinterpret_cast<const uint32_t*>(data_.data());
auto b = reinterpret_cast<const uint32_t*>(h.data_.data());
constexpr unsigned n = N / sizeof(uint32_t);
for (unsigned i=0; i < n; i++)
if (a[i] != b[i])
return false;
return true;
return data_ == h.data_;
}
bool operator!=(const Hash& h) const { return !(*this == h); }
@@ -234,6 +230,11 @@ public:
return get(data.data(), data.size());
}
template <size_t H>
static Hash get(const Hash<H>& o) {
return get(o.data(), o.size());
}
/**
* Computes the hash from a given data buffer of size data_len.
*/
@@ -255,6 +256,8 @@ public:
template <size_t M>
OPENDHT_PUBLIC friend std::istream& operator>> (std::istream& s, Hash<M>& h);
/** Returns view to thread-allocated memory, only valid until the next call to this function. */
std::string_view to_view() const { return std::string_view(to_c_str(), N*2); }
const char* to_c_str() const;
std::string toString() const;
@@ -320,7 +323,7 @@ Hash<N>
Hash<N>::getRandom()
{
Hash h;
crypto::random_device rdev;
std::random_device rdev;
std::uniform_int_distribution<uint32_t> rand_int;
auto a = reinterpret_cast<uint32_t*>(h.data());
auto b = reinterpret_cast<uint32_t*>(h.data() + h.size());
@@ -391,25 +394,11 @@ Hash<N>::toString() const
return std::string(to_c_str(), N*2);
}
struct OPENDHT_PUBLIC NodeExport {
InfoHash id;
sockaddr_storage ss;
socklen_t sslen;
template <typename Packer>
void msgpack_pack(Packer& pk) const
{
pk.pack_map(2);
pk.pack(std::string("id"));
pk.pack(id);
pk.pack(std::string("addr"));
pk.pack_bin(sslen);
pk.pack_bin_body((char*)&ss, sslen);
}
void msgpack_unpack(msgpack::object o);
OPENDHT_PUBLIC friend std::ostream& operator<< (std::ostream& s, const NodeExport& h);
};
}
template <size_t N>
struct fmt::formatter<dht::Hash<N>>: formatter<string_view> {
constexpr auto format(const dht::Hash<N>& c, format_context& ctx) const {
return formatter<string_view>::format(c.to_view(), ctx);
}
};
+2 -38
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
*
* Author: Adrien Béraud <adrien.beraud@savoirfairelinux.com>
*
@@ -20,7 +20,7 @@
#pragma once
#include "def.h"
#include "log_enable.h"
#include "logger.h"
#include <iostream>
#include <memory>
@@ -34,42 +34,6 @@ class DhtRunner;
*/
namespace log {
/**
* Terminal colors for logging
*/
namespace Color {
enum Code {
FG_RED = 31,
FG_GREEN = 32,
FG_YELLOW = 33,
FG_BLUE = 34,
FG_DEFAULT = 39,
BG_RED = 41,
BG_GREEN = 42,
BG_BLUE = 44,
BG_DEFAULT = 49
};
class Modifier {
const Code code;
public:
constexpr Modifier(Code pCode) : code(pCode) {}
friend std::ostream&
operator<<(std::ostream& os, const Modifier& mod) {
return os << "\033[" << mod.code << 'm';
}
};
}
constexpr const Color::Modifier def(Color::FG_DEFAULT);
constexpr const Color::Modifier red(Color::FG_RED);
constexpr const Color::Modifier yellow(Color::FG_YELLOW);
/**
* Print va_list to std::ostream (used for logging).
*/
OPENDHT_PUBLIC void
printLog(std::ostream &s, char const *m, va_list args);
OPENDHT_PUBLIC
std::shared_ptr<Logger> getStdLogger();
-191
View File
@@ -1,191 +0,0 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Author : Adrien Béraud <adrien.beraud@savoirfairelinux.com>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
#pragma once
#ifdef HAVE_CONFIG_H
#include "config.h"
#endif
#include "infohash.h"
#include <cstdarg>
#ifndef OPENDHT_LOG
#define OPENDHT_LOG true
#endif
namespace dht {
// Logging related utility functions
/**
* Wrapper for logging methods
*/
struct LogMethod {
LogMethod() = default;
LogMethod(LogMethod&& l) : func(std::move(l.func)) {}
LogMethod(const LogMethod& l) : func(l.func) {}
LogMethod& operator=(dht::LogMethod&& l) {
func = std::forward<LogMethod>(l.func);
return *this;
}
LogMethod& operator=(const dht::LogMethod& l) {
func = l.func;
return *this;
}
template<typename T>
explicit LogMethod(T&& t) : func(std::forward<T>(t)) {}
template<typename T>
LogMethod(const T& t) : func(t) {}
void operator()(char const* format, ...) const {
va_list args;
va_start(args, format);
func(format, args);
va_end(args);
}
inline void log(char const* format, va_list args) const {
func(format, args);
}
explicit operator bool() const {
return (bool)func;
}
void logPrintable(const uint8_t *buf, size_t buflen) const {
std::string buf_clean(buflen, '\0');
for (size_t i=0; i<buflen; i++)
buf_clean[i] = isprint(buf[i]) ? buf[i] : '.';
(*this)("%s", buf_clean.c_str());
}
private:
std::function<void(char const*, va_list)> func;
};
struct Logger {
LogMethod ERR = {};
LogMethod WARN = {};
LogMethod DBG = {};
Logger() = default;
Logger(LogMethod&& err, LogMethod&& warn, LogMethod&& dbg)
: ERR(std::move(err)), WARN(std::move(warn)), DBG(std::move(dbg)) {}
void setFilter(const InfoHash& f) {
filter_ = f;
filterEnable_ = static_cast<bool>(filter_);
}
inline void log0(const LogMethod& logger, char const* format, va_list args) const {
#if OPENDHT_LOG
if (logger and not filterEnable_)
logger.log(format, args);
#endif
}
inline void log1(const LogMethod& logger, const InfoHash& f, char const* format, va_list args) const {
#if OPENDHT_LOG
if (logger and (not filterEnable_ or f == filter_))
logger.log(format, args);
#endif
}
inline void log2(const LogMethod& logger, const InfoHash& f1, const InfoHash& f2, char const* format, va_list args) const {
#if OPENDHT_LOG
if (logger and (not filterEnable_ or f1 == filter_ or f2 == filter_))
logger.log(format, args);
#endif
}
inline void d(char const* format, ...) const {
#if OPENDHT_LOG
va_list args;
va_start(args, format);
log0(DBG, format, args);
va_end(args);
#endif
}
inline void d(const InfoHash& f, char const* format, ...) const {
#if OPENDHT_LOG
va_list args;
va_start(args, format);
log1(DBG, f, format, args);
va_end(args);
#endif
}
inline void d(const InfoHash& f1, const InfoHash& f2, char const* format, ...) const {
#if OPENDHT_LOG
va_list args;
va_start(args, format);
log2(DBG, f1, f2, format, args);
va_end(args);
#endif
}
inline void w(char const* format, ...) const {
#if OPENDHT_LOG
va_list args;
va_start(args, format);
log0(WARN, format, args);
va_end(args);
#endif
}
inline void w(const InfoHash& f, char const* format, ...) const {
#if OPENDHT_LOG
va_list args;
va_start(args, format);
log1(WARN, f, format, args);
va_end(args);
#endif
}
inline void w(const InfoHash& f1, const InfoHash& f2, char const* format, ...) const {
#if OPENDHT_LOG
va_list args;
va_start(args, format);
log2(WARN, f1, f2, format, args);
va_end(args);
#endif
}
inline void e(char const* format, ...) const {
#if OPENDHT_LOG
va_list args;
va_start(args, format);
log0(ERR, format, args);
va_end(args);
#endif
}
inline void e(const InfoHash& f, char const* format, ...) const {
#if OPENDHT_LOG
va_list args;
va_start(args, format);
log1(ERR, f, format, args);
va_end(args);
#endif
}
inline void e(const InfoHash& f1, const InfoHash& f2, char const* format, ...) const {
#if OPENDHT_LOG
va_list args;
va_start(args, format);
log2(ERR, f1, f2, format, args);
va_end(args);
#endif
}
private:
bool filterEnable_ {false};
InfoHash filter_ {};
};
}
+118
View File
@@ -0,0 +1,118 @@
/*
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author : Adrien Béraud <adrien.beraud@savoirfairelinux.com>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
#pragma once
#include "infohash.h"
#include <fmt/format.h>
#include <fmt/printf.h>
#include <functional>
#include <string_view>
#include <cstdarg>
namespace dht {
namespace log {
enum class LogLevel {
debug, warning, error
};
using LogMethod = std::function<void(LogLevel, std::string&&)>;
struct OPENDHT_PUBLIC Logger {
Logger() = delete;
Logger(LogMethod&& l)
: logger(std::move(l)) {
if (!logger)
throw std::invalid_argument{"logger and loggerf must be set"};
}
void setFilter(const InfoHash& f) {
filter_ = f;
filterEnable_ = static_cast<bool>(filter_);
}
inline void log0(LogLevel level, fmt::string_view format, fmt::printf_args args) const {
if (not filterEnable_)
logger(level, fmt::vsprintf(format, args));
}
inline void log1(LogLevel level, const InfoHash& f, fmt::string_view format, fmt::printf_args args) const {
if (not filterEnable_ or f == filter_)
logger(level, fmt::vsprintf(format, args));
}
inline void log2(LogLevel level, const InfoHash& f1, const InfoHash& f2, fmt::string_view format, fmt::printf_args args) const {
if (not filterEnable_ or f1 == filter_ or f2 == filter_)
logger(level, fmt::vsprintf(format, args));
}
template<typename S, typename... Args>
inline void debug(S&& format, Args&&... args) const {
logger(LogLevel::debug, fmt::format(format, args...));
}
template<typename S, typename... Args>
inline void warn(S&& format, Args&&... args) const {
logger(LogLevel::warning, fmt::format(format, args...));
}
template<typename S, typename... Args>
inline void error(S&& format, Args&&... args) const {
logger(LogLevel::error, fmt::format(format, args...));
}
template <typename... T>
inline void d(fmt::format_string<T...> format, T&&... args) const {
log0(LogLevel::debug, format, fmt::make_printf_args(args...));
}
template <typename... T>
inline void d(const InfoHash& f, fmt::format_string<T...> format, T&&... args) const {
log1(LogLevel::debug, f, format, fmt::make_printf_args(args...));
}
template <typename... T>
inline void d(const InfoHash& f1, const InfoHash& f2, fmt::format_string<T...> format, T&&... args) const {
log2(LogLevel::debug, f1, f2, format, fmt::make_printf_args(args...));
}
template <typename... T>
inline void w(fmt::format_string<T...> format, T&&... args) const {
log0(LogLevel::warning, format, fmt::make_printf_args(args...));
}
template <typename... T>
inline void w(const InfoHash& f, fmt::format_string<T...> format, T&&... args) const {
log1(LogLevel::warning, f, format, fmt::make_printf_args(args...));
}
template <typename... T>
inline void w(const InfoHash& f1, const InfoHash& f2, fmt::format_string<T...> format, T&&... args) const {
log2(LogLevel::warning, f1, f2, format, fmt::make_printf_args(args...));
}
template <typename... T>
inline void e(fmt::format_string<T...> format, T&&... args) const {
log0(LogLevel::error, format, fmt::make_printf_args(args...));
}
template <typename... T>
inline void e(const InfoHash& f, fmt::format_string<T...> format, T&&... args) const {
log1(LogLevel::error, f, format, fmt::make_printf_args(args...));
}
template <typename... T>
inline void e(const InfoHash& f1, const InfoHash& f2, fmt::format_string<T...> format, T&&... args) const {
log2(LogLevel::error, f1, f2, format, fmt::make_printf_args(args...));
}
private:
LogMethod logger = {};
bool filterEnable_ {false};
InfoHash filter_ {};
};
}
using Logger = log::Logger;
}
+17 -8
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author(s) : Adrien Béraud <adrien.beraud@savoirfairelinux.com>
* Simon Désaulniers <simon.desaulniers@savoirfairelinux.com>
*
@@ -27,7 +27,7 @@
#include "utils.h"
#include "rng.h"
#include "rate_limiter.h"
#include "log_enable.h"
#include "logger.h"
#include "network_utils.h"
#include <vector>
@@ -416,12 +416,19 @@ public:
*
* @return the request with information concerning its success.
*/
Sp<Request> sendUpdateValues(const Sp<Node>& n,
void sendUpdateValues(const Sp<Node>& n,
const InfoHash& infohash,
const std::vector<Sp<Value>>& values,
std::vector<Sp<Value>>&& values,
time_point created,
const Blob& token,
const size_t& sid);
size_t sid);
Sp<Request> sendUpdateValues(const Sp<Node>& n,
const InfoHash& infohash,
std::vector<Sp<Value>>::iterator begin,
std::vector<Sp<Value>>::iterator end,
time_point created,
const Blob& token,
size_t sid);
/**
* Parses a message and calls appropriate callbacks.
@@ -492,8 +499,7 @@ private:
static constexpr size_t MTU {1280};
static constexpr size_t MAX_PACKET_VALUE_SIZE {600};
static const std::string my_v;
static constexpr size_t MAX_MESSAGE_VALUE_SIZE {56 * 1024};
void process(std::unique_ptr<ParsedMessage>&&, const SockAddr& from);
@@ -525,7 +531,10 @@ private:
int send(const SockAddr& addr, const char *buf, size_t len, bool confirmed = false);
void sendValueParts(Tid tid, const std::vector<Blob>& svals, const SockAddr& addr);
std::vector<Blob> packValueHeader(msgpack::sbuffer&, const std::vector<Sp<Value>>&);
std::vector<Blob> packValueHeader(msgpack::sbuffer&, std::vector<Sp<Value>>::const_iterator, std::vector<Sp<Value>>::const_iterator) const;
std::vector<Blob> packValueHeader(msgpack::sbuffer& buf, const std::vector<Sp<Value>>& values) const {
return packValueHeader(buf, values.begin(), values.end());
}
void maintainRxBuffer(Tid tid);
/*************
+3 -3
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author(s) : Adrien Béraud <adrien.beraud@savoirfairelinux.com>
*
* This program is free software; you can redistribute it and/or modify
@@ -21,7 +21,7 @@
#include "sockaddr.h"
#include "utils.h"
#include "log_enable.h"
#include "logger.h"
#ifdef _WIN32
#include <ws2tcpip.h>
@@ -42,7 +42,7 @@ namespace dht {
namespace net {
static const constexpr in_port_t DHT_DEFAULT_PORT = 4222;
static const constexpr size_t RX_QUEUE_MAX_SIZE = 1024 * 16;
static const constexpr size_t RX_QUEUE_MAX_SIZE = 1024 * 64;
static const constexpr std::chrono::milliseconds RX_QUEUE_MAX_DELAY(650);
int bindSocket(const SockAddr& addr, SockAddr& bound);
+5 -5
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author(s) : Adrien Béraud <adrien.beraud@savoirfairelinux.com>
* Simon Désaulniers <simon.desaulniers@savoirfairelinux.com>
*
@@ -22,6 +22,7 @@
#include "infohash.h" // includes socket structures
#include "utils.h"
#include "sockaddr.h"
#include "node_export.h"
#include <list>
#include <map>
@@ -52,7 +53,7 @@ struct Node {
Node(const InfoHash& id, const sockaddr* sa, socklen_t salen, std::mt19937_64& rd)
: Node(id, SockAddr(sa, salen), rd) {}
InfoHash getId() const {
const InfoHash& getId() const {
return id;
}
const SockAddr& getAddr() const { return addr; }
@@ -92,8 +93,7 @@ struct Node {
NodeExport exportNode() const {
NodeExport ne;
ne.id = id;
ne.sslen = addr.getLength();
std::memcpy(&ne.ss, addr.get(), ne.sslen);
ne.addr = addr;
return ne;
}
sa_family_t getFamily() const { return addr.getFamily(); }
@@ -139,7 +139,7 @@ struct Node {
*/
Tid getNewTid() {
++transaction_id;
return transaction_id ? ++transaction_id : transaction_id;
return transaction_id ? transaction_id : ++transaction_id;
}
std::string toString() const;
+1 -1
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author(s) : Adrien Béraud <adrien.beraud@savoirfairelinux.com>
*
* This program is free software; you can redistribute it and/or modify
+48
View File
@@ -0,0 +1,48 @@
/*
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
#pragma once
#include "def.h"
#include "infohash.h"
#include "sockaddr.h"
#include <string_view>
namespace dht {
using namespace std::literals;
struct OPENDHT_PUBLIC NodeExport {
InfoHash id;
SockAddr addr;
template <typename Packer>
void msgpack_pack(Packer& pk) const
{
pk.pack_map(2);
pk.pack("id"sv);
pk.pack(id);
pk.pack("addr"sv);
pk.pack_bin(addr.getLength());
pk.pack_bin_body((const char*)addr.get(), (size_t)addr.getLength());
}
void msgpack_unpack(msgpack::object o);
OPENDHT_PUBLIC friend std::ostream& operator<< (std::ostream& s, const NodeExport& h);
};
}
+2 -2
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author(s) : Mingrui Zhang <mingrui.zhang@savoirfairelinux.com>
* Vsevolod Ivanov <vsevolod.ivanov@savoirfairelinux.com>
*
@@ -22,7 +22,7 @@
#include "def.h"
#include "sockaddr.h"
#include "infohash.h"
#include "log_enable.h"
#include "logger.h"
#include <thread>
+1 -1
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author: Adrien Béraud <adrien.beraud@savoirfairelinux.com>
*
* This program is free software; you can redistribute it and/or modify
+1 -1
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author : Adrien Béraud <adrien.beraud@savoirfairelinux.com>
*
* This program is free software; you can redistribute it and/or modify
+38 -93
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author : Adrien Béraud <adrien.beraud@savoirfairelinux.com>
*
* This program is free software; you can redistribute it and/or modify
@@ -27,104 +27,49 @@
namespace dht {
namespace crypto {
#ifndef _MSC_VER
#ifdef _WIN32
/**
* Hardware random number generator using Intel RDRAND/RDSEED,
* API-compatible with std::random_device.
/**
* Generate a seeded random engine.
*/
class random_device {
public:
using result_type = std::random_device::result_type;
using pseudo_engine = std::mt19937_64;
/**
* Current implementation assumption : result_type must be of a size
* supported by Intel RDRAND/RDSEED.
* result_type is unsigned int so this is currently safe.
*/
static_assert(
sizeof(result_type) == 2 ||
sizeof(result_type) == 4 ||
sizeof(result_type) == 8,
"result_type must be 16, 32 or 64 bits");
random_device();
result_type operator()();
static constexpr result_type min() {
return std::numeric_limits<result_type>::lowest();
}
static constexpr result_type max() {
return std::numeric_limits<result_type>::max();
}
double entropy() const {
if (hasRdrand() or hasRdseed())
return 1.;
return 0.;
}
static bool hasRdrand() {
static const bool hasrdrand = _hasRdrand();
return hasrdrand;
}
static bool hasRdseed() {
static const bool hasrdseed = _hasRdseed();
return hasrdseed;
}
private:
random_device& operator=(random_device&) = delete;
pseudo_engine gen;
std::uniform_int_distribution<result_type> dis {};
static bool hasIntelCpu();
static bool _hasRdrand();
static bool _hasRdseed();
struct CPUIDinfo {
unsigned int EAX;
unsigned int EBX;
unsigned int ECX;
unsigned int EDX;
CPUIDinfo(const unsigned int func, const unsigned int subfunc);
template<class T = std::mt19937, std::size_t N = T::state_size+1>
auto getSeededRandomEngine() -> typename std::enable_if<!!N, T>::type {
std::array<typename T::result_type, N> random_data;
constexpr auto gen = [](std::random_device& source) -> typename T::result_type {
for (unsigned j=0; j<64; j++) {
try {
return source();
} catch (...) {
std::this_thread::sleep_for(std::chrono::microseconds(500));
}
}
throw std::runtime_error("Can't generate random number");
};
bool rdrandStep(result_type* r);
bool rdrand(result_type* r);
bool rdseedStep(result_type* r);
bool rdseed(result_type* r);
};
#else
using random_device = std::random_device;
#endif
#else
using random_device = std::random_device;
#endif
template<class T = std::mt19937, std::size_t N = T::state_size>
auto getSeededRandomEngine () -> typename std::enable_if<!!N, T>::type {
typename T::result_type random_data[N];
for (unsigned i=0; i<256; i++) {
for (unsigned i=0; i<8; i++) {
try {
random_device source;
std::generate(std::begin(random_data), std::end(random_data), std::ref(source));
std::seed_seq seeds(std::begin(random_data), std::end(random_data));
T seededEngine (seeds);
return seededEngine;
std::random_device source;
for (auto& r : random_data)
r = gen(source);
std::seed_seq seed(
(std::seed_seq::result_type*)random_data.data(),
(std::seed_seq::result_type*)(random_data.data() + random_data.size()));
return T(seed);
} catch (...) {
std::this_thread::sleep_for(std::chrono::milliseconds(1));
std::this_thread::sleep_for(std::chrono::microseconds(500));
}
}
throw std::runtime_error("Can't seed random engine");
throw std::runtime_error("Can't seed random seed");
}
/**
* Generate a random engine from another source.
*/
template<class T = std::mt19937, std::size_t N = T::state_size+1>
auto getDerivedRandomEngine(T& source) -> typename std::enable_if<!!N, T>::type {
std::array<typename T::result_type, N> random_data;
std::generate(random_data.begin(), random_data.end(), std::ref(source));
std::seed_seq seed(
(std::seed_seq::result_type*)random_data.data(),
(std::seed_seq::result_type*)(random_data.data() + random_data.size()));
return T(seed);
}
}} // dht::crypto
+1 -1
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author(s) : Adrien Béraud <adrien.beraud@savoirfairelinux.com>
* Simon Désaulniers <simon.desaulniers@savoirfairelinux.com>
*
+1 -1
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author(s) : Adrien Béraud <adrien.beraud@savoirfairelinux.com>
* Simon Désaulniers <simon.desaulniers@savoirfairelinux.com>
*
+9 -2
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Authors: Adrien Béraud <adrien.beraud@savoirfairelinux.com>
* Simon Désaulniers <simon.desaulniers@savoirfairelinux.com>
* Sébastien Blin <sebastien.blin@savoirfairelinux.com>
@@ -149,6 +149,9 @@ public:
void setLocalCertificateStore(CertificateStoreQuery&& query_method) {
localQueryMethod_ = std::move(query_method);
}
void setOnPublicAddressChanged(PublicAddressChangedCb cb) override {
dht_->setOnPublicAddressChanged(cb);
}
/**
* SecureDht to Dht proxy
@@ -173,6 +176,10 @@ public:
void setStorageLimit(size_t limit = DEFAULT_STORAGE_LIMIT) override {
dht_->setStorageLimit(limit);
}
size_t getStorageLimit() const override {
return dht_->getStorageLimit();
}
std::vector<NodeExport> exportNodes() const override {
return dht_->exportNodes();
}
@@ -374,7 +381,7 @@ const ValueType CERTIFICATE_TYPE = {
try {
crypto::Certificate crt(v->data);
// TODO check certificate signature
return crt.getPublicKey().getId() == id;
return crt.getPublicKey().getId() == id || InfoHash::get(crt.getPublicKey().getLongId()) == id;
} catch (const std::exception& e) {}
return false;
},
+24 -1
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author : Adrien Béraud <adrien.beraud@savoirfairelinux.com>
*
* This program is free software; you can redistribute it and/or modify
@@ -20,6 +20,10 @@
#include "def.h"
#include <fmt/core.h>
#include <fmt/format.h>
#include <fmt/ostream.h>
#ifndef _WIN32
#include <sys/socket.h>
#include <netinet/in.h>
@@ -48,6 +52,7 @@ typedef uint16_t in_port_t;
namespace dht {
OPENDHT_PUBLIC void print_addr(std::ostream& os, const sockaddr* sa, socklen_t slen);
OPENDHT_PUBLIC std::string print_addr(const sockaddr* sa, socklen_t slen);
OPENDHT_PUBLIC std::string print_addr(const sockaddr_storage& ss, socklen_t sslen);
@@ -236,6 +241,15 @@ public:
return *reinterpret_cast<sockaddr_in6*>(get());
}
/**
* Releases the ownership of the managed object, if any.
* The caller is responsible for deleting the object with free().
*/
inline sockaddr* release() {
len = 0;
return addr.release();
}
/**
* Return true if address is a loopback IP address.
*/
@@ -280,6 +294,11 @@ public:
(uint8_t*)b.get()+start, len) < 0;
}
};
OPENDHT_PUBLIC friend std::ostream& operator<< (std::ostream& s, const SockAddr& h) {
print_addr(s, h.get(), h.getLength());
return s;
}
private:
struct free_delete { void operator()(void* p) { ::free(p); } };
std::unique_ptr<sockaddr, free_delete> addr {};
@@ -300,3 +319,7 @@ private:
OPENDHT_PUBLIC bool operator==(const SockAddr& a, const SockAddr& b);
}
#if FMT_VERSION >= 90000
template <> struct fmt::formatter<dht::SockAddr> : ostream_formatter {};
#endif
+9 -3
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
*
* Author: Adrien Béraud <adrien.beraud@savoirfairelinux.com>
*
@@ -37,7 +37,7 @@ public:
static ThreadPool& io();
ThreadPool();
ThreadPool(size_t maxThreads);
ThreadPool(unsigned minThreads, unsigned maxThreads = 0);
~ThreadPool();
void run(std::function<void()>&& cb);
@@ -61,8 +61,9 @@ public:
return get(std::move(cb));
}
void stop();
void stop(bool wait = true);
void join();
void detach();
private:
std::mutex lock_ {};
@@ -72,7 +73,12 @@ private:
unsigned readyThreads_ {0};
bool running_ {true};
unsigned minThreads_;
const unsigned maxThreads_;
std::chrono::steady_clock::duration threadExpirationDelay {std::chrono::minutes(5)};
double threadDelayRatio_ {2};
void threadEnded(std::thread&);
};
class OPENDHT_PUBLIC Executor : public std::enable_shared_from_this<Executor> {
+1 -1
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author : Adrien Béraud <adrien.beraud@savoirfairelinux.com>
*
* This program is free software; you can redistribute it and/or modify
+63 -62
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author(s) : Adrien Béraud <adrien.beraud@savoirfairelinux.com>
* Simon Désaulniers <simon.desaulniers@savoirfairelinux.com>
*
@@ -171,29 +171,29 @@ struct OPENDHT_PUBLIC Value
template<typename Functor>
Filter(Functor f) : std::function<bool(const Value&)>::function(f) {}
Filter chain(Filter&& f2) {
inline Filter chain(Filter&& f2) {
auto f1 = *this;
return chain(std::move(f1), std::move(f2));
}
Filter chainOr(Filter&& f2) {
inline Filter chainOr(Filter&& f2) {
auto f1 = *this;
return chainOr(std::move(f1), std::move(f2));
}
static Filter chain(Filter&& f1, Filter&& f2) {
static inline Filter chain(Filter&& f1, Filter&& f2) {
if (not f1) return std::move(f2);
if (not f2) return std::move(f1);
return [f1 = std::move(f1), f2 = std::move(f2)](const Value& v) {
return f1(v) and f2(v);
};
}
static Filter chain(const Filter& f1, const Filter& f2) {
static inline Filter chain(const Filter& f1, const Filter& f2) {
if (not f1) return f2;
if (not f2) return f1;
return [f1,f2](const Value& v) {
return f1(v) and f2(v);
};
}
static Filter chainAll(std::vector<Filter>&& set) {
static inline Filter chainAll(std::vector<Filter>&& set) {
if (set.empty()) return {};
return [set = std::move(set)](const Value& v) {
for (const auto& f : set)
@@ -202,18 +202,18 @@ struct OPENDHT_PUBLIC Value
return true;
};
}
static Filter chain(std::initializer_list<Filter> l) {
static inline Filter chain(std::initializer_list<Filter> l) {
return chainAll(std::vector<Filter>(l.begin(), l.end()));
}
static Filter chainOr(Filter&& f1, Filter&& f2) {
static inline Filter chainOr(Filter&& f1, Filter&& f2) {
if (not f1 or not f2) return {};
return [f1=std::move(f1),f2=std::move(f2)](const Value& v) {
return f1(v) or f2(v);
};
}
static Filter notFilter(Filter&& f) {
static inline Filter notFilter(Filter&& f) {
if (not f) return [](const Value&) { return false; };
return [f](const Value& v) { return not f(v); };
return [f = std::move(f)](const Value& v) { return not f(v); };
}
std::vector<Sp<Value>> filter(const std::vector<Sp<Value>>& values) {
if (not (*this))
@@ -228,50 +228,50 @@ struct OPENDHT_PUBLIC Value
/* Sneaky functions disguised in classes */
static const Filter AllFilter() {
static inline Filter AllFilter() {
return {};
}
static Filter TypeFilter(const ValueType& t) {
static inline Filter TypeFilter(const ValueType& t) {
return [tid = t.id](const Value& v) {
return v.type == tid;
};
}
static Filter TypeFilter(const ValueType::Id& tid) {
static inline Filter TypeFilter(const ValueType::Id& tid) {
return [tid](const Value& v) {
return v.type == tid;
};
}
static Filter IdFilter(const Id id) {
static inline Filter IdFilter(const Id id) {
return [id](const Value& v) {
return v.id == id;
};
}
static Filter RecipientFilter(const InfoHash& r) {
static inline Filter RecipientFilter(const InfoHash& r) {
return [r](const Value& v) {
return v.recipient == r;
};
}
static Filter OwnerFilter(const crypto::PublicKey& pk) {
static inline Filter OwnerFilter(const crypto::PublicKey& pk) {
return OwnerFilter(pk.getId());
}
static Filter OwnerFilter(const InfoHash& pkh) {
static inline Filter OwnerFilter(const InfoHash& pkh) {
return [pkh](const Value& v) {
return v.owner and v.owner->getId() == pkh;
};
}
static Filter SeqNumFilter(uint16_t seq_no) {
static inline Filter SeqNumFilter(uint16_t seq_no) {
return [seq_no](const Value& v) {
return v.seq == seq_no;
};
}
static Filter UserTypeFilter(std::string ut) {
static inline Filter UserTypeFilter(std::string ut) {
return [ut = std::move(ut)](const Value& v) {
return v.user_type == ut;
};
@@ -343,10 +343,10 @@ struct OPENDHT_PUBLIC Value
return unpack<T>(*this);
}
bool isEncrypted() const {
inline bool isEncrypted() const {
return not cypher.empty();
}
bool isSigned() const {
inline bool isSigned() const {
return owner and not signature.empty();
}
@@ -355,37 +355,24 @@ struct OPENDHT_PUBLIC Value
* Afterward, checkSignature() will return true and owner will
* be set to the corresponding public key.
*/
void sign(const crypto::PrivateKey& key) {
if (isEncrypted())
throw DhtException("Can't sign encrypted data.");
owner = key.getSharedPublicKey();
signature = key.sign(getToSign());
}
void sign(const crypto::PrivateKey& key);
/**
* Check that the value is signed and that the signature matches.
* If true, the owner field will contain the signer public key.
*/
bool checkSignature() const {
inline bool checkSignature() const {
return isSigned() and owner->checkSignature(getToSign(), signature);
}
std::shared_ptr<crypto::PublicKey> getOwner() const {
return std::static_pointer_cast<crypto::PublicKey>(owner);
inline std::shared_ptr<crypto::PublicKey> getOwner() const {
return owner;
}
/**
* Sign the value with from and returns the encrypted version for to.
*/
Value encrypt(const crypto::PrivateKey& from, const crypto::PublicKey& to) {
if (isEncrypted())
throw DhtException("Data is already encrypted.");
setRecipient(to.getId());
sign(from);
Value nv {id};
nv.setCypher(to.encrypt(getToEncrypt()));
return nv;
}
Value encrypt(const crypto::PrivateKey& from, const crypto::PublicKey& to);
Value() {}
@@ -440,7 +427,7 @@ struct OPENDHT_PUBLIC Value
/**
* Returns true if value contents are equals (not considering the value ID)
*/
inline bool contentEquals(const Value& o) {
inline bool contentEquals(const Value& o) const {
return isEncrypted() ? cypher == o.cypher :
((owner == o.owner || (owner and o.owner and *owner == *o.owner))
&& type == o.type
@@ -449,22 +436,25 @@ struct OPENDHT_PUBLIC Value
&& signature == o.signature);
}
inline bool operator== (const Value& o) {
inline bool operator== (const Value& o) const {
return id == o.id and contentEquals(o);
}
inline bool operator!= (const Value& o) const {
return !(*this == o);
}
void setRecipient(const InfoHash& r) {
inline void setRecipient(const InfoHash& r) {
recipient = r;
}
void setCypher(Blob&& c) {
inline void setCypher(Blob&& c) {
cypher = std::move(c);
}
/**
* Pack part of the data to be signed (must always be done the same way)
*/
Blob getToSign() const {
inline Blob getToSign() const {
msgpack::sbuffer buffer;
msgpack::packer<msgpack::sbuffer> pk(&buffer);
msgpack_pack_to_sign(pk);
@@ -474,7 +464,7 @@ struct OPENDHT_PUBLIC Value
/**
* Pack part of the data to be encrypted
*/
Blob getToEncrypt() const {
inline Blob getToEncrypt() const {
msgpack::sbuffer buffer;
msgpack::packer<msgpack::sbuffer> pk(&buffer);
msgpack_pack_to_encrypt(pk);
@@ -484,8 +474,8 @@ struct OPENDHT_PUBLIC Value
/** print value for debugging */
OPENDHT_PUBLIC friend std::ostream& operator<< (std::ostream& s, const Value& v);
std::string toString() const {
std::stringstream ss;
inline std::string toString() const {
std::ostringstream ss;
ss << *this;
return ss.str();
}
@@ -638,8 +628,16 @@ struct OPENDHT_PUBLIC Value
*/
unsigned priority {0};
inline bool isSignatureChecked() const {
return signatureChecked;
}
inline bool isDecrypted() const {
return decrypted;
}
bool checkSignature();
Sp<Value> decrypt(const crypto::PrivateKey& key);
private:
friend class SecureDht;
/* Cache for crypto ops */
bool signatureChecked {false};
bool signatureValid {false};
@@ -776,7 +774,7 @@ struct OPENDHT_PUBLIC Select
}
std::string toString() const {
std::stringstream ss;
std::ostringstream ss;
ss << *this;
return ss.str();
}
@@ -809,11 +807,11 @@ struct OPENDHT_PUBLIC Where
*
* @return the resulting Where instance.
*/
Where& id(Value::Id id) {
Where&& id(Value::Id id) {
FieldValue fv {Value::Field::Id, id};
if (std::find(filters_.begin(), filters_.end(), fv) == filters_.end())
filters_.emplace_back(std::move(fv));
return *this;
return std::move(*this);
}
/**
@@ -823,11 +821,11 @@ struct OPENDHT_PUBLIC Where
*
* @return the resulting Where instance.
*/
Where& valueType(ValueType::Id type) {
Where&& valueType(ValueType::Id type) {
FieldValue fv {Value::Field::ValueType, type};
if (std::find(filters_.begin(), filters_.end(), fv) == filters_.end())
filters_.emplace_back(std::move(fv));
return *this;
return std::move(*this);
}
/**
@@ -837,11 +835,11 @@ struct OPENDHT_PUBLIC Where
*
* @return the resulting Where instance.
*/
Where& owner(InfoHash owner_pk_hash) {
Where&& owner(InfoHash owner_pk_hash) {
FieldValue fv {Value::Field::OwnerPk, owner_pk_hash};
if (std::find(filters_.begin(), filters_.end(), fv) == filters_.end())
filters_.emplace_back(std::move(fv));
return *this;
return std::move(*this);
}
/**
@@ -851,11 +849,11 @@ struct OPENDHT_PUBLIC Where
*
* @return the resulting Where instance.
*/
Where& seq(uint16_t seq_no) {
Where&& seq(uint16_t seq_no) {
FieldValue fv {Value::Field::SeqNum, seq_no};
if (std::find(filters_.begin(), filters_.end(), fv) == filters_.end())
filters_.emplace_back(std::move(fv));
return *this;
return std::move(*this);
}
/**
@@ -865,11 +863,11 @@ struct OPENDHT_PUBLIC Where
*
* @return the resulting Where instance.
*/
Where& userType(std::string_view user_type) {
Where&& userType(std::string_view user_type) {
FieldValue fv {Value::Field::UserType, Blob {user_type.begin(), user_type.end()}};
if (std::find(filters_.begin(), filters_.end(), fv) == filters_.end())
filters_.emplace_back(std::move(fv));
return *this;
return std::move(*this);
}
/**
@@ -878,7 +876,10 @@ struct OPENDHT_PUBLIC Where
* @return the resulting Value::Filter.
*/
Value::Filter getFilter() const {
if (filters_.empty()) return {};
if (filters_.empty())
return {};
if (filters_.size() == 1)
return filters_[0].getLocalFilter();
std::vector<Value::Filter> fset;
fset.reserve(filters_.size());
for (const auto& f : filters_) {
@@ -896,7 +897,7 @@ struct OPENDHT_PUBLIC Where
}
std::string toString() const {
std::stringstream ss;
std::ostringstream ss;
ss << *this;
return ss.str();
}
@@ -962,7 +963,7 @@ struct OPENDHT_PUBLIC Query
void msgpack_unpack(const msgpack::object& o);
std::string toString() const {
std::stringstream ss;
std::ostringstream ss;
ss << *this;
return ss.str();
}
+237
View File
@@ -0,0 +1,237 @@
project('opendht', 'c', 'cpp', 'cython',
version: '3.2.0',
default_options: [
'cpp_std=c++17',
'warning_level=3'
])
gnutls = dependency('gnutls')
nettle = dependency('nettle')
msgpack = dependency('msgpack-cxx', required : false)
argon2 = dependency('libargon2')
openssl = dependency('openssl', required: get_option('proxy_client'))
jsoncpp = dependency('jsoncpp', required: get_option('proxy_client'))
fmt = dependency('fmt')
llhttp = dependency('llhttp', 'libllhttp', required: get_option('proxy_client'))
deps = [fmt, gnutls, nettle, msgpack, argon2, openssl, jsoncpp, llhttp]
add_project_arguments('-DMSGPACK_NO_BOOST', language : 'cpp')
add_project_arguments(['-Wno-return-type','-Wno-deprecated','-Wnon-virtual-dtor','-pedantic-errors','-fvisibility=hidden'], language : 'cpp')
opendht_inc = include_directories('include/opendht')
opendht_interface_inc = include_directories('include', is_system: true)
opendht_src = [
'src/utils.cpp',
'src/crypto.cpp',
'src/default_types.cpp',
'src/node.cpp',
'src/value.cpp',
'src/dht.cpp',
'src/callbacks.cpp',
'src/routing_table.cpp',
'src/node_cache.cpp',
'src/network_engine.cpp',
'src/securedht.cpp',
'src/dhtrunner.cpp',
'src/log.cpp',
'src/op_cache.cpp',
'src/network_utils.cpp',
'src/thread_pool.cpp',
]
conf_data = configuration_data()
if get_option('indexation').enabled()
opendht_src += 'src/indexation/pht.cpp'
add_project_arguments('-DOPENDHT_INDEXATION', language : 'cpp')
endif
if jsoncpp.found()
opendht_src += ['src/base64.cpp']
add_project_arguments('-DOPENDHT_JSONCPP', language : 'cpp')
conf_data.set('jsoncpp_lib', ', jsoncpp')
endif
if llhttp.found()
opendht_src += ['src/http.cpp', 'src/compat/os_cert.cpp']
if host_machine.system() == 'darwin'
deps+=dependency('appleframeworks', modules : ['CoreFoundation', 'Security'])
endif
conf_data.set('http_lib', '-lllhttp')
endif
if openssl.found()
conf_data.set('openssl_lib', ', openssl')
endif
if get_option('proxy_client').enabled()
opendht_src += ['src/dht_proxy_client.cpp']
add_project_arguments('-DOPENDHT_PROXY_CLIENT', language : 'cpp')
endif
if get_option('proxy_server').enabled()
opendht_src += 'src/dht_proxy_server.cpp'
add_project_arguments('-DOPENDHT_PROXY_SERVER', language : 'cpp')
endif
if get_option('push_notifications').enabled()
add_project_arguments('-DOPENDHT_PUSH_NOTIFICATIONS', language : 'cpp')
endif
if get_option('peer_discovery').enabled()
opendht_src += 'src/peer_discovery.cpp'
add_project_arguments('-DOPENDHT_PEER_DISCOVERY', language : 'cpp')
endif
opendht = library('opendht',
opendht_src,
include_directories : opendht_inc,
dependencies : deps,
cpp_args : ['-DOPENHT_BUILD', '-Dopendht_EXPORTS'],
install : true)
conf_data.set('VERSION', meson.project_version())
conf_data.set('prefix', get_option('prefix'))
conf_data.set('exec_prefix', get_option('prefix'))
conf_data.set('libdir', join_paths(get_option('prefix'), get_option('libdir')))
conf_data.set('includedir', join_paths(get_option('prefix'), get_option('includedir')))
conf_data.set('argon2_lib', ', libargon2')
configure_file(input : 'opendht.pc.in',
output : 'opendht.pc',
configuration : conf_data,
install: true,
install_dir: join_paths(get_option('prefix'), get_option('libdir'), 'pkgconfig'))
readline = meson.get_compiler('c').find_library('readline', required: get_option('tools'))
if get_option('c').enabled()
opendht_c = library('opendht-c',
['c/opendht.cpp', 'c/opendht_c.h'],
include_directories : opendht_interface_inc,
link_with : opendht,
dependencies : [msgpack, jsoncpp, fmt],
cpp_args : ['-DOPENDHT_C_BUILD', '-Dopendht_c_EXPORTS'],
install : true)
configure_file(input : 'opendht-c.pc.in',
output : 'opendht-c.pc',
configuration : conf_data,
install: true,
install_dir: join_paths(get_option('prefix'), get_option('libdir'), 'pkgconfig'))
if get_option('tools').enabled()
dhtcnode = executable('dhtcnode', 'tools/dhtcnode.c',
include_directories : ['c', opendht_interface_inc],
link_with : opendht_c,
dependencies : [readline],
install : true)
endif
endif
if get_option('python').require(get_option('indexation').enabled(), error_message: 'python requires indexation').allowed()
py = import('python').find_installation()
dep_py = py.dependency()
py.extension_module(
'opendht',
'python/opendht.pyx',
include_directories : opendht_interface_inc,
override_options : ['cython_language=cpp'],
link_with : [opendht],
dependencies : [msgpack, jsoncpp, fmt, dep_py],
install : true
)
endif
if get_option('tools').enabled()
dhtnode = executable('dhtnode', 'tools/dhtnode.cpp',
include_directories : opendht_interface_inc,
link_with : opendht,
dependencies : [readline, jsoncpp, msgpack, fmt, openssl],
install : true)
dhtchat = executable('dhtchat', 'tools/dhtchat.cpp',
include_directories : opendht_interface_inc,
link_with : opendht,
dependencies : [readline, jsoncpp, msgpack, fmt, openssl],
install : true)
dhtscanner = executable('dhtscanner', 'tools/dhtscanner.cpp',
include_directories : opendht_interface_inc,
link_with : opendht,
dependencies : [readline, jsoncpp, msgpack, fmt, openssl],
install : true)
if llhttp.found()
durl = executable('durl', 'tools/durl.cpp',
include_directories : opendht_interface_inc,
link_with : opendht,
dependencies : [msgpack, openssl])
endif
endif
# Unit tests
if get_option('tests').enabled()
cppunit = dependency('cppunit')
test_infohash = executable('test_infohash',
'tests/infohashtester.cpp', 'tests/tests_runner.cpp',
include_directories : opendht_interface_inc,
link_with : opendht,
dependencies : [cppunit, jsoncpp, fmt, openssl, msgpack])
test('Infohash', test_infohash)
test_value = executable('test_value',
'tests/valuetester.cpp', 'tests/tests_runner.cpp',
include_directories : opendht_interface_inc,
link_with : opendht,
dependencies : [cppunit, jsoncpp, fmt, openssl, msgpack])
test('Value', test_value)
test_crypto = executable('test_crypto',
'tests/cryptotester.cpp', 'tests/tests_runner.cpp',
include_directories : opendht_interface_inc,
link_with : opendht,
dependencies : [cppunit, jsoncpp, fmt, openssl, msgpack])
test('Crypto', test_crypto)
test_dhtrunner = executable('test_dhtrunner',
'tests/dhtrunnertester.cpp', 'tests/tests_runner.cpp',
include_directories : opendht_interface_inc,
link_with : opendht,
dependencies : [cppunit, jsoncpp, fmt, openssl, msgpack])
test('DhtRunner', test_dhtrunner)
test_threadpool = executable('test_threadpool',
'tests/threadpooltester.cpp', 'tests/tests_runner.cpp',
include_directories : opendht_interface_inc,
link_with : opendht,
dependencies : [cppunit, jsoncpp, fmt, openssl, msgpack])
test('ThreadPool', test_threadpool)
if get_option('proxy_client').enabled() or get_option('proxy_server').enabled()
test_http = executable('test_http',
'tests/httptester.cpp', 'tests/tests_runner.cpp',
include_directories : opendht_interface_inc,
link_with : opendht,
dependencies : [cppunit, jsoncpp, fmt, openssl, msgpack])
test('Http', test_http)
endif
if get_option('peer_discovery').enabled()
test_peerdiscovery = executable('test_peerdiscovery',
'tests/peerdiscoverytester.cpp', 'tests/tests_runner.cpp',
include_directories : opendht_interface_inc,
link_with : opendht,
dependencies : [cppunit, jsoncpp, fmt, openssl, msgpack])
test('PeerDiscovery', test_peerdiscovery)
endif
if get_option('proxy_server').enabled()
test_proxy = executable('test_dhtproxy',
'tests/dhtproxytester.cpp', 'tests/tests_runner.cpp',
include_directories : opendht_interface_inc,
link_with : opendht,
dependencies : [cppunit, jsoncpp, fmt, openssl, msgpack])
test('DhtProxy', test_proxy)
if get_option('long_tests').enabled()
test_proxystress = executable('test_dhtproxystress',
'tests/dhtproxy_stress_tester.cpp', 'tests/tests_runner.cpp',
include_directories : opendht_interface_inc,
link_with : opendht,
dependencies : [cppunit, jsoncpp, fmt, openssl, msgpack])
test('DhtProxy Stress', test_proxystress)
endif
endif
endif
+10
View File
@@ -0,0 +1,10 @@
option('proxy_client', type : 'feature', value : 'disabled')
option('proxy_server', type : 'feature', value : 'disabled')
option('push_notifications', type : 'feature', value : 'disabled')
option('peer_discovery', type : 'feature', value : 'enabled')
option('tools', type : 'feature', value : 'enabled')
option('c', type : 'feature', value : 'enabled')
option('indexation', type : 'feature', value : 'enabled')
option('python', type : 'feature', value : 'enabled')
option('tests', type : 'feature', value : 'enabled')
option('long_tests', type : 'feature', value : 'disabled')
+4 -3
View File
@@ -3,9 +3,10 @@ exec_prefix=@exec_prefix@
libdir=@libdir@
includedir=@includedir@
Name: OpenDHT
Description: C++14 Distributed Hash Table library
Description: C++17 Distributed Hash Table library
Version: @VERSION@
Libs: -L${libdir} -lopendht
Libs.private: @http_parser_lib@ -pthread
Requires.private: gnutls >= 3.3, nettle >= 2.4@argon2_lib@@jsoncpp_lib@@openssl_lib@
Libs.private: @http_lib@ -pthread
Requires: gnutls >= 3.3@jsoncpp_lib@@openssl_lib@
Requires.private: nettle >= 2.4@argon2_lib@
Cflags: -I${includedir}
+6 -1
View File
@@ -3,10 +3,15 @@ set(CURRENT_SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR})
set(CURRENT_BINARY_DIR ${CMAKE_CURRENT_BINARY_DIR})
configure_file(setup.py.in setup.py)
configure_file(pyproject.toml pyproject.toml COPYONLY)
add_custom_target(python ALL
COMMAND python3 setup.py build
DEPENDS opendht opendht_cpp.pxd opendht.pyx)
DEPENDS opendht opendht_cpp.pxd opendht.pyx pyproject.toml)
add_custom_target(dist
COMMAND python3 -m build
DEPENDS opendht opendht_cpp.pxd opendht.pyx pyproject.toml)
install(CODE "execute_process(COMMAND python3 setup.py install --root=\$ENV{DESTDIR}/ WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR})")
if (OPENDHT_TOOLS)
+2 -1
View File
@@ -2,7 +2,8 @@ if USE_CYTHON
noinst_HEADERS = \
opendht.pyx \
opendht_cpp.pxd
opendht_cpp.pxd \
pyproject.toml
PYTHON_INSTALL_RECORD = $(builddir)/install_record.txt
+123 -10
View File
@@ -5,7 +5,7 @@
# distutils: libraries = opendht gnutls
# cython: language_level=3
#
# Copyright (c) 2015-2022 Savoir-faire Linux Inc.
# Copyright (c) 2015-2023 Savoir-faire Linux Inc.
# Author(s): Guillaume Roguez <guillaume.roguez@savoirfairelinux.com>
# Adrien Béraud <adrien.beraud@savoirfairelinux.com>
# Simon Désaulniers <sim.desaulniers@gmail.com>
@@ -42,7 +42,7 @@ cimport opendht_cpp as cpp
import threading
cdef inline void lookup_callback(cpp.vector[cpp.shared_ptr[cpp.IndexValue]]* values, cpp.Prefix* p, void *user_data) with gil:
cdef inline void lookup_callback(cpp.vector[cpp.shared_ptr[cpp.IndexValue]]* values, cpp.Prefix* p, void *user_data) noexcept with gil:
cbs = <object>user_data
if 'lookup' in cbs and cbs['lookup']:
vals = []
@@ -52,13 +52,13 @@ cdef inline void lookup_callback(cpp.vector[cpp.shared_ptr[cpp.IndexValue]]* val
vals.append(v)
cbs['lookup'](vals, p.toString())
cdef inline void shutdown_callback(void* user_data) with gil:
cdef inline void shutdown_callback(void* user_data) noexcept with gil:
cbs = <object>user_data
if 'shutdown' in cbs and cbs['shutdown']:
cbs['shutdown']()
ref.Py_DECREF(cbs)
cdef inline bool get_callback(shared_ptr[cpp.Value] value, void *user_data) with gil:
cdef inline bool get_callback(shared_ptr[cpp.Value] value, void *user_data) noexcept with gil:
cbs = <object>user_data
cb = cbs['get']
f = cbs['filter'] if 'filter' in cbs else None
@@ -66,7 +66,7 @@ cdef inline bool get_callback(shared_ptr[cpp.Value] value, void *user_data) with
pv._value = value
return cb(pv) if not f or f(pv) else True
cdef inline bool value_callback(shared_ptr[cpp.Value] value, bool expired, void *user_data) with gil:
cdef inline bool value_callback(shared_ptr[cpp.Value] value, bool expired, void *user_data) noexcept with gil:
cbs = <object>user_data
cb = cbs['valcb']
f = cbs['filter'] if 'filter' in cbs else None
@@ -74,7 +74,7 @@ cdef inline bool value_callback(shared_ptr[cpp.Value] value, bool expired, void
pv._value = value
return cb(pv, expired) if not f or f(pv) else True
cdef inline void done_callback(bool done, cpp.vector[shared_ptr[cpp.Node]]* nodes, void *user_data) with gil:
cdef inline void done_callback(bool done, cpp.vector[shared_ptr[cpp.Node]]* nodes, void *user_data) noexcept with gil:
node_ids = []
for n in deref(nodes):
h = NodeEntry()
@@ -86,7 +86,7 @@ cdef inline void done_callback(bool done, cpp.vector[shared_ptr[cpp.Node]]* node
cbs['done'](done, node_ids)
ref.Py_DECREF(cbs)
cdef inline void done_callback_simple(bool done, void *user_data) with gil:
cdef inline void done_callback_simple(bool done, void *user_data) noexcept with gil:
cbs = <object>user_data
if 'done' in cbs and cbs['done']:
cbs['done'](done)
@@ -158,6 +158,15 @@ cdef class SockAddr(object):
return self.toString().decode()
def __repr__(self):
return "<%s '%s'>" % (self.__class__.__name__, str(self))
@staticmethod
def resolve(str host, str service=None):
vals = []
result = cpp.SockAddr.resolve(host.encode(), service.encode() if service else b'')
for val in result:
a = SockAddr()
a._addr = val
vals.append(a)
return vals
cdef class Node(_WithID):
cdef shared_ptr[cpp.Node] _node
@@ -453,11 +462,31 @@ cdef class Identity(object):
k._key = self._id.first
return k
def aesEncrypt(bytes data, str password) -> bytes :
cdef size_t d_len = len(data)
cdef cpp.uint8_t* d_ptr = <cpp.uint8_t*>data
cdef cpp.Blob indat
indat.assign(d_ptr, <cpp.uint8_t*>(d_ptr + d_len))
cdef cpp.Blob encrypted = cpp.aesEncrypt(indat, password.encode())
cdef char* encrypted_c_str = <char *>encrypted.data()
cdef Py_ssize_t length = encrypted.size()
return encrypted_c_str[:length]
def aesDecrypt(bytes data, str password) -> bytes :
cdef size_t d_len = len(data)
cdef cpp.uint8_t* d_ptr = <cpp.uint8_t*>data
cdef cpp.Blob indat
indat.assign(d_ptr, <cpp.uint8_t*>(d_ptr + d_len))
cdef cpp.Blob decrypted = cpp.aesDecrypt(indat, password.encode())
cdef char* decrypted_c_str = <char *>decrypted.data()
cdef Py_ssize_t length = decrypted.size()
return decrypted_c_str[:length]
cdef class DhtConfig(object):
cdef cpp.DhtRunnerConfig _config
def __init__(self):
self._config = cpp.DhtRunnerConfig()
self._config.threaded = True;
self._config.threaded = True
def setIdentity(self, Identity id):
self._config.dht_config.id = id._id
def setBootstrapMode(self, bool bootstrap):
@@ -471,6 +500,19 @@ cdef class DhtConfig(object):
def setRateLimit(self, ssize_t max_req_per_sec, ssize_t max_peer_req_per_sec):
self._config.dht_config.node_config.max_req_per_sec = max_req_per_sec
self._config.dht_config.node_config.max_peer_req_per_sec = max_peer_req_per_sec
def setProxyInfo(self, str proxy_server, str push_node_id="", str push_token="", str push_topic="", str push_platform="", Certificate server_ca = Certificate()):
self._config.proxy_server = proxy_server.encode()
self._config.push_node_id = push_node_id.encode()
self._config.push_token = push_token.encode()
self._config.push_topic = push_topic.encode()
self._config.push_platform = push_platform.encode()
self._config.server_ca = server_ca._cert
def setPeerDiscovery(self, bool peer_discovery, bool peer_publish):
self._config.peer_discovery = peer_discovery
self._config.peer_publish = peer_publish
def setBound(self, SockAddr bind4, SockAddr bind6):
self._config.bind4 = bind4._addr
self._config.bind6 = bind6._addr
cdef class DhtRunner(_WithID):
cdef cpp.shared_ptr[cpp.DhtRunner] thisptr
@@ -587,7 +629,8 @@ cdef class DhtRunner(_WithID):
while pending > 0:
lock.wait()
return res
def put(self, InfoHash key, Value val, done_cb=None):
def put(self, InfoHash key, Value val, done_cb=None, permanent=False):
"""Publish a new value on the DHT at key.
key -- the DHT key where to put the value
@@ -597,7 +640,7 @@ cdef class DhtRunner(_WithID):
if done_cb:
cb_obj = {'done':done_cb}
ref.Py_INCREF(cb_obj)
self.thisptr.get().put(key._infohash, val._value, cpp.bindDoneCb(done_callback, <void*>cb_obj))
self.thisptr.get().put(key._infohash, val._value, cpp.bindDoneCb(done_callback, <void*>cb_obj), cpp.time_point.max(), permanent)
else:
lock = threading.Condition()
pending = 0
@@ -614,6 +657,76 @@ cdef class DhtRunner(_WithID):
while pending > 0:
lock.wait()
return ok
def putSigned(self, InfoHash key, Value val, done_cb=None, permanent=False):
if done_cb:
cb_obj = {'done':done_cb}
ref.Py_INCREF(cb_obj)
self.thisptr.get().putSigned(key._infohash, val._value, cpp.bindDoneCb(done_callback, <void*>cb_obj), permanent)
else:
lock = threading.Condition()
pending = 0
ok = False
def tmp_done(ok_ret, nodes):
nonlocal pending, ok, lock
with lock:
ok = ok_ret
pending -= 1
lock.notify()
with lock:
pending += 1
self.putSigned(key, val, done_cb=tmp_done)
while pending > 0:
lock.wait()
return ok
def putEncrypted(self, InfoHash key, InfoHash to, Value val, done_cb=None, bool permanent=False):
if done_cb:
cb_obj = {'done':done_cb}
ref.Py_INCREF(cb_obj)
self.thisptr.get().putEncrypted(key._infohash, to._infohash, val._value, cpp.bindDoneCb(done_callback, <void*>cb_obj), permanent)
else:
lock = threading.Condition()
pending = 0
ok = False
def tmp_done(ok_ret, nodes):
nonlocal pending, ok, lock
with lock:
ok = ok_ret
pending -= 1
lock.notify()
with lock:
pending += 1
self.putEncrypted(key, to, val, done_cb=tmp_done, permanent=permanent)
while pending > 0:
lock.wait()
return ok
def putEncrypted(self, InfoHash key, PublicKey to, Value val, done_cb=None, bool permanent=False):
if done_cb:
cb_obj = {'done':done_cb}
ref.Py_INCREF(cb_obj)
self.thisptr.get().putEncrypted(key._infohash, to._key, val._value, cpp.bindDoneCb(done_callback, <void*>cb_obj), permanent)
else:
lock = threading.Condition()
pending = 0
ok = False
def tmp_done(ok_ret, nodes):
nonlocal pending, ok, lock
with lock:
ok = ok_ret
pending -= 1
lock.notify()
with lock:
pending += 1
self.putEncrypted(key, to, val, done_cb=tmp_done, permanent=permanent)
while pending > 0:
lock.wait()
return ok
def cancelPut(self, InfoHash key, Value val):
self.thisptr.get().cancelPut(key._infohash, val._value)
def listen(self, InfoHash key, value_cb):
t = ListenToken()
t._h = key._infohash
+31 -13
View File
@@ -1,4 +1,4 @@
# Copyright (c) 2015-2022 Savoir-faire Linux Inc.
# Copyright (c) 2015-2023 Savoir-faire Linux Inc.
# Author(s): Adrien Béraud <adrien.beraud@savoirfairelinux.com>
# Simon Désaulniers <sim.desaulniers@gmail.com>
#
@@ -21,12 +21,13 @@ from libcpp.string cimport string
from libcpp.vector cimport vector
from libcpp.utility cimport pair
from libcpp.map cimport map
from libcpp.memory cimport shared_ptr, make_shared
from libc.string cimport const_char, const_uchar
ctypedef uint16_t in_port_t
ctypedef unsigned short int sa_family_t;
cdef extern from "<iostream>" namespace "std::chrono" nogil:
cdef extern from "<chrono>" namespace "std::chrono" nogil:
cdef cppclass duration[ulong]:
duration() except +
@@ -34,16 +35,6 @@ cdef extern from "<iostream>" namespace "std::chrono" nogil:
duration seconds(uint64_t) except +
duration seconds()
cdef extern from "<memory>" namespace "std" nogil:
cdef cppclass shared_ptr[T]:
shared_ptr() except +
shared_ptr(T*) except +
T* get()
T operator*()
bool operator bool() const
void reset(T*)
shared_ptr[T] make_shared[T](...) except +
cdef extern from "<functional>" namespace "std" nogil:
cdef cppclass hash[T]:
hash() except +
@@ -59,6 +50,13 @@ cdef extern from "<future>" namespace "std" nogil:
bool valid() const
shared_future[T] share()
cdef extern from "opendht/utils.h" namespace "dht":
cdef cppclass time_point:
@staticmethod
time_point min() const
@staticmethod
time_point max() const
cdef extern from "opendht/infohash.h" namespace "dht":
cdef cppclass InfoHash:
InfoHash() except +
@@ -88,12 +86,16 @@ cdef extern from "opendht/sockaddr.h" namespace "dht":
bool isLoopback() const
bool isPrivate() const
bool isUnspecified() const
@staticmethod
vector[SockAddr] resolve(string host, string service) except +
ctypedef vector[uint8_t] Blob
cdef extern from "opendht/crypto.h" namespace "dht::crypto":
ctypedef pair[shared_ptr[PrivateKey], shared_ptr[Certificate]] Identity
cdef Identity generateIdentity(string name, Identity ca, unsigned bits)
cdef Blob aesEncrypt(Blob data, string password) except +
cdef Blob aesDecrypt(Blob encrypted, string password) except +
cdef cppclass PrivateKey:
PrivateKey()
@@ -239,6 +241,18 @@ cdef extern from "opendht/dhtrunner.h" namespace "dht":
cppclass Config:
SecureDhtConfig dht_config
bool threaded
string proxy_server
string push_node_id
string push_token
string push_topic
string push_platform
bool peer_discovery
bool peer_publish
shared_ptr[Certificate] server_ca
Identity client_identity
SockAddr bind4
SockAddr bind6
InfoHash getId() const
InfoHash getNodeId() const
void bootstrap(const_char*, const_char*)
@@ -254,7 +268,11 @@ cdef extern from "opendht/dhtrunner.h" namespace "dht":
string getRoutingTablesLog(sa_family_t af) const
string getSearchesLog(sa_family_t af) const
void get(InfoHash key, GetCallback get_cb, DoneCallback done_cb, nullptr_t f, Where w)
void put(InfoHash key, shared_ptr[Value] val, DoneCallback done_cb)
void put(InfoHash key, shared_ptr[Value] val, DoneCallback done_cb, time_point created, bool permanent)
void putSigned(InfoHash key, shared_ptr[Value] val, DoneCallback done_cb, bool permanent)
void putEncrypted(InfoHash key, InfoHash to, shared_ptr[Value] val, DoneCallback done_cb, bool permanent)
void putEncrypted(InfoHash key, shared_ptr[PublicKey] to, shared_ptr[Value] val, DoneCallback done_cb, bool permanent)
void cancelPut(InfoHash key, shared_ptr[Value] val)
ListenToken listen(InfoHash key, ValueCallback get_cb)
void cancelListen(InfoHash key, SharedListenToken token)
vector[unsigned] getNodeMessageStats(bool i)
+3
View File
@@ -0,0 +1,3 @@
[build-system]
requires = ["setuptools", "wheel", "Cython"]
build-backend = "setuptools.build_meta"
+3 -6
View File
@@ -1,4 +1,4 @@
# Copyright (C) 2014-2022 Savoir-faire Linux Inc.
# Copyright (C) 2014-2023 Savoir-faire Linux Inc.
# Author: Guillaume Roguez <guillaume.roguez@savoirfairelinux.com>
# Author: Adrien Béraud <adrien.beraud@savoirfairelinux.com>
#
@@ -32,17 +32,14 @@ setup(name="opendht",
author="Adrien Béraud, Guillaume Roguez, Simon Désaulniers",
license="GPLv3",
classifiers=[
'Development Status :: 3 - Alpha',
'Development Status :: 5 - Production/Stable',
'Intended Audience :: Developers',
'Topic :: Software Development :: Libraries :: Python Modules',
'Topic :: System :: Distributed Computing',
'Topic :: System :: Networking'
'License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.2',
'Programming Language :: Python :: 3.3',
'Programming Language :: Python :: 3.4',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.7',
],
cmdclass = { 'build_ext' : build_ext },
ext_modules = cythonize(Extension(
+1 -1
View File
@@ -20,7 +20,7 @@ help).
## Python dependencies
- pyroute2 >=0.3.14
- pyroute2 >=0.6.9
- matplotlib
- GeoIP (used by `scanner.py` for drawing map of the world)
- ipaddress
+1 -1
View File
@@ -1,6 +1,6 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
# Copyright (C) 2014-2022 Savoir-faire Linux Inc.
# Copyright (C) 2014-2023 Savoir-faire Linux Inc.
# Author(s): Adrien Béraud <adrien.beraud@savoirfairelinux.com>
# Simon Désaulniers <sim.desaulniers@gmail.com>
#
+2 -2
View File
@@ -1,6 +1,6 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
# Copyright (c) 2015-2022 Savoir-faire Linux Inc.
# Copyright (c) 2015-2023 Savoir-faire Linux Inc.
# Author(s): Adrien Béraud <adrien.beraud@savoirfairelinux.com>
# Simon Désaulniers <sim.desaulniers@gmail.com>
#
@@ -30,7 +30,7 @@ import traceback
import ipaddress
import netifaces
import numpy as np
from pyroute2.netns.process.proxy import NSPopen
from pyroute2 import NSPopen
import msgpack
from opendht import *
+1 -1
View File
@@ -1,5 +1,5 @@
# -*- coding: utf-8 -*-
# Copyright (C) 2015-2022 Savoir-Faire Linux Inc.
# Copyright (C) 2015-2023 Savoir-Faire Linux Inc.
# Author(s): Adrien Béraud <adrien.beraud@savoirfairelinux.com>
# Simon Désaulniers <sim.desaulniers@gmail.com>
+234 -88
View File
@@ -1,6 +1,6 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
# Copyright (c) 2015-2022 Savoir-faire Linux Inc.
# Copyright (c) 2015-2023 Savoir-faire Linux Inc.
# Author: Adrien Béraud <adrien.beraud@savoirfairelinux.com>
#
# This program is free software; you can redistribute it and/or modify
@@ -15,107 +15,253 @@
#
# You should have received a copy of the GNU General Public License
# along with this program; If not, see <http://www.gnu.org/licenses/>.
import argparse
import os
import subprocess
import argparse, subprocess
from pyroute2 import NDB, NSPopen
from pyroute2 import IPDB, NetNS
from pyroute2.netns.process.proxy import NSPopen
if __name__ == "__main__":
parser = argparse.ArgumentParser(description='Creates a virtual network topology for testing')
parser.add_argument('-i', '--ifname', help='interface name', default='ethdht')
parser.add_argument('-n', '--ifnum', type=int, help='number of isolated interfaces to create', default=1)
parser.add_argument('-r', '--remove', help='remove instead of adding network interfaces', action="store_true")
parser.add_argument('-l', '--loss', help='simulated packet loss (percent)', type=int, default=0)
parser.add_argument('-d', '--delay', help='simulated latency (ms)', type=int, default=0)
parser.add_argument('-4', '--ipv4', help='Enable IPv4', action="store_true")
parser.add_argument('-6', '--ipv6', help='Enable IPv6', action="store_true")
def int_range(mini, maxi):
def check_ifnum(arg):
try:
ret = int(arg)
except ValueError:
raise argparse.ArgumentTypeError('must be an integer')
if ret > maxi or ret < mini:
raise argparse.ArgumentTypeError(
f'must be {mini} <= int <= {maxi}'
)
return ret
return check_ifnum
if __name__ == '__main__':
parser = argparse.ArgumentParser(
description='Creates a virtual network topology for testing'
)
parser.add_argument(
'-i', '--ifname', help='interface name', default='ethdht'
)
parser.add_argument(
'-n',
'--ifnum',
type=int_range(1, 245),
help='number of isolated interfaces to create',
default=1,
)
parser.add_argument(
'-r',
'--remove',
help='remove instead of adding network interfaces',
action='store_true',
)
parser.add_argument(
'-l',
'--loss',
help='simulated packet loss (percent)',
type=int,
default=0,
)
parser.add_argument(
'-d', '--delay', help='simulated latency (ms)', type=int, default=0
)
parser.add_argument(
'-4', '--ipv4', help='Enable IPv4', action='store_true'
)
parser.add_argument(
'-6', '--ipv6', help='Enable IPv6', action='store_true'
)
parser.add_argument(
'-b',
'--debug',
help='Turn on debug logging and dump topology databases',
action='store_true',
)
parser.add_argument(
'-v',
'--verbose',
help='Turn on verbose output on netns and interfaces operations',
action='store_true',
)
args = parser.parse_args()
local_addr4 = '10.0.42.'
local_addr6 = '2001:db9::'
brige_name = 'br'+args.ifname
bripv4 = f'{local_addr4}1/24'
bripv6 = f'{local_addr6}1/64'
bridge_name = f'br{args.ifname}'
tap_name = f'tap{args.ifname}'
veth_names = []
namespaces = []
ipv4addrs = []
ipv6addrs = []
for ifn in range(args.ifnum):
namespaces.append(f'node{ifn}')
veth_names.append(f'{args.ifname}{ifn}')
ipv4addrs.append(f'{local_addr4}{ifn+8}/24' if args.ipv4 else None)
ipv6addrs.append(f'{local_addr6}{ifn+8}/64' if args.ipv6 else None)
ip = None
try:
ip = IPDB()
with NDB(log='debug' if args.debug else None) as ndb:
if args.remove:
# cleanup interfaces
for ifn in range(args.ifnum):
iface = args.ifname+str(ifn)
if iface in ip.interfaces:
with ip.interfaces[iface] as i:
i.remove()
if 'tap'+args.ifname in ip.interfaces:
with ip.interfaces['tap'+args.ifname] as i:
i.remove()
if brige_name in ip.interfaces:
with ip.interfaces[brige_name] as i:
i.remove()
for ifn in range(args.ifnum):
netns = NetNS('node'+str(ifn))
netns.close()
netns.remove()
else:
for ifn in range(args.ifnum):
iface = args.ifname+str(ifn)
if not iface in ip.interfaces:
ip.create(kind='veth', ifname=iface, peer=iface+'.1').commit()
# cleanup interfaces in the main namespace
for iface in veth_names + [bridge_name] + [tap_name]:
if iface in ndb.interfaces:
ndb.interfaces[iface].remove().commit()
if args.verbose:
print(f'link: del main/{iface}')
ip.create(kind='tuntap', ifname='tap'+args.ifname, mode='tap').commit()
with ip.create(kind='bridge', ifname=brige_name) as i:
for ifn in range(args.ifnum):
iface = args.ifname+str(ifn)
i.add_port(ip.interfaces[iface])
i.add_port(ip.interfaces['tap'+args.ifname])
if args.ipv4:
i.add_ip(local_addr4+'1/24')
if args.ipv6:
i.add_ip(local_addr6+'1/64')
i.up()
with ip.interfaces['tap'+args.ifname] as tap:
tap.up()
for ifn in range(args.ifnum):
iface = args.ifname+str(ifn)
nns = NetNS('node'+str(ifn))
iface1 = iface+'.1'
with ip.interfaces[iface1] as i:
i.net_ns_fd = nns.netns
with ip.interfaces[iface] as i:
i.up()
ip_ns = IPDB(nl=nns)
# cleanup namespaces
for nsname in namespaces:
try:
with ip_ns.interfaces.lo as lo:
lo.up()
with ip_ns.interfaces[iface1] as i:
if args.ipv4:
i.add_ip(local_addr4+str(ifn+8)+'/24')
if args.ipv6:
i.add_ip(local_addr6+str(ifn+8)+'/64')
i.up()
finally:
ip_ns.release()
nsp = NSPopen(nns.netns, ["tc", "qdisc", "add", "dev", iface1, "root", "netem", "delay", str(args.delay)+"ms", str(int(args.delay/2))+"ms", "loss", str(args.loss)+"%", "25%"], stdout=subprocess.PIPE)
#print(nsp.communicate()[0].decode())
ndb.netns[nsname].remove().commit()
if args.verbose:
print(f'netns: del {nsname}')
except KeyError:
pass
else:
# create ports
for veth, nsname, ipv4addr, ipv6addr in zip(
veth_names, namespaces, ipv4addrs, ipv6addrs
):
# create a network namespace and launch NDB for it
#
# another possible solution could be simply to attach
# the namespace to the main NDB instance, but it can
# take a lot of memory in case of many interfaces, thus
# launch and discard netns NDB instances
netns = NDB(
log='debug' if args.debug else None,
sources=[
{
'target': 'localhost',
'netns': nsname,
'kind': 'netns',
}
],
)
if args.verbose:
print(f'netns: add {nsname}')
# create the port and push the peer into the namespace
(
ndb.interfaces.create(
**{
'ifname': veth,
'kind': 'veth',
'state': 'up',
'peer': {'ifname': veth, 'net_ns_fd': nsname},
}
).commit()
)
if args.verbose:
print(f'link: add main/{veth} <-> {nsname}/{veth}')
# bring up namespace's loopback
(
netns.interfaces.wait(ifname='lo', timeout=3)
.set('state', 'up')
.commit()
)
if args.verbose:
print(f'link: set {nsname}/lo')
# bring up the peer
with netns.interfaces.wait(ifname=veth, timeout=3) as i:
i.set('state', 'up')
if args.ipv4:
i.add_ip(ipv4addr)
if args.ipv6:
i.add_ip(ipv6addr)
if args.verbose:
print(f'link: set {nsname}/{veth}, {ipv4addr}, {ipv6addr}')
# disconnect the namespace NDB agent, not removing the NS
if args.debug:
fname = f'{nsname}-ndb.db'
print(f'dump: netns topology database {fname}')
netns.schema.backup(fname)
netns.close()
# set up the emulation QDisc
nsp = NSPopen(
nsname,
[
'tc',
'qdisc',
'add',
'dev',
veth,
'root',
'netem',
'delay',
f'{args.delay}ms',
f'{int(args.delay)/2}ms',
'loss',
f'{args.loss}%',
'25%',
],
stdout=subprocess.PIPE,
)
nsp.communicate()
nsp.wait()
nsp.release()
if args.verbose:
print(
f'netem: add {nsname}/{veth}, '
f'{args.delay}, {args.loss}'
)
if args.ipv4:
subprocess.call(["sysctl", "-w", "net.ipv4.conf."+brige_name+".forwarding=1"])
if args.ipv6:
subprocess.call(["sysctl", "-w", "net.ipv6.conf."+brige_name+".forwarding=1"])
# create the tap
#
# for some reason we should create the tap inteface first,
# and only then bring it up, thus two commit() calls
(
ndb.interfaces.create(
kind='tuntap', ifname=tap_name, mode='tap'
)
.commit()
.set('state', 'up')
.commit()
)
if args.verbose:
print(f'link: add main/{tap_name}')
except Exception as e:
print('Error',e)
finally:
if ip:
ip.release()
# create the bridge and add all the ports
with ndb.interfaces.create(
ifname=bridge_name, kind='bridge', state='up'
) as i:
if args.ipv4:
i.add_ip(bripv4)
if args.ipv6:
i.add_ip(bripv6)
for iface in veth_names + [tap_name]:
i.add_port(iface)
if args.verbose:
print(f'link: add main/{bridge_name}, {bripv4}, {bripv6}')
with open(os.devnull, 'w') as fnull:
if args.ipv4:
subprocess.call(
[
'sysctl',
'-w',
f'net.ipv4.conf.{bridge_name}.forwarding=1',
],
stdout=fnull,
)
if args.verbose:
print(f'sysctl: set {bridge_name} ipv4 forwarding')
if args.ipv6:
subprocess.call(
[
'sysctl',
'-w',
f'net.ipv6.conf.{bridge_name}.forwarding=1',
],
stdout=fnull,
)
if args.verbose:
print(f'sysctl: set {bridge_name} ipv4 forwarding')
if args.debug:
fname = 'main-ndb.db'
print('dump: the main netns topology database')
ndb.schema.backup(fname)
+1 -1
View File
@@ -1,6 +1,6 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
# Copyright (C) 2014-2022 Savoir-faire Linux Inc.
# Copyright (C) 2014-2023 Savoir-faire Linux Inc.
# Author(s): Adrien Béraud <adrien.beraud@savoirfairelinux.com>
#
# This program is free software; you can redistribute it and/or modify
+2 -2
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2016-2022 Savoir-faire Linux Inc.
# Copyright (c) 2016-2023 Savoir-faire Linux Inc.
# Author: Adrien Béraud <adrien.beraud@savoirfairelinux.com>
#
# This program is free software; you can redistribute it and/or modify
@@ -72,7 +72,7 @@ if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Launch an OpenDHT node with an HTTP control interface')
parser.add_argument('-p', '--port', help='OpenDHT port to bind', type=int, default=4222)
parser.add_argument('-hp', '--http-port', help='HTTP port to bind', type=int, default=8080)
parser.add_argument('-b', '--bootstrap', help='bootstrap address', default="bootstrap.ring.cx:4222")
parser.add_argument('-b', '--bootstrap', help='bootstrap address', default="bootstrap.jami.net:4222")
args = parser.parse_args()
endpoints.serverFromString(reactor, "tcp:"+str(args.http_port)).listen(server.Site(DhtServer(args.port, args.bootstrap)))
reactor.run()
+2 -2
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2015-2022 Savoir-faire Linux Inc.
# Copyright (c) 2015-2023 Savoir-faire Linux Inc.
# Author: Adrien Béraud <adrien.beraud@savoirfairelinux.com>
#
# This program is free software; you can redistribute it and/or modify
@@ -24,7 +24,7 @@ from datetime import datetime
import opendht as dht
parser = argparse.ArgumentParser(description='Create a dht network of -n nodes')
parser.add_argument('-b', '--bootstrap', help='bootstrap address', default='bootstrap.ring.cx')
parser.add_argument('-b', '--bootstrap', help='bootstrap address', default='bootstrap.jami.net')
parser.add_argument('-n', '--num-ops', help='number of concurrent operations on the DHT', type=int, default=8)
parser.add_argument('-p', '--period', help='duration between each test (seconds)', type=int, default=60)
parser.add_argument('-t', '--timeout', help='timeout for a test to complete (seconds)', type=float, default=15)
+2 -3
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2015-2022 Savoir-faire Linux Inc.
# Copyright (c) 2015-2023 Savoir-faire Linux Inc.
# Author: Adrien Béraud <adrien.beraud@savoirfairelinux.com>
#
# This program is free software; you can redistribute it and/or modify
@@ -25,8 +25,7 @@ config.setRateLimit(-1, -1)
ping_node = dht.DhtRunner()
ping_node.run(config=config)
#ping_node.enableLogging()
#ping_node.bootstrap("bootstrap.ring.cx", "4222")
#ping_node.bootstrap("bootstrap.jami.net", "4222")
pong_node = dht.DhtRunner()
pong_node.run(config=config)
#pong_node.enableLogging()
+1 -1
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2015-2022 Savoir-faire Linux Inc.
# Copyright (c) 2015-2023 Savoir-faire Linux Inc.
# Author: Adrien Béraud <adrien.beraud@savoirfairelinux.com>
#
# This program is free software; you can redistribute it and/or modify
+1 -1
View File
@@ -8,4 +8,4 @@ edition = "2018"
[dependencies]
libc="0.2.0"
os_socketaddr="0.1.0"
os_socketaddr="0.2.3"
+1 -1
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author: Sébastien Blin <sebastien.blin@savoirfairelinux.com>
*
* This program is free software; you can redistribute it and/or modify
+1 -1
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author: Sébastien Blin <sebastien.blin@savoirfairelinux.com>
*
* This program is free software; you can redistribute it and/or modify
+1 -1
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author: Sébastien Blin <sebastien.blin@savoirfairelinux.com>
*
* This program is free software; you can redistribute it and/or modify
+11 -1
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author: Sébastien Blin <sebastien.blin@savoirfairelinux.com>
*
* This program is free software; you can redistribute it and/or modify
@@ -47,6 +47,8 @@ impl DhtRunnerConfig {
proxy_server: ptr::null(),
push_node_id: ptr::null(),
push_token: ptr::null(),
push_topic: ptr::null(),
push_platform: ptr::null(),
peer_discovery: false,
peer_publish: false,
server_ca: ptr::null_mut(),
@@ -73,6 +75,14 @@ impl DhtRunnerConfig {
self.push_token = CString::new(push_token).unwrap().as_ptr();
}
pub fn set_push_topic(&mut self, push_topic: &str) {
self.push_topic = CString::new(push_topic).unwrap().as_ptr();
}
pub fn set_push_platform(&mut self, push_platform: &str) {
self.push_platform = CString::new(push_platform).unwrap().as_ptr();
}
pub fn set_identity(&mut self, certificate: Box<DhtCertificate>, privatekey: Box<PrivateKey>) {
self.dht_config.id.privatekey = Box::into_raw(privatekey);
self.dht_config.id.certificate = Box::into_raw(certificate);
+3 -1
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author: Sébastien Blin <sebastien.blin@savoirfairelinux.com>
*
* This program is free software; you can redistribute it and/or modify
@@ -118,6 +118,8 @@ pub struct DhtRunnerConfig
pub proxy_server: *const c_char,
pub push_node_id: *const c_char,
pub push_token: *const c_char,
pub push_topic: *const c_char,
pub push_platform: *const c_char,
pub peer_discovery: bool,
pub peer_publish: bool,
pub server_ca: *mut DhtCertificate,
+1 -1
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author: Sébastien Blin <sebastien.blin@savoirfairelinux.com>
*
* This program is free software; you can redistribute it and/or modify
+1 -1
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author: Sébastien Blin <sebastien.blin@savoirfairelinux.com>
*
* This program is free software; you can redistribute it and/or modify
+1 -1
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author: Sébastien Blin <sebastien.blin@savoirfairelinux.com>
*
* This program is free software; you can redistribute it and/or modify
+1 -1
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author: Sébastien Blin <sebastien.blin@savoirfairelinux.com>
*
* This program is free software; you can redistribute it and/or modify
+2 -9
View File
@@ -19,7 +19,6 @@ libopendht_la_SOURCES = \
routing_table.cpp \
network_engine.cpp \
utils.cpp \
infohash.cpp \
node.cpp \
value.cpp \
crypto.cpp \
@@ -30,16 +29,13 @@ libopendht_la_SOURCES = \
network_utils.cpp \
thread_pool.cpp
if WIN32
libopendht_la_SOURCES += rng.cpp
endif
nobase_include_HEADERS = \
../include/opendht.h \
../include/opendht/def.h \
../include/opendht/dht.h \
../include/opendht/callbacks.h \
../include/opendht/node_cache.h \
../include/opendht/node_export.h \
../include/opendht/routing_table.h \
../include/opendht/network_engine.h \
../include/opendht/scheduler.h \
@@ -54,7 +50,7 @@ nobase_include_HEADERS = \
../include/opendht/dhtrunner.h \
../include/opendht/default_types.h \
../include/opendht/log.h \
../include/opendht/log_enable.h \
../include/opendht/logger.h \
../include/opendht/network_utils.h \
../include/opendht/rng.h \
../include/opendht/thread_pool.h
@@ -84,6 +80,3 @@ if ENABLE_INDEXATION
libopendht_la_SOURCES += indexation/pht.cpp
nobase_include_HEADERS += ../include/opendht/indexation/pht.h
endif
clean-local:
rm -rf libargon2.la
+1 -1
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
+1 -1
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
+2 -2
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author(s) : Adrien Béraud <adrien.beraud@savoirfairelinux.com>
*
* This program is free software; you can redistribute it and/or modify
@@ -92,7 +92,7 @@ bindDoneCbSimple(DoneCallbackSimpleRaw raw_cb, void* user_data) {
std::string
NodeStats::toString() const
{
std::stringstream ss;
std::ostringstream ss;
ss << "Known nodes: " << good_nodes << " good, " << dubious_nodes << " dubious, " << incoming_nodes << " incoming." << std::endl;
ss << searches << " searches, " << node_cache_size << " total cached nodes" << std::endl;
if (table_depth > 1) {
+1 -1
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
*
* Author: Andreas Traczyk <andreas.traczyk@savoirfairelinux.com>
*
+2 -93
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author: Andreas Traczyk <andreas.traczyk@savoirfairelinux.com>
*
* This program is free software; you can redistribute it and/or modify
@@ -17,8 +17,7 @@
*/
#include "os_cert.h"
#include "log_enable.h"
#include "logger.h"
#ifdef _WIN32
#include <windows.h>
@@ -32,96 +31,6 @@
#include <Security/Security.h>
#endif /*TARGET_OS_OSX*/
#if EMBEDDED_ASN1_TIME_PARSE
#undef X509_NAME
#undef OCSP_REQUEST
#undef OCSP_RESPONSE
#define GENTIME_LENGTH 15
#define UTCTIME_LENGTH 13
#define ATOI2(ar) ((ar) += 2, ((ar)[-2] - '0') * 10 + ((ar)[-1] - '0'))
extern "C" {
// Taken from libressl/src/crypto/asn1/a_time_tm.c
int
ASN1_time_parse(const char* bytes, size_t len, struct tm* tm, int mode)
{
size_t i;
int type = 0;
struct tm ltm;
struct tm* lt;
const char* p;
if (bytes == NULL)
return (-1);
/* Constrain to valid lengths. */
if (len != UTCTIME_LENGTH && len != GENTIME_LENGTH)
return (-1);
lt = tm;
if (lt == NULL) {
memset(&ltm, 0, sizeof(ltm));
lt = &ltm;
}
/* Timezone is required and must be GMT (Zulu). */
if (bytes[len - 1] != 'Z')
return (-1);
/* Make sure everything else is digits. */
for (i = 0; i < len - 1; i++) {
if (isdigit((unsigned char) bytes[i]))
continue;
return (-1);
}
/*
* Validate and convert the time
*/
p = bytes;
switch (len) {
case GENTIME_LENGTH:
if (mode == V_ASN1_UTCTIME)
return (-1);
lt->tm_year = (ATOI2(p) * 100) - 1900; /* cc */
type = V_ASN1_GENERALIZEDTIME;
/* FALLTHROUGH */
case UTCTIME_LENGTH:
if (type == 0) {
if (mode == V_ASN1_GENERALIZEDTIME)
return (-1);
type = V_ASN1_UTCTIME;
}
lt->tm_year += ATOI2(p); /* yy */
if (type == V_ASN1_UTCTIME) {
if (lt->tm_year < 50)
lt->tm_year += 100;
}
lt->tm_mon = ATOI2(p) - 1; /* mm */
if (lt->tm_mon < 0 || lt->tm_mon > 11)
return (-1);
lt->tm_mday = ATOI2(p); /* dd */
if (lt->tm_mday < 1 || lt->tm_mday > 31)
return (-1);
lt->tm_hour = ATOI2(p); /* HH */
if (lt->tm_hour < 0 || lt->tm_hour > 23)
return (-1);
lt->tm_min = ATOI2(p); /* MM */
if (lt->tm_min < 0 || lt->tm_min > 59)
return (-1);
lt->tm_sec = ATOI2(p); /* SS */
/* Leap second 60 is not accepted. Reconsider later? */
if (lt->tm_sec < 0 || lt->tm_sec > 59)
return (-1);
break;
default:
return (-1);
}
return type;
}
}
#endif /* EMBEDDED_ASN1_TIME_PARSE */
namespace dht {
namespace http {
+2 -32
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author: Andreas Traczyk <andreas.traczyk@savoirfairelinux.com>
*
* This program is free software; you can redistribute it and/or modify
@@ -18,42 +18,12 @@
#pragma once
#include "log_enable.h"
#include "logger.h"
#include <memory>
#include <openssl/x509.h>
#if (defined(LIBRESSL_VERSION_NUMBER) && (LIBRESSL_VERSION_NUMBER > 0x20501000L))
#define EMBEDDED_ASN1_TIME_PARSE 0
#else
#define EMBEDDED_ASN1_TIME_PARSE 1
#endif
#if EMBEDDED_ASN1_TIME_PARSE
#define V_ASN1_UTCTIME 23
#define V_ASN1_GENERALIZEDTIME 24
extern "C" {
/*
* Parse an RFC 5280 format ASN.1 time string.
*
* mode must be:
* 0 if we expect to parse a time as specified in RFC 5280 for an X509 object.
* V_ASN1_UTCTIME if we wish to parse an RFC5280 format UTC time.
* V_ASN1_GENERALIZEDTIME if we wish to parse an RFC5280 format Generalized time.
*
* Returns:
* -1 if the string was invalid.
* V_ASN1_UTCTIME if the string validated as a UTC time string.
* V_ASN1_GENERALIZEDTIME if the string validated as a Generalized time string.
*
* Fills in *tm with the corresponding time if tm is non NULL.
*/
int ASN1_time_parse(const char* bytes, size_t len, struct tm* tm, int mode);
}
#endif
namespace dht {
namespace http {
+147 -130
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author : Adrien Béraud <adrien.beraud@savoirfairelinux.com>
* Vsevolod Ivanov <vsevolod.ivanov@savoirfairelinux.com>
*
@@ -43,20 +43,6 @@ static std::uniform_int_distribution<int> rand_byte{ 0, std::numeric_limits<uint
static std::uniform_int_distribution<uint8_t> rand_byte;
#endif
// support for GnuTLS < 3.4.
#if GNUTLS_VERSION_NUMBER < 0x030400
#define GNUTLS_PKCS_PKCS12_3DES GNUTLS_PKCS_USE_PKCS12_3DES
#define GNUTLS_PKCS_PKCS12_ARCFOUR GNUTLS_PKCS_USE_PKCS12_ARCFOUR
#define GNUTLS_PKCS_PKCS12_RC2_40 GNUTLS_PKCS_USE_PKCS12_RC2_40
#define GNUTLS_PKCS_PBES2_3DES GNUTLS_PKCS_USE_PBES2_3DES
#define GNUTLS_PKCS_PBES2_AES_128 GNUTLS_PKCS_USE_PBES2_AES_128
#define GNUTLS_PKCS_PBES2_AES_192 GNUTLS_PKCS_USE_PBES2_AES_192
#define GNUTLS_PKCS_PBES2_AES_256 GNUTLS_PKCS_USE_PBES2_AES_256
#endif
#define DHT_AES_LEGACY_ENCRYPT 0
#define DHT_AES_LEGACY_DECRYPT 1
namespace dht {
namespace crypto {
@@ -104,28 +90,22 @@ Blob aesEncrypt(const uint8_t* data, size_t data_length, const Blob& key)
Blob ret(data_length + GCM_IV_SIZE + GCM_DIGEST_SIZE);
{
crypto::random_device rdev;
std::random_device rdev;
std::generate_n(ret.begin(), GCM_IV_SIZE, std::bind(rand_byte, std::ref(rdev)));
}
struct gcm_aes_ctx aes;
gcm_aes_set_key(&aes, key.size(), key.data());
gcm_aes_set_iv(&aes, GCM_IV_SIZE, ret.data());
#if DHT_AES_LEGACY_ENCRYPT
gcm_aes_update(&aes, data_length, data);
#endif
gcm_aes_encrypt(&aes, data_length, ret.data() + GCM_IV_SIZE, data);
gcm_aes_digest(&aes, GCM_DIGEST_SIZE, ret.data() + GCM_IV_SIZE + data_length);
return ret;
}
Blob aesEncrypt(const Blob& data, const std::string& password)
Blob aesEncrypt(const Blob& data, std::string_view password, const Blob& salt)
{
Blob salt;
Blob key = stretchKey(password, salt, 256 / 8);
Blob encrypted = aesEncrypt(data, key);
encrypted.insert(encrypted.begin(), salt.begin(), salt.end());
return encrypted;
Blob salt_actual = salt;
Blob key = stretchKey(password, salt_actual, 256 / 8);
return aesBuildEncrypted(aesEncrypt(data, key), salt_actual);
}
Blob aesDecrypt(const uint8_t* data, size_t data_length, const Blob& key)
@@ -148,40 +128,54 @@ Blob aesDecrypt(const uint8_t* data, size_t data_length, const Blob& key)
gcm_aes_digest(&aes, GCM_DIGEST_SIZE, digest.data());
if (not std::equal(digest.begin(), digest.end(), data + data_length - GCM_DIGEST_SIZE)) {
#if DHT_AES_LEGACY_DECRYPT
//gcm_aes_decrypt(&aes, data_sz, ret.data(), data + GCM_IV_SIZE);
Blob ret_tmp(data_sz);
struct gcm_aes_ctx aes_d;
gcm_aes_set_key(&aes_d, key.size(), key.data());
gcm_aes_set_iv(&aes_d, GCM_IV_SIZE, data);
gcm_aes_update(&aes_d, ret.size(), ret.data());
gcm_aes_encrypt(&aes_d, ret.size(), ret_tmp.data(), ret.data());
gcm_aes_digest(&aes_d, GCM_DIGEST_SIZE, digest.data());
if (not std::equal(digest.begin(), digest.end(), data + data_length - GCM_DIGEST_SIZE))
throw DecryptError("Can't decrypt data");
#else
throw DecryptError("Can't decrypt data");
#endif
}
return ret;
}
Blob aesDecrypt(const uint8_t* data, size_t data_len, const std::string& password)
Blob aesDecrypt(const uint8_t* data, size_t data_length, std::string_view password)
{
if (data_len <= PASSWORD_SALT_LENGTH)
throw DecryptError("Wrong data size");
Blob salt {data, data+PASSWORD_SALT_LENGTH};
Blob key = stretchKey(password, salt, 256/8);
return aesDecrypt(data+PASSWORD_SALT_LENGTH, data_len - PASSWORD_SALT_LENGTH, key);
return aesDecrypt(
aesGetEncrypted(data, data_length),
aesGetKey(data, data_length, password)
);
}
Blob stretchKey(const std::string& password, Blob& salt, size_t key_length)
Blob aesGetSalt(const uint8_t* data, size_t data_length)
{
if (data_length <= PASSWORD_SALT_LENGTH)
throw DecryptError("Wrong data size");
return Blob {data, data+PASSWORD_SALT_LENGTH};
}
std::string_view aesGetEncrypted(const uint8_t* data, size_t data_length)
{
if (data_length <= PASSWORD_SALT_LENGTH)
throw DecryptError("Wrong data size");
return std::string_view((const char*)(data+PASSWORD_SALT_LENGTH), data_length - PASSWORD_SALT_LENGTH);
}
Blob aesBuildEncrypted(const uint8_t* data, size_t data_length, const Blob& salt)
{
Blob ret;
ret.reserve(data_length + salt.size());
ret.insert(ret.end(), salt.begin(), salt.end());
ret.insert(ret.end(), data, data + data_length);
return ret;
}
Blob aesGetKey(const uint8_t* data, size_t data_length, std::string_view password)
{
Blob salt = aesGetSalt(data, data_length);
return stretchKey(password, salt, 256/8);
}
Blob stretchKey(std::string_view password, Blob& salt, size_t key_length)
{
if (salt.empty()) {
salt.resize(PASSWORD_SALT_LENGTH);
crypto::random_device rdev;
std::random_device rdev;
std::generate_n(salt.begin(), salt.size(), std::bind(rand_byte, std::ref(rdev)));
}
Blob res;
@@ -297,7 +291,7 @@ PrivateKey::sign(const uint8_t* data, size_t data_length) const
throw CryptoException("Can't sign data: no private key set !");
if (std::numeric_limits<unsigned>::max() < data_length)
throw CryptoException("Can't sign data: too large !");
gnutls_datum_t sig;
gnutls_datum_t sig {nullptr, 0};
const gnutls_datum_t dat {(unsigned char*)data, (unsigned)data_length};
if (gnutls_privkey_sign_data(key, GNUTLS_DIG_SHA512, 0, &dat, &sig) != GNUTLS_E_SUCCESS)
throw CryptoException("Can't sign data !");
@@ -310,7 +304,7 @@ Blob
PrivateKey::decryptBloc(const uint8_t* src, size_t src_size) const
{
const gnutls_datum_t dat {(uint8_t*)src, (unsigned)src_size};
gnutls_datum_t out;
gnutls_datum_t out {nullptr, 0};
int err = gnutls_privkey_decrypt_data(key, 0, &dat, &out);
if (err != GNUTLS_E_SUCCESS)
throw DecryptError(std::string("Can't decrypt data: ") + gnutls_strerror(err));
@@ -377,12 +371,13 @@ PrivateKey::getPublicKey() const
const std::shared_ptr<PublicKey>&
PrivateKey::getSharedPublicKey() const
{
if (publicKey_)
return publicKey_;
auto pk = std::make_shared<PublicKey>();
if (auto err = gnutls_pubkey_import_privkey(pk->pk, key, GNUTLS_KEY_KEY_CERT_SIGN | GNUTLS_KEY_CRL_SIGN, 0))
throw CryptoException(std::string("Can't retreive public key: ") + gnutls_strerror(err));
publicKey_ = pk;
std::lock_guard<std::mutex> lock(publicKeyMutex_);
if (not publicKey_) {
auto pk = std::make_shared<PublicKey>();
if (auto err = gnutls_pubkey_import_privkey(pk->pk, key, GNUTLS_KEY_KEY_CERT_SIGN | GNUTLS_KEY_CRL_SIGN, 0))
throw CryptoException(std::string("Can't retreive public key: ") + gnutls_strerror(err));
publicKey_ = std::move(pk);
}
return publicKey_;
}
@@ -487,7 +482,7 @@ void
PublicKey::encryptBloc(const uint8_t* src, size_t src_size, uint8_t* dst, size_t dst_size) const
{
const gnutls_datum_t key_dat {(uint8_t*)src, (unsigned)src_size};
gnutls_datum_t encrypted;
gnutls_datum_t encrypted {nullptr, 0};
auto err = gnutls_pubkey_encrypt_data(pk, 0, &key_dat, &encrypted);
if (err != GNUTLS_E_SUCCESS)
throw CryptoException(std::string("Can't encrypt data: ") + gnutls_strerror(err));
@@ -528,7 +523,7 @@ PublicKey::encrypt(const uint8_t* data, size_t data_len) const
throw CryptoException("Key is not long enough for AES128");
Blob key(aes_key_sz);
{
crypto::random_device rdev;
std::random_device rdev;
std::generate_n(key.begin(), key.size(), std::bind(rand_byte, std::ref(rdev)));
}
auto data_encrypted = aesEncrypt(data, data_len, key);
@@ -542,36 +537,36 @@ PublicKey::encrypt(const uint8_t* data, size_t data_len) const
return ret;
}
InfoHash
const InfoHash&
PublicKey::getId() const
{
if (not pk)
return {};
InfoHash id;
size_t sz = id.size();
if (auto err = gnutls_pubkey_get_key_id(pk, 0, id.data(), &sz))
throw CryptoException(std::string("Can't get public key ID: ") + gnutls_strerror(err));
if (sz != id.size())
throw CryptoException("Can't get public key ID: wrong output length.");
return id;
if (pk and not idCached_.load()) {
InfoHash id;
size_t sz = id.size();
if (auto err = gnutls_pubkey_get_key_id(pk, 0, id.data(), &sz))
throw CryptoException(std::string("Can't get public key ID: ") + gnutls_strerror(err));
if (sz != id.size())
throw CryptoException("Can't get public key ID: wrong output length.");
cachedId_ = id;
idCached_.store(true);
}
return cachedId_;
}
PkId
const PkId&
PublicKey::getLongId() const
{
if (not pk)
return {};
#if GNUTLS_VERSION_NUMBER < 0x030401
throw CryptoException("Can't get 256 bits public key ID: GnuTLS 3.4.1 or higher required.");
#else
PkId h;
size_t sz = h.size();
if (auto err = gnutls_pubkey_get_key_id(pk, GNUTLS_KEYID_USE_SHA256, h.data(), &sz))
throw CryptoException(std::string("Can't get 256 bits public key ID: ") + gnutls_strerror(err));
if (sz != h.size())
throw CryptoException("Can't get 256 bits public key ID: wrong output length.");
return h;
#endif
if (pk and not longIdCached_.load()) {
PkId h;
size_t sz = h.size();
if (auto err = gnutls_pubkey_get_key_id(pk, GNUTLS_KEYID_USE_SHA256, h.data(), &sz))
throw CryptoException(std::string("Can't get 256 bits public key ID: ") + gnutls_strerror(err));
if (sz != h.size())
throw CryptoException("Can't get 256 bits public key ID: wrong output length.");
cachedLongId_ = h;
longIdCached_.store(true);
}
return cachedLongId_;
}
gnutls_digest_algorithm_t
@@ -839,51 +834,55 @@ Certificate::~Certificate()
}
}
PublicKey
const PublicKey&
Certificate::getPublicKey() const
{
PublicKey pk_ret;
if (auto err = gnutls_pubkey_import_x509(pk_ret.pk, cert, 0))
throw CryptoException(std::string("Can't get certificate public key: ") + gnutls_strerror(err));
return pk_ret;
return *getSharedPublicKey();
}
InfoHash
const std::shared_ptr<PublicKey>&
Certificate::getSharedPublicKey() const
{
std::lock_guard<std::mutex> lock(publicKeyMutex_);
if (not publicKey_) {
auto pk = std::make_shared<PublicKey>();
if (auto err = gnutls_pubkey_import_x509(pk->pk, cert, 0))
throw CryptoException(std::string("Can't get certificate public key: ") + gnutls_strerror(err));
publicKey_ = std::move(pk);
}
return publicKey_;
}
const InfoHash&
Certificate::getId() const
{
if (not cert)
return {};
if (cachedId_)
return cachedId_;
InfoHash id;
size_t sz = id.size();
if (auto err = gnutls_x509_crt_get_key_id(cert, 0, id.data(), &sz))
throw CryptoException(std::string("Can't get certificate public key ID: ") + gnutls_strerror(err));
if (sz != id.size())
throw CryptoException("Can't get certificate public key ID: wrong output length.");
cachedId_ = id;
return id;
if (cert and not idCached_.load()) {
InfoHash id;
size_t sz = id.size();
if (auto err = gnutls_x509_crt_get_key_id(cert, 0, id.data(), &sz))
throw CryptoException(std::string("Can't get certificate public key ID: ") + gnutls_strerror(err));
if (sz != id.size())
throw CryptoException("Can't get certificate public key ID: wrong output length.");
cachedId_ = id;
idCached_.store(true);
}
return cachedId_;
}
PkId
const PkId&
Certificate::getLongId() const
{
if (not cert)
return {};
if (cachedLongId_)
return cachedLongId_;
#if GNUTLS_VERSION_NUMBER < 0x030401
throw CryptoException("Can't get certificate 256 bits public key ID: GnuTLS 3.4.1 or higher required.");
#else
PkId id;
size_t sz = id.size();
if (auto err = gnutls_x509_crt_get_key_id(cert, GNUTLS_KEYID_USE_SHA256, id.data(), &sz))
throw CryptoException(std::string("Can't get certificate 256 bits public key ID: ") + gnutls_strerror(err));
if (sz != id.size())
throw CryptoException("Can't get certificate 256 bits public key ID: wrong output length.");
cachedLongId_ = id;
return id;
#endif
if (cert and not longIdCached_.load()) {
PkId id;
size_t sz = id.size();
if (auto err = gnutls_x509_crt_get_key_id(cert, GNUTLS_KEYID_USE_SHA256, id.data(), &sz))
throw CryptoException(std::string("Can't get certificate 256 bits public key ID: ") + gnutls_strerror(err));
if (sz != id.size())
throw CryptoException("Can't get certificate 256 bits public key ID: wrong output length.");
cachedLongId_ = id;
longIdCached_.store(true);
}
return cachedLongId_;
}
Blob
@@ -986,7 +985,7 @@ Certificate::toString(bool chain) const
std::string
Certificate::print() const
{
gnutls_datum_t out;
gnutls_datum_t out {nullptr, 0};
gnutls_x509_crt_print(cert, GNUTLS_CRT_PRINT_FULL, &out);
std::string ret(out.data, out.data+out.size);
gnutls_free(out.data);
@@ -1062,7 +1061,7 @@ Certificate::generateOcspRequest(gnutls_x509_crt_t& issuer)
err = gnutls_ocsp_req_set_nonce(req.get(), 0, &nonce);
if (err < 0)
throw CryptoException(gnutls_strerror(err));
gnutls_datum_t rdata;
gnutls_datum_t rdata {nullptr, 0};
err = gnutls_ocsp_req_export(req.get(), &rdata);
if (err != 0)
throw CryptoException(gnutls_strerror(err));
@@ -1143,6 +1142,26 @@ saveIdentity(const Identity& id, const std::string& path, const std::string& pri
}
}
Identity
loadIdentity(const std::string &path,const std::string &privkey_password)
{
std::ifstream pkStream(path + ".pem", std::ios::in | std::ios::binary);
std::vector<uint8_t> pkContent((std::istreambuf_iterator<char>(pkStream)),
std::istreambuf_iterator<char>());
auto key = std::make_shared<PrivateKey>(pkContent, privkey_password);
pkStream.close();
// Create a certificate
gnutls_x509_crt_t gnuCert;
if (gnutls_x509_crt_init(&gnuCert) != GNUTLS_E_SUCCESS)
throw std::runtime_error("Failed to initialize gnutls certificate struct");
gnutls_datum_t crtContent {nullptr, 0};
// Read the certificate file
gnutls_load_file((path + ".crt").c_str(), &crtContent);
gnutls_x509_crt_import(gnuCert, &crtContent, GNUTLS_X509_FMT_PEM);
auto cert = std::make_shared<Certificate>(gnuCert);
return {std::move(key), std::move(cert)};
}
void
setValidityPeriod(gnutls_x509_crt_t cert, int64_t validity)
{
@@ -1158,7 +1177,7 @@ setValidityPeriod(gnutls_x509_crt_t cert, int64_t validity)
void
setRandomSerial(gnutls_x509_crt_t cert)
{
random_device rdev;
std::random_device rdev;
std::uniform_int_distribution<int64_t> dist{1};
int64_t cert_serial = dist(rdev);
gnutls_x509_crt_set_serial(cert, &cert_serial, sizeof(cert_serial));
@@ -1305,7 +1324,7 @@ std::string
OcspRequest::toString(const bool compact) const
{
int ret;
gnutls_datum_t dat;
gnutls_datum_t dat {nullptr, 0};
ret = gnutls_ocsp_req_print(request, compact ? GNUTLS_OCSP_PRINT_COMPACT : GNUTLS_OCSP_PRINT_FULL, &dat);
std::string str;
@@ -1320,7 +1339,7 @@ OcspRequest::toString(const bool compact) const
Blob
OcspRequest::pack() const
{
gnutls_datum_t dat;
gnutls_datum_t dat {nullptr, 0};
int err = gnutls_ocsp_req_export(request, &dat);
if (err < 0)
throw CryptoException(gnutls_strerror(err));
@@ -1332,7 +1351,7 @@ OcspRequest::pack() const
Blob
OcspRequest::getNonce() const
{
gnutls_datum_t dat;
gnutls_datum_t dat {nullptr, 0};
unsigned critical;
int err = gnutls_ocsp_req_get_nonce(request, &critical, &dat);
if (err < 0)
@@ -1365,7 +1384,7 @@ OcspResponse::~OcspResponse()
Blob
OcspResponse::pack() const
{
gnutls_datum_t dat;
gnutls_datum_t dat {nullptr, 0};
int err = gnutls_ocsp_resp_export(response, &dat);
if (err < 0)
throw CryptoException(gnutls_strerror(err));
@@ -1379,7 +1398,7 @@ OcspResponse::toString(const bool compact) const
{
int ret;
std::string str;
gnutls_datum_t dat;
gnutls_datum_t dat {nullptr, 0};
ret = gnutls_ocsp_resp_print(response, compact ? GNUTLS_OCSP_PRINT_COMPACT : GNUTLS_OCSP_PRINT_FULL, &dat);
if (ret == 0)
str = std::string((const char*)dat.data, (size_t)dat.size);
@@ -1414,7 +1433,7 @@ OcspResponse::verifyDirect(const Certificate& crt, const Blob& nonce)
if (not nonce.empty()) {
// Ensure no replay attack has been done
gnutls_datum_t rnonce;
gnutls_datum_t rnonce {nullptr, 0};
ret = gnutls_ocsp_resp_get_nonce(response, NULL, &rnonce);
if (ret < 0)
throw CryptoException(gnutls_strerror(ret));
@@ -1631,7 +1650,7 @@ RevocationList::sign(const PrivateKey& key, const Certificate& ca, duration vali
if (number == 0) {
// initialize to a random number
number_sz = sizeof(number);
random_device rdev;
std::random_device rdev;
std::generate_n((uint8_t*)&number, sizeof(number), std::bind(rand_byte, std::ref(rdev)));
} else
number = endian(endian(number) + 1);
@@ -1672,7 +1691,7 @@ RevocationList::getNumber() const
std::string
RevocationList::toString() const
{
gnutls_datum_t out;
gnutls_datum_t out {nullptr, 0};
gnutls_x509_crl_print(crl, GNUTLS_CRT_PRINT_FULL, &out);
std::string ret(out.data, out.data+out.size);
gnutls_free(out.data);
@@ -1771,10 +1790,8 @@ std::ostream& operator<< (std::ostream& o, const TrustList::VerifyResult& h)
o << "* Certificate has expired" << std::endl;
if (h.result & GNUTLS_CERT_UNEXPECTED_OWNER)
o << "* The owner is not the expected one" << std::endl;
#if GNUTLS_VERSION_NUMBER >= 0x030401
if (h.result & GNUTLS_CERT_PURPOSE_MISMATCH)
o << "* Certificate or an intermediate does not match the intended purpose" << std::endl;
#endif
if (h.result & GNUTLS_CERT_MISMATCH)
o << "* Certificate presented isn't the expected one" << std::endl;
} else {
+1 -1
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author : Adrien Béraud <adrien.beraud@savoirfairelinux.com>
*
* This program is free software; you can redistribute it and/or modify
+170 -137
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author(s) : Adrien Béraud <adrien.beraud@savoirfairelinux.com>
* Simon Désaulniers <simon.desaulniers@savoirfairelinux.com>
*
@@ -154,10 +154,18 @@ Dht::sendCachedPing(Bucket& b)
std::vector<SockAddr>
Dht::getPublicAddress(sa_family_t family)
{
std::sort(reported_addr.begin(), reported_addr.end(), [](const ReportedAddr& a, const ReportedAddr& b) {
return a.first > b.first;
});
std::vector<SockAddr> ret;
if (family == AF_UNSPEC) {
auto& d4 = dht(AF_INET).reported_addr;
auto& d6 = dht(AF_INET6).reported_addr;
ret.reserve(d4.size() + d6.size());
for (const auto& a : d4)
ret.emplace_back(a.second);
for (const auto& a : d6)
ret.emplace_back(a.second);
return ret;
}
auto& reported_addr = dht(family).reported_addr;
ret.reserve(!family ? reported_addr.size() : reported_addr.size()/2);
for (const auto& addr : reported_addr)
if (!family || family == addr.second.getFamily())
@@ -200,6 +208,8 @@ Dht::trySearchInsert(const Sp<Node>& node)
void
Dht::reportedAddr(const SockAddr& addr)
{
auto& reported_addr = dht(addr.getFamily()).reported_addr;
auto firstBefore = reported_addr.empty() ? nullptr : reported_addr.begin()->second.get();
auto it = std::find_if(reported_addr.begin(), reported_addr.end(), [&](const ReportedAddr& a){
return a.second == addr;
});
@@ -208,6 +218,21 @@ Dht::reportedAddr(const SockAddr& addr)
reported_addr.emplace_back(1, addr);
} else
it->first++;
std::sort(reported_addr.begin(), reported_addr.end(), [](const ReportedAddr& a, const ReportedAddr& b) {
return a.first > b.first;
});
if (publicAddressChangedCb_) {
auto firstAfter = reported_addr.begin()->second.get();
if (firstBefore != firstAfter) {
auto& otherDht = dht(addr.getFamily() == AF_INET ? AF_INET6 : AF_INET).reported_addr;
std::vector<SockAddr> v;
v.reserve(otherDht.empty() ? 1 : 2);
v.emplace_back(reported_addr.begin()->second);
if (not otherDht.empty())
v.emplace_back(otherDht.begin()->second);
publicAddressChangedCb_(std::move(v));
}
}
}
/* We just learnt about a node, not necessarily a new one. Confirm is 1 if
@@ -437,10 +462,9 @@ Dht::searchSendGetValues(Sp<Search> sr, SearchNode* pn, bool update)
return nullptr;
}
void Dht::searchSendAnnounceValue(const Sp<Search>& sr) {
void Dht::searchSendAnnounceValue(const Sp<Search>& sr, unsigned syncLevel) {
if (sr->announce.empty())
return;
unsigned i = 0;
std::weak_ptr<Search> ws = sr;
auto onDone = [this,ws](const net::Request& req, net::RequestAnswer&& answer)
@@ -552,6 +576,7 @@ void Dht::searchSendAnnounceValue(const Sp<Search>& sr) {
static const auto PROBE_QUERY = std::make_shared<Query>(Select {}.field(Value::Field::Id).field(Value::Field::SeqNum));
const auto& now = scheduler.time();
unsigned i = 0;
for (auto& np : sr->nodes) {
auto& n = *np;
if (not n.isSynced(now))
@@ -592,7 +617,7 @@ void Dht::searchSendAnnounceValue(const Sp<Search>& sr) {
std::bind(&Dht::searchNodeGetExpired, this, _1, _2, ws, PROBE_QUERY));
n.getStatus[PROBE_QUERY] = std::move(req);
}
if (not n.candidate and ++i == TARGET_NODES)
if (not n.candidate and ++i == syncLevel)
break;
}
}
@@ -675,6 +700,10 @@ Dht::searchStep(std::weak_ptr<Search> ws)
if (not sr or sr->expired or sr->done) return;
const auto& now = scheduler.time();
auto level = sr->syncLevel(now);
constexpr auto MARGIN = 1;
bool preSynced = level > MARGIN;
auto syncLevel = preSynced ? level - MARGIN : 0;
/*if (auto req_count = sr->currentlySolicitedNodeCount())
if (logger_)
logger_->d(sr->id, "[search %s IPv%c] step (%d requests)",
@@ -735,6 +764,21 @@ Dht::searchStep(std::weak_ptr<Search> ws)
if (sr->callbacks.empty() && sr->announce.empty() && sr->listeners.empty())
sr->setDone();
} else if (preSynced) {
if (not sr->listeners.empty()) {
auto nl = std::min(syncLevel, LISTEN_NODES);
unsigned i = 0;
for (auto& n : sr->nodes) {
if (not n->isSynced(now))
break;
searchSynchedNodeListen(sr, *n);
if (++i == nl)
break;
}
}
// Announce requests
searchSendAnnounceValue(sr, syncLevel);
}
while (sr->currentlySolicitedNodeCount() < MAX_REQUESTED_SEARCH_NODES and searchSendGetValues(sr));
@@ -870,7 +914,7 @@ Dht::listenTo(const InfoHash& id, sa_family_t af, ValueCallback cb, Value::Filte
throw DhtException("Can't create search");
if (logger_)
logger_->w(id, "[search %s IPv%c] listen", id.to_c_str(), (af == AF_INET) ? '4' : '6');
return sr->listen(cb, f, q, scheduler);
return sr->listen(cb, std::move(f), q, scheduler);
}
size_t
@@ -889,7 +933,7 @@ Dht::listen(const InfoHash& id, ValueCallback cb, Value::Filter f, Where where)
});
auto query = std::make_shared<Query>(Select{}, std::move(where));
auto filter = f.chain(query->where.getFilter());
auto filter = Value::Filter::chain(std::move(f), query->where.getFilter());
auto st = store.find(id);
if (st == store.end() && store.size() < max_store_keys)
st = store.emplace(id, scheduler.time() + MAX_STORAGE_MAINTENANCE_EXPIRE_TIME).first;
@@ -1194,15 +1238,20 @@ Dht::cancelPut(const InfoHash& id, const Value::Id& vid)
};
canceled |= sr_cancel_put(dht4.searches);
canceled |= sr_cancel_put(dht6.searches);
if (canceled)
storageErase(id, vid);
if (canceled) {
auto st = store.find(id);
if (st != store.end()) {
if (auto value = st->second.remove(id, vid))
storageRemoved(id, st->second, {value}, value->size());
}
}
return canceled;
}
// Storage
void
Dht::storageChanged(const InfoHash& id, Storage& st, ValueStorage& v, bool newValue)
Dht::storageChanged(const InfoHash& id, Storage& st, const Sp<Value>& v, bool newValue)
{
if (newValue) {
if (not st.local_listeners.empty()) {
@@ -1212,8 +1261,8 @@ Dht::storageChanged(const InfoHash& id, Storage& st, ValueStorage& v, bool newVa
cbs.reserve(st.local_listeners.size());
for (const auto& l : st.local_listeners) {
std::vector<Sp<Value>> vals;
if (not l.second.filter or l.second.filter(*v.data))
vals.push_back(v.data);
if (not l.second.filter or l.second.filter(*v))
vals.push_back(v);
if (not vals.empty()) {
if (logger_)
logger_->d(id, "[store %s] sending update local listener with token %lu",
@@ -1234,14 +1283,14 @@ Dht::storageChanged(const InfoHash& id, Storage& st, ValueStorage& v, bool newVa
for (const auto& node_listeners : st.listeners) {
for (const auto& l : node_listeners.second) {
auto f = l.second.query.where.getFilter();
if (f and not f(*v.data))
if (f and not f(*v))
continue;
if (logger_)
logger_->w(id, node_listeners.first->id, "[store %s] [node %s] sending update",
id.toString().c_str(),
node_listeners.first->toString().c_str());
std::vector<Sp<Value>> vals {};
vals.push_back(v.data);
vals.push_back(v);
Blob ntoken = makeToken(node_listeners.first->getAddr(), false);
network_engine.tellListener(node_listeners.first, l.first, id, 0, ntoken, {}, {},
std::move(vals), l.second.query, l.second.version);
@@ -1282,26 +1331,19 @@ Dht::storageStore(const InfoHash& id, const Sp<Value>& value, time_point created
vs->expiration_job = scheduler.add(expiration, std::bind(&Dht::expireStorage, this, id));
}
if (total_store_size > max_store_size) {
auto value = vs->data;
auto value_diff = store.second.values_diff;
auto value_edit = store.second.edited_values;
expireStore();
storageChanged(id, st->second, value, value_diff > 0 || value_edit > 0);
} else {
storageChanged(id, st->second, vs->data, store.second.values_diff > 0 || store.second.edited_values > 0);
}
storageChanged(id, st->second, *vs, store.second.values_diff > 0);
}
return std::get<0>(store);
}
bool
Dht::storageErase(const InfoHash& id, Value::Id vid)
{
auto st = store.find(id);
if (st == store.end())
return false;
auto ret = st->second.remove(id, vid);
total_store_size += ret.size_diff;
total_values += ret.values_diff;
return ret.values_diff;
}
void
Dht::storageAddListener(const InfoHash& id, const Sp<Node>& node, size_t socket_id, Query&& query, int version)
{
@@ -1333,36 +1375,8 @@ Dht::expireStore(decltype(store)::iterator i)
const auto& id = i->first;
auto& st = i->second;
auto stats = st.expire(id, scheduler.time());
total_store_size += stats.first;
total_values -= stats.second.size();
if (not stats.second.empty()) {
if (logger_)
logger_->d(id, "[store %s] discarded %ld expired values (%ld bytes)",
id.toString().c_str(), stats.second.size(), -stats.first);
if (not st.listeners.empty()) {
if (logger_)
logger_->d(id, "[store %s] %lu remote listeners", id.toString().c_str(), st.listeners.size());
std::vector<Value::Id> ids;
ids.reserve(stats.second.size());
for (const auto& v : stats.second)
ids.emplace_back(v->id);
for (const auto& node_listeners : st.listeners) {
for (const auto& l : node_listeners.second) {
if (logger_)
logger_->w(id, node_listeners.first->id, "[store %s] [node %s] sending expired",
id.toString().c_str(),
node_listeners.first->toString().c_str());
Blob ntoken = makeToken(node_listeners.first->getAddr(), false);
network_engine.tellListenerExpired(node_listeners.first, l.first, id, ntoken, ids, l.second.version);
}
}
}
for (const auto& local_listeners : st.local_listeners) {
local_listeners.second.get_cb(stats.second, true);
}
storageRemoved(id, st, stats.second, -stats.first);
}
}
@@ -1374,6 +1388,41 @@ Dht::expireStorage(InfoHash h)
expireStore(i);
}
void
Dht::storageRemoved(const InfoHash& id, Storage& st, const std::vector<Sp<Value>>& values, size_t totalSize)
{
if (logger_)
logger_->d(id, "[store %s] discarded %ld values (%ld bytes)",
id.toString().c_str(), values.size(), totalSize);
total_store_size -= totalSize;
total_values -= values.size();
if (not st.listeners.empty()) {
if (logger_)
logger_->d(id, "[store %s] %lu remote listeners", id.toString().c_str(), st.listeners.size());
std::vector<Value::Id> ids;
ids.reserve(values.size());
for (const auto& v : values)
ids.emplace_back(v->id);
for (const auto& node_listeners : st.listeners) {
for (const auto& l : node_listeners.second) {
if (logger_)
logger_->w(id, node_listeners.first->id, "[store %s] [node %s] sending expired",
id.toString().c_str(),
node_listeners.first->toString().c_str());
Blob ntoken = makeToken(node_listeners.first->getAddr(), false);
network_engine.tellListenerExpired(node_listeners.first, l.first, id, ntoken, ids, l.second.version);
}
}
}
for (const auto& local_listeners : st.local_listeners) {
local_listeners.second.get_cb(values, true);
}
}
void
Dht::expireStore()
{
@@ -1399,23 +1448,23 @@ Dht::expireStore()
break;
}
auto largest = store_quota.begin();
for (auto it = ++largest; it != store_quota.end(); ++it) {
for (auto it = std::next(largest); it != store_quota.end(); ++it) {
if (it->second.size() > largest->second.size())
largest = it;
}
if (logger_)
logger_->w("No space left: discarding value of largest consumer %s", largest->first.toString().c_str());
while (true) {
auto exp_value = largest->second.getOldest();
auto storage = store.find(exp_value.first);
if (storage != store.end()) {
auto ret = storage->second.remove(exp_value.first, exp_value.second);
total_store_size += ret.size_diff;
total_values += ret.values_diff;
if (logger_)
logger_->w("Discarded %ld bytes, still %ld used", largest->first.toString().c_str(), total_store_size);
if (ret.values_diff)
break;
if (largest != store_quota.end()) {
while (true) {
auto exp_value = largest->second.getOldest();
auto storage = store.find(exp_value.first);
if (storage != store.end()) {
if (logger_)
logger_->w("Storage quota full: discarding value from %s at %s %016" PRIx64, largest->first.toString().c_str(), exp_value.first.to_c_str(), exp_value.second);
if (auto value = storage->second.remove(exp_value.first, exp_value.second)) {
storageRemoved(storage->first, storage->second, {value}, value->size());
break;
}
}
}
}
}
@@ -1434,11 +1483,10 @@ Dht::connectivityChanged(sa_family_t af)
{
const auto& now = scheduler.time();
scheduler.edit(nextNodesConfirmation, now);
buckets(af).connectivityChanged(now);
auto& dht = this->dht(af);
dht.buckets.connectivityChanged(now);
dht.reported_addr.clear();
network_engine.connectivityChanged(af);
reported_addr.erase(std::remove_if(reported_addr.begin(), reported_addr.end(), [&](const ReportedAddr& addr){
return addr.second.getFamily() == af;
}), reported_addr.end());
startBootstrap(); // will only happen if disconnected
}
@@ -1555,21 +1603,20 @@ Dht::dumpSearch(const Search& sr, std::ostream& out) const
const auto& now = scheduler.time();
const auto& listen_expire = getListenExpiration();
using namespace std::chrono;
out << std::endl << "Search IPv" << (sr.af == AF_INET6 ? '6' : '4') << ' ' << sr.id << " gets: " << sr.callbacks.size();
out << ", last step: " << print_time_relative(now, sr.step_time);
if (sr.done)
out << " [done]";
if (sr.expired)
out << " [expired]";
bool synced = sr.isSynced(now);
out << (synced ? " [synced]" : " [not synced]");
if (synced && sr.isListening(now, listen_expire))
out << " [listening]";
out << std::endl;
fmt::print(out, "Search IPv{} {} gets: {} last step: {}{}{}{}{}\n",
(sr.af == AF_INET6 ? '6' : '4'),
sr.id,
sr.callbacks.size(),
print_time_relative(now, sr.step_time),
(sr.done ? " [done]"sv : ""sv),
(sr.expired ? " [expired]"sv : ""sv),
(sr.isSynced(now) ? " [synced]"sv : " [not synced]"sv),
(sr.isListening(now, listen_expire) ? " [listening]"sv : ""sv)
);
/*printing the queries*/
// printing the queries
if (sr.callbacks.size() + sr.listeners.size() > 0)
out << "Queries:" << std::endl;
fmt::print(out, "Queries:\n");
for (const auto& cb : sr.callbacks) {
out << *cb.second.query << std::endl;
}
@@ -1582,62 +1629,46 @@ Dht::dumpSearch(const Search& sr, std::ostream& out) const
out << "Announcement: " << *a.value << (announced ? " [announced]" : "") << std::endl;
}
out << " Common bits InfoHash Conn. Get Ops IP" << std::endl;
fmt::print(out, " Common bits InfoHash Conn. Get Ops IP\n");
auto last_get = sr.getLastGetTime();
for (const auto& np : sr.nodes) {
auto& n = *np;
out << std::setfill (' ') << std::setw(3) << InfoHash::commonBits(sr.id, n.node->id) << ' ' << n.node->id;
out << ' ' << (findNode(n.node->id, sr.af) ? '*' : ' ');
out << " [";
if (auto pendingCount = n.node->getPendingMessageCount())
out << pendingCount;
else
out << ' ';
out << (n.node->isExpired() ? 'x' : ' ') << "]";
// Get status
{
char g_i = n.pending(n.getStatus) ? (n.candidate ? 'c' : 'f') : ' ';
char s_i = n.isSynced(now) ? (n.last_get_reply > last_get ? 'u' : 's') : '-';
out << " [" << s_i << g_i << "] ";
}
// Listen status
if (not sr.listeners.empty()) {
if (n.listenStatus.empty())
out << " ";
else
out << "["
<< (n.isListening(now,listen_expire) ? 'l' : (n.pending(n.listenStatus) ? 'f' : ' ')) << "] ";
}
// Announce status
auto listen = (not sr.listeners.empty() ? (n.listenStatus.empty() ? " " : fmt::format("[{}] ", (n.isListening(now,listen_expire) ? 'l' : (n.pending(n.listenStatus) ? 'f' : ' ')))) : "");
std::string announce;
if (not sr.announce.empty()) {
if (n.acked.empty()) {
out << " ";
for (size_t a=0; a < sr.announce.size(); a++)
out << ' ';
announce = std::string(sr.announce.size() + 3, ' ');
} else {
out << "[";
announce = "[";
for (const auto& a : sr.announce) {
auto ack = n.acked.find(a.value->id);
if (ack == n.acked.end() or not ack->second.req) {
out << ' ';
announce += ' ';
} else {
out << ack->second.req->getStateChar();
announce += ack->second.req->getStateChar();
}
}
out << "] ";
announce += "] ";
}
}
out << n.node->getAddrStr() << std::endl;
fmt::print(out, "{:3} {} {} [{}{}] {}{}{}\n",
InfoHash::commonBits(sr.id, n.node->id),
n.node->id,
(findNode(n.node->id, sr.af) ? '*' : ' '),
(n.pending(n.getStatus) ? (n.candidate ? 'c' : 'f') : ' '),
(n.isSynced(now) ? (n.last_get_reply > last_get ? 'u' : 's') : '-'),
listen,
announce,
n.node->getAddrStr()
);
}
}
void
Dht::dumpTables() const
{
std::stringstream out;
std::ostringstream out;
out << "My id " << myid << std::endl;
out << "Buckets IPv4 :" << std::endl;
@@ -1664,7 +1695,7 @@ Dht::dumpTables() const
std::string
Dht::getStorageLog() const
{
std::stringstream out;
std::ostringstream out;
for (const auto& s : store)
out << printStorageLog(s);
out << std::endl << std::endl;
@@ -1689,7 +1720,7 @@ Dht::getStorageLog(const InfoHash& h) const
{
auto s = store.find(h);
if (s == store.end()) {
std::stringstream out;
std::ostringstream out;
out << "Storage " << h << " empty" << std::endl;
return out.str();
}
@@ -1699,7 +1730,7 @@ Dht::getStorageLog(const InfoHash& h) const
std::string
Dht::printStorageLog(const decltype(store)::value_type& s) const
{
std::stringstream out;
std::ostringstream out;
using namespace std::chrono;
const auto& st = s.second;
out << "Storage " << s.first << " "
@@ -1718,7 +1749,7 @@ Dht::printStorageLog(const decltype(store)::value_type& s) const
std::string
Dht::getRoutingTablesLog(sa_family_t af) const
{
std::stringstream out;
std::ostringstream out;
for (const auto& b : buckets(af))
dumpBucket(b, out);
return out.str();
@@ -1727,7 +1758,7 @@ Dht::getRoutingTablesLog(sa_family_t af) const
std::string
Dht::getSearchesLog(sa_family_t af) const
{
std::stringstream out;
std::ostringstream out;
auto num_searches = dht4.searches.size() + dht6.searches.size();
if (num_searches > 8) {
if (not af or af == AF_INET)
@@ -1752,7 +1783,7 @@ Dht::getSearchesLog(sa_family_t af) const
std::string
Dht::getSearchLog(const InfoHash& id, sa_family_t af) const
{
std::stringstream out;
std::ostringstream out;
if (af == AF_UNSPEC) {
out << getSearchLog(id, AF_INET) << getSearchLog(id, AF_INET6);
} else {
@@ -1784,12 +1815,14 @@ fromDhtConfig(const Config& config)
return netConf;
}
Dht::Dht(std::unique_ptr<net::DatagramSocket>&& sock, const Config& config, const Sp<Logger>& l)
Dht::Dht(std::unique_ptr<net::DatagramSocket>&& sock, const Config& config, const Sp<Logger>& l, std::unique_ptr<std::mt19937_64>&& rda)
: DhtInterface(l),
rd(rda ? std::move(*rda) : crypto::getSeededRandomEngine<std::mt19937_64>()),
myid(config.node_id ? config.node_id : InfoHash::getRandom(rd)),
store(),
store_quota(),
max_store_keys(config.max_store_size ? (int)config.max_store_size : MAX_HASHES),
max_store_keys(config.max_store_keys ? (int)config.max_store_keys : MAX_HASHES),
max_store_size(config.max_store_size ? (int)config.max_store_size : DEFAULT_STORAGE_LIMIT),
max_searches(config.max_searches ? (int)config.max_searches : MAX_SEARCHES),
network_engine(myid, fromDhtConfig(config), std::move(sock), logger_, rd, scheduler,
std::bind(&Dht::onError, this, _1, _2),
@@ -1834,7 +1867,7 @@ Dht::Dht(std::unique_ptr<net::DatagramSocket>&& sock, const Config& config, cons
expire();
if (logger_)
logger_->d("DHT node initialised with ID %s", myid.toString().c_str());
logger_->debug("DHT node initialised with ID {:s}", myid);
}
bool
@@ -2226,7 +2259,7 @@ Dht::pingNode(SockAddr sa, DoneCallbackSimple&& cb)
{
scheduler.syncTime();
if (logger_)
logger_->d("Sending ping to %s", sa.toString().c_str());
logger_->debug("Sending ping to {}", sa);
auto& count = dht(sa.getFamily()).pending_pings;
count++;
network_engine.sendPing(std::move(sa), [&count,cb](const net::Request&, net::RequestAnswer&&) {
@@ -2558,7 +2591,7 @@ Dht::storageRefresh(const InfoHash& id, Value::Id vid)
}
}
auto expiration = s->second.refresh(now, vid, types);
auto expiration = s->second.refresh(id, now, vid, types);
if (expiration.first) {
scheduler.cancel(expiration.first->expiration_job);
if (expiration.second != time_point::max()) {
@@ -2632,7 +2665,7 @@ Dht::loadState(const std::string& path)
std::vector<Sp<Node>> tmpNodes;
tmpNodes.reserve(state.nodes.size());
for (const auto& node : state.nodes)
tmpNodes.emplace_back(network_engine.insertNode(node.id, SockAddr(node.ss, node.sslen)));
tmpNodes.emplace_back(network_engine.insertNode(node.id, node.addr));
importValues(state.values);
}
} catch (const std::exception& e) {
+100 -45
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author: Sébastien Blin <sebastien.blin@savoirfairelinux.com>
* Adrien Béraud <adrien.beraud@savoirfairelinux.com>
* Vsevolod Ivanov <vsevolod.ivanov@savoirfairelinux.com>
@@ -23,7 +23,7 @@
#include "op_cache.h"
#include "utils.h"
#include <http_parser.h>
#include <llhttp.h>
#include <deque>
namespace dht {
@@ -97,7 +97,7 @@ std::string
getRandomSessionId(size_t length = 8) {
static constexpr const char chars[] = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789!#$%&()*+,./:;<=>?@[]^_`{|}~";
std::string str(length, 0);
crypto::random_device rdev;
std::random_device rdev;
std::uniform_int_distribution<> dist(0, (sizeof(chars)/sizeof(char)) - 2);
std::generate_n( str.begin(), length, [&]{ return chars[dist(rdev)]; } );
return str;
@@ -116,6 +116,9 @@ DhtProxyClient::DhtProxyClient(
, loopSignal_(signal)
, jsonReader_(Json::CharReaderBuilder{}.newCharReader())
{
localAddrv4_.setFamily(AF_INET);
localAddrv6_.setFamily(AF_INET6);
jsonBuilder_["commentStyle"] = "None";
jsonBuilder_["indentation"] = "";
if (logger_) {
@@ -187,7 +190,12 @@ void
DhtProxyClient::stop()
{
if (not isDestroying_.exchange(true)) {
resolver_.reset();
std::shared_ptr<http::Resolver> resolver;
{
std::lock_guard<std::mutex> l(resolverLock_);
resolver = std::move(resolver_);
}
resolver->cancel();
cancelAllListeners();
if (infoState_)
infoState_->cancel = true;
@@ -311,26 +319,30 @@ DhtProxyClient::get(const InfoHash& key, GetCallback cb, DoneCallback donecb, Va
return;
}
try {
auto request = buildRequest("/" + key.toString());
auto request = buildRequest("/key/" + key.toString());
auto reqid = request->id();
//request->set_connection_type(restinio::http_connection_header_t::keep_alive);
request->set_method(restinio::http_method_get());
setHeaderFields(*request);
auto opstate = std::make_shared<OperationState>();
Value::Filter filter = w.empty() ? f : f.chain(w.getFilter());
auto rxBuf = std::make_shared<LineSplit>();
request->add_on_body_callback([this, key, opstate, filter, rxBuf, cb](const char* at, size_t length){
request->add_on_body_callback([
this,
key,
opstate,
filter = Value::Filter::chain(std::move(f), w.getFilter()),
rxBuf = std::make_shared<LineSplit>(),
cb
](const char* at, size_t length){
try {
auto& b = *rxBuf;
b.append(at, length);
rxBuf->append(at, length);
// one value per body line
std::vector<Sp<Value>> values;
while (b.getLine('\n') and !opstate->stop) {
while (rxBuf->getLine('\n') and !opstate->stop) {
std::string err;
Json::Value json;
const auto& line = b.line();
const auto& line = rxBuf->line();
if (!jsonReader_->parse(line.data(), line.data() + line.size(), &json, &err)){
opstate->ok.store(false);
return;
@@ -460,7 +472,9 @@ DhtProxyClient::handleRefreshPut(const asio::error_code &ec, InfoHash key, Value
std::shared_ptr<http::Request>
DhtProxyClient::buildRequest(const std::string& target)
{
std::unique_lock<std::mutex> l(resolverLock_);
auto resolver = resolver_;
l.unlock();
if (not resolver)
resolver = std::make_shared<http::Resolver>(httpContext_, proxyUrl_, logger_);
auto request = target.empty()
@@ -478,10 +492,10 @@ void
DhtProxyClient::doPut(const InfoHash& key, Sp<Value> val, DoneCallbackSimple cb, time_point /*created*/, bool permanent)
{
if (logger_)
logger_->d("[proxy:client] [put] [search %s] executing for %s", key.to_c_str(), val->toString().c_str());
logger_->debug("[proxy:client] [put] [search {}] executing for {}", key, val->toString());
try {
auto request = buildRequest("/" + key.toString());
auto request = buildRequest("/key/" + key.toString());
auto reqid = request->id();
request->set_method(restinio::http_method_post());
setHeaderFields(*request);
@@ -526,12 +540,12 @@ DhtProxyClient::doPut(const InfoHash& key, Sp<Value> val, DoneCallbackSimple cb,
}
} else {
if (logger_)
logger_->e("[proxy:client] [status] failed to parse value from server", response.status_code);
logger_->error("[proxy:client] [put] failed to parse value from server: {}", err);
}
}
} else {
if (logger_)
logger_->e("[proxy:client] [status] failed with code=%i", response.status_code);
logger_->error("[proxy:client] [put] failed with code={:d}", response.status_code);
if (not response.aborted and response.status_code == 0)
opFailed();
}
@@ -550,7 +564,8 @@ DhtProxyClient::doPut(const InfoHash& key, Sp<Value> val, DoneCallbackSimple cb,
}
catch (const std::exception &e){
if (logger_)
logger_->e("[proxy:client] [put %s] error: %s", key.to_c_str(), e.what());
logger_->error("[proxy:client] [put {}] error: {}", key, e.what());
opFailed();
}
}
@@ -626,6 +641,7 @@ DhtProxyClient::getProxyInfos()
auto resolver = std::make_shared<http::Resolver>(httpContext_, proxyUrl_, logger_);
queryProxyInfo(infoState, resolver, AF_INET);
queryProxyInfo(infoState, resolver, AF_INET6);
std::lock_guard<std::mutex> l(resolverLock_);
resolver_ = resolver;
}
@@ -657,6 +673,12 @@ DhtProxyClient::queryProxyInfo(const Sp<InfoState>& infoState, const Sp<http::Re
if (!jsonReader_->parse(response.body.data(), response.body.data() + response.body.size(), &proxyInfos, &err)){
onProxyInfos(Json::Value{}, family);
} else if (not infoState->cancel) {
if (auto req = response.request.lock()) {
if (auto conn = req->get_connection()) {
const auto& localAddr = conn->local_address();
proxyInfos["local_ip"] = localAddr.to_string();
}
}
onProxyInfos(proxyInfos, family);
}
}
@@ -688,31 +710,57 @@ DhtProxyClient::onProxyInfos(const Json::Value& proxyInfos, const sa_family_t fa
std::unique_lock<std::mutex> l(lockCurrentProxyInfos_);
auto oldStatus = std::max(statusIpv4_, statusIpv6_);
auto& status = family == AF_INET ? statusIpv4_ : statusIpv6_;
auto ipChanged = false;
auto& pubAddress = family == AF_INET? publicAddressV4_ : publicAddressV6_;
auto& localAddress = family == AF_INET? localAddrv4_ : localAddrv6_;
if (not proxyInfos.isMember("node_id")) {
if (logger_)
logger_->e("[proxy:client] [info] request failed for %s", family == AF_INET ? "ipv4" : "ipv6");
status = NodeStatus::Disconnected;
if (pubAddress) {
pubAddress = {};
ipChanged = true;
}
} else {
if (logger_)
logger_->d("[proxy:client] [info] got proxy reply for %s",
logger_->debug("[proxy:client] [info] got proxy reply for {}",
family == AF_INET ? "ipv4" : "ipv6");
try {
myid = InfoHash(proxyInfos["node_id"].asString());
stats4_ = NodeStats(proxyInfos["ipv4"]);
stats6_ = NodeStats(proxyInfos["ipv6"]);
if (stats4_.good_nodes + stats6_.good_nodes)
auto publicIp = parsePublicAddress(proxyInfos["public_ip"]);
ipChanged = pubAddress && pubAddress.toString() != publicIp.toString();
bool pub = ipChanged || !pubAddress;
pubAddress = publicIp;
if (proxyInfos.isMember("local_ip")) {
std::string localIp = proxyInfos["local_ip"].asString();
if (localAddress.toString() != localIp) {
localAddress.setAddress(localIp.c_str());
ipChanged = (bool)localAddress;
}
}
if (pub && publicAddressChangedCb_) {
std::vector<SockAddr> addresses;
if (publicAddressV4_)
addresses.emplace_back(publicAddressV4_);
if (publicAddressV6_)
addresses.emplace_back(publicAddressV6_);
std::lock_guard<std::mutex> lock(lockCallbacks_);
callbacks_.emplace_back([cb=publicAddressChangedCb_, addresses = std::move(addresses)](){
cb(std::move(addresses));
});
}
if (!ipChanged && stats4_.good_nodes + stats6_.good_nodes)
status = NodeStatus::Connected;
else if (stats4_.dubious_nodes + stats6_.dubious_nodes)
else if (!ipChanged && stats4_.dubious_nodes + stats6_.dubious_nodes)
status = NodeStatus::Connecting;
else
status = NodeStatus::Disconnected;
auto publicIp = parsePublicAddress(proxyInfos["public_ip"]);
auto publicFamily = publicIp.getFamily();
if (publicFamily == AF_INET)
publicAddressV4_ = publicIp;
else if (publicFamily == AF_INET6)
publicAddressV6_ = publicIp;
} catch (const std::exception& e) {
if (logger_)
logger_->e("[proxy:client] [info] error processing: %s", e.what());
@@ -738,7 +786,10 @@ DhtProxyClient::onProxyInfos(const Json::Value& proxyInfos, const sa_family_t fa
nextProxyConfirmationTimer_->async_wait(std::bind(&DhtProxyClient::handleProxyConfirm, this, std::placeholders::_1));
}
else if (newStatus == NodeStatus::Disconnected) {
nextProxyConfirmationTimer_->expires_at(std::chrono::steady_clock::now() + std::chrono::minutes(1));
auto next = std::chrono::steady_clock::now();
if (!ipChanged)
next += std::chrono::minutes(1);
nextProxyConfirmationTimer_->expires_at(next);
nextProxyConfirmationTimer_->async_wait(std::bind(&DhtProxyClient::handleProxyConfirm, this, std::placeholders::_1));
}
l.unlock();
@@ -776,7 +827,7 @@ DhtProxyClient::listen(const InfoHash& key, ValueCallback cb, Value::Filter filt
std::lock_guard<std::mutex> lock(searchLock_);
auto& search = searches_[key];
auto query = std::make_shared<Query>(Select{}, std::move(where));
return search.ops.listen(cb, query, filter, [this, key](Sp<Query>, ValueCallback cb, SyncCallback) -> size_t {
return search.ops.listen(cb, query, std::move(filter), [this, key](Sp<Query>, ValueCallback cb, SyncCallback) -> size_t {
// Find search
auto search = searches_.find(key);
if (search == searches_.end()) {
@@ -838,7 +889,7 @@ DhtProxyClient::listen(const InfoHash& key, ValueCallback cb, Value::Filter filt
else {
method = ListenMethod::SUBSCRIBE;
header.method(restinio::http_method_subscribe());
header.request_target("/" + key.toString());
header.request_target("/key/" + key.toString());
}
sendListen(header, l->second.cb, opstate, l->second, method);
return token;
@@ -924,7 +975,7 @@ DhtProxyClient::handleExpireListener(const asio::error_code &ec, const InfoHash&
if (not deviceKey_.empty()) {
// UNSUBSCRIBE
auto request = buildRequest("/" + key.toString());
auto request = buildRequest("/key/" + key.toString());
auto reqid = request->id();
try {
request->set_method(restinio::http_method_unsubscribe());
@@ -959,7 +1010,10 @@ DhtProxyClient::handleExpireListener(const asio::error_code &ec, const InfoHash&
}
} else {
// stop the request
listener.request.reset();
if (listener.request) {
listener.request->cancel();
listener.request.reset();
}
}
search->second.listeners.erase(it);
if (logger_)
@@ -1054,6 +1108,12 @@ DhtProxyClient::sendListen(const restinio::http_request_header_t& header,
std::lock_guard<std::mutex> l(requestLock_);
requests_[reqid] = request;
}
request->add_on_status_callback([request, seconds = this->listenKeepIdle()] (unsigned status_code) {
if(status_code == 200) {
// increase TCP_KEEPIDLE to save power
request->get_connection()->set_keepalive(seconds);
}
});
request->send();
}
catch (const std::exception &e){
@@ -1105,7 +1165,7 @@ DhtProxyClient::restartListeners(const asio::error_code &ec)
std::lock_guard<std::mutex> lock(searchLock_);
for (auto& search : searches_) {
auto key = search.first;
const auto& key = search.first;
for (auto& put : search.second.puts) {
doPut(key, put.second.value, [ok = put.second.ok](bool result){
*ok = result;
@@ -1117,6 +1177,10 @@ DhtProxyClient::restartListeners(const asio::error_code &ec)
put.second.refreshPutTimer->async_wait(std::bind(&DhtProxyClient::handleRefreshPut, this,
std::placeholders::_1, key, put.first));
}
// Restart failed pending puts
for (auto& pput : search.second.pendingPuts) {
doPut(key, pput, {}, time_point::max(), true);
}
}
if (not deviceKey_.empty()) {
if (logger_)
@@ -1157,7 +1221,7 @@ DhtProxyClient::restartListeners(const asio::error_code &ec)
}
void
DhtProxyClient::pushNotificationReceived(const std::map<std::string, std::string>& notification)
DhtProxyClient::pushNotificationReceived([[maybe_unused]] const std::map<std::string, std::string>& notification)
{
#ifdef OPENDHT_PUSH_NOTIFICATIONS
{
@@ -1256,13 +1320,11 @@ DhtProxyClient::pushNotificationReceived(const std::map<std::string, std::string
}
if (launchLoop)
loopSignal_();
#else
(void) notification;
#endif
}
void
DhtProxyClient::resubscribe(const InfoHash& key, const size_t token, Listener& listener)
DhtProxyClient::resubscribe([[maybe_unused]] const InfoHash& key, [[maybe_unused]] const size_t token, [[maybe_unused]] Listener& listener)
{
#ifdef OPENDHT_PUSH_NOTIFICATIONS
if (deviceKey_.empty())
@@ -1286,9 +1348,6 @@ DhtProxyClient::resubscribe(const InfoHash& key, const size_t token, Listener& l
std::placeholders::_1, key, token, opstate));
auto vcb = listener.cb;
sendListen(header, vcb, opstate, listener, ListenMethod::RESUBSCRIBE);
#else
(void) key;
(void) listener;
#endif
}
@@ -1299,12 +1358,8 @@ DhtProxyClient::getPushRequest(Json::Value& body) const
body["key"] = deviceKey_;
body["client_id"] = pushClientId_;
body["session_id"] = pushSessionId_;
#ifdef __ANDROID__
body["platform"] = "android";
#endif
#ifdef __APPLE__
body["platform"] = "apple";
#endif
body["topic"] = notificationTopic_;
body["platform"] = platform_;
}
std::string
+240 -116
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author: Sébastien Blin <sebastien.blin@savoirfairelinux.com>
* Adrien Béraud <adrien.beraud@savoirfairelinux.com>
* Vsevolod Ivanov <vsevolod.ivanov@savoirfairelinux.com>
@@ -58,6 +58,19 @@ constexpr const std::chrono::minutes PRINT_STATS_PERIOD {2};
using ResponseByParts = restinio::chunked_output_t;
using ResponseByPartsBuilder = restinio::response_builder_t<ResponseByParts>;
namespace http {
/**
* Session value associated with a connection_id_t key.
*/
struct ListenerSession
{
ListenerSession() = default;
dht::InfoHash hash;
std::future<size_t> token;
std::shared_ptr<restinio::response_builder_t<restinio::chunked_output_t>> response;
};
}
class opendht_logger_t
{
public:
@@ -116,7 +129,7 @@ private:
void
DhtProxyServer::ConnectionListener::state_changed(const restinio::connection_state::notice_t& notice) noexcept
{
if (restinio::holds_alternative<restinio::connection_state::closed_t>(notice.cause())) {
if (std::holds_alternative<restinio::connection_state::closed_t>(notice.cause())) {
onClosed_(notice.connection_id());
}
}
@@ -173,6 +186,9 @@ DhtProxyServer::PermanentPut::msgpack_unpack(const msgpack::object& o)
if (auto val = findMapValue(o, "value"sv)) {
value = std::make_shared<dht::Value>(*val);
}
if (auto top = findMapValue(o, "top"sv)) {
topic = top->as<std::string>();
}
}
#ifdef OPENDHT_PUSH_NOTIFICATIONS
@@ -194,6 +210,9 @@ DhtProxyServer::Listener::msgpack_unpack(const msgpack::object& o)
if (auto t = findMapValue(o, "t"sv)) {
type = t->as<PushType>();
}
if (auto top = findMapValue(o, "top"sv)) {
topic = top->as<std::string>();
}
}
#endif
@@ -205,7 +224,8 @@ DhtProxyServer::DhtProxyServer(const std::shared_ptr<DhtRunner>& dht,
dht_(dht), persistPath_(config.persistStatePath), logger_(logger),
printStatsTimer_(std::make_unique<asio::steady_timer>(*ioContext_, 3s)),
connListener_(std::make_shared<ConnectionListener>(std::bind(&DhtProxyServer::onConnectionClosed, this, std::placeholders::_1))),
pushServer_(config.pushServer)
pushServer_(config.pushServer),
bundleId_(config.bundleId)
{
if (not dht_)
throw std::invalid_argument("A DHT instance must be provided");
@@ -264,6 +284,8 @@ DhtProxyServer::DhtProxyServer(const std::shared_ptr<DhtRunner>& dht,
// build http server
auto settings = restinio::run_on_this_thread_settings_t<RestRouterTraitsTls>();
addServerSettings(settings);
if (not config.address.empty())
settings.address(config.address);
settings.port(config.port);
settings.tls_context(std::move(tls_context));
httpsServer_ = std::make_unique<restinio::http_server_t<RestRouterTraitsTls>>(
@@ -281,6 +303,8 @@ DhtProxyServer::DhtProxyServer(const std::shared_ptr<DhtRunner>& dht,
else {
auto settings = restinio::run_on_this_thread_settings_t<RestRouterTraits>();
addServerSettings(settings);
if (not config.address.empty())
settings.address(config.address);
settings.port(config.port);
httpServer_ = std::make_unique<restinio::http_server_t<RestRouterTraits>>(
ioContext_,
@@ -372,7 +396,7 @@ DhtProxyServer::loadState(Is& is, size_t size) {
pput.second.expireNotifyTimer = std::make_unique<asio::steady_timer>(io_context(), pput.second.expiration - proxy::OP_MARGIN);
pput.second.expireNotifyTimer->async_wait(std::bind(
&DhtProxyServer::handleNotifyPushListenExpire, this,
std::placeholders::_1, pput.second.pushToken, std::move(jsonProvider), pput.second.type));
std::placeholders::_1, pput.second.pushToken, std::move(jsonProvider), pput.second.type, pput.second.topic));
}
#endif
dht_->put(put.first, pput.second.value, DoneCallbackSimple{}, time_point::max(), true);
@@ -391,31 +415,12 @@ DhtProxyServer::loadState(Is& is, size_t size) {
for (auto& pushListener : pushListeners_) {
for (auto& listeners : pushListener.second.listeners) {
for (auto& listener : listeners.second) {
// start listening
listener.internalToken = dht_->listen(listeners.first,
[this, infoHash=listeners.first, pushToken=pushListener.first, type=listener.type, clientId=listener.clientId, sessionCtx = listener.sessionCtx]
(const std::vector<std::shared_ptr<Value>>& values, bool expired) {
// Build message content
Json::Value json;
json["key"] = infoHash.toString();
json["to"] = clientId;
json["t"] = Json::Value::Int64(std::chrono::duration_cast<std::chrono::milliseconds>(system_clock::now().time_since_epoch()).count());
{
std::lock_guard<std::mutex> l(sessionCtx->lock);
json["s"] = sessionCtx->sessionId;
}
if (expired and values.size() < 2){
std::stringstream ss;
for(size_t i = 0; i < values.size(); ++i){
if(i != 0) ss << ",";
ss << values[i]->id;
}
json["exp"] = ss.str();
}
auto maxPrio = 1000u;
for (const auto& v : values)
maxPrio = std::min(maxPrio, v->priority);
sendPushNotification(pushToken, std::move(json), type, !expired and maxPrio == 0);
return true;
[this, infoHash=listeners.first, pushToken=pushListener.first, type=listener.type,
clientId=listener.clientId, sessionCtx = listener.sessionCtx, topic=listener.topic]
(const std::vector<Sp<Value>>& values, bool expired) {
return this->handlePushListen(infoHash, pushToken, type, clientId, sessionCtx, topic, values, expired);
}
);
// expire notify
@@ -429,7 +434,7 @@ DhtProxyServer::loadState(Is& is, size_t size) {
return json;
};
listener.expireNotifyTimer->async_wait(std::bind(&DhtProxyServer::handleNotifyPushListenExpire, this,
std::placeholders::_1, pushListener.first, std::move(jsonProvider), listener.type));
std::placeholders::_1, pushListener.first, std::move(jsonProvider), listener.type, listener.topic));
// cancel push listen
listener.expireTimer = std::make_unique<asio::steady_timer>(io_context(), listener.expiration);
listener.expireTimer->async_wait(std::bind(&DhtProxyServer::handleCancelPushListen, this,
@@ -620,6 +625,8 @@ DhtProxyServer::createRestRouter()
// key.listen
router->http_get("/key/:hash/listen", std::bind(&DhtProxyServer::listen, this, _1, _2));
#ifdef OPENDHT_PUSH_NOTIFICATIONS
// node.pingPush
router->http_post("/node/pingPush", std::bind(&DhtProxyServer::pingPush, this, _1, _2));
// key.subscribe
router->add_handler(restinio::http_method_subscribe(),
"/key/:hash", std::bind(&DhtProxyServer::subscribe, this, _1, _2));
@@ -691,7 +698,7 @@ DhtProxyServer::get(restinio::request_handle_t request,
initHttpResponse(request->create_response<ResponseByParts>()));
response->flush();
dht_->get(infoHash, [this, response](const std::vector<Sp<Value>>& values) {
std::stringstream output;
std::ostringstream output;
for (const auto& value : values) {
output << Json::writeString(jsonBuilder_, value->toJson()) << "\n";
}
@@ -745,6 +752,59 @@ DhtProxyServer::listen(restinio::request_handle_t request,
#ifdef OPENDHT_PUSH_NOTIFICATIONS
PushType
DhtProxyServer::getTypeFromString(const std::string& type) {
if (type == "android")
return PushType::Android;
else if (type == "ios")
return PushType::iOS;
else if (type == "unifiedpush")
return PushType::UnifiedPush;
return PushType::None;
}
std::string
DhtProxyServer::getDefaultTopic(PushType) {
return bundleId_;
}
RequestStatus
DhtProxyServer::pingPush(restinio::request_handle_t request,
restinio::router::route_params_t /*params*/)
{
requestNum_++;
try {
std::string err;
Json::Value r;
auto* char_data = reinterpret_cast<const char*>(request->body().data());
auto reader = std::unique_ptr<Json::CharReader>(jsonReaderBuilder_.newCharReader());
if (!reader->parse(char_data, char_data + request->body().size(), &r, &err)){
auto response = initHttpResponse(request->create_response(restinio::status_bad_request()));
response.set_body(RESP_MSG_JSON_INCORRECT);
return response.done();
}
const Json::Value& root(r); // parse using const Json so [] never creates element
auto pushToken = root["key"].asString();
if (pushToken.empty()){
auto response = initHttpResponse(request->create_response(restinio::status_bad_request()));
response.set_body(RESP_MSG_NO_TOKEN);
return response.done();
}
auto type = getTypeFromString(root["platform"].asString());
auto topic = root["topic"].asString();
if (topic.empty()) {
topic = getDefaultTopic(type);
}
Json::Value json;
json["to"] = root["client_id"];
json["pong"] = true;
sendPushNotification(pushToken, std::move(json), type, true, topic);
} catch (...) {
return serverError(*request);
}
return restinio::request_handling_status_t::accepted;
}
RequestStatus
DhtProxyServer::subscribe(restinio::request_handle_t request,
restinio::router::route_params_t params)
@@ -771,7 +831,11 @@ DhtProxyServer::subscribe(restinio::request_handle_t request,
response.set_body(RESP_MSG_NO_TOKEN);
return response.done();
}
auto type = root["platform"].asString() == "android" ? PushType::Android : PushType::iOS;
auto type = getTypeFromString(root["platform"].asString());
auto topic = root["topic"].asString();
if (topic.empty()) {
topic = getDefaultTopic(type);
}
auto clientId = root["client_id"].asString();
auto sessionId = root["session_id"].asString();
@@ -802,6 +866,7 @@ DhtProxyServer::subscribe(restinio::request_handle_t request,
auto timeout = std::chrono::steady_clock::now() + proxy::OP_TIMEOUT;
listener.expiration = timeout;
listener.type = type;
listener.topic = topic;
if (listener.expireNotifyTimer)
listener.expireNotifyTimer->expires_at(timeout - proxy::OP_MARGIN);
else
@@ -815,7 +880,7 @@ DhtProxyServer::subscribe(restinio::request_handle_t request,
return json;
};
listener.expireNotifyTimer->async_wait(std::bind(&DhtProxyServer::handleNotifyPushListenExpire, this,
std::placeholders::_1, pushToken, std::move(jsonProvider), listener.type));
std::placeholders::_1, pushToken, std::move(jsonProvider), listener.type, listener.topic));
if (!listener.expireTimer)
listener.expireTimer = std::make_unique<asio::steady_timer>(io_context(), timeout);
else
@@ -826,13 +891,13 @@ DhtProxyServer::subscribe(restinio::request_handle_t request,
// Send response
if (not newListener) {
if (logger_)
logger_->d("[proxy:server] [subscribe] found [client %s]", listener.clientId.c_str());
logger_->d("[proxy:server] [subscribe %s] found [client %s]", infoHash.toString().c_str(), listener.clientId.c_str());
// Send response header
auto response = std::make_shared<ResponseByPartsBuilder>(initHttpResponse(request->create_response<ResponseByParts>()));
response->flush();
if (!root["refresh"].asBool()) {
// No Refresh
dht_->get(infoHash, [this, response](const Sp<Value>& value){
dht_->get(infoHash, [this, response](const Sp<Value>& value) {
auto output = Json::writeString(jsonBuilder_, value->toJson()) + "\n";
response->append_chunk(output);
response->flush();
@@ -849,33 +914,15 @@ DhtProxyServer::subscribe(restinio::request_handle_t request,
} else {
// =========== No existing listener for an infoHash ============
// Add listen on dht
if (logger_)
logger_->d("[proxy:server] [subscribe %s] new", infoHash.toString().c_str());
listener.internalToken = dht_->listen(infoHash,
[this, infoHash, pushToken, type, clientId, sessionCtx = listener.sessionCtx]
(const std::vector<std::shared_ptr<Value>>& values, bool expired){
// Build message content
Json::Value json;
json["key"] = infoHash.toString();
json["to"] = clientId;
json["t"] = Json::Value::Int64(std::chrono::duration_cast<std::chrono::milliseconds>(system_clock::now().time_since_epoch()).count());
{
std::lock_guard<std::mutex> l(sessionCtx->lock);
json["s"] = sessionCtx->sessionId;
}
if (expired and values.size() < 2){
std::stringstream ss;
for(size_t i = 0; i < values.size(); ++i){
if(i != 0) ss << ",";
ss << values[i]->id;
}
json["exp"] = ss.str();
}
auto maxPrio = 1000u;
for (const auto& v : values)
maxPrio = std::min(maxPrio, v->priority);
sendPushNotification(pushToken, std::move(json), type, !expired and maxPrio == 0);
return true;
[this, infoHash, pushToken, type, clientId, sessionCtx = listener.sessionCtx, topic]
(const std::vector<Sp<Value>>& values, bool expired) {
return this->handlePushListen(infoHash, pushToken, type, clientId, sessionCtx, topic, values, expired);
}
);
// Send response header
auto response = initHttpResponse(request->create_response());
response.set_body("{}\n");
return response.done();
@@ -928,7 +975,7 @@ DhtProxyServer::unsubscribe(restinio::request_handle_t request,
void
DhtProxyServer::handleNotifyPushListenExpire(const asio::error_code &ec, const std::string pushToken,
std::function<Json::Value()> jsonProvider, PushType type)
std::function<Json::Value()> jsonProvider, PushType type, const std::string& topic)
{
if (ec == asio::error::operation_aborted)
return;
@@ -937,8 +984,8 @@ DhtProxyServer::handleNotifyPushListenExpire(const asio::error_code &ec, const s
logger_->e("[proxy:server] [subscribe] error sending put refresh: %s", ec.message().c_str());
}
if (logger_)
logger_->d("[proxy:server] [subscribe] sending put refresh to %s token", pushToken.c_str());
sendPushNotification(pushToken, jsonProvider(), type, false);
logger_->d("[proxy:server] [subscribe] sending refresh to %s token", pushToken.c_str());
sendPushNotification(pushToken, jsonProvider(), type, false, topic);
}
void
@@ -979,46 +1026,117 @@ DhtProxyServer::handleCancelPushListen(const asio::error_code &ec, const std::st
pushListeners_.erase(pushListener);
}
void
DhtProxyServer::sendPushNotification(const std::string& token, Json::Value&& json, PushType type, bool highPriority)
bool
DhtProxyServer::handlePushListen(const InfoHash& infoHash, const std::string& pushToken, PushType type, const std::string& clientId,
const std::shared_ptr<DhtProxyServer::PushSessionContext>& sessionCtx, const std::string& topic,
const std::vector<std::shared_ptr<Value>>& values, bool expired)
{
if (pushServer_.empty())
Json::Value json;
json["key"] = infoHash.toString();
json["to"] = clientId;
using namespace std::chrono;
json["t"] = Json::Value::Int64(duration_cast<milliseconds>(system_clock::now().time_since_epoch()).count());
{
std::lock_guard<std::mutex> l(sessionCtx->lock);
json["s"] = sessionCtx->sessionId;
}
// Build a comma-separated list of ids from the values. This is intended to be used when
// streaming from the iOS notification extension in order to filter out unwanted values.
std::string ids;
for(const auto& value : values) {
if (!ids.empty()) ids += ",";
ids += std::to_string(value->id);
}
json["ids"] = ids;
// If message is expired copy the value ID to the `exp` field.
// This is presumably used by the Android notification system.
if (expired && values.size() < 2) {
json["exp"] = json["ids"];
}
auto minPriority = 1000u;
for (const auto& v : values)
minPriority = std::min(minPriority, v->priority);
if (logger_)
logger_->d("[proxy:server] [listen %s] [client %s] [session %s] [expired %i] [priority %i] [values %zu]",
infoHash.toString().c_str(), clientId.c_str(), sessionCtx->sessionId.c_str(), expired, minPriority, values.size());
sendPushNotification(pushToken, std::move(json), type, !expired and minPriority == 0, topic);
return true;
}
void
DhtProxyServer::sendPushNotification(const std::string& token, Json::Value&& json, PushType type, bool highPriority, const std::string& topic)
{
if (pushServer_.empty() && type != PushType::UnifiedPush)
return;
unsigned reqid = 0;
try {
auto request = std::make_shared<http::Request>(io_context(), pushHostPort_.first, pushHostPort_.second,
httpsServer_ ? true : false, logger_);
std::shared_ptr<http::Request> request;
http::Url tokenUrl(token);
if (type == PushType::UnifiedPush)
request = std::make_shared<http::Request>(io_context(), tokenUrl.protocol + "://" + tokenUrl.host + (tokenUrl.service.empty() ? "" : (":" + tokenUrl.service)), logger_);
else
request = std::make_shared<http::Request>(io_context(), pushHostPort_.first, pushHostPort_.second, pushHostPort_.first.find("https://") == 0, logger_);;
reqid = request->id();
request->set_target("/api/push");
request->set_target(type == PushType::UnifiedPush ? (tokenUrl.target) : "/api/push");
request->set_method(restinio::http_method_post());
request->set_header_field(restinio::http_field_t::host, pushServer_.c_str());
request->set_header_field(restinio::http_field_t::host, type == PushType::UnifiedPush ? tokenUrl.host.c_str() : pushServer_.c_str());
request->set_header_field(restinio::http_field_t::user_agent, "RESTinio client");
request->set_header_field(restinio::http_field_t::accept, "*/*");
request->set_header_field(restinio::http_field_t::content_type, "application/json");
// NOTE: see https://github.com/appleboy/gorush
Json::Value notification(Json::objectValue);
Json::Value tokens(Json::arrayValue);
tokens[0] = token;
notification["tokens"] = std::move(tokens);
notification["platform"] = type == PushType::Android ? 2 : 1;
notification["data"] = std::move(json);
notification["priority"] = highPriority ? "high" : "normal";
if (type == PushType::Android)
notification["time_to_live"] = 3600 * 24; // 24 hours
else {
const auto expiration = std::chrono::system_clock::now() + std::chrono::hours(24);
uint32_t exp = std::chrono::duration_cast<std::chrono::seconds>(expiration.time_since_epoch()).count();
notification["expiration"] = exp;
if (type == PushType::UnifiedPush) {
Json::Value notification(Json::objectValue);
notification["message"] = Json::writeString(jsonBuilder_, std::move(json));
notification["topic"] = token;
notification["priority"] = highPriority ? 5 : 1;
request->set_body(Json::writeString(jsonBuilder_, std::move(json)));
} else {
// NOTE: see https://github.com/appleboy/gorush
auto isResubscribe = json.isMember("timeout");
Json::Value notification(Json::objectValue);
Json::Value tokens(Json::arrayValue);
tokens[0] = token;
notification["tokens"] = std::move(tokens);
notification["platform"] = type == PushType::Android ? 2 : 1;
notification["data"] = std::move(json);
notification["priority"] = highPriority ? "high" : "normal";
if (type == PushType::Android)
notification["time_to_live"] = 3600 * 24; // 24 hours
else {
const auto expiration = std::chrono::system_clock::now() + std::chrono::hours(24);
uint32_t exp = std::chrono::duration_cast<std::chrono::seconds>(expiration.time_since_epoch()).count();
notification["expiration"] = exp;
if (!topic.empty())
notification["topic"] = topic;
if (highPriority || isResubscribe) {
Json::Value alert(Json::objectValue);
alert["title"]="hello";
notification["push_type"] = "alert";
notification["alert"] = alert;
notification["mutable_content"] = true;
notification["priority"] = "high";
} else {
notification["push_type"] = "background";
notification["content_available"] = true;
}
}
Json::Value notifications(Json::arrayValue);
notifications[0] = notification;
Json::Value content;
content["notifications"] = std::move(notifications);
request->set_body(Json::writeString(jsonBuilder_, content));
}
Json::Value notifications(Json::arrayValue);
notifications[0] = notification;
Json::Value content;
content["notifications"] = std::move(notifications);
request->set_body(Json::writeString(jsonBuilder_, content));
request->add_on_state_change_callback([this, reqid]
(http::Request::State state, const http::Response& response){
if (state == http::Request::State::DONE){
@@ -1036,7 +1154,7 @@ DhtProxyServer::sendPushNotification(const std::string& token, Json::Value&& jso
}
catch (const std::exception &e){
if (logger_)
logger_->e("[proxy:server] [notification] error send push: %i", e.what());
logger_->e("[proxy:server] [notification] error send push: %s", e.what());
if (reqid) {
std::lock_guard<std::mutex> l(requestLock_);
requests_.erase(reqid);
@@ -1104,13 +1222,14 @@ DhtProxyServer::put(restinio::request_handle_t request,
logger_->d("[proxy:server] [put %s] %s %s", infoHash.toString().c_str(),
value->toString().c_str(), (permanent ? "permanent" : ""));
if (permanent) {
std::string pushToken, clientId, sessionId, platform;
std::string pushToken, clientId, sessionId, platform, topic;
auto& pVal = root["permanent"];
if (pVal.isObject()){
pushToken = pVal["key"].asString();
clientId = pVal["client_id"].asString();
platform = pVal["platform"].asString();
sessionId = pVal["session_id"].asString();
topic = pVal["topic"].asString();
}
std::lock_guard<std::mutex> lock(lockSearchPuts_);
auto timeout = std::chrono::steady_clock::now() + proxy::OP_TIMEOUT;
@@ -1133,7 +1252,7 @@ DhtProxyServer::put(restinio::request_handle_t request,
}
}
auto response = initHttpResponse(request->create_response());
response.append_body(Json::writeString(jsonBuilder_, value->toJson()) + "\n");
response.append_body(Json::writeString(jsonBuilder_, pp.second.value->toJson()) + "\n");
return response.done();
}
}
@@ -1145,34 +1264,17 @@ DhtProxyServer::put(restinio::request_handle_t request,
pput.value = value;
pput.expiration = timeout;
if (not pput.expireTimer) {
auto &ctx = io_context();
// cancel permanent put
pput.expireTimer = std::make_unique<asio::steady_timer>(ctx, timeout);
pput.expireTimer = std::make_unique<asio::steady_timer>(io_context(), timeout);
#ifdef OPENDHT_PUSH_NOTIFICATIONS
if (not pushToken.empty()){
bool isAndroid = platform == "android";
if (not pushToken.empty()) {
pput.pushToken = pushToken;
pput.clientId = clientId;
pput.type = isAndroid ? PushType::Android : PushType::iOS;
pput.type = getTypeFromString(platform);
if (topic.empty())
topic = getDefaultTopic(pput.type);
pput.topic = topic;
pput.sessionCtx = std::make_shared<PushSessionContext>(sessionId);
// notify push listen expire
auto jsonProvider = [infoHash, clientId, vid, sessionCtx = pput.sessionCtx](){
Json::Value json;
json["timeout"] = infoHash.toString();
json["to"] = clientId;
json["vid"] = std::to_string(vid);
std::lock_guard<std::mutex> l(sessionCtx->lock);
json["s"] = sessionCtx->sessionId;
return json;
};
if (!pput.expireNotifyTimer)
pput.expireNotifyTimer = std::make_unique<asio::steady_timer>(ctx,
timeout - proxy::OP_MARGIN);
else
pput.expireNotifyTimer->expires_at(timeout - proxy::OP_MARGIN);
pput.expireNotifyTimer->async_wait(std::bind(
&DhtProxyServer::handleNotifyPushListenExpire, this,
std::placeholders::_1, pushToken, std::move(jsonProvider), pput.type));
}
#endif
} else {
@@ -1185,12 +1287,34 @@ DhtProxyServer::put(restinio::request_handle_t request,
}
}
pput.expireTimer->expires_at(timeout);
if (pput.expireNotifyTimer)
pput.expireNotifyTimer->expires_at(timeout - proxy::OP_MARGIN);
}
pput.expireTimer->async_wait(std::bind(&DhtProxyServer::handleCancelPermamentPut, this,
std::placeholders::_1, infoHash, vid));
#ifdef OPENDHT_PUSH_NOTIFICATIONS
// notify put permanent expiration
if (pput.sessionCtx) {
auto jsonProvider = [infoHash, clientId, vid, sessionCtx = pput.sessionCtx](){
Json::Value json;
json["timeout"] = infoHash.toString();
json["to"] = clientId;
json["vid"] = std::to_string(vid);
std::lock_guard<std::mutex> l(sessionCtx->lock);
json["s"] = sessionCtx->sessionId;
return json;
};
if (!pput.expireNotifyTimer)
pput.expireNotifyTimer = std::make_unique<asio::steady_timer>(io_context(),
timeout - proxy::OP_MARGIN);
else
pput.expireNotifyTimer->expires_at(timeout - proxy::OP_MARGIN);
pput.expireNotifyTimer->async_wait(std::bind(
&DhtProxyServer::handleNotifyPushListenExpire, this,
std::placeholders::_1, pushToken, std::move(jsonProvider), pput.type, pput.topic));
}
#endif
}
dht_->put(infoHash, value, [this, request, value](bool ok){
if (ok){
auto response = initHttpResponse(request->create_response());
+167 -266
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Authors: Adrien Béraud <adrien.beraud@savoirfairelinux.com>
* Simon Désaulniers <simon.desaulniers@savoirfairelinux.com>
* Sébastien Blin <sebastien.blin@savoirfairelinux.com>
@@ -38,15 +38,6 @@ namespace dht {
static const std::string PEER_DISCOVERY_DHT_SERVICE = "dht";
struct DhtRunner::Listener {
size_t tokenClassicDht {0};
size_t tokenProxyDht {0};
ValueCallback gcb;
InfoHash hash {};
Value::Filter f;
Where w;
};
struct NodeInsertionPack {
dht::InfoHash nodeId;
in_port_t port;
@@ -55,9 +46,6 @@ struct NodeInsertionPack {
};
DhtRunner::DhtRunner() : dht_()
#ifdef OPENDHT_PROXY_CLIENT
, dht_via_proxy_()
#endif //OPENDHT_PROXY_CLIENT
{
#ifdef _WIN32
WSADATA wsd;
@@ -105,7 +93,7 @@ DhtRunner::run(const Config& config, Context&& context)
auto expected = State::Idle;
if (not running.compare_exchange_strong(expected, State::Running)) {
if (context.logger)
context.logger->w("[runner %p] Node is already running. Call join() first before calling run() again.", this);
context.logger->w("[runner %p] Node is already running. Call join() first before calling run() again.", fmt::ptr(this));
return;
}
@@ -114,27 +102,28 @@ DhtRunner::run(const Config& config, Context&& context)
auto local6 = config.bind6;
if (not local4 and not local6) {
if (context.logger)
context.logger->w("[runner %p] No address to bind specified in the configuration, using default addresses", this);
context.logger->w("[runner %p] No address to bind specified in the configuration, using default addresses", fmt::ptr(this));
local4.setFamily(AF_INET);
local6.setFamily(AF_INET6);
}
auto state_path = config.dht_config.node_config.persist_path;
if (not state_path.empty() && (local4.getPort() == 0 || local6.getPort() == 0)) {
if (not state_path.empty())
state_path += "_port.txt";
if (not state_path.empty() && (local4.getPort() == 0 || local6.getPort() == 0)) {
std::ifstream inConfig(state_path);
if (inConfig.is_open()) {
in_port_t port;
if (inConfig >> port) {
if (local4.getPort() == 0) {
if (context.logger)
context.logger->d("[runner %p] Using IPv4 port %hu from saved configuration", this, port);
context.logger->d("[runner %p] Using IPv4 port %hu from saved configuration", fmt::ptr(this), port);
local4.setPort(port);
}
}
if (inConfig >> port) {
if (local6.getPort() == 0) {
if (context.logger)
context.logger->d("[runner %p] Using IPv6 port %hu from saved configuration", this, port);
context.logger->d("[runner %p] Using IPv6 port %hu from saved configuration", fmt::ptr(this), port);
local6.setPort(port);
}
}
@@ -143,68 +132,62 @@ DhtRunner::run(const Config& config, Context&& context)
if (context.logger) {
logger_ = context.logger;
logger_->d("[runner %p] state changed to Running", fmt::ptr(this));
}
if (not context.sock) {
context.sock.reset(new net::UdpSocket(local4, local6, context.logger));
}
if (not state_path.empty()) {
std::ofstream outConfig(state_path);
outConfig << context.sock->getBoundRef(AF_INET).getPort() << std::endl;
outConfig << context.sock->getBoundRef(AF_INET6).getPort() << std::endl;
}
if (context.logger) {
logger_->d("[runner %p] state changed to Running", this);
}
context.sock->setOnReceive([&] (net::PacketList&& pkts) {
net::PacketList ret;
{
std::lock_guard<std::mutex> lck(sock_mtx);
auto maxSize = net::RX_QUEUE_MAX_SIZE - pkts.size();
while (rcv.size() > maxSize) {
if (logger_)
logger_->e("Dropping packet: queue is full!");
rcv.pop_front();
}
rcv.splice(rcv.end(), std::move(pkts));
ret = std::move(rcv_free);
}
cv.notify_all();
return ret;
});
#ifdef OPENDHT_PROXY_CLIENT
config_ = config;
identityAnnouncedCb_ = context.identityAnnouncedCb;
#endif
auto dht = std::make_unique<Dht>(std::move(context.sock), SecureDht::getConfig(config.dht_config), context.logger);
dht_ = std::make_unique<SecureDht>(std::move(dht), config.dht_config, std::move(context.identityAnnouncedCb), context.logger);
enableProxy(not config.proxy_server.empty());
if (config.proxy_server.empty()) {
if (not context.sock) {
context.sock.reset(new net::UdpSocket(local4, local6, context.logger));
}
context.sock->setOnReceive([&] (net::PacketList&& pkts) {
net::PacketList ret;
{
std::lock_guard<std::mutex> lck(sock_mtx);
rcv.splice(rcv.end(), std::move(pkts));
size_t dropped = 0;
while (rcv.size() > net::RX_QUEUE_MAX_SIZE) {
rcv.pop_front();
dropped++;
}
if (dropped and logger_) {
logger_->w("[runner %p] dropped %zu packets: queue is full!", fmt::ptr(this), dropped);
}
ret = std::move(rcv_free);
}
cv.notify_all();
return ret;
});
if (not state_path.empty()) {
std::ofstream outConfig(state_path);
outConfig << context.sock->getBoundRef(AF_INET).getPort() << std::endl;
outConfig << context.sock->getBoundRef(AF_INET6).getPort() << std::endl;
}
auto dht = std::make_unique<Dht>(std::move(context.sock), SecureDht::getConfig(config.dht_config), context.logger, std::move(context.rng));
dht_ = std::make_unique<SecureDht>(std::move(dht), config.dht_config, std::move(context.identityAnnouncedCb), context.logger);
} else {
enableProxy(true);
}
} catch(const std::exception& e) {
config_ = {};
identityAnnouncedCb_ = {};
dht_.reset();
#ifdef OPENDHT_PROXY_CLIENT
dht_via_proxy_.reset();
#endif
running = State::Idle;
throw;
}
if (context.logger and dht_via_proxy_) {
dht_via_proxy_->setLogger(context.logger);
}
if (context.statusChangedCallback) {
statusCb = std::move(context.statusChangedCallback);
}
if (context.certificateStore) {
dht_->setLocalCertificateStore(std::move(context.certificateStore));
if (dht_via_proxy_)
dht_via_proxy_->setLocalCertificateStore(std::move(context.certificateStore));
}
if (context.publicAddressChangedCb) {
dht_->setOnPublicAddressChanged(std::move(context.publicAddressChangedCb));
}
if (not config.threaded)
@@ -227,7 +210,7 @@ DhtRunner::run(const Config& config, Context&& context)
if (not pending_ops_prio.empty())
return true;
auto s = getStatus();
if (not pending_ops.empty() and (s == NodeStatus::Connected or s == NodeStatus::Disconnected))
if (not pending_ops.empty() and (s == NodeStatus::Connected or s == NodeStatus::Disconnected or running == State::Stopping))
return true;
}
return false;
@@ -239,46 +222,50 @@ DhtRunner::run(const Config& config, Context&& context)
}
});
if (config.peer_discovery or config.peer_publish) {
if (config.proxy_server.empty()) {
if (config.peer_discovery or config.peer_publish) {
#ifdef OPENDHT_PEER_DISCOVERY
peerDiscovery_ = context.peerDiscovery ?
std::move(context.peerDiscovery) :
std::make_shared<PeerDiscovery>();
peerDiscovery_ = context.peerDiscovery ?
std::move(context.peerDiscovery) :
std::make_shared<PeerDiscovery>();
#else
std::cerr << "Peer discovery requested but OpenDHT built without peer discovery support." << std::endl;
std::cerr << "Peer discovery requested but OpenDHT built without peer discovery support." << std::endl;
#endif
}
}
#ifdef OPENDHT_PEER_DISCOVERY
auto netId = config.dht_config.node_config.network;
if (config.peer_discovery) {
peerDiscovery_->startDiscovery<NodeInsertionPack>(PEER_DISCOVERY_DHT_SERVICE, [this, netId](NodeInsertionPack&& v, SockAddr&& addr){
addr.setPort(v.port);
if (v.nodeId != dht_->getNodeId() && netId == v.net){
bootstrap(v.nodeId, addr);
auto netId = config.dht_config.node_config.network;
if (config.peer_discovery) {
peerDiscovery_->startDiscovery<NodeInsertionPack>(PEER_DISCOVERY_DHT_SERVICE, [this, netId](NodeInsertionPack&& v, SockAddr&& addr){
addr.setPort(v.port);
if (v.nodeId != dht_->getNodeId() && netId == v.net){
bootstrap(v.nodeId, addr);
}
});
}
if (config.peer_publish) {
msgpack::sbuffer sbuf_node;
NodeInsertionPack adc;
adc.net = netId;
adc.nodeId = dht_->getNodeId();
if (auto socket = dht_->getSocket()) {
// IPv4
if (const auto& bound4 = socket->getBoundRef(AF_INET)) {
adc.port = bound4.getPort();
msgpack::pack(sbuf_node, adc);
peerDiscovery_->startPublish(AF_INET, PEER_DISCOVERY_DHT_SERVICE, sbuf_node);
}
// IPv6
if (const auto& bound6 = socket->getBoundRef(AF_INET6)) {
adc.port = bound6.getPort();
sbuf_node.clear();
msgpack::pack(sbuf_node, adc);
peerDiscovery_->startPublish(AF_INET6, PEER_DISCOVERY_DHT_SERVICE, sbuf_node);
}
}
});
}
if (config.peer_publish) {
msgpack::sbuffer sbuf_node;
NodeInsertionPack adc;
adc.net = netId;
adc.nodeId = dht_->getNodeId();
// IPv4
if (const auto& bound4 = dht_->getSocket()->getBoundRef(AF_INET)) {
adc.port = bound4.getPort();
msgpack::pack(sbuf_node, adc);
peerDiscovery_->startPublish(AF_INET, PEER_DISCOVERY_DHT_SERVICE, sbuf_node);
}
// IPv6
if (const auto& bound6 = dht_->getSocket()->getBoundRef(AF_INET6)) {
adc.port = bound6.getPort();
sbuf_node.clear();
msgpack::pack(sbuf_node, adc);
peerDiscovery_->startPublish(AF_INET6, PEER_DISCOVERY_DHT_SERVICE, sbuf_node);
}
}
#endif
}
}
void
@@ -287,7 +274,8 @@ DhtRunner::shutdown(ShutdownCallback cb, bool stop) {
auto expected = State::Running;
if (not running.compare_exchange_strong(expected, State::Stopping)) {
if (expected == State::Stopping and ongoing_ops) {
shutdownCallbacks_.emplace_back(std::move(cb));
if (cb)
shutdownCallbacks_.emplace_back(std::move(cb));
}
else if (cb) {
lck.unlock();
@@ -296,17 +284,15 @@ DhtRunner::shutdown(ShutdownCallback cb, bool stop) {
return;
}
if (logger_)
logger_->d("[runner %p] state changed to Stopping, %zu ongoing ops", this, ongoing_ops.load());
logger_->debug("[runner {:p}] state changed to Stopping, {:d} ongoing ops", fmt::ptr(this), ongoing_ops.load());
ongoing_ops++;
shutdownCallbacks_.emplace_back(std::move(cb));
pending_ops.emplace([=](SecureDht&) mutable {
auto onShutdown = [this]{ opEnded(); };
#ifdef OPENDHT_PROXY_CLIENT
if (dht_via_proxy_)
dht_via_proxy_->shutdown(onShutdown, stop);
#endif
if (dht_)
dht_->shutdown(onShutdown, stop);
else
opEnded();
});
cv.notify_all();
}
@@ -363,7 +349,7 @@ DhtRunner::join()
if (auto sock = dht_->getSocket())
sock->stop();
if (logger_)
logger_->d("[runner %p] state changed to Idle", this);
logger_->debug("[runner {:p}] state changed to Idle", fmt::ptr(this));
}
if (dht_thread.joinable())
@@ -372,7 +358,7 @@ DhtRunner::join()
{
std::lock_guard<std::mutex> lck(storage_mtx);
if (ongoing_ops and logger_) {
logger_->w("[runner %p] stopping with %zu remaining ops", this, ongoing_ops.load());
logger_->warn("[runner {:p}] stopping with {:d} remaining ops", fmt::ptr(this), ongoing_ops.load());
}
pending_ops = decltype(pending_ops)();
pending_ops_prio = decltype(pending_ops_prio)();
@@ -409,31 +395,25 @@ void
DhtRunner::dumpTables() const
{
std::lock_guard<std::mutex> lck(dht_mtx);
activeDht()->dumpTables();
dht_->dumpTables();
}
InfoHash
DhtRunner::getId() const
{
if (auto dht = activeDht())
return dht->getId();
return {};
return dht_ ? dht_->getId() : InfoHash{};
}
std::shared_ptr<crypto::PublicKey>
DhtRunner::getPublicKey() const
{
if (auto dht = activeDht())
return dht->getPublicKey();
return {};
return dht_ ? dht_->getPublicKey() : std::shared_ptr<crypto::PublicKey>{};
}
InfoHash
DhtRunner::getNodeId() const
{
if (auto dht = activeDht())
return dht->getNodeId();
return {};
return dht_ ? dht_->getNodeId() : InfoHash{};
}
@@ -475,16 +455,6 @@ DhtRunner::setLogger(const Sp<Logger>& logger) {
logger_ = logger;
if (dht_)
dht_->setLogger(logger);
#ifdef OPENDHT_PROXY_CLIENT
if (dht_via_proxy_)
dht_via_proxy_->setLogger(logger);
#endif
}
void
DhtRunner::setLoggers(LogMethod error, LogMethod warn, LogMethod debug) {
Logger logger {std::move(error), std::move(warn), std::move(debug)};
setLogger(logger);
}
void
@@ -492,16 +462,12 @@ DhtRunner::setLogFilter(const InfoHash& f) {
std::lock_guard<std::mutex> lck(dht_mtx);
if (dht_)
dht_->setLogFilter(f);
#ifdef OPENDHT_PROXY_CLIENT
if (dht_via_proxy_)
dht_via_proxy_->setLogFilter(f);
#endif
}
void
DhtRunner::registerType(const ValueType& type) {
std::lock_guard<std::mutex> lck(dht_mtx);
activeDht()->registerType(type);
dht_->registerType(type);
}
void
@@ -514,7 +480,7 @@ unsigned
DhtRunner::getNodesStats(sa_family_t af, unsigned *good_return, unsigned *dubious_return, unsigned *cached_return, unsigned *incoming_return) const
{
std::lock_guard<std::mutex> lck(dht_mtx);
const auto stats = activeDht()->getNodesStats(af);
const auto stats = dht_->getNodesStats(af);
if (good_return)
*good_return = stats.good_nodes;
if (dubious_return)
@@ -530,19 +496,19 @@ NodeStats
DhtRunner::getNodesStats(sa_family_t af) const
{
std::lock_guard<std::mutex> lck(dht_mtx);
return activeDht()->getNodesStats(af);
return dht_->getNodesStats(af);
}
NodeInfo
DhtRunner::getNodeInfo() const {
std::lock_guard<std::mutex> lck(dht_mtx);
NodeInfo info {};
if (auto dht = activeDht()) {
info.id = dht->getId();
info.node_id = dht->getNodeId();
info.ipv4 = dht->getNodesStats(AF_INET);
info.ipv6 = dht->getNodesStats(AF_INET6);
if (auto sock = dht->getSocket()) {
if (dht_) {
info.id = dht_->getId();
info.node_id = dht_->getNodeId();
info.ipv4 = dht_->getNodesStats(AF_INET);
info.ipv6 = dht_->getNodesStats(AF_INET6);
if (auto sock = dht_->getSocket()) {
info.bound4 = sock->getBoundRef(AF_INET).getPort();
info.bound6 = sock->getBoundRef(AF_INET6).getPort();
}
@@ -563,6 +529,7 @@ DhtRunner::getNodeInfo(std::function<void(std::shared_ptr<NodeInfo>)> cb)
info.node_id = dht.getNodeId();
info.ipv4 = dht.getNodesStats(AF_INET);
info.ipv6 = dht.getNodesStats(AF_INET6);
std::tie(info.storage_size, info.storage_values) = dht.getStoreSize();
if (auto sock = dht.getSocket()) {
info.bound4 = sock->getBoundRef(AF_INET).getPort();
info.bound6 = sock->getBoundRef(AF_INET6).getPort();
@@ -578,49 +545,49 @@ std::vector<unsigned>
DhtRunner::getNodeMessageStats(bool in) const
{
std::lock_guard<std::mutex> lck(dht_mtx);
return activeDht()->getNodeMessageStats(in);
return dht_->getNodeMessageStats(in);
}
std::string
DhtRunner::getStorageLog() const
{
std::lock_guard<std::mutex> lck(dht_mtx);
return activeDht()->getStorageLog();
return dht_->getStorageLog();
}
std::string
DhtRunner::getStorageLog(const InfoHash& f) const
{
std::lock_guard<std::mutex> lck(dht_mtx);
return activeDht()->getStorageLog(f);
return dht_->getStorageLog(f);
}
std::string
DhtRunner::getRoutingTablesLog(sa_family_t af) const
{
std::lock_guard<std::mutex> lck(dht_mtx);
return activeDht()->getRoutingTablesLog(af);
return dht_->getRoutingTablesLog(af);
}
std::string
DhtRunner::getSearchesLog(sa_family_t af) const
{
std::lock_guard<std::mutex> lck(dht_mtx);
return activeDht()->getSearchesLog(af);
return dht_->getSearchesLog(af);
}
std::string
DhtRunner::getSearchLog(const InfoHash& f, sa_family_t af) const
{
std::lock_guard<std::mutex> lck(dht_mtx);
return activeDht()->getSearchLog(f, af);
return dht_->getSearchLog(f, af);
}
std::vector<SockAddr>
DhtRunner::getPublicAddress(sa_family_t af)
DhtRunner::getPublicAddress(sa_family_t af) const
{
std::lock_guard<std::mutex> lck(dht_mtx);
if (auto dht = activeDht())
return dht->getPublicAddress(af);
if (dht_)
return dht_->getPublicAddress(af);
return {};
}
std::vector<std::string>
DhtRunner::getPublicAddressStr(sa_family_t af)
DhtRunner::getPublicAddressStr(sa_family_t af) const
{
auto addrs = getPublicAddress(af);
std::vector<std::string> ret(addrs.size());
@@ -643,15 +610,12 @@ DhtRunner::getPublicAddress(std::function<void(std::vector<SockAddr>&&)> cb, sa_
void
DhtRunner::registerCertificate(std::shared_ptr<crypto::Certificate> cert) {
std::lock_guard<std::mutex> lck(dht_mtx);
activeDht()->registerCertificate(cert);
dht_->registerCertificate(cert);
}
void
DhtRunner::setLocalCertificateStore(CertificateStoreQuery&& query_method) {
std::lock_guard<std::mutex> lck(dht_mtx);
#ifdef OPENDHT_PROXY_CLIENT
if (dht_via_proxy_)
dht_via_proxy_->setLocalCertificateStore(std::forward<CertificateStoreQuery>(query_method));
#endif
if (dht_)
dht_->setLocalCertificateStore(std::forward<CertificateStoreQuery>(query_method));
}
@@ -659,19 +623,18 @@ DhtRunner::setLocalCertificateStore(CertificateStoreQuery&& query_method) {
time_point
DhtRunner::loop_()
{
auto dht = activeDht();
if (not dht)
if (not dht_)
return {};
decltype(pending_ops) ops {};
{
std::lock_guard<std::mutex> lck(storage_mtx);
auto s = getStatus();
ops = (pending_ops_prio.empty() && (s == NodeStatus::Connected or s == NodeStatus::Disconnected)) ?
ops = (pending_ops_prio.empty() && (s == NodeStatus::Connected or s == NodeStatus::Disconnected or running == State::Stopping)) ?
std::move(pending_ops) : std::move(pending_ops_prio);
}
while (not ops.empty()) {
ops.front()(*dht);
ops.front()(*dht_);
ops.pop();
}
@@ -704,13 +667,13 @@ DhtRunner::loop_()
if (now - pkt.received > net::RX_QUEUE_MAX_DELAY)
dropped++;
else
wakeup = dht->periodic(pkt.data.data(), pkt.data.size(), std::move(pkt.from), now);
wakeup = dht_->periodic(pkt.data.data(), pkt.data.size(), std::move(pkt.from), now);
pkt.data.clear();
}
received_treated.splice(received_treated.end(), std::move(received));
} else {
// Or just run the scheduler
wakeup = dht->periodic(nullptr, 0, nullptr, 0, clock::now());
wakeup = dht_->periodic(nullptr, 0, nullptr, 0, clock::now());
}
if (not received_treated.empty()) {
@@ -720,10 +683,10 @@ DhtRunner::loop_()
}
if (dropped && logger_)
logger_->e("[runner %p] Dropped %zu packets with high delay.", this, dropped);
logger_->e("[runner %p] Dropped %zu packets with high delay.", fmt::ptr(this), dropped);
NodeStatus nstatus4 = dht->updateStatus(AF_INET);
NodeStatus nstatus6 = dht->updateStatus(AF_INET6);
NodeStatus nstatus4 = dht_->updateStatus(AF_INET);
NodeStatus nstatus6 = dht_->updateStatus(AF_INET6);
if (nstatus4 != status4 || nstatus6 != status6) {
status4 = nstatus4;
status6 = nstatus6;
@@ -781,27 +744,7 @@ DhtRunner::listen(InfoHash hash, ValueCallback vcb, Value::Filter f, Where w)
return ret_token->get_future();
}
pending_ops.emplace([=](SecureDht& dht) mutable {
#ifdef OPENDHT_PROXY_CLIENT
auto tokenbGlobal = listener_token_++;
auto& listener = listeners_[tokenbGlobal];
listener.hash = hash;
listener.f = std::move(f);
listener.w = std::move(w);
listener.gcb = [hash,vcb,tokenbGlobal,this](const std::vector<Sp<Value>>& vals, bool expired) {
if (not vcb(vals, expired)) {
cancelListen(hash, tokenbGlobal);
return false;
}
return true;
};
if (auto token = dht.listen(hash, listener.gcb, listener.f, listener.w)) {
if (use_proxy) listener.tokenProxyDht = token;
else listener.tokenClassicDht = token;
}
ret_token->set_value(tokenbGlobal);
#else
ret_token->set_value(dht.listen(hash, std::move(vcb), std::move(f), std::move(w)));
#endif
});
cv.notify_all();
return ret_token->get_future();
@@ -820,23 +763,10 @@ DhtRunner::cancelListen(InfoHash h, size_t token)
if (running != State::Running)
return;
ongoing_ops++;
#ifdef OPENDHT_PROXY_CLIENT
pending_ops.emplace([=](SecureDht&) {
auto it = listeners_.find(token);
if (it == listeners_.end()) return;
if (it->second.tokenClassicDht)
dht_->cancelListen(h, it->second.tokenClassicDht);
if (it->second.tokenProxyDht and dht_via_proxy_)
dht_via_proxy_->cancelListen(h, it->second.tokenProxyDht);
listeners_.erase(it);
opEnded();
});
#else
pending_ops.emplace([=](SecureDht& dht) {
dht.cancelListen(h, token);
opEnded();
});
#endif // OPENDHT_PROXY_CLIENT
cv.notify_all();
}
@@ -847,23 +777,10 @@ DhtRunner::cancelListen(InfoHash h, std::shared_future<size_t> ftoken)
if (running != State::Running)
return;
ongoing_ops++;
#ifdef OPENDHT_PROXY_CLIENT
pending_ops.emplace([this, h, ftoken = std::move(ftoken)](SecureDht&) {
auto it = listeners_.find(ftoken.get());
if (it == listeners_.end()) return;
if (it->second.tokenClassicDht)
dht_->cancelListen(h, it->second.tokenClassicDht);
if (it->second.tokenProxyDht and dht_via_proxy_)
dht_via_proxy_->cancelListen(h, it->second.tokenProxyDht);
listeners_.erase(it);
opEnded();
});
#else
pending_ops.emplace([this, h, ftoken = std::move(ftoken)](SecureDht& dht) {
dht.cancelListen(h, ftoken.get());
opEnded();
});
#endif // OPENDHT_PROXY_CLIENT
cv.notify_all();
}
@@ -1152,23 +1069,9 @@ void
DhtRunner::resetDht()
{
peerDiscovery_.reset();
#ifdef OPENDHT_PROXY_CLIENT
listeners_.clear();
dht_via_proxy_.reset();
#endif // OPENDHT_PROXY_CLIENT
dht_.reset();
}
SecureDht*
DhtRunner::activeDht() const
{
#ifdef OPENDHT_PROXY_CLIENT
return use_proxy? dht_via_proxy_.get() : dht_.get();
#else
return dht_.get();
#endif // OPENDHT_PROXY_CLIENT
}
void
DhtRunner::setProxyServer(const std::string& proxy, const std::string& pushNodeId)
{
@@ -1181,7 +1084,8 @@ DhtRunner::setProxyServer(const std::string& proxy, const std::string& pushNodeI
enableProxy(use_proxy and not config_.proxy_server.empty());
#else
if (not proxy.empty())
std::cerr << "DHT proxy requested but OpenDHT built without proxy support." << std::endl;
throw std::runtime_error("DHT proxy requested but OpenDHT built without proxy support.");
(void) pushNodeId;
#endif
}
@@ -1189,54 +1093,37 @@ void
DhtRunner::enableProxy(bool proxify)
{
#ifdef OPENDHT_PROXY_CLIENT
if (dht_via_proxy_) {
dht_via_proxy_->shutdown({});
if (dht_) {
dht_->shutdown({});
}
if (proxify) {
// Init the proxy client
auto dht_via_proxy = std::unique_ptr<DhtInterface>(
new DhtProxyClient(
auto dht_via_proxy = std::make_unique<DhtProxyClient>(
config_.server_ca,
config_.client_identity,
[this]{
if (config_.threaded) {
{
std::lock_guard<std::mutex> lck(storage_mtx);
pending_ops_prio.emplace([=](SecureDht&) mutable {});
}
std::lock_guard<std::mutex> lck(storage_mtx);
pending_ops_prio.emplace([=](SecureDht&) mutable {});
cv.notify_all();
}
},
config_.proxy_server, config_.push_node_id, logger_)
);
#ifdef OPENDHT_PUSH_NOTIFICATIONS
config_.proxy_server, config_.push_node_id, logger_);
if (not config_.push_token.empty())
dht_via_proxy->setPushNotificationToken(config_.push_token);
#endif
dht_via_proxy_ = std::unique_ptr<SecureDht>(new SecureDht(std::move(dht_via_proxy), config_.dht_config, identityAnnouncedCb_, logger_));
// add current listeners
for (auto& l: listeners_)
l.second.tokenProxyDht = dht_via_proxy_->listen(l.second.hash, l.second.gcb, l.second.f, l.second.w);
if (not config_.push_topic.empty())
dht_via_proxy->setPushNotificationTopic(config_.push_topic);
if (not config_.push_platform.empty())
dht_via_proxy->setPushNotificationPlatform(config_.push_platform);
dht_ = std::make_unique<SecureDht>(std::move(dht_via_proxy), config_.dht_config, identityAnnouncedCb_, logger_);
// and use it
use_proxy = proxify;
} else {
use_proxy = proxify;
std::lock_guard<std::mutex> lck(storage_mtx);
if (not listeners_.empty()) {
pending_ops.emplace([this](SecureDht& /*dht*/) mutable {
if (not dht_)
return;
for (auto& l : listeners_) {
if (not l.second.tokenClassicDht) {
l.second.tokenClassicDht = dht_->listen(l.second.hash, l.second.gcb, l.second.f, l.second.w);
}
}
});
}
}
#else
if (proxify)
std::cerr << "DHT proxy requested but OpenDHT built without proxy support." << std::endl;
throw std::runtime_error("DHT proxy requested but OpenDHT built without proxy support.");
#endif
}
@@ -1244,16 +1131,8 @@ void
DhtRunner::forwardAllMessages(bool forward)
{
std::lock_guard<std::mutex> lck(dht_mtx);
#ifdef OPENDHT_PROXY_SERVER
#ifdef OPENDHT_PROXY_CLIENT
if (dht_via_proxy_)
dht_via_proxy_->forwardAllMessages(forward);
#endif // OPENDHT_PROXY_CLIENT
if (dht_)
dht_->forwardAllMessages(forward);
#else
(void) forward;
#endif // OPENDHT_PROXY_SERVER
}
/**
@@ -1264,24 +1143,46 @@ DhtRunner::setPushNotificationToken(const std::string& token) {
std::lock_guard<std::mutex> lck(dht_mtx);
#if defined(OPENDHT_PROXY_CLIENT) && defined(OPENDHT_PUSH_NOTIFICATIONS)
config_.push_token = token;
if (dht_via_proxy_)
dht_via_proxy_->setPushNotificationToken(token);
if (dht_)
dht_->setPushNotificationToken(token);
#else
(void) token;
#endif
}
void
DhtRunner::setPushNotificationTopic(const std::string& topic) {
std::lock_guard<std::mutex> lck(dht_mtx);
#if defined(OPENDHT_PROXY_CLIENT) && defined(OPENDHT_PUSH_NOTIFICATIONS)
config_.push_topic = topic;
if (dht_)
dht_->setPushNotificationTopic(topic);
#else
(void) topic;
#endif
}
void
DhtRunner::setPushNotificationPlatform(const std::string& platform) {
std::lock_guard<std::mutex> lck(dht_mtx);
#if defined(OPENDHT_PROXY_CLIENT) && defined(OPENDHT_PUSH_NOTIFICATIONS)
config_.push_platform = platform;
if (dht_)
dht_->setPushNotificationPlatform(platform);
#else
(void) platform;
#endif
}
void
DhtRunner::pushNotificationReceived(const std::map<std::string, std::string>& data)
{
#if defined(OPENDHT_PROXY_CLIENT) && defined(OPENDHT_PUSH_NOTIFICATIONS)
{
std::lock_guard<std::mutex> lck(storage_mtx);
pending_ops_prio.emplace([=](SecureDht&) {
if (dht_via_proxy_)
dht_via_proxy_->pushNotificationReceived(data);
});
}
std::lock_guard<std::mutex> lck(storage_mtx);
pending_ops_prio.emplace([=](SecureDht&) {
if (dht_)
dht_->pushNotificationReceived(data);
});
cv.notify_all();
#else
(void) data;
+221 -199
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author: Vsevolod Ivanov <vsevolod.ivanov@savoirfairelinux.com>
*
* This program is free software; you can redistribute it and/or modify
@@ -17,14 +17,14 @@
*/
#include "http.h"
#include "log_enable.h"
#include "logger.h"
#include "crypto.h"
#include "base64.h"
#include "compat/os_cert.h"
#include <asio.hpp>
#include <restinio/impl/tls_socket.hpp>
#include <http_parser.h>
#include <llhttp.h>
#include <json/json.h>
#include <openssl/ocsp.h>
@@ -33,6 +33,10 @@
#include <openssl/x509.h>
#include <openssl/x509_vfy.h>
#include <cctype>
#include <iomanip>
#include <sstream>
#define MAXAGE_SEC (14*24*60*60)
#define JITTER_SEC (60)
#define OCSP_MAX_RESPONSE_SIZE (20480)
@@ -87,7 +91,7 @@ Url::Url(const std::string& url): url(url)
std::string
Url::toString() const
{
std::stringstream ss;
std::ostringstream ss;
if (not protocol.empty()) {
ss << protocol << "://";
}
@@ -111,11 +115,11 @@ newTlsClientContext(const std::shared_ptr<dht::Logger>& logger)
if (char* path = getenv("CA_ROOT_FILE")) {
if (logger)
logger->d("Using CA file: %s", path);
logger->debug("Using CA file: {}", path);
ctx->load_verify_file(path);
} else if (char* path = getenv("CA_ROOT_PATH")) {
if (logger)
logger->d("Using CA path: %s", path);
logger->debug("Using CA path: {}", path);
ctx->add_verify_path(path);
} else {
#ifdef __ANDROID__
@@ -140,12 +144,12 @@ Connection::Connection(asio::io_context& ctx, const bool ssl, std::shared_ptr<dh
ssl_ctx_ = newTlsClientContext(l);
ssl_socket_ = std::make_unique<ssl_socket_t>(ctx_, ssl_ctx_);
if (logger_)
logger_->d("[connection:%i] start https session with system CA", id_);
logger_->debug("[connection:{:d}] start https session with system CA", id_);
}
else {
socket_ = std::make_unique<socket_t>(ctx);
if (logger_)
logger_->d("[connection:%i] start http session", id_);
logger_->debug("[connection:{:d}] start http session", id_);
}
}
@@ -162,11 +166,11 @@ Connection::Connection(asio::io_context& ctx, std::shared_ptr<dht::crypto::Certi
if (ec)
throw std::runtime_error("Error adding certificate authority: " + ec.message());
else if (logger_)
logger_->d("[connection:%i] start https with custom CA %s", id_, server_ca->getUID().c_str());
logger_->debug("[connection:{:d}] start https with custom CA {:s}", id_, server_ca->getUID());
} else {
ssl_ctx_ = newTlsClientContext(l);
if (logger_)
logger_->d("[connection:%i] start https session with system CA", id_);
logger_->debug("[connection:{:d}] start https session with system CA", id_);
}
if (identity.first){
auto key = identity.first->serialize();
@@ -181,7 +185,7 @@ Connection::Connection(asio::io_context& ctx, std::shared_ptr<dht::crypto::Certi
if (ec)
throw std::runtime_error("Error adding client certificate: " + ec.message());
else if (logger_)
logger_->d("[connection:%i] client certificate %s", id_, identity.second->getUID().c_str());
logger_->debug("[connection:{:d}] client certificate {:s}", id_, identity.second->getUID());
}
ssl_socket_ = std::make_unique<ssl_socket_t>(ctx_, ssl_ctx_);
}
@@ -204,7 +208,7 @@ Connection::close()
socket_->close(ec);
}
if (ec and logger_)
logger_->e("[connection:%i] error closing: %s", id_, ec.message().c_str());
logger_->error("[connection:{:d}] error closing: {:s}", id_, ec.message());
}
bool
@@ -221,20 +225,25 @@ Connection::is_ssl() const
return ssl_ctx_ ? true : false;
}
static time_t
parse_ocsp_time(ASN1_GENERALIZEDTIME* gt)
{
struct tm tm;
time_t rv = -1;
std::string asn1ToString(ASN1_GENERALIZEDTIME* time) {
if (!time) {
return "(null)";
}
BIO* memBio = BIO_new(BIO_s_mem());
if (!memBio) {
throw std::runtime_error("Failed to create BIO");
}
if (gt == nullptr)
return -1;
// RFC 6960 specifies that all times in OCSP must be GENERALIZEDTIME
if (ASN1_time_parse((const char*)gt->data, gt->length, &tm, V_ASN1_GENERALIZEDTIME) == -1)
return -1;
if ((rv = timegm(&tm)) == -1)
return -1;
return rv;
if (ASN1_GENERALIZEDTIME_print(memBio, time) <= 0) {
BIO_free(memBio);
throw std::runtime_error("Failed to print ASN1_GENERALIZEDTIME");
}
char* bioData;
long bioLength = BIO_get_mem_data(memBio, &bioData);
std::string result(bioData, bioLength);
BIO_free(memBio);
return result;
}
static inline X509*
@@ -272,26 +281,26 @@ ocspRequestFromCert(STACK_OF(X509)* fullchain, const std::shared_ptr<Logger>& lo
if (sk_X509_num(fullchain) <= 1) {
if (logger)
logger->e("Cert does not contain a cert chain");
logger->error("Cert does not contain a cert chain");
return {};
}
X509* cert = cert_from_chain(fullchain);
if (cert == nullptr) {
if (logger)
logger->e("No certificate found");
logger->error("No certificate found");
return {};
}
X509* issuer = issuer_from_chain(fullchain);
if (issuer == nullptr) {
if (logger)
logger->e("Unable to find issuer for cert");
logger->error("Unable to find issuer for cert");
return {};
}
auto urls = X509_get1_ocsp(cert);
if (urls == nullptr || sk_OPENSSL_STRING_num(urls) <= 0) {
if (logger)
logger->e("Certificate contains no OCSP url");
logger->error("Certificate contains no OCSP url");
return {};
}
auto url = sk_OPENSSL_STRING_value(urls, 0);
@@ -306,12 +315,12 @@ ocspRequestFromCert(STACK_OF(X509)* fullchain, const std::shared_ptr<Logger>& lo
OCSP_CERTID* id = OCSP_cert_to_id(EVP_sha1(), cert, issuer);
if (id == nullptr) {
if (logger)
logger->e("Unable to get certificate id from cert");
logger->error("Unable to get certificate id from cert");
return {};
}
if (OCSP_request_add0_id(request->req.get(), id) == nullptr) {
if (logger)
logger->e("Unable to add certificate id to request");
logger->error("Unable to add certificate id to request");
return {};
}
@@ -322,12 +331,12 @@ ocspRequestFromCert(STACK_OF(X509)* fullchain, const std::shared_ptr<Logger>& lo
uint8_t* data {nullptr};
if ((size = i2d_OCSP_REQUEST(request->req.get(), &data)) <= 0) {
if (logger)
logger->e("Unable to encode ocsp request");
logger->error("Unable to encode ocsp request");
return {};
}
if (data == nullptr) {
if (logger)
logger->e("Unable to allocte memory");
logger->error("Unable to allocte memory");
return {};
}
request->data = std::string((char*)data, (char*)data+size);
@@ -338,36 +347,32 @@ ocspRequestFromCert(STACK_OF(X509)* fullchain, const std::shared_ptr<Logger>& lo
bool
ocspValidateResponse(const OscpRequestInfo& info, STACK_OF(X509)* fullchain, const std::string& response, X509_STORE *store, const std::shared_ptr<Logger>& logger)
{
ASN1_GENERALIZEDTIME *revtime = nullptr, *thisupd = nullptr, *nextupd = nullptr;
const uint8_t* p = (const uint8_t*)response.data();
int status, cert_status=0, crl_reason=0;
time_t now, rev_t = -1, this_t, next_t;
X509* cert = cert_from_chain(fullchain);
if (cert == nullptr) {
if (logger)
logger->e("No certificate found");
logger->error("ocsp: no certificate found");
return false;
}
X509* issuer = issuer_from_chain(fullchain);
if (issuer == nullptr) {
if (logger)
logger->e("Unable to find issuer for cert");
logger->error("ocsp: unable to find issuer for cert");
return false;
}
OCSP_CERTID *cidr;
if ((cidr = OCSP_cert_to_id(nullptr, cert, issuer)) == nullptr) {
if (logger)
logger->e("Unable to get issuer cert/CID");
logger->error("ocsp: unable to get issuer cert/CID");
return false;
}
std::unique_ptr<OCSP_CERTID, decltype(&OCSP_CERTID_free)> cid(cidr, &OCSP_CERTID_free);
const uint8_t* resp_data = (const uint8_t*)response.data();
OCSP_RESPONSE *r;
if ((r = d2i_OCSP_RESPONSE(nullptr, &p, response.size())) == nullptr) {
if ((r = d2i_OCSP_RESPONSE(nullptr, &resp_data, response.size())) == nullptr) {
if (logger)
logger->e("OCSP response unserializable");
logger->error("OCSP response unserializable");
return false;
}
std::unique_ptr<OCSP_RESPONSE, decltype(&OCSP_RESPONSE_free)> resp(r, &OCSP_RESPONSE_free);
@@ -375,99 +380,59 @@ ocspValidateResponse(const OscpRequestInfo& info, STACK_OF(X509)* fullchain, con
OCSP_BASICRESP *brespr;
if ((brespr = OCSP_response_get1_basic(resp.get())) == nullptr) {
if (logger)
logger->e("Failed to load OCSP response");
logger->error("Failed to load OCSP response");
return false;
}
std::unique_ptr<OCSP_BASICRESP, decltype(&OCSP_BASICRESP_free)> bresp(brespr, &OCSP_BASICRESP_free);
if (OCSP_basic_verify(bresp.get(), fullchain, store, OCSP_TRUSTOTHER) != 1) {
if (logger)
logger->w("OCSP verify failed");
logger->warn("OCSP verify failed");
return false;
}
printf("OCSP response signature validated\n");
status = OCSP_response_status(resp.get());
int status = OCSP_response_status(resp.get());
if (status != OCSP_RESPONSE_STATUS_SUCCESSFUL) {
if (logger)
logger->w("OCSP Failure: code %d (%s)", status, OCSP_response_status_str(status));
logger->warn("OCSP Failure: code {:d} ({:s})", status, OCSP_response_status_str(status));
return false;
}
// Check the nonce if we sent one
if (OCSP_check_nonce(info.req.get(), bresp.get()) <= 0) {
if (logger)
logger->w("No OCSP nonce, or mismatch");
logger->warn("No OCSP nonce, or mismatch");
return false;
}
ASN1_GENERALIZEDTIME *revtime = nullptr, *thisupd = nullptr, *nextupd = nullptr;
int cert_status=0, crl_reason=0;
if (OCSP_resp_find_status(bresp.get(), cid.get(), &cert_status, &crl_reason,
&revtime, &thisupd, &nextupd) != 1) {
if (logger)
logger->w("OCSP verify failed: no result for cert");
logger->warn("OCSP verify failed: no result for cert");
return false;
}
if (revtime && (rev_t = parse_ocsp_time(revtime)) == -1) {
if (logger)
logger->w("Unable to parse revocation time in OCSP reply");
return false;
}
// Belt and suspenders, Treat it as revoked if there is either
// a revocation time, or status revoked.
if (rev_t != -1 || cert_status == V_OCSP_CERTSTATUS_REVOKED) {
if (logger)
logger->w("Invalid OCSP reply: certificate is revoked");
if (rev_t != -1) {
if (logger)
logger->w("Certificate revoked at: %s", ctime(&rev_t));
if (revtime || cert_status == V_OCSP_CERTSTATUS_REVOKED) {
if (logger) {
logger->warn("OCSP verify failed: certificate revoked since {}", asn1ToString(revtime));
}
return false;
}
if ((this_t = parse_ocsp_time(thisupd)) == -1) {
if (logger)
logger->w("unable to parse this update time in OCSP reply");
return false;
}
if ((next_t = parse_ocsp_time(nextupd)) == -1) {
if (logger)
logger->w("unable to parse next update time in OCSP reply");
return false;
}
// Don't allow this update to precede next update
if (this_t >= next_t) {
if (OCSP_check_validity(thisupd, nextupd, 1, -1) == 0) {
if (logger)
logger->w("Invalid OCSP reply: this update >= next update");
return false;
}
now = time(nullptr);
// Check that this update is not more than JITTER seconds in the future.
if (this_t > now + JITTER_SEC) {
if (logger)
logger->e("Invalid OCSP reply: this update is in the future (%s)", ctime(&this_t));
return false;
}
// Check that this update is not more than MAXSEC in the past.
if (this_t < now - MAXAGE_SEC) {
if (logger)
logger->e("Invalid OCSP reply: this update is too old (%s)", ctime(&this_t));
return false;
}
// Check that next update is still valid
if (next_t < now - JITTER_SEC) {
if (logger)
logger->w("Invalid OCSP reply: reply has expired (%s)", ctime(&next_t));
logger->warn("OCSP reply is expired or not yet valid");
return false;
}
if (logger) {
logger->d("OCSP response validated");
logger->d(" This Update: %s", ctime(&this_t));
logger->d(" Next Update: %s", ctime(&next_t));
logger->debug("OCSP response validated");
logger->debug(" This Update: {}", asn1ToString(thisupd));
logger->debug(" Next Update: {}", asn1ToString(nextupd));
}
return true;
}
@@ -488,7 +453,7 @@ Connection::set_ssl_verification(const std::string& hostname, const asio::ssl::v
if (logger) {
char subject_name[1024];
X509_NAME_oneline(X509_get_subject_name(cert), subject_name, 1024);
logger->d("[connection:%i] verify %s compliance to RFC 2818:\n%s", id, hostname.c_str(), subject_name);
logger->debug("[connection:{:d}] verify {:s} compliance to RFC 2818:\n{:s}", id, hostname, subject_name);
}
// starts from CA and goes down the presented chain
@@ -496,14 +461,14 @@ Connection::set_ssl_verification(const std::string& hostname, const asio::ssl::v
bool verified = verifier(preverified, ctx);
auto verify_ec = X509_STORE_CTX_get_error(ctx.native_handle());
if (verify_ec != 0 /*X509_V_OK*/ and logger)
logger->e("[http::connection:%i] ssl verification error=%i %d", id, verify_ec, verified);
logger->error("[http::connection:{:d}] ssl verification error={:d} %d", id, verify_ec, verified);
if (verified and checkOcsp) {
std::unique_ptr<stack_st_X509, void(*)(stack_st_X509*)> chain(
X509_STORE_CTX_get1_chain(ctx.native_handle()),
[](stack_st_X509* c){ sk_X509_pop_free(c, X509_free); });
if (auto ocspInfo = ocspRequestFromCert(chain.get(), logger)) {
if (logger)
logger->w("[http::connection:%i] TLS OCSP server: %s, request size: %zu", id, ocspInfo->url.c_str(), ocspInfo->data.size());
logger->warn("[http::connection:{:d}] TLS OCSP server: {:s}, request size: {:d}", id, ocspInfo->url, ocspInfo->data.size());
bool ocspVerified = false;
asio::io_context io_ctx;
auto ocspReq = std::make_shared<Request>(io_ctx, ocspInfo->url, [&](const Response& ocspResp){
@@ -511,7 +476,7 @@ Connection::set_ssl_verification(const std::string& hostname, const asio::ssl::v
ocspVerified = ocspValidateResponse(*ocspInfo, chain.get(), ocspResp.body, X509_STORE_CTX_get0_store(ctx.native_handle()), logger);
} else {
if (logger)
logger->w("[http::connection:%i] TLS OCSP check error", id);
logger->warn("[http::connection:{:d}] TLS OCSP check error", id);
}
}, logger);
ocspReq->set_method(restinio::http_method_post());
@@ -567,57 +532,25 @@ Connection::async_connect(std::vector<asio::ip::tcp::endpoint>&& endpoints, Conn
}
auto& base = ssl_socket_? ssl_socket_->lowest_layer() : *socket_;
ConnectHandlerCb wcb = [&base, cb=std::move(cb)](const asio::error_code& ec, const asio::ip::tcp::endpoint& endpoint) {
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wunused-variable"
ConnectHandlerCb wcb = [this, &base, cb=std::move(cb)](const asio::error_code& ec, const asio::ip::tcp::endpoint& endpoint) {
if (!ec) {
auto socket = base.native_handle();
local_address_ = base.local_endpoint().address();
// Once connected, set a keep alive on the TCP socket with 30 seconds delay
// This will generate broken pipes as soon as possible.
// Note this needs to be done once connected to have a valid native_handle()
uint32_t start = 30;
uint32_t interval = 30;
uint32_t cnt = 1;
#ifdef _WIN32
std::string val = "1";
setsockopt(socket, SOL_SOCKET, SO_KEEPALIVE, val.c_str(), sizeof(val));
// TCP_KEEPIDLE and TCP_KEEPINTVL are available since Win 10 version 1709
// TCP_KEEPCNT since Win 10 version 1703
#ifdef TCP_KEEPIDLE
std::string start_str = std::to_string(start);
setsockopt(socket, IPPROTO_TCP, TCP_KEEPIDLE,
start_str.c_str(), sizeof(start_str));
#endif
#ifdef TCP_KEEPINTVL
std::string interval_str = std::to_string(interval);
setsockopt(socket, IPPROTO_TCP, TCP_KEEPINTVL,
interval_str.c_str(), sizeof(interval_str));
#endif
#ifdef TCP_KEEPCNT
std::string cnt_str = std::to_string(cnt);
setsockopt(socket, IPPROTO_TCP, TCP_KEEPCNT,
cnt_str.c_str(), sizeof(cnt_str));
#endif
#else
uint32_t val = 1;
setsockopt(socket, SOL_SOCKET, SO_KEEPALIVE, &val, sizeof(uint32_t));
#ifdef __APPLE__
// Apple devices only have one parameter
setsockopt(socket, IPPROTO_TCP, TCP_KEEPALIVE, &start, sizeof(uint32_t));
#else
// Linux based systems
setsockopt(socket, SOL_TCP, TCP_KEEPIDLE, &start, sizeof(uint32_t));
setsockopt(socket, SOL_TCP, TCP_KEEPINTVL, &interval, sizeof(uint32_t));
setsockopt(socket, SOL_TCP, TCP_KEEPCNT, &cnt, sizeof(uint32_t));
#endif
#endif
this->set_keepalive(30);
}
if (cb)
cb(ec, endpoint);
};
#pragma GCC diagnostic pop
if (ssl_socket_)
asio::async_connect(ssl_socket_->lowest_layer(), std::move(endpoints), wrapCallabck(std::move(wcb)));
asio::async_connect(ssl_socket_->lowest_layer(), std::move(endpoints), wrapCallback(std::move(wcb)));
else
asio::async_connect(*socket_, std::move(endpoints), wrapCallabck(std::move(wcb)));
asio::async_connect(*socket_, std::move(endpoints), wrapCallback(std::move(wcb)));
}
void
@@ -637,11 +570,11 @@ Connection::async_handshake(HandlerCb cb)
if (this_.logger_) {
if (verify_ec == X509_V_ERR_DEPTH_ZERO_SELF_SIGNED_CERT /*18*/
|| verify_ec == X509_V_ERR_SELF_SIGNED_CERT_IN_CHAIN /*19*/)
this_.logger_->d("[connection:%i] self-signed certificate in handshake: %i", this_.id_, verify_ec);
this_.logger_->debug("[connection:{:d}] self-signed certificate in handshake: {:d}", this_.id_, verify_ec);
else if (verify_ec != X509_V_OK)
this_.logger_->e("[connection:%i] verify handshake error: %i", this_.id_, verify_ec);
this_.logger_->error("[connection:{:d}] verify handshake error: {:d}", this_.id_, verify_ec);
else
this_.logger_->w("[connection:%i] verify handshake success", this_.id_);
this_.logger_->warn("[connection:{:d}] verify handshake success", this_.id_);
}
}
if (cb)
@@ -662,8 +595,8 @@ Connection::async_write(BytesHandlerCb cb)
if (cb) ctx_.post([cb](){ cb(asio::error::broken_pipe, 0); });
return;
}
if (ssl_socket_) asio::async_write(*ssl_socket_, write_buf_, wrapCallabck(std::move(cb)));
else if (socket_) asio::async_write(*socket_, write_buf_, wrapCallabck(std::move(cb)));
if (ssl_socket_) asio::async_write(*ssl_socket_, write_buf_, wrapCallback(std::move(cb)));
else if (socket_) asio::async_write(*socket_, write_buf_, wrapCallback(std::move(cb)));
else if (cb) ctx_.post([cb](){ cb(asio::error::operation_aborted, 0); });
}
@@ -675,8 +608,8 @@ Connection::async_read_until(const char* delim, BytesHandlerCb cb)
if (cb) ctx_.post([cb](){ cb(asio::error::broken_pipe, 0); });
return;
}
if (ssl_socket_) asio::async_read_until(*ssl_socket_, read_buf_, delim, wrapCallabck(std::move(cb)));
else if (socket_) asio::async_read_until(*socket_, read_buf_, delim, wrapCallabck(std::move(cb)));
if (ssl_socket_) asio::async_read_until(*ssl_socket_, read_buf_, delim, wrapCallback(std::move(cb)));
else if (socket_) asio::async_read_until(*socket_, read_buf_, delim, wrapCallback(std::move(cb)));
else if (cb) ctx_.post([cb](){ cb(asio::error::operation_aborted, 0); });
}
@@ -688,8 +621,8 @@ Connection::async_read_until(char delim, BytesHandlerCb cb)
if (cb) ctx_.post([cb](){ cb(asio::error::broken_pipe, 0); });
return;
}
if (ssl_socket_) asio::async_read_until(*ssl_socket_, read_buf_, delim, wrapCallabck(std::move(cb)));
else if (socket_) asio::async_read_until(*socket_, read_buf_, delim, wrapCallabck(std::move(cb)));
if (ssl_socket_) asio::async_read_until(*ssl_socket_, read_buf_, delim, wrapCallback(std::move(cb)));
else if (socket_) asio::async_read_until(*socket_, read_buf_, delim, wrapCallback(std::move(cb)));
else if (cb) ctx_.post([cb](){ cb(asio::error::operation_aborted, 0); });
}
@@ -701,8 +634,8 @@ Connection::async_read(size_t bytes, BytesHandlerCb cb)
if (cb) ctx_.post([cb](){ cb(asio::error::broken_pipe, 0); });
return;
}
if (ssl_socket_) asio::async_read(*ssl_socket_, read_buf_, asio::transfer_exactly(bytes), wrapCallabck(std::move(cb)));
else if (socket_) asio::async_read(*socket_, read_buf_, asio::transfer_exactly(bytes), wrapCallabck(std::move(cb)));
if (ssl_socket_) asio::async_read(*ssl_socket_, read_buf_, asio::transfer_exactly(bytes), wrapCallback(std::move(cb)));
else if (socket_) asio::async_read(*socket_, read_buf_, asio::transfer_exactly(bytes), wrapCallback(std::move(cb)));
else if (cb) ctx_.post([cb](){ cb(asio::error::operation_aborted, 0); });
}
@@ -724,15 +657,72 @@ Connection::async_read_some(size_t bytes, BytesHandlerCb cb)
}
void
Connection::timeout(const std::chrono::seconds timeout, HandlerCb cb)
Connection::set_keepalive(uint32_t seconds)
{
std::lock_guard<std::mutex> lock(mutex_);
if (!ssl_socket_ && !socket_) return;
auto& base = ssl_socket_? ssl_socket_->lowest_layer() : *socket_;
auto socket = base.native_handle();
uint32_t interval = 1;
uint32_t cnt = 10;
#ifdef _WIN32
// TCP_KEEPIDLE and TCP_KEEPINTVL are available since Win 10 version 1709
// TCP_KEEPCNT since Win 10 version 1703
#if defined(TCP_KEEPIDLE) && defined(TCP_KEEPINTVL) && defined(TCP_KEEPCNT)
std::string val = "1";
setsockopt(socket, SOL_SOCKET, SO_KEEPALIVE, val.c_str(), sizeof(val));
std::string seconds_str = std::to_string(seconds);
setsockopt(socket, IPPROTO_TCP, TCP_KEEPIDLE,
seconds_str.c_str(), sizeof(seconds_str));
std::string interval_str = std::to_string(interval);
setsockopt(socket, IPPROTO_TCP, TCP_KEEPINTVL,
interval_str.c_str(), sizeof(interval_str));
std::string cnt_str = std::to_string(cnt);
setsockopt(socket, IPPROTO_TCP, TCP_KEEPCNT,
cnt_str.c_str(), sizeof(cnt_str));
#else
struct {
uint32_t onoff;
uint32_t keepalivetime;
uint32_t keepaliveinterval;
} keepalive;
keepalive.onoff = 1;
keepalive.keepalivetime = seconds * 1000;
keepalive.keepaliveinterval = interval * 1000;
int32_t out = 0;
WSAIoctl(socket, SIO_KEEPALIVE_VALS, keepalive, sizeof(tcp_keepalive),
nullptr, 0, &out, nullptr, nullptr);
#endif
#else
uint32_t val = 1;
setsockopt(socket, SOL_SOCKET, SO_KEEPALIVE, &val, sizeof(uint32_t));
#ifdef __APPLE__
// Old Apple devices only have one parameter
setsockopt(socket, IPPROTO_TCP, TCP_KEEPALIVE, &seconds, sizeof(uint32_t));
#if defined(TCP_KEEPINTVL) && defined(TCP_KEEPCNT)
setsockopt(socket, IPPROTO_TCP, TCP_KEEPINTVL, &interval, sizeof(uint32_t));
setsockopt(socket, IPPROTO_TCP, TCP_KEEPCNT, &cnt, sizeof(uint32_t));
#endif
#else
// Linux based systems
setsockopt(socket, SOL_TCP, TCP_KEEPIDLE, &seconds, sizeof(uint32_t));
setsockopt(socket, SOL_TCP, TCP_KEEPINTVL, &interval, sizeof(uint32_t));
setsockopt(socket, SOL_TCP, TCP_KEEPCNT, &cnt, sizeof(uint32_t));
#endif
#endif
}
const asio::ip::address&
Connection::local_address() const
{
return local_address_;
}
void
Connection::timeout(const std::chrono::seconds& timeout, HandlerCb cb)
{
if (!is_open()){
if (logger_)
logger_->e("[connection:%i] closed, can't timeout", id_);
if (cb)
cb(asio::error::operation_aborted);
return;
}
if (!timeout_timer_)
timeout_timer_ = std::make_unique<asio::steady_timer>(ctx_);
timeout_timer_->expires_at(std::chrono::steady_clock::now() + timeout);
@@ -741,7 +731,7 @@ Connection::timeout(const std::chrono::seconds timeout, HandlerCb cb)
return;
else if (ec){
if (logger)
logger->e("[connection:%i] timeout error: %s", id, ec.message().c_str());
logger->error("[connection:{:d}] timeout error: {:s}", id, ec.message());
}
if (cb)
cb(ec);
@@ -791,6 +781,12 @@ Resolver::~Resolver()
*destroyed_ = true;
}
void
Resolver::cancel()
{
resolver_.cancel();
}
inline
std::vector<asio::ip::tcp::endpoint>
filter(const std::vector<asio::ip::tcp::endpoint>& epts, sa_family_t family)
@@ -832,9 +828,8 @@ Resolver::resolve(const std::string& host, const std::string& service)
if (ec == asio::error::operation_aborted or *destroyed)
return;
if (logger_) {
if (ec)
logger_->e("[http:client] [resolver] error for %s:%s: %s",
host.c_str(), service.c_str(), ec.message().c_str());
logger_->error("[http:client] [resolver] result for {:s}:{:s}: {:s}",
host, service, ec.message());
}
decltype(cbs_) cbs;
{
@@ -876,7 +871,7 @@ Request::Request(asio::io_context& ctx, const std::string& url, const Json::Valu
Json::CharReaderBuilder rbuilder;
auto reader = std::unique_ptr<Json::CharReader>(rbuilder.newCharReader());
if (!reader->parse(response.body.data(), response.body.data() + response.body.size(), &json, &err) and logger_)
logger_->e("[http:request:%i] can't parse response to json", id_, err.c_str());
logger_->error("[http:request:{:d}] can't parse response to json: {:s}", id_, err);
}
if (jsoncb)
jsoncb(std::move(json), response);
@@ -898,7 +893,7 @@ Request::Request(asio::io_context& ctx, const std::string& url, OnJsonCb jsoncb,
Json::CharReaderBuilder rbuilder;
auto reader = std::unique_ptr<Json::CharReader>(rbuilder.newCharReader());
if (!reader->parse(response.body.data(), response.body.data() + response.body.size(), &json, &err) and logger_)
logger_->e("[http:request:%i] can't parse response to json", id_, err.c_str());
logger_->error("[http:request:{:d}] can't parse response to json: {:s}", id_, err);
}
if (jsoncb)
jsoncb(std::move(json), response);
@@ -966,6 +961,8 @@ Request::init_default_headers()
void
Request::cancel()
{
if (auto r = resolver_)
r->cancel();
if (auto c = conn_)
c->close();
}
@@ -1040,7 +1037,7 @@ Request::set_auth(const std::string& username, const std::string& password)
void
Request::build()
{
std::stringstream request;
std::ostringstream request;
bool append_body = !body_.empty();
// first header
@@ -1062,7 +1059,7 @@ Request::build()
break;
case restinio::http_connection_header_t::upgrade:
if (logger_)
logger_->e("Unsupported connection type 'upgrade', fallback to 'close'");
logger_->error("Unsupported connection type 'upgrade', fallback to 'close'");
// fallthrough
case restinio::http_connection_header_t::close:
conn_str = "close"; // default
@@ -1080,6 +1077,30 @@ Request::build()
request_ = request.str();
}
// https://stackoverflow.com/a/17708801
std::string
Request::url_encode(std::string_view value)
{
std::ostringstream escaped;
escaped.fill('0');
escaped << std::hex;
for (const char& c : value) {
// Keep alphanumeric and other accepted characters intact
if (std::isalnum(c) || c == '-' || c == '_' || c == '.' || c == '~') {
escaped << c;
continue;
}
// Any other characters are percent-encoded
escaped << std::uppercase;
escaped << '%' << std::setw(2) << static_cast<int>(static_cast<unsigned char>(c));
escaped << std::nouppercase;
}
return escaped.str();
}
void
Request::add_on_status_callback(OnStatusCb cb) {
cbs_.on_status = std::move(cb);
@@ -1116,13 +1137,11 @@ Request::init_parser()
response_.request = shared_from_this();
if (!parser_)
parser_ = std::make_unique<http_parser>();
http_parser_init(parser_.get(), HTTP_RESPONSE);
parser_->data = static_cast<void*>(this);
parser_ = std::make_unique<llhttp_t>();
if (!parser_s_)
parser_s_ = std::make_unique<http_parser_settings>();
http_parser_settings_init(parser_s_.get());
parser_s_ = std::make_unique<llhttp_settings_t>();
llhttp_settings_init(parser_s_.get());
cbs_.on_status = [this, statusCb = std::move(cbs_.on_status)](unsigned int status_code){
response_.status_code = status_code;
@@ -1137,31 +1156,33 @@ Request::init_parser()
response_.headers[*header_field] = std::string(at, length);
};
// http_parser raw c callback (note: no context can be passed into them)
parser_s_->on_status = [](http_parser* parser, const char* /*at*/, size_t /*length*/) -> int {
// llhttp raw c callback (note: no context can be passed into them)
parser_s_->on_status = [](llhttp_t* parser, const char* /*at*/, size_t /*length*/) -> int {
static_cast<Request*>(parser->data)->cbs_.on_status(parser->status_code);
return 0;
};
parser_s_->on_header_field = [](http_parser* parser, const char* at, size_t length) -> int {
parser_s_->on_header_field = [](llhttp_t* parser, const char* at, size_t length) -> int {
static_cast<Request*>(parser->data)->cbs_.on_header_field(at, length);
return 0;
};
parser_s_->on_header_value = [](http_parser* parser, const char* at, size_t length) -> int {
parser_s_->on_header_value = [](llhttp_t* parser, const char* at, size_t length) -> int {
static_cast<Request*>(parser->data)->cbs_.on_header_value(at, length);
return 0;
};
parser_s_->on_body = [](http_parser* parser, const char* at, size_t length) -> int {
parser_s_->on_body = [](llhttp_t* parser, const char* at, size_t length) -> int {
static_cast<Request*>(parser->data)->onBody(at, length);
return 0;
};
parser_s_->on_headers_complete = [](http_parser* parser) -> int {
parser_s_->on_headers_complete = [](llhttp_t* parser) -> int {
static_cast<Request*>(parser->data)->onHeadersComplete();
return 0;
};
parser_s_->on_message_complete = [](http_parser* parser) -> int {
parser_s_->on_message_complete = [](llhttp_t* parser) -> int {
static_cast<Request*>(parser->data)->onComplete();
return 0;
};
llhttp_init(parser_.get(), HTTP_RESPONSE, parser_s_.get());
parser_->data = static_cast<void*>(this);
}
void
@@ -1169,7 +1190,7 @@ Request::connect(std::vector<asio::ip::tcp::endpoint>&& endpoints, HandlerCb cb)
{
if (endpoints.empty()){
if (logger_)
logger_->e("[http:request:%i] connect: no endpoints provided", id_);
logger_->error("[http:request:{:d}] connect: no endpoints provided", id_);
if (cb)
cb(asio::error::connection_aborted);
return;
@@ -1178,7 +1199,7 @@ Request::connect(std::vector<asio::ip::tcp::endpoint>&& endpoints, HandlerCb cb)
std::string eps = "";
for (const auto& endpoint : endpoints)
eps.append(endpoint.address().to_string() + ":" + std::to_string(endpoint.port()) + " ");
logger_->d("[http:request:%i] connect begin: %s", id_, eps.c_str());
logger_->debug("[http:request:{:d}] connect begin: {:s}", id_, eps);
}
bool isHttps = get_url().protocol == "https";
if (isHttps) {
@@ -1191,6 +1212,9 @@ Request::connect(std::vector<asio::ip::tcp::endpoint>&& endpoints, HandlerCb cb)
else
conn_ = std::make_shared<Connection>(ctx_, false/*ssl*/, logger_);
if (conn_ && timeoutCb_)
conn_->timeout(timeout_, std::move(timeoutCb_));
// try to connect to any until one works
std::weak_ptr<Request> wthis = shared_from_this();
conn_->async_connect(std::move(endpoints), [wthis, cb, isHttps]
@@ -1206,7 +1230,7 @@ Request::connect(std::vector<asio::ip::tcp::endpoint>&& endpoints, HandlerCb cb)
}
else if (ec) {
if (this_.logger_)
this_.logger_->e("[http:request:%i] connect failed with all endpoints: %s", this_.id_, ec.message().c_str());
this_.logger_->error("[http:request:{:d}] connect failed with all endpoints: {:s}", this_.id_, ec.message());
} else {
const auto& url = this_.get_url();
auto port = endpoint.port();
@@ -1222,9 +1246,9 @@ Request::connect(std::vector<asio::ip::tcp::endpoint>&& endpoints, HandlerCb cb)
if (ec == asio::error::operation_aborted)
return;
if (ec and logger)
logger->e("[http:request:%i] handshake error: %s", id, ec.message().c_str());
logger->error("[http:request:{:d}] handshake error: {:s}", id, ec.message());
//else if (logger)
// logger->d("[http:request:%i] handshake success", id);
// logger->d("[http:request:{:d}] handshake success", id);
if (cb)
cb(ec);
});
@@ -1252,7 +1276,7 @@ Request::send()
std::lock_guard<std::mutex> lock(this_.mutex_);
if (ec){
if (this_.logger_)
this_.logger_->e("[http:request:%i] resolve error: %s", this_.id_, ec.message().c_str());
this_.logger_->error("[http:request:{:d}] resolve error: {:s}", this_.id_, ec.message());
this_.terminate(asio::error::connection_aborted);
}
else if (!this_.conn_ or !this_.conn_->is_open()) {
@@ -1282,7 +1306,7 @@ Request::post()
init_parser();
if (logger_)
logger_->d("[http:request:%i] sending %zu bytes", id_, request_.size());
logger_->debug("[http:request:{}] sending {} bytes", id_, request_.size());
// write the request to buffer
std::ostream request_stream(&conn_->input());
@@ -1310,12 +1334,12 @@ Request::terminate(const asio::error_code& ec)
if (logger_) {
if (ec and ec != asio::error::eof and !response_.aborted)
logger_->e("[http:request:%i] end with error: %s", id_, ec.message().c_str());
logger_->error("[http:request:{:d}] end with error: {:s}", id_, ec.message());
else
logger_->d("[http:request:%i] done with status code %u", id_, response_.status_code);
logger_->debug("[http:request:{:d}] done with status code {:d}", id_, response_.status_code);
}
if (!parser_ or !http_should_keep_alive(parser_.get()))
if (!parser_ or !llhttp_should_keep_alive(parser_.get()))
if (auto c = conn_)
c->close();
notify_state_change(State::DONE);
@@ -1354,17 +1378,15 @@ Request::handle_response(const asio::error_code& ec, size_t /* n_bytes */)
return;
}
auto request = (ec == asio::error::eof) ? std::string{} : conn_->read_bytes();
size_t ret = http_parser_execute(parser_.get(), parser_s_.get(), request.c_str(), request.size());
if (ret != request.size()) {
enum llhttp_errno ret = llhttp_execute(parser_.get(), request.c_str(), request.size());
if (ret != HPE_OK && ret != HPE_PAUSED) {
if (logger_)
logger_->e("Error parsing HTTP: %zu %s %s", ret,
http_errno_name(HTTP_PARSER_ERRNO(parser_)),
http_errno_description(HTTP_PARSER_ERRNO(parser_)));
logger_->e("Error parsing HTTP: %zu %s %d", (int)ret, llhttp_errno_name(ret), llhttp_get_error_reason(parser_.get()));
terminate(asio::error::basic_errors::broken_pipe);
return;
}
if (state_ != State::DONE and parser_ and not http_body_is_final(parser_.get())) {
if (state_ != State::DONE and parser_ and not llhttp_message_needs_eof(parser_.get())) {
auto toRead = parser_->content_length ? std::min<uint64_t>(parser_->content_length, 64 * 1024) : 64 * 1024;
std::weak_ptr<Request> wthis = shared_from_this();
conn_->async_read_some(toRead, [wthis](const asio::error_code& ec, size_t bytes){
+4 -4
View File
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
* Author(s) : Adrien Béraud <adrien.beraud@savoirfairelinux.com>
* Simon Désaulniers <simon.desaulniers@savoirfairelinux.com>
* Nicolas Reynaud <nicolas.reynaud@savoirfairelinux.com>
@@ -32,7 +32,7 @@ namespace indexation {
* @return string that represent the blob into a readable way
*/
static std::string blobToString(const Blob &bl) {
std::stringstream ss;
std::ostringstream ss;
auto bn = bl.size() % 8;
auto n = bl.size() / 8;
@@ -46,7 +46,7 @@ static std::string blobToString(const Blob &bl) {
}
std::string Prefix::toString() const {
std::stringstream ss;
std::ostringstream ss;
ss << "Prefix : " << std::endl << "\tContent_ : \"";
ss << blobToString(content_);
@@ -316,7 +316,7 @@ void Pht::updateCanary(Prefix p) {
dht_->put(p.hash(), std::move(canary_value),
[=](bool){
static std::bernoulli_distribution d(0.5);
crypto::random_device rd;
std::random_device rd;
if (p.size_ and d(rd))
updateCanary(p.getPrefix(-1));
}
-58
View File
@@ -1,58 +0,0 @@
/*
* Copyright (C) 2014-2022 Savoir-faire Linux Inc.
* Author : Adrien Béraud <adrien.beraud@savoirfairelinux.com>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
#include "infohash.h"
#include <functional>
#include <sstream>
#include <cstdio>
using namespace std::literals;
namespace dht {
const HexMap hex_map = {};
void
NodeExport::msgpack_unpack(msgpack::object o)
{
if (o.type != msgpack::type::MAP)
throw msgpack::type_error();
if (o.via.map.size < 2)
throw msgpack::type_error();
if (o.via.map.ptr[0].key.as<std::string_view>() != "id"sv)
throw msgpack::type_error();
if (o.via.map.ptr[1].key.as<std::string_view>() != "addr"sv)
throw msgpack::type_error();
const auto& addr = o.via.map.ptr[1].val;
if (addr.type != msgpack::type::BIN)
throw msgpack::type_error();
if (addr.via.bin.size > sizeof(sockaddr_storage))
throw msgpack::type_error();
id.msgpack_unpack(o.via.map.ptr[0].val);
sslen = addr.via.bin.size;
std::copy_n(addr.via.bin.ptr, addr.via.bin.size, (char*)&ss);
}
std::ostream& operator<< (std::ostream& s, const NodeExport& h)
{
msgpack::pack(s, h);
return s;
}
}
Loaded 100 of 151 files, more files were not shown because too many files have changed in this diff. Show more