Compare commits

..
74 Commits
Author SHA1 Message Date
Adrien Beraud 134b37861d proxy client: fix leak in sendListen 2024-11-19 16:58:08 -05:00
Adrien Beraud 078e0a0c3e search: combine callbacks in case of multiple equivalent puts 2024-11-19 16:58:08 -05:00
Adrien Beraud c4857bf595 tests: add testPutDuplicate 2024-11-18 11:08:01 -05:00
Adrien Beraud 3f27cd7069 proxy client: resubscribe on token change 2024-11-18 11:07:55 -05:00
Adrien Beraud b222efe638 proxy client: pass push parameters to constructor 2024-11-18 11:07:55 -05:00
Adrien Béraud 3c62026b5c proxy server: move time_point using statement before use 2024-11-05 20:20:45 -05:00
Adrien Béraud 7cf43f53ac build/cmake: accept both llhttp and libllhttp 2024-11-05 20:12:13 -05:00
François-Simon Fauteux-Chapleau df189e3e12 dht_proxy_server: add push notification statistics 2024-11-05 15:13:39 -05:00
Adrien Beraud 7e0e7d5483 github/ci: update macOS job to macOS 15 2024-11-04 16:44:36 -05:00
Adrien Beraud 1aeb596b04 crypto: fix warning 2024-11-04 16:39:35 -05:00
Adrien Beraud 950194f68e proxy client: allow to configure user agent for proxy 2024-11-04 16:34:02 -05:00
AmnaSnene 5dc3d3cfd7 Raise an error if any Identity file fails to save 2024-11-04 12:08:34 -05:00
François-Simon Fauteux-Chapleau 5397f80b87 value: preserve priority when encrypting
All encrypted values put on the DHT currently have the default priority
of 0 because the priority information is ignored by the Value::encrypt
function.
2024-10-22 15:22:43 -04:00
Adrien Beraud 88d0af9e3d meson: fix conf_data used before definition 2024-10-03 15:33:03 -04:00
François-Simon Fauteux-Chapleau 792b851d77 dht_proxy_server: fix Android push notifications
This commit updates the format of the push notification requests sent by
the DHT proxy server to conform to FCM's HTTP v1 API. (The previously
used API was deprecated in 2023, see:
https://firebase.google.com/docs/cloud-messaging/migrate-v1)
The "priority" and "ttl" (time to live) options are specific to Android
and therefore need to be put in the "android" block, as documented here:
https://firebase.google.com/docs/reference/fcm/rest/v1/projects.messages
2024-09-19 12:48:36 -04:00
Adrien Béraud 9da832c1d1 http: use appropriate string format syntax 2024-08-11 11:44:47 -04:00
liufei 44f6d21c83 DecryptError cause by low level error 2024-07-23 11:37:26 -04:00
AmnaSnene 38454eaf6e peer discovery: add a timer to schedule connectivityChanged with a maximum interval of one minute 2024-07-16 14:25:01 -04:00
Matthew Webb 76a2bd234a Get off of deprecated GCM AES methods
Multiple compiler warnings note that the gcm_aes_* family of functions are deprecated. They have been replaced with gcm_aes<key_length>_*. This change uses the correct set of functions based on the given key size. Resolves #571.

Sample compiler warning:
/home/noviv/opendht/src/crypto.cpp: In function ‘dht::Blob dht::crypto::aesEncrypt(const uint8_t*, size_t, const dht::Blob&)’: /home/noviv/opendht/src/crypto.cpp:97:20: warning: ‘void nettle_gcm_aes_set_key(gcm_aes_ctx*, size_t, const uint8_t*)’ is deprecated [-Wdeprecated-declarations]
   97 |     gcm_aes_set_key(&aes, key.size(), key.data());
         |                    ^
         In file included from /home/noviv/opendht/src/crypto.cpp:27:
         /usr/include/nettle/gcm.h:276:1: note: declared here
           276 | gcm_aes_set_key(struct gcm_aes_ctx *ctx,
                 | ^~~~~~~~~~~~~~~)

https://github.com/gnutls/nettle/commit/6a19845e6f71791ca98765d490ec08e776494bee marked the functions as deprecated.
2024-07-11 10:07:05 -04:00
Adrien Beraud 972d4d28a1 pkgconfig: add optional liburing dependency 2024-07-05 11:38:32 -04:00
Matthew Webb 72e0a4fb6b Fix an unused parameter warning in dhtrunnertester
This change eliminates the following compiler warning without reducing readability:

/home/noviv/opendht/tests/dhtrunnertester.cpp: In lambda function:
/home/noviv/opendht/tests/dhtrunnertester.cpp:292:89: warning: unused parameter ‘expired’ [-Wunused-parameter]
  292 |     node2.listen(key2, [&](const std::vector<std::shared_ptr<dht::Value>>& values, bool expired){
      |                                                                                    ~~~~~^~~~~~~
2024-07-03 21:59:07 -04:00
AmnaSnene 3e2fd5e8fa peer discovery: address requested changes 2024-06-25 13:27:35 -04:00
AmnaSnene 8b3f7f2480 peer discovery: re-publish and re-discover immediately when a node disconnects, instead of on the connectivityChanged signal. 2024-06-25 13:27:35 -04:00
AmnaSnene 57c855e1cb peer discovery: post reDiscover() and publish() to the ioContext when connectivity changes. 2024-06-25 13:27:35 -04:00
Adrien Beraud 56e7cd94a5 cmake, meson: allow to use io_uring 2024-06-21 12:09:29 -04:00
Adrien Beraud 074e05cc32 ci/github: update to latest action versions 2024-06-10 16:39:55 -04:00
Lanius-collaris 251f66dfce fix getif_workaround_android.h
char may be signed
2024-06-10 08:21:57 -04:00
Lucas Dias 227a7aa0d9 make crypto error messages more readable
Add verifications to gnutls calls and throw expcetion
with readable messages.
For example, if call loadIdentity and crt file has name
different of private key exception message was
Can't get certificate public key ID: ASN1 parser: Value was not found.

However, the real problem is that crt file was not found.
Now, for the same problem the exception message will be:
Error while reading file.

Signed-off-by: Lucas Dias <lucas_dias@redes.ufsm.br>
2024-06-09 16:56:29 -03:00
Lucas Dias e12e344a79 make crypto error messages more readable
Add verifications to gnutls calls and throw expcetion
with readable messages.
For example, if call loadIdentity and crt file has name
different of private key exception message was
Can't get certificate public key ID: ASN1 parser: Value was not found.

However, the real problem is that crt file was not found.
Now, for the same problem the exception message will be:
Error while reading file.

Signed-off-by: Lucas Dias <lucas_dias@redes.ufsm.br>
2024-06-09 16:55:10 -03:00
Lucas Dias d4fc4fd658 Merge branch 'crypto_error_messages' of https://github.com/lucassdiass/opendht into crypto_error_messages
Signed-off-by: Lucas Dias <lucas_dias@redes.ufsm.br>
2024-06-09 16:53:32 -03:00
Lucas Dias 08d67cc8d8 Merge branch 'crypto_error_messages' of https://github.com/lucassdiass/opendht into crypto_error_messages
Signed-off-by: Lucas Dias <lucas_dias@redes.ufsm.br>
2024-06-09 16:52:09 -03:00
Lucas Dias 208afd22f2 Merge branch 'crypto_error_messages' of https://github.com/lucassdiass/opendht into crypto_error_messages 2024-06-09 15:55:31 -03:00
Lucas Dias 9eea01202a make crypto error messages more readable
Add verifications to gnutls calls and throw expcetion
with readable messages.
For example, if call loadIdentity and crt file has name
different of private key exception message was
Can't get certificate public key ID: ASN1 parser: Value was not found.

However, the real problem is that crt file was not found.
Now, for the same problem the exception message will be:
Error while reading file.

Signed-off-by: Lucas Dias <lucas_dias@redes.ufsm.br>
2024-06-09 15:46:34 -03:00
Lucas Dias 304a9992fe make crypto error messages more readable
Add verifications to gnutls calls and throw expcetion
with readable messages.
For example, if call loadIdentity and crt file has name
different of private key exception message was
Can't get certificate public key ID: ASN1 parser: Value was not found.

However, the real problem is that crt file was not found.
Now, for the same problem the exception message will be:
Error while reading file.

Signed-off-by: Lucas Dias <lucas_dias@redes.ufsm.br>
2024-06-09 09:45:21 -03:00
Adrien Beraud ae62216bf4 rust: update to 2021, fix memory issues 2024-06-07 10:29:26 -04:00
Adrien Beraud cd92eeaae5 dht: add support to behave as a client node 2024-06-07 09:52:01 -04:00
Adrien Beraud 9dad7c69fd peer discovery: cleanup 2024-06-07 09:31:48 -04:00
Lanius-collaris 5cc5b98aeb fix android workaround 2024-06-07 09:21:05 -04:00
Lanius-collaris c03ce419ca peer_discovery: add a workaround for Android hotspot 2024-06-07 09:21:05 -04:00
Adrien Béraud ceef3f048c README: add llhttp dependency 2024-06-03 08:32:17 -04:00
Adrien Béraud 8dd926182c ci/github: update before install 2024-06-03 08:26:52 -04:00
Adrien Béraud c8b528b3f4 update version to 3.2.0 2024-05-31 06:58:09 -04:00
Adrien Béraud a65148cd08 ci: build proxy client for Autotools build 2024-05-31 06:58:09 -04:00
Adrien Béraud 5a2a20a32f http: migrate from http-parser to llhttp 2024-05-31 06:58:09 -04:00
Lanius-collaris 318d02c55a update http, dht_proxy_client
http: simplify Connection::async_connect()
dht_proxy_client: add listenKeepIdle setting
2024-05-25 23:57:52 -04:00
Lanius-collaris a69da52f89 update http.h, http.cpp 2024-05-25 23:57:52 -04:00
Lanius-collaris ac0ea6ac57 proxy client: decrease the frequency of keepalive for listen op 2024-05-25 23:57:52 -04:00
Lanius-collaris 88d65a55a7 http: allow to modify keepalive option 2024-05-25 23:57:52 -04:00
Lanius-collaris dffe7d91ee return to 386943de 2024-05-25 23:57:52 -04:00
Lanius-collaris da638b3ae6 cleanup 2024-05-25 23:57:52 -04:00
Lanius-collaris 379568ae0d Update http.cpp 2024-05-25 23:57:52 -04:00
Lanius-collaris 24adfbf2c4 Adjust TCP keepalive options for listen op 2024-05-25 23:57:52 -04:00
François-Simon Fauteux-Chapleau 89179cad4f network_utils: use poll instead of select
select is limited to file descriptor numbers below FD_SETSIZE (1024),
and calling FD_SET with a higher value can cause a buffer overflow. This
can be problematic for applications using OpenDHT as a library.
2024-05-22 11:57:38 -04:00
François-Simon Fauteux-Chapleau f2cee8e9ce thread pool: add function to detach threads 2024-04-23 14:02:32 -04:00
Adrien Béraud 386943de39 tests/proxy: use random port 2024-04-06 23:29:38 -04:00
Adrien Béraud 60b810af7e meson: disable long tests by default 2024-04-06 23:29:35 -04:00
Adrien Béraud aa2c0c2da9 meson: add test for PeerDiscovery 2024-04-06 23:01:41 -04:00
Adrien Béraud 62debfa58a meson: add test prefix for unit tests 2024-04-06 22:59:12 -04:00
Adrien Béraud 9741dba974 update version to 3.1.11 2024-04-06 22:26:56 -04:00
Adrien Béraud 93ac28ead2 meson: configure pkgconfig files 2024-04-06 20:10:33 -04:00
Adrien Beraud a0a3cd3028 proxy: add long test 2024-04-06 19:42:42 -04:00
Adrien Béraud 611ecbd06a meson: add ability to build static or shared 2024-04-06 19:18:27 -04:00
Adrien Béraud 70c465b006 meson: link tests with msgpack 2024-04-06 18:43:36 -04:00
Adrien Beraud 59c8d9b9be op cache: return cache result in onValue 2024-04-06 18:24:16 -04:00
Adrien Beraud 7f70fcd316 proxy client: cancel http request on listen cancel 2024-04-06 18:24:16 -04:00
Adrien Beraud 17e9ad5093 meson: fix build with proxy server 2024-04-06 15:51:11 -04:00
Adrien Beraud 42435bdd4f meson: allow to build unit tests 2024-04-06 15:51:11 -04:00
Adrien Beraud c125a2af1f meson: build thread pool 2024-04-06 15:51:11 -04:00
Andreas Traczyk c0a1356049 dht_proxy_server: use lambda instead of std::bind for listen callback
Avoids issues compiling with MSC.

Also move PushSessionContext definition outside the OPENDHT_PUSH_NOTIFICATIONS condition.
2024-04-04 14:20:01 -04:00
Adrien Béraud 5e6679139b update version to 3.1.10 2024-04-03 14:35:42 -04:00
Andreas Traczyk c118f85886 Merge pull request #694 from savoirfairelinux/push-value-IDs
provide value IDs for push notifications
2024-04-03 11:52:01 -04:00
Andreas Traczyk b40d604ef7 provide value IDs for push notifications 2024-04-02 17:34:16 -04:00
SASANO Takayoshi f630b013ee replace CPPFLAGS += ... (2) 2024-04-02 13:04:52 -04:00
SASANO Takayoshi 36167c3e45 replace CPPFLAGS += ... 2024-04-02 13:04:52 -04:00
52 changed files with 1409 additions and 437 deletions

No files matched your search

+49 -17
View File
@@ -7,18 +7,42 @@ jobs:
name: Ubuntu/GCC Autotools build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Dependencies
run: |
sudo apt update && \
sudo apt install libncurses5-dev libreadline-dev nettle-dev libasio-dev \
libgnutls28-dev libuv1-dev python3-pip python3-dev python3-setuptools libcppunit-dev libjsoncpp-dev \
autotools-dev autoconf libfmt-dev libhttp-parser-dev libmsgpack-dev libargon2-0-dev
autotools-dev autoconf libfmt-dev libmsgpack-dev libargon2-0-dev
- name: Cython
run: sudo pip3 install Cython
- name: llhttp
run: |
mkdir llhttp
wget https://github.com/nodejs/llhttp/archive/refs/tags/release/v9.2.0.tar.gz -O llhttp.tar.gz
tar -xzf llhttp.tar.gz -C llhttp --strip-components=1
cd llhttp && cmake -DCMAKE_INSTALL_PREFIX=/usr . && make -j2 && sudo make install
cd ../ && rm -rf llhttp*
- name: expected-lite
run: |
sudo mkdir /usr/local/include/nonstd && \
sudo wget https://raw.githubusercontent.com/martinmoene/expected-lite/master/include/nonstd/expected.hpp \
-O /usr/local/include/nonstd/expected.hpp
- name: restinio
run: |
mkdir restinio && cd restinio
wget https://github.com/Stiffstream/restinio/releases/download/v.0.7.2/restinio-0.7.2.tar.bz2
ls -l && tar -xjf restinio-0.7.2.tar.bz2
cd restinio-0.7.2/dev
cmake -DCMAKE_INSTALL_PREFIX=/usr/local -DRESTINIO_TEST=Off -DRESTINIO_SAMPLE=Off -DRESTINIO_BENCHMARK=Off \
-DRESTINIO_WITH_SOBJECTIZER=Off -DRESTINIO_DEP_STANDALONE_ASIO=system -DRESTINIO_DEP_LLHTTP=system \
-DRESTINIO_DEP_FMT=system -DRESTINIO_DEP_EXPECTED_LITE=system .
sudo make install
cd ../../ && rm -rf restinio*
- name: Configure
run: |
./autogen.sh
./configure
./configure --enable-proxy-client
- name: Build
run: make
@@ -26,12 +50,13 @@ jobs:
name: Ubuntu/GCC Meson build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Dependencies
run: |
sudo apt update && \
sudo apt install ninja-build libncurses5-dev libreadline-dev nettle-dev libasio-dev \
libgnutls28-dev libuv1-dev python3-dev python3-setuptools python3-pip \
libcppunit-dev libjsoncpp-dev libfmt-dev libhttp-parser-dev libmsgpack-dev libargon2-0-dev
libcppunit-dev libjsoncpp-dev libfmt-dev libmsgpack-dev libargon2-0-dev
- name: Python dependencies
run: sudo pip3 install meson Cython
- name: Configure
@@ -43,9 +68,10 @@ jobs:
name: Ubuntu/GCC minimal build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Dependencies
run: |
sudo apt update && \
sudo apt install libncurses5-dev libreadline-dev nettle-dev libfmt-dev \
libgnutls28-dev libcppunit-dev libmsgpack-dev libargon2-0-dev
- name: Configure
@@ -61,24 +87,30 @@ jobs:
build-macos:
name: macOS/Clang build
runs-on: macos-12
runs-on: macOS-15
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: Dependencies
run: |
brew install msgpack-cxx asio gnutls nettle readline fmt jsoncpp argon2 openssl http-parser cppunit
brew install msgpack-cxx asio gnutls nettle readline fmt jsoncpp argon2 openssl cppunit
- name: expected-lite
run: |
sudo mkdir -p /usr/local/include/nonstd && \
sudo wget https://raw.githubusercontent.com/martinmoene/expected-lite/master/include/nonstd/expected.hpp \
-O /usr/local/include/nonstd/expected.hpp
- name: restinio
run: |
mkdir restinio && cd restinio
wget https://github.com/aberaud/restinio/archive/6fd08b65f6f15899dd0de3c801f6a5462b811c64.tar.gz
ls -l && tar -xzf 6fd08b65f6f15899dd0de3c801f6a5462b811c64.tar.gz
cd restinio-6fd08b65f6f15899dd0de3c801f6a5462b811c64/dev
cmake -DCMAKE_INSTALL_PREFIX=/usr/local -DRESTINIO_TEST=OFF -DRESTINIO_SAMPLE=OFF \
-DRESTINIO_INSTALL_SAMPLES=OFF -DRESTINIO_BENCH=OFF -DRESTINIO_INSTALL_BENCHES=OFF \
-DRESTINIO_FIND_DEPS=ON -DRESTINIO_ALLOW_SOBJECTIZER=Off -DRESTINIO_USE_BOOST_ASIO=none .
make -j8 && sudo make install
cd ../../.. && rm -rf restinio
wget https://github.com/Stiffstream/restinio/releases/download/v.0.7.2/restinio-0.7.2.tar.bz2
ls -l && tar -xjf restinio-0.7.2.tar.bz2
cd restinio-0.7.2/dev
cmake -DCMAKE_INSTALL_PREFIX=/usr/local -DRESTINIO_TEST=Off -DRESTINIO_SAMPLE=Off -DRESTINIO_BENCHMARK=Off \
-DRESTINIO_WITH_SOBJECTIZER=Off -DRESTINIO_DEP_STANDALONE_ASIO=system -DRESTINIO_DEP_LLHTTP=system \
-DRESTINIO_DEP_FMT=system -DRESTINIO_DEP_EXPECTED_LITE=system .
make -j2 && sudo make install
cd ../../ && rm -rf restinio*
- name: Configure
run: |
+18 -11
View File
@@ -6,26 +6,33 @@ jobs:
name: Clang Static Analysis
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
- name: deps
run: |
sudo apt update && \
sudo apt install libncurses5-dev libreadline-dev nettle-dev \
libgnutls28-dev libuv1-dev python3-dev python3-setuptools libcppunit-dev libjsoncpp-dev \
autotools-dev autoconf libfmt-dev libhttp-parser-dev libmsgpack-dev libargon2-0-dev libasio-dev \
autotools-dev autoconf libfmt-dev libmsgpack-dev libargon2-0-dev libasio-dev \
llvm llvm-dev clang clang-tools && \
sudo apt remove gcc g++
- name: expected-lite
run: |
sudo mkdir /usr/include/nonstd && \
sudo wget https://raw.githubusercontent.com/martinmoene/expected-lite/master/include/nonstd/expected.hpp \
-O /usr/include/nonstd/expected.hpp
- name: restinio
run: |
mkdir restinio && cd restinio \
&& wget https://github.com/aberaud/restinio/archive/6fd08b65f6f15899dd0de3c801f6a5462b811c64.tar.gz \
&& ls -l && tar -xzf 6fd08b65f6f15899dd0de3c801f6a5462b811c64.tar.gz \
&& cd restinio-6fd08b65f6f15899dd0de3c801f6a5462b811c64/dev \
&& cmake -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=OFF -DRESTINIO_SAMPLE=OFF \
-DRESTINIO_INSTALL_SAMPLES=OFF -DRESTINIO_BENCH=OFF -DRESTINIO_INSTALL_BENCHES=OFF \
-DRESTINIO_FIND_DEPS=ON -DRESTINIO_ALLOW_SOBJECTIZER=Off -DRESTINIO_USE_BOOST_ASIO=none . \
&& make -j8 && sudo make install \
&& cd ../../.. && rm -rf restinio
mkdir restinio && cd restinio
wget https://github.com/Stiffstream/restinio/releases/download/v.0.7.2/restinio-0.7.2.tar.bz2
ls -l && tar -xjf restinio-0.7.2.tar.bz2
cd restinio-0.7.2/dev
cmake -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=Off -DRESTINIO_SAMPLE=Off -DRESTINIO_BENCHMARK=Off \
-DRESTINIO_WITH_SOBJECTIZER=Off -DRESTINIO_DEP_STANDALONE_ASIO=system -DRESTINIO_DEP_LLHTTP=system \
-DRESTINIO_DEP_FMT=system -DRESTINIO_DEP_EXPECTED_LITE=system .
make -j2 && sudo make install
cd ../../ && rm -rf restinio*
- name: cmake
run: |
+1 -1
View File
@@ -26,7 +26,7 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v3
uses: actions/checkout@v4
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
+31 -31
View File
@@ -25,10 +25,10 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v3
uses: actions/checkout@v4
- name: Log in to the Container registry
uses: docker/login-action@v2
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
@@ -36,12 +36,12 @@ jobs:
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v4
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME_DEPS }}
- name: Build and push Docker image
uses: docker/build-push-action@v3
uses: docker/build-push-action@v5
with:
context: .
file: docker/DockerfileDeps
@@ -58,10 +58,10 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v3
uses: actions/checkout@v4
- name: Log in to the Container registry
uses: docker/login-action@v2
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
@@ -69,13 +69,13 @@ jobs:
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v4
uses: docker/metadata-action@v5
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
- name: Build and push Docker image
uses: docker/build-push-action@v3
uses: docker/build-push-action@v5
with:
context: .
file: docker/Dockerfile
@@ -92,10 +92,10 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v3
uses: actions/checkout@v4
- name: Log in to the Container registry
uses: docker/login-action@v2
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
@@ -103,12 +103,12 @@ jobs:
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v4
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME_DEPS_LLVM }}
- name: Build and push Docker image
uses: docker/build-push-action@v3
uses: docker/build-push-action@v5
with:
context: .
file: docker/DockerfileDepsLlvm
@@ -125,10 +125,10 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v3
uses: actions/checkout@v4
- name: Log in to the Container registry
uses: docker/login-action@v2
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
@@ -136,13 +136,13 @@ jobs:
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v4
uses: docker/metadata-action@v5
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME_LLVM }}
- name: Build and push Docker image
uses: docker/build-push-action@v3
uses: docker/build-push-action@v5
with:
context: .
file: docker/DockerfileLlvm
@@ -159,10 +159,10 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v3
uses: actions/checkout@v4
- name: Log in to the Container registry
uses: docker/login-action@v2
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
@@ -170,13 +170,13 @@ jobs:
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v4
uses: docker/metadata-action@v5
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME_DHTNODE }}
- name: Build and push Docker image
uses: docker/build-push-action@v3
uses: docker/build-push-action@v5
with:
context: .
file: docker/DockerfileDhtnode
@@ -193,10 +193,10 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v3
uses: actions/checkout@v4
- name: Log in to the Container registry
uses: docker/login-action@v2
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
@@ -204,13 +204,13 @@ jobs:
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v4
uses: docker/metadata-action@v5
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME_ALPINE_DEPS }}
- name: Build and push Docker image
uses: docker/build-push-action@v3
uses: docker/build-push-action@v5
with:
context: .
file: docker/DockerfileDepsAlpine
@@ -227,10 +227,10 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v3
uses: actions/checkout@v4
- name: Log in to the Container registry
uses: docker/login-action@v2
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
@@ -238,13 +238,13 @@ jobs:
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v4
uses: docker/metadata-action@v5
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME_ALPINE }}
- name: Build and push Docker image
uses: docker/build-push-action@v3
uses: docker/build-push-action@v5
with:
context: .
file: docker/DockerfileAlpine
@@ -262,7 +262,7 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@v3
uses: actions/checkout@v4
- name: cmake
run: |
@@ -280,14 +280,14 @@ jobs:
- name: build
run: cd build && make dist
- uses: actions/upload-artifact@v3
- uses: actions/upload-artifact@v4
with:
name: opendht-wheels-linux
path: build/python/dist/*.whl
- name: Create Release
id: create_release
uses: softprops/action-gh-release@v1
uses: softprops/action-gh-release@v2
with:
draft: true
files: build/python/dist/*.whl
+49 -5
View File
@@ -1,4 +1,7 @@
cmake_minimum_required (VERSION 3.16..3.28)
if(POLICY CMP0135)
cmake_policy(SET CMP0135 NEW)
endif()
project (opendht)
include(CMakePackageConfigHelpers)
@@ -7,9 +10,10 @@ include(CheckIncludeFileCXX)
include(FindPkgConfig)
include(cmake/CheckAtomic.cmake)
include(CTest)
include(FetchContent)
set (opendht_VERSION_MAJOR 3)
set (opendht_VERSION_MINOR 1.9)
set (opendht_VERSION_MINOR 2.0)
set (opendht_VERSION ${opendht_VERSION_MAJOR}.${opendht_VERSION_MINOR})
set (PACKAGE_VERSION ${opendht_VERSION})
set (VERSION "${opendht_VERSION}")
@@ -81,15 +85,44 @@ if (NOT MSVC)
if (OPENDHT_HTTP OR OPENDHT_PEER_DISCOVERY)
find_path(ASIO_INCLUDE_DIR asio.hpp REQUIRED)
message(STATUS "Found ASIO ${ASIO_INCLUDE_DIR}")
else()
message(STATUS "ASIO not required")
endif ()
find_package(fmt)
if (OPENDHT_HTTP)
find_package(Restinio REQUIRED)
find_library(HTTP_PARSER_LIBRARY http_parser)
add_library(http_parser SHARED IMPORTED)
set(http_parser_lib "-lhttp_parser")
# llhttp
find_path(LLHTTP_INCLUDE_DIR llhttp.h)
find_library(LLHTTP_LIBRARY NAMES llhttp libllhttp)
if (LLHTTP_INCLUDE_DIR AND LLHTTP_LIBRARY)
message(STATUS "Found llhttp ${LLHTTP_INCLUDE_DIR} ${LLHTTP_LIBRARY}")
add_library(llhttp_static STATIC IMPORTED)
set_target_properties(llhttp_static PROPERTIES
IMPORTED_LOCATION ${LLHTTP_LIBRARY}
INTERFACE_INCLUDE_DIRECTORIES ${LLHTTP_INCLUDE_DIR}
)
set(llhttp_target llhttp_static)
else()
FetchContent_Declare(llhttp-local URL "https://github.com/nodejs/llhttp/archive/refs/tags/release/v9.2.0.tar.gz")
if (BUILD_SHARED_LIBS)
set(BUILD_SHARED_LIBS ON CACHE INTERNAL "")
else()
set(BUILD_SHARED_LIBS OFF CACHE INTERNAL "")
set(BUILD_STATIC_LIBS ON CACHE INTERNAL "")
endif()
FetchContent_MakeAvailable(llhttp-local)
if (BUILD_SHARED_LIBS)
set(llhttp_target llhttp_shared)
else()
set(llhttp_target llhttp_static)
endif()
endif()
set(http_lib "-lllhttp")
if (NOT Jsoncpp_FOUND)
message(SEND_ERROR "Jsoncpp is required for DHT proxy support")
endif()
@@ -130,6 +163,9 @@ endif ()
if (OPENDHT_HTTP OR OPENDHT_PEER_DISCOVERY)
add_definitions(-DASIO_STANDALONE)
if (UNIX AND NOT APPLE)
pkg_search_module(liburing IMPORTED_TARGET liburing)
endif ()
endif()
if (NOT MSVC)
@@ -204,6 +240,7 @@ list (APPEND opendht_SOURCES
)
list (APPEND opendht_HEADERS
include/opendht.h
include/opendht/def.h
include/opendht/utils.h
include/opendht/sockaddr.h
@@ -356,18 +393,25 @@ else()
PRIVATE
PkgConfig::argon2
PkgConfig::Nettle
${HTTP_PARSER_LIBRARY}
PUBLIC
${CMAKE_THREAD_LIBS_INIT}
PkgConfig::GnuTLS
fmt::fmt
)
if (OPENDHT_HTTP)
target_link_libraries(opendht PUBLIC ${llhttp_target})
endif()
if (Jsoncpp_FOUND)
target_link_libraries(opendht PUBLIC PkgConfig::Jsoncpp)
endif()
if (OPENDHT_PROXY_OPENSSL)
target_link_libraries(opendht PUBLIC PkgConfig::OPENSSL)
endif()
if (liburing_FOUND)
set(iouring_lib ", liburing")
target_link_libraries(opendht PUBLIC PkgConfig::liburing)
target_compile_definitions(opendht PUBLIC ASIO_HAS_IO_URING ASIO_DISABLE_EPOLL)
endif()
endif()
if (BUILD_SHARED_LIBS)
+1
View File
@@ -98,6 +98,7 @@ for r in results:
- Nettle 2.4+, a GnuTLS dependency for crypto.
- {fmt} 9.0+, for log formatting.
- (optional) restinio used for the REST API.
- (optional) llhttp used for the REST API.
- (optional) jsoncpp 1.7.4-3+, used for the REST API.
- Build tested with GCC 7+ (GNU/Linux, Windows with MinGW), Clang/LLVM (GNU/Linux, Android, macOS, iOS).
- Build tested with Microsoft Visual Studio 2019, 2022
+1
View File
@@ -344,6 +344,7 @@ int dht_runner_run_config(dht_runner* r, in_port_t port, const dht_runner_config
config.push_token = conf->push_token ? std::string(conf->push_token) : std::string{};
config.push_topic = conf->push_topic ? std::string(conf->push_topic) : std::string{};
config.push_platform = conf->push_platform ? std::string(conf->push_platform) : std::string{};
config.proxy_user_agent = conf->proxy_user_agent ? std::string(conf->proxy_user_agent) : std::string{};
config.peer_discovery = conf->peer_discovery;
config.peer_publish = conf->peer_publish;
+1
View File
@@ -148,6 +148,7 @@ struct OPENDHT_PUBLIC dht_runner_config {
const char* push_token;
const char* push_topic;
const char* push_platform;
const char* proxy_user_agent;
bool peer_discovery;
bool peer_publish;
dht_certificate* server_ca;
+12 -12
View File
@@ -1,7 +1,7 @@
dnl define macros
m4_define([opendht_major_version], 3)
m4_define([opendht_minor_version], 1)
m4_define([opendht_patch_version], 9)
m4_define([opendht_minor_version], 2)
m4_define([opendht_patch_version], 0)
m4_define([opendht_version],
[opendht_major_version.opendht_minor_version.opendht_patch_version])
@@ -21,10 +21,10 @@ AS_IF([test "x$enable_debug" = "xyes"],
[CXXFLAGS="${CXXFLAGS} -g -Og -Wno-return-type -Wall -Wextra -Wnon-virtual-dtor -Wno-deprecated -pedantic-errors"],
[CXXFLAGS="${CXXFLAGS} -O3 -Wno-deprecated -pedantic-errors -fvisibility=hidden"])
CPPFLAGS+=" -DOPENDHT_BUILD"
CPPFLAGS="${CPPFLAGS} -DOPENDHT_BUILD"
AM_CONDITIONAL([OPENDHT_SHARED], [test "x$enable_shared" != xno])
AM_COND_IF(OPENDHT_SHARED, [
CPPFLAGS+=" -Dopendht_EXPORTS"
CPPFLAGS="${CPPFLAGS} -Dopendht_EXPORTS"
])
AM_PROG_AR
@@ -147,7 +147,7 @@ AS_IF([test "x$with_jsoncpp" != "xno"],
[have_jsoncpp=no])
AS_IF([test "x$have_jsoncpp" = "xyes"], [
AC_MSG_NOTICE([Using JsonCpp])
CPPFLAGS+=" -DOPENDHT_JSONCPP"
CPPFLAGS="${CPPFLAGS} -DOPENDHT_JSONCPP"
AC_SUBST(jsoncpp_lib, [", jsoncpp"])
], [
AC_MSG_NOTICE([Not using JsonCpp])
@@ -175,11 +175,11 @@ AS_IF([test "x$have_fmt" = "xyes"], [
AM_COND_IF([PROXY_CLIENT_OR_SERVER], [
AC_CHECK_HEADERS([asio.hpp],, AC_MSG_ERROR([Missing Asio headers files]))
CXXFLAGS="${CXXFLAGS} -DASIO_STANDALONE"
# http_parser has no pkgconfig, instead we check with:
AC_CHECK_LIB(http_parser, exit,, AC_MSG_ERROR([Missing HttpParser library files]))
AC_CHECK_HEADERS([http_parser.h], [http_parser_headers=yes; break;])
AC_SUBST(http_parser_lib, ["-lhttp_parser"])
AS_IF([test "x$http_parser_headers" != "xyes"], AC_MSG_ERROR([Missing HttpParser headers files]))
# llhttp has no pkgconfig, instead we check with:
AC_CHECK_LIB(llhttp, exit,, AC_MSG_ERROR([Missing llhttp library files]))
AC_CHECK_HEADERS([llhttp.h], [llhttp_headers=yes; break;])
AC_SUBST(http_lib, ["-lllhttp"])
AS_IF([test "x$llhttp_headers" != "xyes"], AC_MSG_ERROR([Missing llhttp headers files]))
])
CXXFLAGS="${CXXFLAGS} -DMSGPACK_NO_BOOST -DMSGPACK_DISABLE_LEGACY_NIL -DMSGPACK_DISABLE_LEGACY_CONVERT"
@@ -194,8 +194,8 @@ AM_COND_IF([ENABLE_TOOLS], [
AM_COND_IF(ENABLE_PROXY_SERVER, AC_DEFINE([OPENDHT_PROXY_SERVER], [], [Building with proxy server]))
AM_COND_IF(ENABLE_PROXY_CLIENT, AC_DEFINE([OPENDHT_PROXY_CLIENT], [], [Building with proxy client]))
AM_COND_IF(ENABLE_PUSH_NOTIFICATIONS, [CPPFLAGS+=" -DOPENDHT_PUSH_NOTIFICATIONS"], [])
AM_COND_IF(ENABLE_PROXY_SERVER_IDENTITY, [CPPFLAGS+=" -DOPENDHT_PROXY_SERVER_IDENTITY"], [])
AM_COND_IF(ENABLE_PUSH_NOTIFICATIONS, [CPPFLAGS="${CPPFLAGS} -DOPENDHT_PUSH_NOTIFICATIONS"], [])
AM_COND_IF(ENABLE_PROXY_SERVER_IDENTITY, [CPPFLAGS="${CPPFLAGS} -DOPENDHT_PROXY_SERVER_IDENTITY"], [])
dnl Configure setup.py if we build the python module
AC_SUBST(CURRENT_SOURCE_DIR, ".")
+2 -2
View File
@@ -19,7 +19,7 @@ RUN cd opendht && mkdir build && cd build \
-DOPENDHT_SYSTEMD=On \
&& make -j8 && make install
FROM alpine:3.18 AS install
FROM alpine:3.19 AS install
COPY --from=build /install /
RUN apk add --no-cache \
libstdc++ \
@@ -28,8 +28,8 @@ RUN apk add --no-cache \
openssl \
argon2-dev \
jsoncpp \
llhttp \
fmt \
http-parser \
readline \
ncurses
CMD ["dhtnode", "-b", "bootstrap.jami.net", "-p", "4222", "--proxyserver", "8080"]
+13 -9
View File
@@ -13,18 +13,22 @@ RUN apt-get update && apt-get install -y \
python3-pip python3-dev python3-setuptools python3-build python3-virtualenv \
libncurses5-dev libreadline-dev nettle-dev libcppunit-dev \
libgnutls28-dev libuv1-dev libjsoncpp-dev libargon2-dev \
libssl-dev libfmt-dev libhttp-parser-dev libasio-dev libmsgpack-dev \
libssl-dev libfmt-dev libasio-dev libmsgpack-dev \
&& apt-get clean && rm -rf /var/lib/apt/lists/* /var/cache/apt/*
RUN pip3 install meson Cython
RUN mkdir /usr/include/nonstd \
&& wget https://raw.githubusercontent.com/martinmoene/expected-lite/master/include/nonstd/expected.hpp \
-O /usr/include/nonstd/expected.hpp
RUN echo "*** Downloading RESTinio ***" \
&& mkdir restinio && cd restinio \
&& wget https://github.com/aberaud/restinio/archive/6fd08b65f6f15899dd0de3c801f6a5462b811c64.tar.gz \
&& ls -l && tar -xzf 6fd08b65f6f15899dd0de3c801f6a5462b811c64.tar.gz \
&& cd restinio-6fd08b65f6f15899dd0de3c801f6a5462b811c64/dev \
&& cmake -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=OFF -DRESTINIO_SAMPLE=OFF \
-DRESTINIO_INSTALL_SAMPLES=OFF -DRESTINIO_BENCH=OFF -DRESTINIO_INSTALL_BENCHES=OFF \
-DRESTINIO_FIND_DEPS=ON -DRESTINIO_ALLOW_SOBJECTIZER=Off -DRESTINIO_USE_BOOST_ASIO=none . \
&& make -j8 && make install \
&& cd ../../.. && rm -rf restinio
&& wget https://github.com/Stiffstream/restinio/releases/download/v.0.7.2/restinio-0.7.2.tar.bz2 \
&& ls -l && tar -xjf restinio-0.7.2.tar.bz2 \
&& cd restinio-0.7.2/dev \
&& cmake -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=Off -DRESTINIO_SAMPLE=Off -DRESTINIO_BENCHMARK=Off \
-DRESTINIO_WITH_SOBJECTIZER=Off -DRESTINIO_DEP_STANDALONE_ASIO=system -DRESTINIO_DEP_LLHTTP=system \
-DRESTINIO_DEP_FMT=system -DRESTINIO_DEP_EXPECTED_LITE=system . \
&& make -j2 && make install \
&& cd ../../ && rm -rf restinio*
+14 -10
View File
@@ -1,4 +1,4 @@
FROM alpine:3.18
FROM alpine:3.19
LABEL maintainer="Adrien Béraud <adrien.beraud@savoirfairelinux.com>"
LABEL org.opencontainers.image.source https://github.com/savoirfairelinux/opendht
@@ -8,16 +8,20 @@ RUN apk add --no-cache \
ncurses-dev readline-dev nettle-dev \
cppunit-dev gnutls-dev jsoncpp-dev \
argon2-dev openssl-dev fmt-dev \
http-parser-dev asio-dev msgpack-cxx-dev \
llhttp-dev asio-dev msgpack-cxx-dev \
&& rm -rf /var/cache/apk/*
RUN mkdir /usr/include/nonstd \
&& wget https://raw.githubusercontent.com/martinmoene/expected-lite/master/include/nonstd/expected.hpp \
-O /usr/include/nonstd/expected.hpp
RUN echo "*** Downloading RESTinio ***" \
&& mkdir restinio && cd restinio \
&& wget https://github.com/aberaud/restinio/archive/6fd08b65f6f15899dd0de3c801f6a5462b811c64.tar.gz \
&& tar -xzf 6fd08b65f6f15899dd0de3c801f6a5462b811c64.tar.gz \
&& cd restinio-6fd08b65f6f15899dd0de3c801f6a5462b811c64/dev \
&& cmake -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=OFF -DRESTINIO_SAMPLE=OFF \
-DRESTINIO_INSTALL_SAMPLES=OFF -DRESTINIO_BENCH=OFF -DRESTINIO_INSTALL_BENCHES=OFF \
-DRESTINIO_FIND_DEPS=ON -DRESTINIO_ALLOW_SOBJECTIZER=Off -DRESTINIO_USE_BOOST_ASIO=none . \
&& make -j8 && make install \
&& cd ../../.. && rm -rf restinio
&& wget https://github.com/Stiffstream/restinio/releases/download/v.0.7.2/restinio-0.7.2.tar.bz2 \
&& ls -l && tar -xjf restinio-0.7.2.tar.bz2 \
&& cd restinio-0.7.2/dev \
&& cmake -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=Off -DRESTINIO_SAMPLE=Off -DRESTINIO_BENCHMARK=Off \
-DRESTINIO_WITH_SOBJECTIZER=Off -DRESTINIO_DEP_STANDALONE_ASIO=system -DRESTINIO_DEP_LLHTTP=system \
-DRESTINIO_DEP_FMT=system -DRESTINIO_DEP_EXPECTED_LITE=system . \
&& make -j2 && make install \
&& cd ../../ && rm -rf restinio*
+12 -8
View File
@@ -7,7 +7,7 @@ RUN apt-get update && apt-get install -y \
apt-transport-https build-essential pkg-config git wget libncurses5-dev libreadline-dev nettle-dev \
libgnutls28-dev libuv1-dev python3-dev python3-setuptools libcppunit-dev libjsoncpp-dev \
libargon2-0-dev \
autotools-dev autoconf libfmt-dev libhttp-parser-dev libmsgpack-dev libssl-dev python3-pip \
autotools-dev autoconf libfmt-dev libmsgpack-dev libssl-dev python3-pip \
&& apt-get clean && rm -rf /var/lib/apt/lists/* /var/cache/apt/*
RUN pip3 install --upgrade cmake meson Cython
@@ -20,13 +20,17 @@ RUN echo "** Building a recent version of asio ***" \
&& make install \
&& cd ../../ && rm -rf asio*
RUN mkdir /usr/include/nonstd \
&& wget https://raw.githubusercontent.com/martinmoene/expected-lite/master/include/nonstd/expected.hpp \
-O /usr/include/nonstd/expected.hpp
RUN echo "*** Downloading RESTinio ***" \
&& mkdir restinio && cd restinio \
&& wget https://github.com/aberaud/restinio/archive/8d5d3e8237e0947adb9ba1ffc8281f4ad7cb2a59.tar.gz \
&& ls -l && tar -xzf 8d5d3e8237e0947adb9ba1ffc8281f4ad7cb2a59.tar.gz \
&& cd restinio-8d5d3e8237e0947adb9ba1ffc8281f4ad7cb2a59/dev \
&& cmake -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=OFF -DRESTINIO_SAMPLE=OFF \
-DRESTINIO_INSTALL_SAMPLES=OFF -DRESTINIO_BENCH=OFF -DRESTINIO_INSTALL_BENCHES=OFF \
-DRESTINIO_FIND_DEPS=ON -DRESTINIO_ALLOW_SOBJECTIZER=Off -DRESTINIO_USE_BOOST_ASIO=none . \
&& make -j8 && make install \
&& wget https://github.com/Stiffstream/restinio/releases/download/v.0.7.2/restinio-0.7.2.tar.bz2 \
&& ls -l && tar -xjf restinio-0.7.2.tar.bz2 \
&& cd restinio-0.7.2/dev \
&& cmake -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=Off -DRESTINIO_SAMPLE=Off -DRESTINIO_BENCHMARK=Off \
-DRESTINIO_WITH_SOBJECTIZER=Off -DRESTINIO_DEP_STANDALONE_ASIO=system -DRESTINIO_DEP_LLHTTP=system \
-DRESTINIO_DEP_FMT=system -DRESTINIO_DEP_EXPECTED_LITE=system . \
&& make -j2 && make install \
&& cd ../../ && rm -rf restinio*
+13 -9
View File
@@ -13,18 +13,22 @@ RUN apt-get update && apt-get install -y \
python3-dev python3-setuptools python3-pip \
libncurses5-dev libreadline-dev nettle-dev libcppunit-dev \
libgnutls28-dev libuv1-dev libjsoncpp-dev libargon2-dev \
libssl-dev libfmt-dev libhttp-parser-dev libasio-dev libmsgpack-dev \
libssl-dev libfmt-dev libasio-dev libmsgpack-dev \
&& apt-get clean && rm -rf /var/lib/apt/lists/* /var/cache/apt/*
RUN pip3 install meson Cython
RUN mkdir /usr/include/nonstd \
&& wget https://raw.githubusercontent.com/martinmoene/expected-lite/master/include/nonstd/expected.hpp \
-O /usr/include/nonstd/expected.hpp
RUN echo "*** Downloading RESTinio ***" \
&& mkdir restinio && cd restinio \
&& wget https://github.com/aberaud/restinio/archive/e0a261dd8488246a3cb8bbb3ea781ea5139c3c94.tar.gz \
&& ls -l && tar -xzf e0a261dd8488246a3cb8bbb3ea781ea5139c3c94.tar.gz \
&& cd restinio-e0a261dd8488246a3cb8bbb3ea781ea5139c3c94/dev \
&& cmake -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=OFF -DRESTINIO_SAMPLE=OFF \
-DRESTINIO_INSTALL_SAMPLES=OFF -DRESTINIO_BENCH=OFF -DRESTINIO_INSTALL_BENCHES=OFF \
-DRESTINIO_FIND_DEPS=ON -DRESTINIO_ALLOW_SOBJECTIZER=Off -DRESTINIO_USE_BOOST_ASIO=none . \
&& make -j8 && make install \
&& cd ../../.. && rm -rf restinio
&& wget https://github.com/Stiffstream/restinio/releases/download/v.0.7.2/restinio-0.7.2.tar.bz2 \
&& ls -l && tar -xjf restinio-0.7.2.tar.bz2 \
&& cd restinio-0.7.2/dev \
&& cmake -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=Off -DRESTINIO_SAMPLE=Off -DRESTINIO_BENCHMARK=Off \
-DRESTINIO_WITH_SOBJECTIZER=Off -DRESTINIO_DEP_STANDALONE_ASIO=system -DRESTINIO_DEP_LLHTTP=system \
-DRESTINIO_DEP_FMT=system -DRESTINIO_DEP_EXPECTED_LITE=system . \
&& make -j2 && make install \
&& cd ../../ && rm -rf restinio*
+12 -8
View File
@@ -13,7 +13,7 @@ RUN apt-get update \
nettle-dev libgnutls28-dev libuv1-dev libmsgpack-dev libjsoncpp-dev python3-dev \
python3-setuptools libcppunit-dev python3-pip python3-build python3-virtualenv \
autotools-dev autoconf libssl-dev libargon2-dev \
libfmt-dev libhttp-parser-dev libasio-dev \
libfmt-dev libasio-dev \
&& apt-get remove -y gcc g++ && apt-get autoremove -y \
&& apt-get clean && rm -rf /var/lib/apt/lists/* /var/cache/apt/*
@@ -22,13 +22,17 @@ RUN pip3 install meson Cython
ENV CC cc
ENV CXX c++
RUN mkdir /usr/include/nonstd \
&& wget https://raw.githubusercontent.com/martinmoene/expected-lite/master/include/nonstd/expected.hpp \
-O /usr/include/nonstd/expected.hpp
RUN echo "*** Downloading RESTinio ***" \
&& mkdir restinio && cd restinio \
&& wget https://github.com/aberaud/restinio/archive/6fd08b65f6f15899dd0de3c801f6a5462b811c64.tar.gz \
&& ls -l && tar -xzf 6fd08b65f6f15899dd0de3c801f6a5462b811c64.tar.gz \
&& cd restinio-6fd08b65f6f15899dd0de3c801f6a5462b811c64/dev \
&& cmake -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=OFF -DRESTINIO_SAMPLE=OFF \
-DRESTINIO_INSTALL_SAMPLES=OFF -DRESTINIO_BENCH=OFF -DRESTINIO_INSTALL_BENCHES=OFF \
-DRESTINIO_FIND_DEPS=ON -DRESTINIO_ALLOW_SOBJECTIZER=Off -DRESTINIO_USE_BOOST_ASIO=none . \
&& make -j8 && make install \
&& wget https://github.com/Stiffstream/restinio/releases/download/v.0.7.2/restinio-0.7.2.tar.bz2 \
&& ls -l && tar -xjf restinio-0.7.2.tar.bz2 \
&& cd restinio-0.7.2/dev \
&& cmake -DCMAKE_INSTALL_PREFIX=/usr -DRESTINIO_TEST=Off -DRESTINIO_SAMPLE=Off -DRESTINIO_BENCHMARK=Off \
-DRESTINIO_WITH_SOBJECTIZER=Off -DRESTINIO_DEP_STANDALONE_ASIO=system -DRESTINIO_DEP_LLHTTP=system \
-DRESTINIO_DEP_FMT=system -DRESTINIO_DEP_EXPECTED_LITE=system . \
&& make -j2 && make install \
&& cd ../../ && rm -rf restinio*
+3
View File
@@ -143,6 +143,9 @@ struct OPENDHT_PUBLIC Config {
* - Larger listen refresh time
*/
bool public_stable {false};
/* Client mode, node will not be used by other nodes to store data. */
bool client_mode {false};
};
/**
+23 -11
View File
@@ -53,19 +53,21 @@ public:
DhtProxyClient();
explicit DhtProxyClient(
std::shared_ptr<crypto::Certificate> serverCA, crypto::Identity clientIdentity,
std::function<void()> loopSignal, const std::string& serverHost,
const std::string& pushClientId = "", std::shared_ptr<Logger> logger = {});
std::shared_ptr<crypto::Certificate> serverCA,
crypto::Identity clientIdentity,
std::function<void()> loopSignal,
const std::string& serverHost,
const std::string& userAgent = "",
const std::string& pushClientId = "",
const std::string& pushToken = "",
const std::string& pushTopic = "",
const std::string& pushPlatform = "",
std::shared_ptr<Logger> logger = {}
);
void setHeaderFields(http::Request& request);
virtual void setPushNotificationToken(const std::string& token) override {
#ifdef OPENDHT_PUSH_NOTIFICATIONS
deviceKey_ = token;
#else
(void) token;
#endif
}
virtual void setPushNotificationToken(const std::string& token) override;
virtual void setPushNotificationTopic(const std::string& topic) override {
#ifdef OPENDHT_PUSH_NOTIFICATIONS
@@ -291,6 +293,11 @@ public:
loopSignal_();
}
void listenKeepIdle(uint32_t seconds) {
listenKeepIdle_ = seconds;
}
inline uint32_t listenKeepIdle() { return listenKeepIdle_; }
private:
/**
* Start the connection with a server.
@@ -346,7 +353,7 @@ private:
std::string proxyUrl_;
dht::crypto::Identity clientIdentity_;
std::shared_ptr<dht::crypto::Certificate> serverCertificate_;
//std::pair<std::string, std::string> serverHostService_;
std::string userAgent_ {"OpenDHT"};
std::string pushClientId_;
std::string pushSessionId_;
@@ -391,6 +398,11 @@ private:
size_t listenerToken_ {0};
std::map<InfoHash, ProxySearch> searches_;
/**
* Keepalive idle time for listen
*/
uint32_t listenKeepIdle_ {120};
/**
* Callbacks should be executed in the main thread.
*/
+80 -14
View File
@@ -46,17 +46,17 @@ enum class PushType {
}
MSGPACK_ADD_ENUM(dht::PushType)
namespace http {
class Request;
struct ListenerSession;
}
namespace Json {
class Value;
}
namespace dht {
namespace http {
class Request;
struct ListenerSession;
}
class DhtRunner;
using RestRouter = restinio::router::express_router_t<>;
@@ -98,6 +98,32 @@ public:
asio::io_context& io_context() const;
using clock = std::chrono::steady_clock;
using time_point = clock::time_point;
struct PushStats {
uint64_t highPriorityCount {0};
uint64_t normalPriorityCount {0};
void increment(bool highPriority) {
if (highPriority)
highPriorityCount++;
else
normalPriorityCount++;
}
Json::Value toJson() const {
Json::Value val;
val["highPriorityCount"] = static_cast<Json::UInt64>(highPriorityCount);
val["normalPriorityCount"] = static_cast<Json::UInt64>(normalPriorityCount);
return val;
}
std::string toString() const {
return fmt::format("{} high priority, {} normal priority", highPriorityCount, normalPriorityCount);
}
};
struct ServerStats {
/** Current number of listen operations */
size_t listenCount {0};
@@ -107,6 +133,17 @@ public:
size_t totalPermanentPuts {0};
/** Current number of push tokens with at least one listen operation */
size_t pushListenersCount {0};
/** Time at which the server was started */
time_point serverStartTime;
/** Last time at which the stats were updated */
time_point lastUpdated;
/** Total number of push notification requests that the server attempted to
* send since being started, broken down by type and priority level */
PushStats androidPush;
PushStats iosPush;
PushStats unifiedPush;
/** Average requests per second */
double requestRate {0};
/** Node Info **/
@@ -115,6 +152,10 @@ public:
std::string toString() const {
std::ostringstream ss;
ss << "Listens: " << listenCount << " Puts: " << putCount << " PushListeners: " << pushListenersCount << std::endl;
ss << "Push requests in the last " << print_duration(lastUpdated - serverStartTime) << ": "
<< "[Android: " << androidPush.toString() << "], "
<< "[iOS: " << iosPush.toString() << "], "
<< "[Unified: " << unifiedPush.toString() << "]" << std::endl;
ss << "Requests: " << requestRate << " per second." << std::endl;
if (nodeInfo) {
auto& ipv4 = nodeInfo->ipv4;
@@ -136,6 +177,11 @@ public:
result["putCount"] = static_cast<Json::UInt64>(putCount);
result["totalPermanentPuts"] = static_cast<Json::UInt64>(totalPermanentPuts);
result["pushListenersCount"] = static_cast<Json::UInt64>(pushListenersCount);
result["serverStartTime"] = static_cast<Json::LargestInt>(to_time_t(serverStartTime));
result["lastUpdated"] = static_cast<Json::LargestInt>(to_time_t(lastUpdated));
result["androidPush"] = androidPush.toJson();
result["iosPush"] = iosPush.toJson();
result["unifiedPush"] = unifiedPush.toJson();
result["requestRate"] = requestRate;
if (nodeInfo)
result["nodeInfo"] = nodeInfo->toJson();
@@ -275,6 +321,12 @@ private:
RequestStatus options(restinio::request_handle_t request,
restinio::router::route_params_t params);
struct PushSessionContext {
std::mutex lock;
std::string sessionId;
PushSessionContext(const std::string& id) : sessionId(id) {}
};
#ifdef OPENDHT_PUSH_NOTIFICATIONS
PushType getTypeFromString(const std::string& type);
std::string getDefaultTopic(PushType type);
@@ -331,6 +383,24 @@ private:
void handleCancelPushListen(const asio::error_code &ec, const std::string pushToken,
const InfoHash key, const std::string clientId);
/**
* Handles a push listen request.
*
* @param infoHash The information hash associated with the push listen request.
* @param pushToken The push token associated with the push listen request.
* @param type The type of the push listen request.
* @param clientId The client ID associated with the push listen request.
* @param sessionCtx The shared pointer to the push session context associated with the push listen request.
* @param topic The topic associated with the push listen request.
* @param values The vector of shared pointers to values associated with the push listen request.
* @param expired A boolean indicating whether the push listen request has expired.
* @return true.
*/
bool handlePushListen(const InfoHash& infoHash, const std::string& pushToken,
PushType type, const std::string& clientId,
const std::shared_ptr<DhtProxyServer::PushSessionContext>& sessionCtx, const std::string& topic,
const std::vector<std::shared_ptr<Value>>& values, bool expired);
#endif //OPENDHT_PUSH_NOTIFICATIONS
void handlePrintStats(const asio::error_code &ec);
@@ -342,9 +412,6 @@ private:
template <typename Is>
void loadState(Is& is, size_t size);
using clock = std::chrono::steady_clock;
using time_point = clock::time_point;
std::shared_ptr<asio::io_context> ioContext_;
std::shared_ptr<DhtRunner> dht_;
Json::StreamWriterBuilder jsonBuilder_;
@@ -369,6 +436,11 @@ private:
std::shared_ptr<ServerStats> stats_;
std::shared_ptr<NodeInfo> nodeInfo_ {};
std::unique_ptr<asio::steady_timer> printStatsTimer_;
const time_point serverStartTime_;
mutable std::mutex pushStatsMutex_;
PushStats androidPush_;
PushStats iosPush_;
PushStats unifiedPush_;
// Thread-safe access to listeners map.
std::mutex lockListener_;
@@ -376,12 +448,6 @@ private:
std::map<restinio::connection_id_t, http::ListenerSession> listeners_;
// Connection Listener observing conn state changes.
std::shared_ptr<ConnectionListener> connListener_;
struct PushSessionContext {
std::mutex lock;
std::string sessionId;
PushSessionContext(const std::string& id) : sessionId(id) {}
};
struct PermanentPut {
time_point expiration;
std::string pushToken;
+5 -2
View File
@@ -64,6 +64,7 @@ public:
std::string push_token {};
std::string push_topic {};
std::string push_platform {};
std::string proxy_user_agent {};
bool peer_discovery {false};
bool peer_publish {false};
std::shared_ptr<dht::crypto::Certificate> server_ca;
@@ -409,7 +410,8 @@ public:
void run(const Config& config, Context&& context);
void setOnStatusChanged(StatusCallback&& cb) {
statusCb = std::move(cb);
if (cb)
statusCbs.emplace_back(std::move(cb));
}
/**
@@ -519,7 +521,8 @@ private:
NodeStatus status4 {NodeStatus::Disconnected},
status6 {NodeStatus::Disconnected};
StatusCallback statusCb {nullptr};
std::vector<StatusCallback> statusCbs {};
/** PeerDiscovery Parameters */
std::shared_ptr<PeerDiscovery> peerDiscovery_;
+8 -20
View File
@@ -29,23 +29,20 @@
# define snprintf snprintf
#endif
#include <asio/ip/tcp.hpp>
#include <asio/streambuf.hpp>
#include <asio/ssl/context.hpp>
#include <restinio/http_headers.hpp>
#include <restinio/message_builders.hpp>
#include <memory>
#include <queue>
#include <mutex>
#include <future>
namespace Json {
class Value;
}
extern "C" {
struct http_parser;
struct http_parser_settings;
}
namespace restinio {
namespace impl {
class tls_socket_t;
@@ -116,6 +113,8 @@ public:
void async_read(size_t bytes, BytesHandlerCb cb);
void async_read_some(size_t bytes, BytesHandlerCb cb);
void set_keepalive(uint32_t seconds);
const asio::ip::address& local_address() const;
void timeout(const std::chrono::seconds& timeout, HandlerCb cb = {});
@@ -154,17 +153,6 @@ private:
bool checkOcsp_ {false};
};
/**
* Session value associated with a connection_id_t key.
*/
struct ListenerSession
{
ListenerSession() = default;
dht::InfoHash hash;
std::future<size_t> token;
std::shared_ptr<restinio::response_builder_t<restinio::chunked_output_t>> response;
};
/* @class Resolver
* @brief The purpose is to only resolve once to avoid mutliple dns requests per operation.
*/
@@ -336,7 +324,7 @@ private:
void init_default_headers();
/**
* Initialized and wraps the http_parser callbacks with our user defined callbacks.
* Initialized and wraps the llhttp callbacks with our user defined callbacks.
*/
void init_parser();
@@ -378,8 +366,8 @@ private:
Response response_ {};
std::string request_;
std::atomic<bool> finishing_ {false};
std::unique_ptr<http_parser> parser_;
std::unique_ptr<http_parser_settings> parser_s_;
std::unique_ptr<llhttp_t> parser_;
std::unique_ptr<llhttp_settings_t> parser_s_;
// Next request in case of redirect following
std::shared_ptr<Request> next_;
+1
View File
@@ -52,6 +52,7 @@ struct NetworkConfig {
NetId network {0};
ssize_t max_req_per_sec {0};
ssize_t max_peer_req_per_sec {0};
bool is_client {false};
};
class DhtProtocolException : public DhtException {
+21 -14
View File
@@ -23,7 +23,9 @@
#include "sockaddr.h"
#include "infohash.h"
#include "logger.h"
#include "utils.h"
#include <asio/steady_timer.hpp>
#include <thread>
namespace asio {
@@ -36,31 +38,34 @@ class OPENDHT_PUBLIC PeerDiscovery
{
public:
static constexpr in_port_t DEFAULT_PORT = 8888;
using ServiceDiscoveredCallback = std::function<void(msgpack::object&&, SockAddr&&)>;
using ServiceDiscoveredCallback = std::function<void(msgpack::object &&, SockAddr &&)>;
PeerDiscovery(in_port_t port = DEFAULT_PORT, std::shared_ptr<asio::io_context> ioContext = {}, std::shared_ptr<Logger> logger = {});
PeerDiscovery(in_port_t port = DEFAULT_PORT,
std::shared_ptr<asio::io_context> ioContext = {},
std::shared_ptr<Logger> logger = {});
~PeerDiscovery();
/**
* startDiscovery - Keep Listening data from the sender until node is joinned or stop is called
*/
* startDiscovery - Keep Listening data from the sender until node is joinned
* or stop is called
*/
void startDiscovery(const std::string &type, ServiceDiscoveredCallback callback);
template<typename T>
void startDiscovery(const std::string &type, std::function<void(T&&, SockAddr&&)> cb) {
startDiscovery(type, [cb](msgpack::object&& ob, SockAddr&& addr) {
template <typename T>
void startDiscovery(const std::string &type, std::function<void(T &&, SockAddr &&)> cb) {
startDiscovery(type, [cb](msgpack::object &&ob, SockAddr &&addr) {
cb(ob.as<T>(), std::move(addr));
});
}
/**
* startPublish - Keeping sending data until node is joinned or stop is called
*/
*/
void startPublish(const std::string &type, const msgpack::sbuffer &pack_buf);
void startPublish(sa_family_t domain, const std::string &type, const msgpack::sbuffer &pack_buf);
template<typename T>
void startPublish(const std::string &type, const T& object) {
template <typename T>
void startPublish(const std::string &type, const T &object) {
msgpack::sbuffer buf;
msgpack::pack(buf, object);
startPublish(type, buf);
@@ -68,22 +73,24 @@ public:
/**
* Thread Stopper
*/
*/
void stop();
/**
* Remove possible callBack to discovery
*/
*/
bool stopDiscovery(const std::string &type);
/**
* Remove different serivce message to send
*/
*/
bool stopPublish(const std::string &type);
bool stopPublish(sa_family_t domain, const std::string &type);
void connectivityChanged();
void stopConnectivityChanged();
private:
class DomainPeerDiscovery;
std::unique_ptr<DomainPeerDiscovery> peerDiscovery4_;
@@ -92,4 +99,4 @@ private:
std::thread ioRunnner_;
};
}
} // namespace dht
+1
View File
@@ -63,6 +63,7 @@ public:
void stop(bool wait = true);
void join();
void detach();
private:
std::mutex lock_ {};
+121 -20
View File
@@ -1,5 +1,5 @@
project('opendht', 'c', 'cpp', 'cython',
version: '3.1.9',
version: '3.2.0',
default_options: [
'cpp_std=c++17',
'warning_level=3'
@@ -12,17 +12,17 @@ argon2 = dependency('libargon2')
openssl = dependency('openssl', required: get_option('proxy_client'))
jsoncpp = dependency('jsoncpp', required: get_option('proxy_client'))
fmt = dependency('fmt')
llhttp = dependency('llhttp', 'libllhttp', required: get_option('proxy_client'))
io_uring = dependency('liburing', required: false)
dirs=[]
if host_machine.system() == 'freebsd'
dirs+='/usr/local/lib'
elif host_machine.system() == 'darwin'
dirs+='/opt/homebrew/lib'
deps = [fmt, gnutls, nettle, msgpack, argon2, openssl, jsoncpp, llhttp, io_uring]
conf_data = configuration_data()
add_project_arguments('-DMSGPACK_NO_BOOST', '-DASIO_STANDALONE', language : 'cpp')
if io_uring.found()
add_project_arguments('-DASIO_HAS_IO_URING', '-DASIO_DISABLE_EPOLL', language : 'cpp')
conf_data.set('iouring_lib', ', liburing')
endif
http_parser = meson.get_compiler('c').find_library('http_parser', dirs: dirs, required: get_option('proxy_client'))
deps = [fmt, gnutls, nettle, msgpack, argon2, openssl, jsoncpp, http_parser]
add_project_arguments('-DMSGPACK_NO_BOOST', language : 'cpp')
add_project_arguments(['-Wno-return-type','-Wno-deprecated','-Wnon-virtual-dtor','-pedantic-errors','-fvisibility=hidden'], language : 'cpp')
opendht_inc = include_directories('include/opendht')
@@ -42,7 +42,8 @@ opendht_src = [
'src/dhtrunner.cpp',
'src/log.cpp',
'src/op_cache.cpp',
'src/network_utils.cpp'
'src/network_utils.cpp',
'src/thread_pool.cpp',
]
if get_option('indexation').enabled()
@@ -52,12 +53,17 @@ endif
if jsoncpp.found()
opendht_src += ['src/base64.cpp']
add_project_arguments('-DOPENDHT_JSONCPP', language : 'cpp')
conf_data.set('jsoncpp_lib', ', jsoncpp')
endif
if http_parser.found()
if llhttp.found()
opendht_src += ['src/http.cpp', 'src/compat/os_cert.cpp']
if host_machine.system() == 'darwin'
deps+=dependency('appleframeworks', modules : ['CoreFoundation', 'Security'])
endif
conf_data.set('http_lib', '-lllhttp')
endif
if openssl.found()
conf_data.set('openssl_lib', ', openssl')
endif
if get_option('proxy_client').enabled()
opendht_src += ['src/dht_proxy_client.cpp']
@@ -74,23 +80,42 @@ if get_option('peer_discovery').enabled()
opendht_src += 'src/peer_discovery.cpp'
add_project_arguments('-DOPENDHT_PEER_DISCOVERY', language : 'cpp')
endif
opendht = shared_library('opendht',
opendht = library('opendht',
opendht_src,
include_directories : opendht_inc,
dependencies : deps,
cpp_args : ['-DOPENHT_BUILD', '-Dopendht_EXPORTS'],
install : true)
conf_data.set('VERSION', meson.project_version())
conf_data.set('prefix', get_option('prefix'))
conf_data.set('exec_prefix', get_option('prefix'))
conf_data.set('libdir', join_paths(get_option('prefix'), get_option('libdir')))
conf_data.set('includedir', join_paths(get_option('prefix'), get_option('includedir')))
conf_data.set('argon2_lib', ', libargon2')
configure_file(input : 'opendht.pc.in',
output : 'opendht.pc',
configuration : conf_data,
install: true,
install_dir: join_paths(get_option('prefix'), get_option('libdir'), 'pkgconfig'))
readline = meson.get_compiler('c').find_library('readline', required: get_option('tools'))
if get_option('c').enabled()
opendht_c = shared_library('opendht-c',
opendht_c = library('opendht-c',
['c/opendht.cpp', 'c/opendht_c.h'],
include_directories : opendht_interface_inc,
link_with : opendht,
dependencies : [msgpack, fmt],
dependencies : [msgpack, jsoncpp, fmt],
cpp_args : ['-DOPENDHT_C_BUILD', '-Dopendht_c_EXPORTS'],
install : true)
configure_file(input : 'opendht-c.pc.in',
output : 'opendht-c.pc',
configuration : conf_data,
install: true,
install_dir: join_paths(get_option('prefix'), get_option('libdir'), 'pkgconfig'))
if get_option('tools').enabled()
dhtcnode = executable('dhtcnode', 'tools/dhtcnode.c',
@@ -110,7 +135,7 @@ if get_option('python').require(get_option('indexation').enabled(), error_messag
include_directories : opendht_interface_inc,
override_options : ['cython_language=cpp'],
link_with : [opendht],
dependencies : [msgpack, fmt, dep_py],
dependencies : [msgpack, jsoncpp, fmt, dep_py],
install : true
)
endif
@@ -119,22 +144,98 @@ if get_option('tools').enabled()
dhtnode = executable('dhtnode', 'tools/dhtnode.cpp',
include_directories : opendht_interface_inc,
link_with : opendht,
dependencies : [readline, msgpack, fmt, openssl],
dependencies : [readline, jsoncpp, msgpack, fmt, openssl],
install : true)
dhtchat = executable('dhtchat', 'tools/dhtchat.cpp',
include_directories : opendht_interface_inc,
link_with : opendht,
dependencies : [readline, msgpack, fmt, openssl],
dependencies : [readline, jsoncpp, msgpack, fmt, openssl],
install : true)
dhtscanner = executable('dhtscanner', 'tools/dhtscanner.cpp',
include_directories : opendht_interface_inc,
link_with : opendht,
dependencies : [readline, msgpack, fmt, openssl],
dependencies : [readline, jsoncpp, msgpack, fmt, openssl],
install : true)
if http_parser.found()
if llhttp.found()
durl = executable('durl', 'tools/durl.cpp',
include_directories : opendht_interface_inc,
link_with : opendht,
dependencies : [msgpack, openssl])
endif
endif
# Unit tests
if get_option('tests').enabled()
cppunit = dependency('cppunit')
test_infohash = executable('test_infohash',
'tests/infohashtester.cpp', 'tests/tests_runner.cpp',
include_directories : opendht_interface_inc,
link_with : opendht,
dependencies : [cppunit, jsoncpp, fmt, openssl, msgpack])
test('Infohash', test_infohash)
test_value = executable('test_value',
'tests/valuetester.cpp', 'tests/tests_runner.cpp',
include_directories : opendht_interface_inc,
link_with : opendht,
dependencies : [cppunit, jsoncpp, fmt, openssl, msgpack])
test('Value', test_value)
test_crypto = executable('test_crypto',
'tests/cryptotester.cpp', 'tests/tests_runner.cpp',
include_directories : opendht_interface_inc,
link_with : opendht,
dependencies : [cppunit, jsoncpp, fmt, openssl, msgpack])
test('Crypto', test_crypto)
test_dhtrunner = executable('test_dhtrunner',
'tests/dhtrunnertester.cpp', 'tests/tests_runner.cpp',
include_directories : opendht_interface_inc,
link_with : opendht,
dependencies : [cppunit, jsoncpp, fmt, openssl, msgpack])
test('DhtRunner', test_dhtrunner)
test_threadpool = executable('test_threadpool',
'tests/threadpooltester.cpp', 'tests/tests_runner.cpp',
include_directories : opendht_interface_inc,
link_with : opendht,
dependencies : [cppunit, jsoncpp, fmt, openssl, msgpack])
test('ThreadPool', test_threadpool)
if get_option('proxy_client').enabled() or get_option('proxy_server').enabled()
test_http = executable('test_http',
'tests/httptester.cpp', 'tests/tests_runner.cpp',
include_directories : opendht_interface_inc,
link_with : opendht,
dependencies : [cppunit, jsoncpp, fmt, openssl, msgpack])
test('Http', test_http)
endif
if get_option('peer_discovery').enabled()
test_peerdiscovery = executable('test_peerdiscovery',
'tests/peerdiscoverytester.cpp', 'tests/tests_runner.cpp',
include_directories : opendht_interface_inc,
link_with : opendht,
dependencies : [cppunit, jsoncpp, fmt, openssl, msgpack])
test('PeerDiscovery', test_peerdiscovery)
endif
if get_option('proxy_server').enabled()
test_proxy = executable('test_dhtproxy',
'tests/dhtproxytester.cpp', 'tests/tests_runner.cpp',
include_directories : opendht_interface_inc,
link_with : opendht,
dependencies : [cppunit, jsoncpp, fmt, openssl, msgpack])
test('DhtProxy', test_proxy)
if get_option('long_tests').enabled()
test_proxystress = executable('test_dhtproxystress',
'tests/dhtproxy_stress_tester.cpp', 'tests/tests_runner.cpp',
include_directories : opendht_interface_inc,
link_with : opendht,
dependencies : [cppunit, jsoncpp, fmt, openssl, msgpack])
test('DhtProxy Stress', test_proxystress)
endif
endif
endif
+2
View File
@@ -6,3 +6,5 @@ option('tools', type : 'feature', value : 'enabled')
option('c', type : 'feature', value : 'enabled')
option('indexation', type : 'feature', value : 'enabled')
option('python', type : 'feature', value : 'enabled')
option('tests', type : 'feature', value : 'enabled')
option('long_tests', type : 'feature', value : 'disabled')
+2 -2
View File
@@ -6,7 +6,7 @@ Name: OpenDHT
Description: C++17 Distributed Hash Table library
Version: @VERSION@
Libs: -L${libdir} -lopendht
Libs.private: @http_parser_lib@ -pthread
Libs.private: @http_lib@ -pthread
Requires: gnutls >= 3.3@jsoncpp_lib@@openssl_lib@
Requires.private: nettle >= 2.4@argon2_lib@
Requires.private: nettle >= 2.4@argon2_lib@@iouring_lib@
Cflags: -I${includedir}
+2 -1
View File
@@ -500,12 +500,13 @@ cdef class DhtConfig(object):
def setRateLimit(self, ssize_t max_req_per_sec, ssize_t max_peer_req_per_sec):
self._config.dht_config.node_config.max_req_per_sec = max_req_per_sec
self._config.dht_config.node_config.max_peer_req_per_sec = max_peer_req_per_sec
def setProxyInfo(self, str proxy_server, str push_node_id="", str push_token="", str push_topic="", str push_platform="", Certificate server_ca = Certificate()):
def setProxyInfo(self, str proxy_server, str push_node_id="", str push_token="", str push_topic="", str push_platform="", str user_agent="", Certificate server_ca = Certificate()):
self._config.proxy_server = proxy_server.encode()
self._config.push_node_id = push_node_id.encode()
self._config.push_token = push_token.encode()
self._config.push_topic = push_topic.encode()
self._config.push_platform = push_platform.encode()
self._config.proxy_user_agent = user_agent.encode()
self._config.server_ca = server_ca._cert
def setPeerDiscovery(self, bool peer_discovery, bool peer_publish):
self._config.peer_discovery = peer_discovery
+1
View File
@@ -246,6 +246,7 @@ cdef extern from "opendht/dhtrunner.h" namespace "dht":
string push_token
string push_topic
string push_platform
string proxy_user_agent
bool peer_discovery
bool peer_publish
shared_ptr[Certificate] server_ca
+5 -5
View File
@@ -2,10 +2,10 @@
name = "opendht"
version = "0.1.0"
authors = ["Sébastien Blin <sebastien.blin@savoirfairelinux.com>"]
edition = "2018"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
edition = "2021"
[dependencies]
libc="0.2.0"
os_socketaddr="0.2.3"
libc="0.2"
os_socketaddr="0.2.3"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+14 -8
View File
@@ -49,6 +49,7 @@ impl DhtRunnerConfig {
push_token: ptr::null(),
push_topic: ptr::null(),
push_platform: ptr::null(),
proxy_user_agent: ptr::null(),
peer_discovery: false,
peer_publish: false,
server_ca: ptr::null_mut(),
@@ -56,6 +57,7 @@ impl DhtRunnerConfig {
privatekey: ptr::null_mut(),
certificate: ptr::null_mut(),
},
log: true,
});
unsafe {
dht_runner_config_default(&mut *config);
@@ -64,23 +66,27 @@ impl DhtRunnerConfig {
}
pub fn set_proxy_server(&mut self, proxy_server: &str) {
self.proxy_server = CString::new(proxy_server).unwrap().as_ptr();
self.proxy_server = CString::new(proxy_server).unwrap().into_raw();
}
pub fn set_push_node_id(&mut self, push_node_id: &str) {
self.push_node_id = CString::new(push_node_id).unwrap().as_ptr();
self.push_node_id = CString::new(push_node_id).unwrap().into_raw();
}
pub fn set_push_token(&mut self, push_token: &str) {
self.push_token = CString::new(push_token).unwrap().as_ptr();
self.push_token = CString::new(push_token).unwrap().into_raw();
}
pub fn set_push_topic(&mut self, push_topic: &str) {
self.push_topic = CString::new(push_topic).unwrap().as_ptr();
self.push_topic = CString::new(push_topic).unwrap().into_raw();
}
pub fn set_push_platform(&mut self, push_platform: &str) {
self.push_platform = CString::new(push_platform).unwrap().as_ptr();
self.push_platform = CString::new(push_platform).unwrap().into_raw();
}
pub fn set_proxy_user_agent(&mut self, proxy_user_agent: &str) {
self.proxy_user_agent = CString::new(proxy_user_agent).unwrap().into_raw();
}
pub fn set_identity(&mut self, certificate: Box<DhtCertificate>, privatekey: Box<PrivateKey>) {
@@ -93,7 +99,7 @@ impl DhtRunnerConfig {
impl DhtNodeConfig
{
pub fn set_persist_path(&mut self, persist_path: &str) {
self.persist_path = CString::new(persist_path).unwrap().as_ptr();
self.persist_path = CString::new(persist_path).unwrap().into_raw();
}
}
@@ -197,8 +203,8 @@ impl DhtRunner {
pub fn bootstrap(&mut self, host: &str, service: u16) {
unsafe {
dht_runner_bootstrap(&mut *self,
CString::new(host).unwrap().as_ptr(),
CString::new(service.to_string()).unwrap().as_ptr())
CString::new(host).unwrap().into_raw(),
CString::new(service.to_string()).unwrap().into_raw())
}
}
+2 -1
View File
@@ -120,13 +120,14 @@ pub struct DhtRunnerConfig
pub push_token: *const c_char,
pub push_topic: *const c_char,
pub push_platform: *const c_char,
pub proxy_user_agent: *const c_char,
pub peer_discovery: bool,
pub peer_publish: bool,
pub server_ca: *mut DhtCertificate,
pub client_identity: DhtIdentity,
pub log: bool,
}
#[link(name = "opendht-c")]
extern {
// dht::Value
+39
View File
@@ -0,0 +1,39 @@
//Code from getifaddrs.3 man
#include <arpa/inet.h>
#include <sys/socket.h>
#include <ifaddrs.h>
#include <stdlib.h>
#include <unistd.h>
#include <net/if.h>
#include <asio.hpp>
#include <stdexcept>
namespace dht{
namespace workaround{
asio::ip::address_v4
get_interface(){
struct ifaddrs *ifaddr;
if(getifaddrs(&ifaddr) == -1){
throw std::runtime_error("Can't getifaddrs");
}
for (struct ifaddrs *if1 = ifaddr; if1 != NULL; if1 = if1->ifa_next){
if (if1->ifa_addr == NULL)
continue;
if (if1->ifa_addr->sa_family != AF_INET)
continue;
if((if1->ifa_flags & IFF_MULTICAST) == IFF_MULTICAST){
auto sa_data = reinterpret_cast<uint8_t*>(if1->ifa_addr->sa_data);
uint32_t value = sa_data[2] << 24
| sa_data[3] << 16
| sa_data[4] << 8
| sa_data[5];
freeifaddrs(ifaddr);
return asio::ip::address_v4(value);
}
}
freeifaddrs(ifaddr);
throw std::runtime_error("Can't find a interface which supports multicast");
}
} /* namespace workaround */
} /* namespace dht */
+83 -22
View File
@@ -93,11 +93,27 @@ Blob aesEncrypt(const uint8_t* data, size_t data_length, const Blob& key)
std::random_device rdev;
std::generate_n(ret.begin(), GCM_IV_SIZE, std::bind(rand_byte, std::ref(rdev)));
}
struct gcm_aes_ctx aes;
gcm_aes_set_key(&aes, key.size(), key.data());
gcm_aes_set_iv(&aes, GCM_IV_SIZE, ret.data());
gcm_aes_encrypt(&aes, data_length, ret.data() + GCM_IV_SIZE, data);
gcm_aes_digest(&aes, GCM_DIGEST_SIZE, ret.data() + GCM_IV_SIZE + data_length);
if (key.size() == AES_LENGTHS[0]) {
struct gcm_aes128_ctx aes;
gcm_aes128_set_key(&aes, key.data());
gcm_aes128_set_iv(&aes, GCM_IV_SIZE, ret.data());
gcm_aes128_encrypt(&aes, data_length, ret.data() + GCM_IV_SIZE, data);
gcm_aes128_digest(&aes, GCM_DIGEST_SIZE, ret.data() + GCM_IV_SIZE + data_length);
} else if (key.size() == AES_LENGTHS[1]) {
struct gcm_aes192_ctx aes;
gcm_aes192_set_key(&aes, key.data());
gcm_aes192_set_iv(&aes, GCM_IV_SIZE, ret.data());
gcm_aes192_encrypt(&aes, data_length, ret.data() + GCM_IV_SIZE, data);
gcm_aes192_digest(&aes, GCM_DIGEST_SIZE, ret.data() + GCM_IV_SIZE + data_length);
} else if (key.size() == AES_LENGTHS[2]) {
struct gcm_aes256_ctx aes;
gcm_aes256_set_key(&aes, key.data());
gcm_aes256_set_iv(&aes, GCM_IV_SIZE, ret.data());
gcm_aes256_encrypt(&aes, data_length, ret.data() + GCM_IV_SIZE, data);
gcm_aes256_digest(&aes, GCM_DIGEST_SIZE, ret.data() + GCM_IV_SIZE + data_length);
}
return ret;
}
@@ -118,14 +134,28 @@ Blob aesDecrypt(const uint8_t* data, size_t data_length, const Blob& key)
std::array<uint8_t, GCM_DIGEST_SIZE> digest;
struct gcm_aes_ctx aes;
gcm_aes_set_key(&aes, key.size(), key.data());
gcm_aes_set_iv(&aes, GCM_IV_SIZE, data);
size_t data_sz = data_length - GCM_IV_SIZE - GCM_DIGEST_SIZE;
Blob ret(data_sz);
gcm_aes_decrypt(&aes, data_sz, ret.data(), data + GCM_IV_SIZE);
gcm_aes_digest(&aes, GCM_DIGEST_SIZE, digest.data());
if (key.size() == AES_LENGTHS[0]) {
struct gcm_aes128_ctx aes;
gcm_aes128_set_key(&aes, key.data());
gcm_aes128_set_iv(&aes, GCM_IV_SIZE, data);
gcm_aes128_decrypt(&aes, data_sz, ret.data(), data + GCM_IV_SIZE);
gcm_aes128_digest(&aes, GCM_DIGEST_SIZE, digest.data());
} else if (key.size() == AES_LENGTHS[1]) {
struct gcm_aes192_ctx aes;
gcm_aes192_set_key(&aes, key.data());
gcm_aes192_set_iv(&aes, GCM_IV_SIZE, data);
gcm_aes192_decrypt(&aes, data_sz, ret.data(), data + GCM_IV_SIZE);
gcm_aes192_digest(&aes, GCM_DIGEST_SIZE, digest.data());
} else if (key.size() == AES_LENGTHS[2]) {
struct gcm_aes256_ctx aes;
gcm_aes256_set_key(&aes, key.data());
gcm_aes256_set_iv(&aes, GCM_IV_SIZE, data);
gcm_aes256_decrypt(&aes, data_sz, ret.data(), data + GCM_IV_SIZE);
gcm_aes256_digest(&aes, GCM_DIGEST_SIZE, digest.data());
}
if (not std::equal(digest.begin(), digest.end(), data + data_length - GCM_DIGEST_SIZE)) {
throw DecryptError("Can't decrypt data");
@@ -227,7 +257,7 @@ PrivateKey::PrivateKey(const uint8_t* src, size_t src_size, const char* password
throw CryptoException("Can't initialize private key !");
const gnutls_datum_t dt {(uint8_t*)src, static_cast<unsigned>(src_size)};
int flags = password_ptr ? GNUTLS_PKCS_PLAIN
int flags = (password_ptr == NULL || strlen(password_ptr)==0) ? GNUTLS_PKCS_PLAIN
: ( GNUTLS_PKCS_PBES2_AES_128 | GNUTLS_PKCS_PBES2_AES_192 | GNUTLS_PKCS_PBES2_AES_256
| GNUTLS_PKCS_PKCS12_3DES | GNUTLS_PKCS_PKCS12_ARCFOUR | GNUTLS_PKCS_PKCS12_RC2_40);
@@ -1134,11 +1164,17 @@ saveIdentity(const Identity& id, const std::string& path, const std::string& pri
auto ca_key_data = id.first->serialize(privkey_password);
std::ofstream key_file(path + ".pem");
key_file.write((char*)ca_key_data.data(), ca_key_data.size());
// Throw error if the file is not written
if (!key_file)
throw CryptoException("Could not write private key file");
}
{
auto ca_key_data = id.second->getPacked();
std::ofstream crt_file(path + ".crt");
crt_file.write((char*)ca_key_data.data(), ca_key_data.size());
// Throw error if the file is not written
if (!crt_file)
throw CryptoException("Could not write certificate file");
}
}
@@ -1152,12 +1188,19 @@ loadIdentity(const std::string &path,const std::string &privkey_password)
pkStream.close();
// Create a certificate
gnutls_x509_crt_t gnuCert;
if (gnutls_x509_crt_init(&gnuCert) != GNUTLS_E_SUCCESS)
if (gnutls_x509_crt_init(&gnuCert))
throw std::runtime_error("Failed to initialize gnutls certificate struct");
gnutls_datum_t crtContent {nullptr, 0};
// Read the certificate file
gnutls_load_file((path + ".crt").c_str(), &crtContent);
gnutls_x509_crt_import(gnuCert, &crtContent, GNUTLS_X509_FMT_PEM);
int err = gnutls_load_file((path + ".crt").c_str(), &crtContent);
if (err)
throw CryptoException(gnutls_strerror(err));
err = gnutls_x509_crt_import(gnuCert, &crtContent, GNUTLS_X509_FMT_PEM);
if (err)
throw CryptoException(gnutls_strerror(err));
auto cert = std::make_shared<Certificate>(gnuCert);
return {std::move(key), std::move(cert)};
}
@@ -1204,31 +1247,49 @@ Certificate::generate(const PrivateKey& key, const std::string& name, const Iden
auto pk_id = pk.getId();
const std::string uid_str = pk_id.toString();
gnutls_x509_crt_set_subject_key_id(cert, &pk_id, sizeof(pk_id));
gnutls_x509_crt_set_dn_by_oid(cert, GNUTLS_OID_X520_COMMON_NAME, 0, name.data(), name.length());
gnutls_x509_crt_set_dn_by_oid(cert, GNUTLS_OID_LDAP_UID, 0, uid_str.data(), uid_str.length());
int err = gnutls_x509_crt_set_subject_key_id(cert, &pk_id, sizeof(pk_id));
if(err) {
throw CryptoException(std::string("Error when setting subject key id ") + gnutls_strerror(err));
}
err = gnutls_x509_crt_set_dn_by_oid(cert, GNUTLS_OID_X520_COMMON_NAME, 0, name.data(), name.length());
if(err) {
throw CryptoException(std::string("Error when setting subject key id ") + gnutls_strerror(err));
}
err = gnutls_x509_crt_set_dn_by_oid(cert, GNUTLS_OID_LDAP_UID, 0, uid_str.data(), uid_str.length());
if(err) {
throw CryptoException(std::string("Error when setting dn by oid ") + gnutls_strerror(err));
}
setRandomSerial(cert);
unsigned key_usage = 0;
if (is_ca) {
gnutls_x509_crt_set_ca_status(cert, 1);
err = gnutls_x509_crt_set_ca_status(cert, 1);
if(err) {
throw CryptoException(std::string("Error when setting ca status ") + gnutls_strerror(err));
}
key_usage |= GNUTLS_KEY_KEY_CERT_SIGN | GNUTLS_KEY_CRL_SIGN;
} else {
key_usage |= GNUTLS_KEY_DIGITAL_SIGNATURE | GNUTLS_KEY_DATA_ENCIPHERMENT;
}
gnutls_x509_crt_set_key_usage(cert, key_usage);
err = gnutls_x509_crt_set_key_usage(cert, key_usage);
if(err) {
throw CryptoException(std::string("Error when setting ca status ") + gnutls_strerror(err));
}
if (ca.first && ca.second) {
if (not ca.second->isCA()) {
throw CryptoException("Signing certificate must be CA");
}
if (int err = gnutls_x509_crt_privkey_sign(cert, ca.second->cert, ca.first->key, pk.getPreferredDigest(), 0)) {
err = gnutls_x509_crt_privkey_sign(cert, ca.second->cert, ca.first->key, pk.getPreferredDigest(), 0);
if (err) {
throw CryptoException(std::string("Error when signing certificate ") + gnutls_strerror(err));
}
ret.issuer = ca.second;
} else {
if (int err = gnutls_x509_crt_privkey_sign(cert, cert, key.key, pk.getPreferredDigest(), 0)) {
err = gnutls_x509_crt_privkey_sign(cert, cert, key.key, pk.getPreferredDigest(), 0);
if (err) {
throw CryptoException(std::string("Error when signing certificate ") + gnutls_strerror(err));
}
}
+2 -1
View File
@@ -93,7 +93,7 @@ Dht::shutdown(ShutdownCallback cb, bool stop)
r.second.done_cb(false, {});
sr.second->callbacks.clear();
for (const auto& a : sr.second->announce) {
if (a.callback) a.callback(false, {});
for (auto& cb : a.callbacks) cb(false, {});
}
sr.second->announce.clear();
sr.second->listeners.clear();
@@ -1812,6 +1812,7 @@ fromDhtConfig(const Config& config)
netConf.max_peer_req_per_sec = config.max_peer_req_per_sec
? config.max_peer_req_per_sec
: netConf.max_req_per_sec/8;
netConf.is_client = config.client_mode;
return netConf;
}
+51 -7
View File
@@ -23,7 +23,7 @@
#include "op_cache.h"
#include "utils.h"
#include <http_parser.h>
#include <llhttp.h>
#include <deque>
namespace dht {
@@ -106,13 +106,26 @@ getRandomSessionId(size_t length = 8) {
DhtProxyClient::DhtProxyClient() {}
DhtProxyClient::DhtProxyClient(
std::shared_ptr<dht::crypto::Certificate> serverCA, dht::crypto::Identity clientIdentity,
std::function<void()> signal, const std::string& serverHost,
const std::string& pushClientId, std::shared_ptr<dht::Logger> logger)
std::shared_ptr<dht::crypto::Certificate> serverCA,
dht::crypto::Identity clientIdentity,
std::function<void()> signal,
const std::string& serverHost,
const std::string& userAgent,
const std::string& pushClientId,
const std::string& pushToken,
const std::string& pushTopic,
const std::string& pushPlatform,
std::shared_ptr<dht::Logger> logger
)
: DhtInterface(logger)
, proxyUrl_(serverHost)
, clientIdentity_(clientIdentity), serverCertificate_(serverCA)
, pushClientId_(pushClientId), pushSessionId_(getRandomSessionId())
, userAgent_(userAgent)
, pushClientId_(pushClientId)
, pushSessionId_(getRandomSessionId())
, deviceKey_(pushToken)
, notificationTopic_(pushTopic)
, platform_(pushPlatform)
, loopSignal_(signal)
, jsonReader_(Json::CharReaderBuilder{}.newCharReader())
{
@@ -484,7 +497,7 @@ DhtProxyClient::buildRequest(const std::string& target)
request->set_certificate_authority(serverCertificate_);
if (clientIdentity_.first and clientIdentity_.second)
request->set_identity(clientIdentity_);
request->set_header_field(restinio::http_field_t::user_agent, "RESTinio client");
request->set_header_field(restinio::http_field_t::user_agent, userAgent_);
return request;
}
@@ -1010,7 +1023,10 @@ DhtProxyClient::handleExpireListener(const asio::error_code &ec, const InfoHash&
}
} else {
// stop the request
listener.request.reset();
if (listener.request) {
listener.request->cancel();
listener.request.reset();
}
}
search->second.listeners.erase(it);
if (logger_)
@@ -1105,6 +1121,14 @@ DhtProxyClient::sendListen(const restinio::http_request_header_t& header,
std::lock_guard<std::mutex> l(requestLock_);
requests_[reqid] = request;
}
request->add_on_status_callback([r=std::weak_ptr(request), seconds = this->listenKeepIdle()] (unsigned status_code) {
if(status_code == 200) {
// increase TCP_KEEPIDLE to save power
if (auto request = r.lock()) {
request->get_connection()->set_keepalive(seconds);
}
}
});
request->send();
}
catch (const std::exception &e){
@@ -1372,4 +1396,24 @@ DhtProxyClient::fillBody(bool resubscribe)
}
#endif // OPENDHT_PUSH_NOTIFICATIONS
void
DhtProxyClient::setPushNotificationToken([[maybe_unused]] const std::string& token) {
#ifdef OPENDHT_PUSH_NOTIFICATIONS
std::unique_lock<std::mutex> l(lockCurrentProxyInfos_);
if (deviceKey_ != token) {
deviceKey_ = token;
if (statusIpv4_ == NodeStatus::Connected || statusIpv6_ == NodeStatus::Connected) {
if (logger_)
logger_->d("[proxy:client] [push] token changed, resubscribing");
for (auto& search : searches_) {
for (auto& listener : search.second.listeners) {
resubscribe(search.first, listener.first, listener.second);
}
}
}
}
#endif
}
} // namespace dht
+102 -54
View File
@@ -58,6 +58,19 @@ constexpr const std::chrono::minutes PRINT_STATS_PERIOD {2};
using ResponseByParts = restinio::chunked_output_t;
using ResponseByPartsBuilder = restinio::response_builder_t<ResponseByParts>;
namespace http {
/**
* Session value associated with a connection_id_t key.
*/
struct ListenerSession
{
ListenerSession() = default;
dht::InfoHash hash;
std::future<size_t> token;
std::shared_ptr<restinio::response_builder_t<restinio::chunked_output_t>> response;
};
}
class opendht_logger_t
{
public:
@@ -116,7 +129,7 @@ private:
void
DhtProxyServer::ConnectionListener::state_changed(const restinio::connection_state::notice_t& notice) noexcept
{
if (restinio::holds_alternative<restinio::connection_state::closed_t>(notice.cause())) {
if (std::holds_alternative<restinio::connection_state::closed_t>(notice.cause())) {
onClosed_(notice.connection_id());
}
}
@@ -210,6 +223,7 @@ DhtProxyServer::DhtProxyServer(const std::shared_ptr<DhtRunner>& dht,
: ioContext_(std::make_shared<asio::io_context>()),
dht_(dht), persistPath_(config.persistStatePath), logger_(logger),
printStatsTimer_(std::make_unique<asio::steady_timer>(*ioContext_, 3s)),
serverStartTime_(clock::now()),
connListener_(std::make_shared<ConnectionListener>(std::bind(&DhtProxyServer::onConnectionClosed, this, std::placeholders::_1))),
pushServer_(config.pushServer),
bundleId_(config.bundleId)
@@ -402,31 +416,12 @@ DhtProxyServer::loadState(Is& is, size_t size) {
for (auto& pushListener : pushListeners_) {
for (auto& listeners : pushListener.second.listeners) {
for (auto& listener : listeners.second) {
// start listening
listener.internalToken = dht_->listen(listeners.first,
[this, infoHash=listeners.first, pushToken=pushListener.first, type=listener.type, clientId=listener.clientId, sessionCtx = listener.sessionCtx, topic=listener.topic]
(const std::vector<std::shared_ptr<Value>>& values, bool expired) {
// Build message content
Json::Value json;
json["key"] = infoHash.toString();
json["to"] = clientId;
json["t"] = Json::Value::Int64(std::chrono::duration_cast<std::chrono::milliseconds>(system_clock::now().time_since_epoch()).count());
{
std::lock_guard<std::mutex> l(sessionCtx->lock);
json["s"] = sessionCtx->sessionId;
}
if (expired and values.size() < 2){
std::ostringstream ss;
for(size_t i = 0; i < values.size(); ++i){
if(i != 0) ss << ",";
ss << values[i]->id;
}
json["exp"] = ss.str();
}
auto maxPrio = 1000u;
for (const auto& v : values)
maxPrio = std::min(maxPrio, v->priority);
sendPushNotification(pushToken, std::move(json), type, !expired and maxPrio == 0, topic);
return true;
[this, infoHash=listeners.first, pushToken=pushListener.first, type=listener.type,
clientId=listener.clientId, sessionCtx = listener.sessionCtx, topic=listener.topic]
(const std::vector<Sp<Value>>& values, bool expired) {
return this->handlePushListen(infoHash, pushToken, type, clientId, sessionCtx, topic, values, expired);
}
);
// expire notify
@@ -548,7 +543,15 @@ DhtProxyServer::updateStats(std::shared_ptr<NodeInfo> info) const
stats.requestRate = count / dt.count();
#ifdef OPENDHT_PUSH_NOTIFICATIONS
stats.pushListenersCount = pushListeners_.size();
{
std::lock_guard lk(pushStatsMutex_);
stats.androidPush = androidPush_;
stats.iosPush = iosPush_;
stats.unifiedPush = unifiedPush_;
}
#endif
stats.serverStartTime = serverStartTime_;
stats.lastUpdated = now;
stats.totalPermanentPuts = 0;
std::for_each(puts_.begin(), puts_.end(), [&stats](const auto& put) {
stats.totalPermanentPuts += put.second.puts.size();
@@ -897,13 +900,13 @@ DhtProxyServer::subscribe(restinio::request_handle_t request,
// Send response
if (not newListener) {
if (logger_)
logger_->d("[proxy:server] [subscribe] found [client %s]", listener.clientId.c_str());
logger_->d("[proxy:server] [subscribe %s] found [client %s]", infoHash.toString().c_str(), listener.clientId.c_str());
// Send response header
auto response = std::make_shared<ResponseByPartsBuilder>(initHttpResponse(request->create_response<ResponseByParts>()));
response->flush();
if (!root["refresh"].asBool()) {
// No Refresh
dht_->get(infoHash, [this, response](const Sp<Value>& value){
dht_->get(infoHash, [this, response](const Sp<Value>& value) {
auto output = Json::writeString(jsonBuilder_, value->toJson()) + "\n";
response->append_chunk(output);
response->flush();
@@ -920,33 +923,15 @@ DhtProxyServer::subscribe(restinio::request_handle_t request,
} else {
// =========== No existing listener for an infoHash ============
// Add listen on dht
if (logger_)
logger_->d("[proxy:server] [subscribe %s] new", infoHash.toString().c_str());
listener.internalToken = dht_->listen(infoHash,
[this, infoHash, pushToken, type, clientId, sessionCtx = listener.sessionCtx, topic]
(const std::vector<std::shared_ptr<Value>>& values, bool expired){
// Build message content
Json::Value json;
json["key"] = infoHash.toString();
json["to"] = clientId;
json["t"] = Json::Value::Int64(std::chrono::duration_cast<std::chrono::milliseconds>(system_clock::now().time_since_epoch()).count());
{
std::lock_guard<std::mutex> l(sessionCtx->lock);
json["s"] = sessionCtx->sessionId;
}
if (expired and values.size() < 2){
std::ostringstream ss;
for(size_t i = 0; i < values.size(); ++i) {
if(i != 0) ss << ",";
ss << values[i]->id;
}
json["exp"] = ss.str();
}
auto maxPrio = 1000u;
for (const auto& v : values)
maxPrio = std::min(maxPrio, v->priority);
sendPushNotification(pushToken, std::move(json), type, !expired and maxPrio == 0, topic);
return true;
(const std::vector<Sp<Value>>& values, bool expired) {
return this->handlePushListen(infoHash, pushToken, type, clientId, sessionCtx, topic, values, expired);
}
);
// Send response header
auto response = initHttpResponse(request->create_response());
response.set_body("{}\n");
return response.done();
@@ -1050,6 +1035,50 @@ DhtProxyServer::handleCancelPushListen(const asio::error_code &ec, const std::st
pushListeners_.erase(pushListener);
}
bool
DhtProxyServer::handlePushListen(const InfoHash& infoHash, const std::string& pushToken, PushType type, const std::string& clientId,
const std::shared_ptr<DhtProxyServer::PushSessionContext>& sessionCtx, const std::string& topic,
const std::vector<std::shared_ptr<Value>>& values, bool expired)
{
Json::Value json;
json["key"] = infoHash.toString();
json["to"] = clientId;
using namespace std::chrono;
json["t"] = Json::Value::Int64(duration_cast<milliseconds>(system_clock::now().time_since_epoch()).count());
{
std::lock_guard<std::mutex> l(sessionCtx->lock);
json["s"] = sessionCtx->sessionId;
}
// Build a comma-separated list of ids from the values. This is intended to be used when
// streaming from the iOS notification extension in order to filter out unwanted values.
std::string ids;
for(const auto& value : values) {
if (!ids.empty()) ids += ",";
ids += std::to_string(value->id);
}
json["ids"] = ids;
// If message is expired copy the value ID to the `exp` field.
// This is presumably used by the Android notification system.
if (expired && values.size() < 2) {
json["exp"] = json["ids"];
}
auto minPriority = 1000u;
for (const auto& v : values)
minPriority = std::min(minPriority, v->priority);
if (logger_)
logger_->d("[proxy:server] [listen %s] [client %s] [session %s] [expired %i] [priority %i] [values %zu]",
infoHash.toString().c_str(), clientId.c_str(), sessionCtx->sessionId.c_str(), expired, minPriority, values.size());
sendPushNotification(pushToken, std::move(json), type, !expired and minPriority == 0, topic);
return true;
}
void
DhtProxyServer::sendPushNotification(const std::string& token, Json::Value&& json, PushType type, bool highPriority, const std::string& topic)
{
@@ -1087,10 +1116,14 @@ DhtProxyServer::sendPushNotification(const std::string& token, Json::Value&& jso
notification["tokens"] = std::move(tokens);
notification["platform"] = type == PushType::Android ? 2 : 1;
notification["data"] = std::move(json);
notification["priority"] = highPriority ? "high" : "normal";
if (type == PushType::Android)
notification["time_to_live"] = 3600 * 24; // 24 hours
else {
auto priority = highPriority ? "high" : "normal";
if (type == PushType::Android) {
Json::Value androidConfig(Json::objectValue);
androidConfig["priority"] = priority;
androidConfig["ttl"] = "86400s"; // time to live = 24 hours
notification["android"] = std::move(androidConfig);
} else {
notification["priority"] = priority;
const auto expiration = std::chrono::system_clock::now() + std::chrono::hours(24);
uint32_t exp = std::chrono::duration_cast<std::chrono::seconds>(expiration.time_since_epoch()).count();
notification["expiration"] = exp;
@@ -1131,6 +1164,21 @@ DhtProxyServer::sendPushNotification(const std::string& token, Json::Value&& jso
requests_[reqid] = request;
}
request->send();
// For monitoring purposes
std::lock_guard lk(pushStatsMutex_);
switch (type) {
case PushType::Android:
androidPush_.increment(highPriority);
break;
case PushType::iOS:
iosPush_.increment(highPriority);
break;
case PushType::UnifiedPush:
unifiedPush_.increment(highPriority);
break;
default:
break;
}
}
catch (const std::exception &e){
if (logger_)
+24 -12
View File
@@ -180,9 +180,9 @@ DhtRunner::run(const Config& config, Context&& context)
throw;
}
if (context.statusChangedCallback) {
statusCb = std::move(context.statusChangedCallback);
}
statusCbs.clear();
if (context.statusChangedCallback)
statusCbs.emplace_back(std::move(context.statusChangedCallback));
if (context.certificateStore) {
dht_->setLocalCertificateStore(std::move(context.certificateStore));
}
@@ -264,6 +264,17 @@ DhtRunner::run(const Config& config, Context&& context)
}
}
}
if (peerDiscovery_) {
statusCbs.emplace_back([this](NodeStatus status4, NodeStatus status6) {
if (status4 == NodeStatus::Disconnected && status6 == NodeStatus::Disconnected) {
peerDiscovery_->connectivityChanged();
}
else if (status4 != NodeStatus::Connected || status6 != NodeStatus::Connected) {
peerDiscovery_->stopConnectivityChanged();
}
});
}
#endif
}
}
@@ -690,8 +701,9 @@ DhtRunner::loop_()
if (nstatus4 != status4 || nstatus6 != status6) {
status4 = nstatus4;
status6 = nstatus6;
if (statusCb)
statusCb(status4, status6);
for (auto& cb : statusCbs){
cb(status4, status6);
}
}
return wakeup;
@@ -1108,13 +1120,13 @@ DhtRunner::enableProxy(bool proxify)
cv.notify_all();
}
},
config_.proxy_server, config_.push_node_id, logger_);
if (not config_.push_token.empty())
dht_via_proxy->setPushNotificationToken(config_.push_token);
if (not config_.push_topic.empty())
dht_via_proxy->setPushNotificationTopic(config_.push_topic);
if (not config_.push_platform.empty())
dht_via_proxy->setPushNotificationPlatform(config_.push_platform);
config_.proxy_server,
config_.proxy_user_agent,
config_.push_node_id,
config_.push_token,
config_.push_topic,
config_.push_platform,
logger_);
dht_ = std::make_unique<SecureDht>(std::move(dht_via_proxy), config_.dht_config, identityAnnouncedCb_, logger_);
// and use it
use_proxy = proxify;
+78 -59
View File
@@ -24,7 +24,7 @@
#include <asio.hpp>
#include <restinio/impl/tls_socket.hpp>
#include <http_parser.h>
#include <llhttp.h>
#include <json/json.h>
#include <openssl/ocsp.h>
@@ -461,7 +461,7 @@ Connection::set_ssl_verification(const std::string& hostname, const asio::ssl::v
bool verified = verifier(preverified, ctx);
auto verify_ec = X509_STORE_CTX_get_error(ctx.native_handle());
if (verify_ec != 0 /*X509_V_OK*/ and logger)
logger->error("[http::connection:{:d}] ssl verification error={:d} %d", id, verify_ec, verified);
logger->error("[http::connection:{:d}] ssl verification error={:d} {}", id, verify_ec, verified);
if (verified and checkOcsp) {
std::unique_ptr<stack_st_X509, void(*)(stack_st_X509*)> chain(
X509_STORE_CTX_get1_chain(ctx.native_handle()),
@@ -536,48 +536,11 @@ Connection::async_connect(std::vector<asio::ip::tcp::endpoint>&& endpoints, Conn
#pragma GCC diagnostic ignored "-Wunused-variable"
ConnectHandlerCb wcb = [this, &base, cb=std::move(cb)](const asio::error_code& ec, const asio::ip::tcp::endpoint& endpoint) {
if (!ec) {
auto socket = base.native_handle();
local_address_ = base.local_endpoint().address();
// Once connected, set a keep alive on the TCP socket with 30 seconds delay
// This will generate broken pipes as soon as possible.
// Note this needs to be done once connected to have a valid native_handle()
uint32_t start = 30;
uint32_t interval = 30;
uint32_t cnt = 1;
#ifdef _WIN32
std::string val = "1";
setsockopt(socket, SOL_SOCKET, SO_KEEPALIVE, val.c_str(), sizeof(val));
// TCP_KEEPIDLE and TCP_KEEPINTVL are available since Win 10 version 1709
// TCP_KEEPCNT since Win 10 version 1703
#ifdef TCP_KEEPIDLE
std::string start_str = std::to_string(start);
setsockopt(socket, IPPROTO_TCP, TCP_KEEPIDLE,
start_str.c_str(), sizeof(start_str));
#endif
#ifdef TCP_KEEPINTVL
std::string interval_str = std::to_string(interval);
setsockopt(socket, IPPROTO_TCP, TCP_KEEPINTVL,
interval_str.c_str(), sizeof(interval_str));
#endif
#ifdef TCP_KEEPCNT
std::string cnt_str = std::to_string(cnt);
setsockopt(socket, IPPROTO_TCP, TCP_KEEPCNT,
cnt_str.c_str(), sizeof(cnt_str));
#endif
#else
uint32_t val = 1;
setsockopt(socket, SOL_SOCKET, SO_KEEPALIVE, &val, sizeof(uint32_t));
#ifdef __APPLE__
// Apple devices only have one parameter
setsockopt(socket, IPPROTO_TCP, TCP_KEEPALIVE, &start, sizeof(uint32_t));
#else
// Linux based systems
setsockopt(socket, SOL_TCP, TCP_KEEPIDLE, &start, sizeof(uint32_t));
setsockopt(socket, SOL_TCP, TCP_KEEPINTVL, &interval, sizeof(uint32_t));
setsockopt(socket, SOL_TCP, TCP_KEEPCNT, &cnt, sizeof(uint32_t));
#endif
#endif
this->set_keepalive(30);
}
if (cb)
cb(ec, endpoint);
@@ -693,6 +656,64 @@ Connection::async_read_some(size_t bytes, BytesHandlerCb cb)
else socket_->async_read_some(buf, onEnd);
}
void
Connection::set_keepalive(uint32_t seconds)
{
std::lock_guard<std::mutex> lock(mutex_);
if (!ssl_socket_ && !socket_) return;
auto& base = ssl_socket_? ssl_socket_->lowest_layer() : *socket_;
auto socket = base.native_handle();
uint32_t interval = 1;
uint32_t cnt = 10;
#ifdef _WIN32
// TCP_KEEPIDLE and TCP_KEEPINTVL are available since Win 10 version 1709
// TCP_KEEPCNT since Win 10 version 1703
#if defined(TCP_KEEPIDLE) && defined(TCP_KEEPINTVL) && defined(TCP_KEEPCNT)
std::string val = "1";
setsockopt(socket, SOL_SOCKET, SO_KEEPALIVE, val.c_str(), sizeof(val));
std::string seconds_str = std::to_string(seconds);
setsockopt(socket, IPPROTO_TCP, TCP_KEEPIDLE,
seconds_str.c_str(), sizeof(seconds_str));
std::string interval_str = std::to_string(interval);
setsockopt(socket, IPPROTO_TCP, TCP_KEEPINTVL,
interval_str.c_str(), sizeof(interval_str));
std::string cnt_str = std::to_string(cnt);
setsockopt(socket, IPPROTO_TCP, TCP_KEEPCNT,
cnt_str.c_str(), sizeof(cnt_str));
#else
struct {
uint32_t onoff;
uint32_t keepalivetime;
uint32_t keepaliveinterval;
} keepalive;
keepalive.onoff = 1;
keepalive.keepalivetime = seconds * 1000;
keepalive.keepaliveinterval = interval * 1000;
int32_t out = 0;
WSAIoctl(socket, SIO_KEEPALIVE_VALS, keepalive, sizeof(tcp_keepalive),
nullptr, 0, &out, nullptr, nullptr);
#endif
#else
uint32_t val = 1;
setsockopt(socket, SOL_SOCKET, SO_KEEPALIVE, &val, sizeof(uint32_t));
#ifdef __APPLE__
// Old Apple devices only have one parameter
setsockopt(socket, IPPROTO_TCP, TCP_KEEPALIVE, &seconds, sizeof(uint32_t));
#if defined(TCP_KEEPINTVL) && defined(TCP_KEEPCNT)
setsockopt(socket, IPPROTO_TCP, TCP_KEEPINTVL, &interval, sizeof(uint32_t));
setsockopt(socket, IPPROTO_TCP, TCP_KEEPCNT, &cnt, sizeof(uint32_t));
#endif
#else
// Linux based systems
setsockopt(socket, SOL_TCP, TCP_KEEPIDLE, &seconds, sizeof(uint32_t));
setsockopt(socket, SOL_TCP, TCP_KEEPINTVL, &interval, sizeof(uint32_t));
setsockopt(socket, SOL_TCP, TCP_KEEPCNT, &cnt, sizeof(uint32_t));
#endif
#endif
}
const asio::ip::address&
Connection::local_address() const
{
@@ -1116,13 +1137,11 @@ Request::init_parser()
response_.request = shared_from_this();
if (!parser_)
parser_ = std::make_unique<http_parser>();
http_parser_init(parser_.get(), HTTP_RESPONSE);
parser_->data = static_cast<void*>(this);
parser_ = std::make_unique<llhttp_t>();
if (!parser_s_)
parser_s_ = std::make_unique<http_parser_settings>();
http_parser_settings_init(parser_s_.get());
parser_s_ = std::make_unique<llhttp_settings_t>();
llhttp_settings_init(parser_s_.get());
cbs_.on_status = [this, statusCb = std::move(cbs_.on_status)](unsigned int status_code){
response_.status_code = status_code;
@@ -1137,31 +1156,33 @@ Request::init_parser()
response_.headers[*header_field] = std::string(at, length);
};
// http_parser raw c callback (note: no context can be passed into them)
parser_s_->on_status = [](http_parser* parser, const char* /*at*/, size_t /*length*/) -> int {
// llhttp raw c callback (note: no context can be passed into them)
parser_s_->on_status = [](llhttp_t* parser, const char* /*at*/, size_t /*length*/) -> int {
static_cast<Request*>(parser->data)->cbs_.on_status(parser->status_code);
return 0;
};
parser_s_->on_header_field = [](http_parser* parser, const char* at, size_t length) -> int {
parser_s_->on_header_field = [](llhttp_t* parser, const char* at, size_t length) -> int {
static_cast<Request*>(parser->data)->cbs_.on_header_field(at, length);
return 0;
};
parser_s_->on_header_value = [](http_parser* parser, const char* at, size_t length) -> int {
parser_s_->on_header_value = [](llhttp_t* parser, const char* at, size_t length) -> int {
static_cast<Request*>(parser->data)->cbs_.on_header_value(at, length);
return 0;
};
parser_s_->on_body = [](http_parser* parser, const char* at, size_t length) -> int {
parser_s_->on_body = [](llhttp_t* parser, const char* at, size_t length) -> int {
static_cast<Request*>(parser->data)->onBody(at, length);
return 0;
};
parser_s_->on_headers_complete = [](http_parser* parser) -> int {
parser_s_->on_headers_complete = [](llhttp_t* parser) -> int {
static_cast<Request*>(parser->data)->onHeadersComplete();
return 0;
};
parser_s_->on_message_complete = [](http_parser* parser) -> int {
parser_s_->on_message_complete = [](llhttp_t* parser) -> int {
static_cast<Request*>(parser->data)->onComplete();
return 0;
};
llhttp_init(parser_.get(), HTTP_RESPONSE, parser_s_.get());
parser_->data = static_cast<void*>(this);
}
void
@@ -1318,7 +1339,7 @@ Request::terminate(const asio::error_code& ec)
logger_->debug("[http:request:{:d}] done with status code {:d}", id_, response_.status_code);
}
if (!parser_ or !http_should_keep_alive(parser_.get()))
if (!parser_ or !llhttp_should_keep_alive(parser_.get()))
if (auto c = conn_)
c->close();
notify_state_change(State::DONE);
@@ -1357,17 +1378,15 @@ Request::handle_response(const asio::error_code& ec, size_t /* n_bytes */)
return;
}
auto request = (ec == asio::error::eof) ? std::string{} : conn_->read_bytes();
size_t ret = http_parser_execute(parser_.get(), parser_s_.get(), request.c_str(), request.size());
if (ret != request.size()) {
enum llhttp_errno ret = llhttp_execute(parser_.get(), request.c_str(), request.size());
if (ret != HPE_OK && ret != HPE_PAUSED) {
if (logger_)
logger_->e("Error parsing HTTP: %zu %s %s", ret,
http_errno_name(HTTP_PARSER_ERRNO(parser_)),
http_errno_description(HTTP_PARSER_ERRNO(parser_)));
logger_->e("Error parsing HTTP: %zu %s %d", (int)ret, llhttp_errno_name(ret), llhttp_get_error_reason(parser_.get()));
terminate(asio::error::basic_errors::broken_pipe);
return;
}
if (state_ != State::DONE and parser_ and not http_body_is_final(parser_.get())) {
if (state_ != State::DONE and parser_ and not llhttp_message_needs_eof(parser_.get())) {
auto toRead = parser_->content_length ? std::min<uint64_t>(parser_->content_length, 64 * 1024) : 64 * 1024;
std::weak_ptr<Request> wthis = shared_from_this();
conn_->async_read_some(toRead, [wthis](const asio::error_code& ec, size_t bytes){
+61 -16
View File
@@ -143,7 +143,7 @@ NetworkEngine::tellListenerRefreshed(const Sp<Node>& n, Tid socket_id, const Inf
msgpack::sbuffer buffer;
msgpack::packer<msgpack::sbuffer> pk(&buffer);
pk.pack_map(4 + (version >= 1 ? 1 : 0) + (config.network?1:0));
pk.pack_map(4 + (version >= 1 ? 1 : 0) + (config.network?1:0)+(config.is_client?1:0));
pk.pack(version >= 1 ? KEY_A : KEY_U);
pk.pack_map(1 + (version >= 1 ? 1 : 0) + (not values.empty()?1:0) + (not token.empty()?1:0));
@@ -166,6 +166,9 @@ NetworkEngine::tellListenerRefreshed(const Sp<Node>& n, Tid socket_id, const Inf
if (config.network) {
pk.pack(KEY_NETID); pk.pack(config.network);
}
if (config.is_client) {
pk.pack(KEY_ISCLIENT); pk.pack(config.is_client);
}
if (version >= 1) {
Tid tid (n->getNewTid());
@@ -194,7 +197,7 @@ NetworkEngine::tellListenerExpired(const Sp<Node>& n, Tid socket_id, const InfoH
msgpack::sbuffer buffer;
msgpack::packer<msgpack::sbuffer> pk(&buffer);
pk.pack_map(4 + (version >= 1 ? 1 : 0) + (config.network?1:0));
pk.pack_map(4 + (version >= 1 ? 1 : 0) + (config.network?1:0)+(config.is_client?1:0));
pk.pack(version >= 1 ? KEY_A : KEY_U);
pk.pack_map(1 + (version >= 1 ? 1 : 0) + (not values.empty()?1:0) + (not token.empty()?1:0));
@@ -217,6 +220,9 @@ NetworkEngine::tellListenerExpired(const Sp<Node>& n, Tid socket_id, const InfoH
if (config.network) {
pk.pack(KEY_NETID); pk.pack(config.network);
}
if (config.is_client) {
pk.pack(KEY_ISCLIENT); pk.pack(config.is_client);
}
if (version >= 1) {
Tid tid (n->getNewTid());
@@ -704,7 +710,7 @@ NetworkEngine::sendPing(const Sp<Node>& node, RequestCb&& on_done, RequestExpire
Tid tid (node->getNewTid());
msgpack::sbuffer buffer;
msgpack::packer<msgpack::sbuffer> pk(&buffer);
pk.pack_map(5+(config.network?1:0));
pk.pack_map(5+(config.network?1:0)+(config.is_client?1:0));
pk.pack(KEY_A); pk.pack_map(1);
pk.pack(KEY_REQ_ID); pk.pack(myid);
@@ -716,6 +722,9 @@ NetworkEngine::sendPing(const Sp<Node>& node, RequestCb&& on_done, RequestExpire
if (config.network) {
pk.pack(KEY_NETID); pk.pack(config.network);
}
if (config.is_client) {
pk.pack(KEY_ISCLIENT); pk.pack(config.is_client);
}
auto req = std::make_shared<Request>(MessageType::Ping, tid, node,
Blob(buffer.data(), buffer.data() + buffer.size()),
@@ -741,7 +750,7 @@ void
NetworkEngine::sendPong(const SockAddr& addr, Tid tid) {
msgpack::sbuffer buffer;
msgpack::packer<msgpack::sbuffer> pk(&buffer);
pk.pack_map(4+(config.network?1:0));
pk.pack_map(4+(config.network?1:0)+(config.is_client?1:0));
pk.pack(KEY_R); pk.pack_map(2);
pk.pack(KEY_REQ_ID); pk.pack(myid);
@@ -753,6 +762,9 @@ NetworkEngine::sendPong(const SockAddr& addr, Tid tid) {
if (config.network) {
pk.pack(KEY_NETID); pk.pack(config.network);
}
if (config.is_client) {
pk.pack(KEY_ISCLIENT); pk.pack(config.is_client);
}
send(addr, buffer.data(), buffer.size());
}
@@ -763,7 +775,7 @@ NetworkEngine::sendFindNode(const Sp<Node>& n, const InfoHash& target, want_t wa
Tid tid (n->getNewTid());
msgpack::sbuffer buffer;
msgpack::packer<msgpack::sbuffer> pk(&buffer);
pk.pack_map(5+(config.network?1:0));
pk.pack_map(5+(config.network?1:0)+(config.is_client?1:0));
pk.pack(KEY_A); pk.pack_map(2 + (want>0?1:0));
pk.pack(KEY_REQ_ID); pk.pack(myid);
@@ -782,6 +794,9 @@ NetworkEngine::sendFindNode(const Sp<Node>& n, const InfoHash& target, want_t wa
if (config.network) {
pk.pack(KEY_NETID); pk.pack(config.network);
}
if (config.is_client) {
pk.pack(KEY_ISCLIENT); pk.pack(config.is_client);
}
auto req = std::make_shared<Request>(MessageType::FindNode, tid, n,
Blob(buffer.data(), buffer.data() + buffer.size()),
@@ -808,7 +823,7 @@ NetworkEngine::sendGetValues(const Sp<Node>& n, const InfoHash& info_hash, const
Tid tid (n->getNewTid());
msgpack::sbuffer buffer;
msgpack::packer<msgpack::sbuffer> pk(&buffer);
pk.pack_map(5+(config.network?1:0));
pk.pack_map(5+(config.network?1:0)+(config.is_client?1:0));
unsigned sendQuery = (not query.where.empty() or not query.select.empty()) ? 1 : 0;
unsigned sendWant = (want > 0) ? 1 : 0;
@@ -835,7 +850,10 @@ NetworkEngine::sendGetValues(const Sp<Node>& n, const InfoHash& info_hash, const
if (config.network) {
pk.pack(KEY_NETID); pk.pack(config.network);
}
if (config.is_client) {
pk.pack(KEY_ISCLIENT); pk.pack(config.is_client);
}
auto req = std::make_shared<Request>(MessageType::GetValues, tid, n,
Blob(buffer.data(), buffer.data() + buffer.size()),
[=](const Request& req_status, ParsedMessage&& msg) { /* on done */
@@ -952,10 +970,13 @@ NetworkEngine::sendValueParts(Tid tid, const std::vector<Blob>& svals, const Soc
end = std::min(start + MTU, v.size());
buffer.clear();
msgpack::packer<msgpack::sbuffer> pk(&buffer);
pk.pack_map(3+(config.network?1:0));
pk.pack_map(3+(config.network?1:0)+(config.is_client?1:0));
if (config.network) {
pk.pack(KEY_NETID); pk.pack(config.network);
}
if (config.is_client) {
pk.pack(KEY_ISCLIENT); pk.pack(config.is_client);
}
pk.pack(KEY_Y); pk.pack(KEY_V);
pk.pack(KEY_TID); pk.pack(tid);
pk.pack(KEY_V); pk.pack_map(1);
@@ -976,7 +997,7 @@ NetworkEngine::sendNodesValues(const SockAddr& addr, Tid tid, const Blob& nodes,
{
msgpack::sbuffer buffer;
msgpack::packer<msgpack::sbuffer> pk(&buffer);
pk.pack_map(4+(config.network?1:0));
pk.pack_map(4+(config.network?1:0)+(config.is_client?1:0));
pk.pack(KEY_R);
pk.pack_map(2 + (not st.empty()?1:0) + (nodes.size()>0?1:0) + (nodes6.size()>0?1:0) + (not token.empty()?1:0));
@@ -1018,6 +1039,9 @@ NetworkEngine::sendNodesValues(const SockAddr& addr, Tid tid, const Blob& nodes,
if (config.network) {
pk.pack(KEY_NETID); pk.pack(config.network);
}
if (config.is_client) {
pk.pack(KEY_ISCLIENT); pk.pack(config.is_client);
}
// send response
send(addr, buffer.data(), buffer.size());
@@ -1089,7 +1113,7 @@ NetworkEngine::sendListen(const Sp<Node>& n,
Tid tid (n->getNewTid());
msgpack::sbuffer buffer;
msgpack::packer<msgpack::sbuffer> pk(&buffer);
pk.pack_map(5+(config.network?1:0));
pk.pack_map(5+(config.network?1:0)+(config.is_client?1:0));
auto has_query = not query.where.empty() or not query.select.empty();
pk.pack(KEY_A); pk.pack_map(5 + has_query);
@@ -1109,6 +1133,9 @@ NetworkEngine::sendListen(const Sp<Node>& n,
if (config.network) {
pk.pack(KEY_NETID); pk.pack(config.network);
}
if (config.is_client) {
pk.pack(KEY_ISCLIENT); pk.pack(config.is_client);
}
auto req = std::make_shared<Request>(MessageType::Listen, tid, n,
Blob(buffer.data(), buffer.data() + buffer.size()),
@@ -1130,7 +1157,7 @@ void
NetworkEngine::sendListenConfirmation(const SockAddr& addr, Tid tid) {
msgpack::sbuffer buffer;
msgpack::packer<msgpack::sbuffer> pk(&buffer);
pk.pack_map(4+(config.network?1:0));
pk.pack_map(4+(config.network?1:0)+(config.is_client?1:0));
pk.pack(KEY_R); pk.pack_map(2);
pk.pack(KEY_REQ_ID); pk.pack(myid);
@@ -1142,6 +1169,9 @@ NetworkEngine::sendListenConfirmation(const SockAddr& addr, Tid tid) {
if (config.network) {
pk.pack(KEY_NETID); pk.pack(config.network);
}
if (config.is_client) {
pk.pack(KEY_ISCLIENT); pk.pack(config.is_client);
}
send(addr, buffer.data(), buffer.size());
}
@@ -1158,7 +1188,7 @@ NetworkEngine::sendAnnounceValue(const Sp<Node>& n,
Tid tid (n->getNewTid());
msgpack::sbuffer buffer;
msgpack::packer<msgpack::sbuffer> pk(&buffer);
pk.pack_map(5+(config.network?1:0));
pk.pack_map(5+(config.network?1:0)+(config.is_client?1:0));
bool add_created = created < scheduler.time();
pk.pack(KEY_A); pk.pack_map(add_created ? 5 : 4);
@@ -1178,6 +1208,9 @@ NetworkEngine::sendAnnounceValue(const Sp<Node>& n,
if (config.network) {
pk.pack(KEY_NETID); pk.pack(config.network);
}
if (config.is_client) {
pk.pack(KEY_ISCLIENT); pk.pack(config.is_client);
}
auto req = std::make_shared<Request>(MessageType::AnnounceValue, tid, n,
Blob(buffer.data(), buffer.data() + buffer.size()),
@@ -1244,7 +1277,7 @@ NetworkEngine::sendUpdateValues(const Sp<Node>& n,
msgpack::sbuffer buffer;
msgpack::packer<msgpack::sbuffer> pk(&buffer);
pk.pack_map(5+(config.network?1:0));
pk.pack_map(5+(config.network?1:0)+(config.is_client?1:0));
pk.pack(KEY_A); pk.pack_map((created < scheduler.time() ? 7 : 6));
pk.pack(KEY_REQ_ID); pk.pack(myid);
@@ -1265,6 +1298,9 @@ NetworkEngine::sendUpdateValues(const Sp<Node>& n,
if (config.network) {
pk.pack(KEY_NETID); pk.pack(config.network);
}
if (config.is_client) {
pk.pack(KEY_ISCLIENT); pk.pack(config.is_client);
}
auto req = std::make_shared<Request>(MessageType::UpdateValue, tid, n,
Blob(buffer.data(), buffer.data() + buffer.size()),
@@ -1289,7 +1325,7 @@ NetworkEngine::sendRefreshValue(const Sp<Node>& n,
Tid tid (n->getNewTid());
msgpack::sbuffer buffer;
msgpack::packer<msgpack::sbuffer> pk(&buffer);
pk.pack_map(5+(config.network?1:0));
pk.pack_map(5+(config.network?1:0)+(config.is_client?1:0));
pk.pack(KEY_A); pk.pack_map(4);
pk.pack(KEY_REQ_ID); pk.pack(myid);
@@ -1304,6 +1340,9 @@ NetworkEngine::sendRefreshValue(const Sp<Node>& n,
if (config.network) {
pk.pack(KEY_NETID); pk.pack(config.network);
}
if (config.is_client) {
pk.pack(KEY_ISCLIENT); pk.pack(config.is_client);
}
auto req = std::make_shared<Request>(MessageType::Refresh, tid, n,
Blob(buffer.data(), buffer.data() + buffer.size()),
@@ -1335,7 +1374,7 @@ void
NetworkEngine::sendValueAnnounced(const SockAddr& addr, Tid tid, Value::Id vid) {
msgpack::sbuffer buffer;
msgpack::packer<msgpack::sbuffer> pk(&buffer);
pk.pack_map(4+(config.network?1:0));
pk.pack_map(4+(config.network?1:0)+(config.is_client?1:0));
pk.pack(KEY_R); pk.pack_map(3);
pk.pack(KEY_REQ_ID); pk.pack(myid);
@@ -1348,6 +1387,9 @@ NetworkEngine::sendValueAnnounced(const SockAddr& addr, Tid tid, Value::Id vid)
if (config.network) {
pk.pack(KEY_NETID); pk.pack(config.network);
}
if (config.is_client) {
pk.pack(KEY_ISCLIENT); pk.pack(config.is_client);
}
send(addr, buffer.data(), buffer.size());
}
@@ -1361,7 +1403,7 @@ NetworkEngine::sendError(const SockAddr& addr,
{
msgpack::sbuffer buffer;
msgpack::packer<msgpack::sbuffer> pk(&buffer);
pk.pack_map(4 + (include_id?1:0) + (config.network?1:0));
pk.pack_map(4 + (include_id?1:0) + (config.network?1:0)+(config.is_client?1:0));
pk.pack(KEY_E); pk.pack_array(2);
pk.pack(code);
@@ -1378,6 +1420,9 @@ NetworkEngine::sendError(const SockAddr& addr,
if (config.network) {
pk.pack(KEY_NETID); pk.pack(config.network);
}
if (config.is_client) {
pk.pack(KEY_ISCLIENT); pk.pack(config.is_client);
}
send(addr, buffer.data(), buffer.size());
}
+20 -16
View File
@@ -25,11 +25,16 @@
#include <cstring>
#define close(x) closesocket(x)
#define write(s, b, f) send(s, b, (int)strlen(b), 0)
#define _poll(fds, nfds, timeout) WSAPoll(fds, nfds, timeout)
#else
#include <sys/select.h>
#include <fcntl.h>
#include <poll.h>
#define _poll(fds, nfds, timeout) poll(fds, nfds, timeout)
#endif
// number of file descriptors to poll in openSockets()
#define NUM_FDS 3
#include <iostream>
namespace dht {
@@ -235,23 +240,23 @@ UdpSocket::openSockets(const SockAddr& bind4, const SockAddr& bind6)
running = true;
rcv_thread = std::thread([this, stop_readfd, ls4=s4, ls6=s6]() mutable {
int selectFd = std::max({ls4, ls6, stop_readfd}) + 1;
struct pollfd fds[NUM_FDS];
for (int i = 0; i < NUM_FDS; i++)
fds[i].events = POLLIN;
constexpr size_t stop_readfd_index = 0, ls4_index = 1, ls6_index = 2;
try {
while (running) {
fd_set readfds;
// ls4 and ls6 can be negative, but this doesn't require any special handling
// because poll() will simply ignore them (and set revents to 0) in that case
fds[stop_readfd_index].fd = stop_readfd;
fds[ls4_index].fd = ls4;
fds[ls6_index].fd = ls6;
FD_ZERO(&readfds);
FD_SET(stop_readfd, &readfds);
if(ls4 >= 0)
FD_SET(ls4, &readfds);
if(ls6 >= 0)
FD_SET(ls6, &readfds);
int rc = select(selectFd, &readfds, nullptr, nullptr, nullptr);
int rc = _poll(fds, NUM_FDS, -1);
if (rc < 0) {
if (errno != EINTR) {
if (logger)
logger->e("Select error: %s", strerror(errno));
logger->e("Poll error: %s", strerror(errno));
std::this_thread::sleep_for(std::chrono::seconds(1));
}
}
@@ -264,16 +269,16 @@ UdpSocket::openSockets(const SockAddr& bind4, const SockAddr& bind6)
sockaddr_storage from;
socklen_t from_len = sizeof(from);
if (FD_ISSET(stop_readfd, &readfds)) {
if (fds[stop_readfd_index].revents & POLLIN) {
if (recv(stop_readfd, (char*)buf.data(), buf.size(), 0) < 0) {
if (logger)
logger->e("Got stop packet error: %s", strerror(errno));
break;
}
}
else if (ls4 >= 0 && FD_ISSET(ls4, &readfds))
else if (fds[ls4_index].revents & POLLIN)
rc = recvfrom(ls4, (char*)buf.data(), buf.size(), 0, (sockaddr*)&from, &from_len);
else if (ls6 >= 0 && FD_ISSET(ls6, &readfds))
else if (fds[ls6_index].revents & POLLIN)
rc = recvfrom(ls6, (char*)buf.data(), buf.size(), 0, (sockaddr*)&from, &from_len);
else
continue;
@@ -316,7 +321,6 @@ UdpSocket::openSockets(const SockAddr& bind4, const SockAddr& bind6)
break;
s4 = ls4;
s6 = ls6;
selectFd = std::max({ls4, ls6, stop_readfd}) + 1;
} else {
break;
}
+1 -2
View File
@@ -97,8 +97,7 @@ public:
}) {}
bool onValue(const std::vector<Sp<Value>>& vals, bool expired) {
cache.onValue(vals, expired);
return not listeners.empty();
return cache.onValue(vals, expired);
}
void onNodeChanged(ListenSyncStatus status) {
cache.onNodeChanged(status);
+61 -3
View File
@@ -22,6 +22,10 @@
#include "network_utils.h"
#include "utils.h"
#ifdef __ANDROID__
#include "compat/getif_workaround_android.h"
#endif
#include <asio.hpp>
using namespace std::literals;
@@ -48,6 +52,7 @@ public:
void connectivityChanged();
void stopConnectivityChanged();
private:
Sp<Logger> logger_;
//dmtx_ for callbackmap_ and drunning_ (write)
@@ -55,6 +60,14 @@ private:
//mtx_ for messages_ and lrunning (listen)
std::mutex mtx_;
std::shared_ptr<asio::io_context> ioContext_;
static constexpr dht::duration PeerDiscovery_PERIOD_MAX{
std::chrono::minutes(1)};
static constexpr std::chrono::seconds PeerDiscovery_PERIOD{10};
asio::steady_timer peerDiscoveryTimer;
std::chrono::steady_clock::duration peerDiscovery_period{
PeerDiscovery_PERIOD};
asio::ip::udp::socket sockFd_;
asio::ip::udp::endpoint sockAddrSend_;
@@ -82,6 +95,7 @@ private:
PeerDiscovery::DomainPeerDiscovery::DomainPeerDiscovery(asio::ip::udp domain, in_port_t port, Sp<asio::io_context> ioContext, Sp<Logger> logger)
: logger_(logger)
, ioContext_(ioContext)
, peerDiscoveryTimer(*ioContext_)
, sockFd_(*ioContext_, domain)
, sockAddrSend_(asio::ip::address::from_string(domain.family() == AF_INET ? MULTICAST_ADDRESS_IPV4
: MULTICAST_ADDRESS_IPV6), port)
@@ -91,8 +105,23 @@ PeerDiscovery::DomainPeerDiscovery::DomainPeerDiscovery(asio::ip::udp domain, in
sockFd_.set_option(asio::ip::udp::socket::reuse_address(true));
sockFd_.bind({domain, port});
} catch (const std::exception& e) {
#ifdef __ANDROID__
if(domain.family() == AF_INET && strcmp(e.what(), "set_option: No such device") == 0) {
try{
sockFd_.set_option(asio::ip::udp::socket::reuse_address(true));
auto mc_interface = workaround::get_interface();
sockFd_.set_option(asio::ip::multicast::outbound_interface(mc_interface));
sockFd_.set_option(asio::ip::multicast::join_group(sockAddrSend_.address().to_v4(), mc_interface));
sockFd_.bind({domain, port});
} catch(const std::exception& e){
if (logger_)
logger_->error("Can't start peer discovery using android workaround: {}", e.what());
}
}
else
#endif
if (logger_)
logger_->e("Can't start peer discovery for %s: %s",
logger_->error("Can't start peer discovery for {}: {}",
domain.family() == AF_INET ? "IPv4" : "IPv6", e.what());
}
}
@@ -298,8 +327,30 @@ PeerDiscovery::DomainPeerDiscovery::reDiscover()
void
PeerDiscovery::DomainPeerDiscovery::connectivityChanged()
{
reDiscover();
publish(sockAddrSend_);
ioContext_->post([this] () {
reDiscover();
publish(sockAddrSend_);
});
if (logger_)
logger_->d("PeerDiscovery: connectivity changed");
if (peerDiscovery_period == PeerDiscovery_PERIOD_MAX ){
peerDiscovery_period = PeerDiscovery_PERIOD;
}
else{
peerDiscoveryTimer.expires_after(peerDiscovery_period);
peerDiscoveryTimer.async_wait([this](const asio::error_code& ec) {
if (ec == asio::error::operation_aborted)
return;
connectivityChanged();
});
peerDiscovery_period = std::min(peerDiscovery_period * 2, PeerDiscovery_PERIOD_MAX);
}
}
void PeerDiscovery::DomainPeerDiscovery::stopConnectivityChanged() {
peerDiscoveryTimer.cancel();
peerDiscovery_period = PeerDiscovery_PERIOD;
}
PeerDiscovery::PeerDiscovery(in_port_t port, Sp<asio::io_context> ioContext, Sp<Logger> logger)
@@ -412,4 +463,11 @@ PeerDiscovery::connectivityChanged()
peerDiscovery6_->connectivityChanged();
}
void PeerDiscovery::stopConnectivityChanged() {
if (peerDiscovery4_)
peerDiscovery4_->stopConnectivityChanged();
if (peerDiscovery6_)
peerDiscovery6_->stopConnectivityChanged();
}
} /* namespace dht */
+26 -19
View File
@@ -45,7 +45,7 @@ struct Dht::Announce {
bool permanent;
Sp<Value> value;
time_point created;
DoneCallback callback;
std::vector<DoneCallback> callbacks;
};
struct Dht::SearchNode {
@@ -442,8 +442,9 @@ struct Dht::Search {
g.second.done_cb = {};
}
for (auto& a : announce) {
a.callback(false, {});
a.callback = {};
for (auto& cb : a.callbacks)
cb(false, {});
a.callbacks.clear();
}
}
@@ -627,7 +628,9 @@ struct Dht::Search {
return a.value->id == value->id;
});
if (a_sr == announce.end()) {
announce.emplace_back(Announce {permanent, value, created, callback});
auto& a = announce.emplace_back(Announce {permanent, value, created, {}} );
if (callback)
a.callbacks.emplace_back(std::move(callback));
for (auto& n : nodes) {
n->probe_query.reset();
n->acked[value->id].req.reset();
@@ -635,24 +638,26 @@ struct Dht::Search {
} else {
a_sr->permanent = permanent;
a_sr->created = created;
if (a_sr->value != value) {
if (a_sr->value != value && *a_sr->value != *value) {
// Value is updated, previous ops are failed
a_sr->value = value;
for (auto& n : nodes) {
n->acked[value->id].req.reset();
n->probe_query.reset();
}
}
if (isAnnounced(value->id)) {
if (a_sr->callback)
a_sr->callback(true, {});
a_sr->callback = {};
for (auto& cb: a_sr->callbacks)
cb(false, {});
a_sr->callbacks.clear();
if (callback)
a_sr->callbacks.emplace_back(std::move(callback));
} else if (isAnnounced(value->id)) {
// Same value, already announced
if (callback)
callback(true, {});
return;
} else {
if (a_sr->callback)
a_sr->callback(false, {});
a_sr->callback = callback;
// Same value, not announced yet
if (callback)
a_sr->callbacks.emplace_back(std::move(callback));
}
}
}
@@ -722,8 +727,8 @@ struct Dht::Search {
std::vector<DoneCallback> a_cbs;
a_cbs.reserve(announce.size());
for (auto ait = announce.begin() ; ait != announce.end(); ) {
if (ait->callback)
a_cbs.emplace_back(std::move(ait->callback));
a_cbs.insert(a_cbs.end(), std::make_move_iterator(ait->callbacks.begin()), std::make_move_iterator(ait->callbacks.end()));
ait->callbacks.clear();
if (not ait->permanent)
ait = announce.erase(ait);
else
@@ -747,9 +752,11 @@ struct Dht::Search {
if (vid != Value::INVALID_ID and (!a.value || a.value->id != vid))
return true;
if (isAnnounced(a.value->id)) {
if (a.callback) {
a.callback(true, getNodes());
a.callback = nullptr;
if (!a.callbacks.empty()) {
const auto& nodes = getNodes();
for (auto& cb : a.callbacks)
cb(true, nodes);
a.callbacks.clear();
}
if (not a.permanent)
return false;
+11 -1
View File
@@ -85,7 +85,7 @@ ThreadPool::run(std::function<void()>&& cb)
auto waitCond = [&](){ return not running_ or not tasks_.empty(); };
if (permanent_thread)
cv_.wait(l, waitCond);
else
else
cv_.wait_for(l, e, waitCond);
readyThreads_--;
if (not running_ or tasks_.empty())
@@ -166,6 +166,16 @@ ThreadPool::join()
tasks_ = {};
}
void
ThreadPool::detach()
{
stop(false);
for (auto& t : threads_)
t->detach();
threads_.clear();
tasks_ = {};
}
void
Executor::run(std::function<void()>&& task)
{
+1
View File
@@ -271,6 +271,7 @@ Value::encrypt(const crypto::PrivateKey& from, const crypto::PublicKey& to)
setRecipient(to.getId());
sign(from);
Value nv {id};
nv.priority = priority;
nv.setCypher(to.encrypt(getToEncrypt()));
return nv;
}
+21
View File
@@ -239,6 +239,27 @@ void CryptoTester::testAesEncryption() {
CPPUNIT_ASSERT(data2 == decrypted2);
}
void CryptoTester::testAesEncryptionWithMultipleKeySizes() {
auto data = std::vector<uint8_t>(rand(), rand());
// Valid key sizes
for (auto key_length : {16, 24, 32}) {
auto key = std::vector<uint8_t>(key_length, rand());
auto encrypted_data = dht::crypto::aesEncrypt(data, key);
auto decrypted_data = dht::crypto::aesDecrypt(encrypted_data, key);
CPPUNIT_ASSERT(data == decrypted_data);
}
// Invalid key sizes
for (auto key_length : {12, 28, 36}) {
auto key = std::vector<uint8_t>(key_length, rand());
CPPUNIT_ASSERT_THROW(dht::crypto::aesEncrypt(data, key), dht::crypto::DecryptError);
}
}
void
CryptoTester::tearDown() {
+2
View File
@@ -34,6 +34,7 @@ class CryptoTester : public CppUnit::TestFixture {
CPPUNIT_TEST(testCertificateSerialNumber);
CPPUNIT_TEST(testOcsp);
CPPUNIT_TEST(testAesEncryption);
CPPUNIT_TEST(testAesEncryptionWithMultipleKeySizes);
CPPUNIT_TEST_SUITE_END();
public:
@@ -69,6 +70,7 @@ class CryptoTester : public CppUnit::TestFixture {
* Test key streching and aes encryption/decryption
*/
void testAesEncryption();
void testAesEncryptionWithMultipleKeySizes();
};
} // namespace test
+179
View File
@@ -0,0 +1,179 @@
/*
* Copyright (C) 2014-2024 Savoir-faire Linux Inc.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
#include "dhtproxy_stress_tester.h"
#include <iostream>
#include <string>
#include <chrono>
#include <condition_variable>
using namespace std::chrono_literals;
namespace test {
CPPUNIT_TEST_SUITE_REGISTRATION(DhtProxyStressTester);
void
DhtProxyStressTester::setUp() {
logger = dht::log::getStdLogger();
dht::DhtRunner::Context ctx;
ctx.logger = logger;
clientConfig.dht_config.node_config.max_peer_req_per_sec = -1;
clientConfig.dht_config.node_config.max_req_per_sec = -1;
nodePeer.run(0, clientConfig, std::move(ctx));
ctx.logger = logger;
nodeProxy = std::make_shared<dht::DhtRunner>();
nodeProxy->run(0, clientConfig, std::move(ctx));
nodeProxy->bootstrap(nodePeer.getBound());
dht::ProxyServerConfig serverConfig;
serverConfig.port = 8084;
serverProxy = std::make_unique<dht::DhtProxyServer>(nodeProxy, serverConfig, logger);
clientConfig.proxy_server = "http://127.0.0.1:8084";
}
void
DhtProxyStressTester::tearDown() {
nodePeer.join();
nodeClient.join();
bool done = false;
std::condition_variable cv;
std::mutex cv_m;
nodeProxy->shutdown([&]{
std::lock_guard<std::mutex> lk(cv_m);
done = true;
cv.notify_all();
});
std::unique_lock<std::mutex> lk(cv_m);
CPPUNIT_ASSERT(cv.wait_for(lk, 15s, [&]{ return done; }));
serverProxy.reset();
nodeProxy.reset();
}
void
DhtProxyStressTester::testRepeatValues() {
dht::DhtRunner::Context ctx;
ctx.logger = logger;
nodeClient.run(0, clientConfig, std::move(ctx));
std::condition_variable cv;
std::mutex cv_m;
std::unique_lock<std::mutex> lk(cv_m);
auto key = dht::InfoHash::get("GLaDOs");
bool done = false;
// If a peer send a value, the listen operation from the client
// should retrieve this value
dht::Value firstVal {"Hey! It's been a long time. How have you been?"};
auto firstVal_data = firstVal.data;
nodePeer.put(key, std::move(firstVal), [&](bool ok) {
CPPUNIT_ASSERT(ok);
std::lock_guard<std::mutex> lk(cv_m);
done = true;
cv.notify_all();
});
CPPUNIT_ASSERT(cv.wait_for(lk, 10s, [&]{ return done; }));
done = false;
std::vector<dht::Blob> values;
auto token = nodeClient.listen(key, [&](const std::vector<std::shared_ptr<dht::Value>>& v, bool expired) {
if (not expired) {
std::lock_guard<std::mutex> lk(cv_m);
for (const auto& value : v)
values.emplace_back(value->data);
done = true;
cv.notify_all();
}
return true;
});
CPPUNIT_ASSERT(cv.wait_for(lk, 10s, [&]{ return done; }));
done = false;
// Here values should contains 1 values
CPPUNIT_ASSERT_EQUAL(static_cast<int>(values.size()), 1);
CPPUNIT_ASSERT(values.front() == firstVal_data);
// And the listen should retrieve futures values
dht::Value secondVal {"You're a monster"};
auto secondVal_data = secondVal.data;
nodePeer.put(key, std::move(secondVal));
CPPUNIT_ASSERT(cv.wait_for(lk, 10s, [&]{ return done; }));
nodeClient.cancelListen(key, std::move(token));
CPPUNIT_ASSERT_EQUAL(static_cast<int>(values.size()), 2);
CPPUNIT_ASSERT(values.back() == secondVal_data);
done = false;
clientConfig.proxy_server = {};
constexpr auto DELAY = 60s;
dht::Value::Id id = 42;
for (auto now = std::chrono::steady_clock::now(), end = now + 12min; now < end; now += DELAY) {
auto tmpNode = std::make_shared<dht::DhtRunner>();
dht::DhtRunner::Context ctx;
ctx.logger = logger;
tmpNode->run(0, clientConfig, std::move(ctx));
tmpNode->bootstrap(nodePeer.getBound());
// 1 minute
std::this_thread::sleep_until(now + DELAY);
auto val = std::make_shared<dht::Value>("I'm doing science and I'm still alive");
val->id = id++;
nodePeer.put(key, val, [&](bool ok) {
CPPUNIT_ASSERT(ok);
std::lock_guard<std::mutex> lk(cv_m);
done = true;
cv.notify_all();
});
CPPUNIT_ASSERT(cv.wait_for(lk, 20s, [&]{ return done; }));
done = false;
// temporary node to make sure the value is retrieved
auto tmpValues = tmpNode->get(key);
auto token = nodeClient.listen(key, [&, val](const std::vector<std::shared_ptr<dht::Value>>& v, bool expired) {
if (not expired) {
for (const auto& value : v) {
if (value->id == val->id) {
std::lock_guard<std::mutex> lk(cv_m);
done = true;
cv.notify_all();
break;
}
}
}
return true;
});
CPPUNIT_ASSERT(cv.wait_for(lk, 25s, [&]{ return done; }));
done = false;
bool found_value = false;
for (const auto& value : tmpValues.get()) {
if (value->data == val->data) {
found_value = true;
break;
}
}
CPPUNIT_ASSERT(found_value);
nodeClient.cancelListen(key, std::move(token));
tmpNode->shutdown();
}
}
} // namespace test
+59
View File
@@ -0,0 +1,59 @@
/*
* Copyright (C) 2014-2023 Savoir-faire Linux Inc.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
#pragma once
// cppunit
#include <cppunit/TestFixture.h>
#include <cppunit/extensions/HelperMacros.h>
#include <opendht/dhtrunner.h>
#include <opendht/dht_proxy_server.h>
#include <opendht/log.h>
namespace test {
class DhtProxyStressTester : public CppUnit::TestFixture {
CPPUNIT_TEST_SUITE(DhtProxyStressTester);
CPPUNIT_TEST(testRepeatValues);
CPPUNIT_TEST_SUITE_END();
public:
/**
* Method automatically called before each test by CppUnit
* Init nodes
*/
void setUp();
/**
* Method automatically called after each test CppUnit
*/
void tearDown();
/**
* Test get and put methods
*/
void testRepeatValues();
private:
std::shared_ptr<dht::log::Logger> logger;
dht::DhtRunner::Config clientConfig {};
dht::DhtRunner nodePeer;
dht::DhtRunner nodeClient;
std::shared_ptr<dht::DhtRunner> nodeProxy;
std::unique_ptr<dht::DhtProxyServer> serverProxy;
};
} // namespace test
+5 -3
View File
@@ -43,18 +43,20 @@ DhtProxyTester::setUp() {
nodeProxy->run(0, clientConfig);
nodeProxy->bootstrap(nodePeer.getBound());
auto serverCAIdentity = dht::crypto::generateEcIdentity("DHT Node CA");
//auto serverCAIdentity = dht::crypto::generateEcIdentity("DHT Node CA");
uint16_t port = 1024 + (std::rand() % (65535 - 1024));
dht::ProxyServerConfig serverConfig;
//serverConfig.identity = dht::crypto::generateIdentity("DHT Node", serverCAIdentity);
serverConfig.port = 8080;
serverConfig.port = port;
// serverConfig.pushServer = "127.0.0.1:8090";
serverProxy = std::make_unique<dht::DhtProxyServer>(nodeProxy, serverConfig);
/*clientConfig.server_ca = serverCAIdentity.second;
clientConfig.client_identity = dht::crypto::generateIdentity("DhtProxyTester");
clientConfig.push_node_id = "dhtnode";*/
clientConfig.proxy_server = "http://127.0.0.1:8080";//"https://127.0.0.1:8080";
clientConfig.proxy_server = "http://127.0.0.1:" + std::to_string(port);//"https://127.0.0.1:8080";
}
void
+54 -1
View File
@@ -90,6 +90,59 @@ DhtRunnerTester::testGetPut() {
CPPUNIT_ASSERT(vals.front()->data == val_data);
}
void
DhtRunnerTester::testPutDuplicate() {
auto key = dht::InfoHash::get("123");
auto val = std::make_shared<dht::Value>("hey");
val->id = 42;
auto val_data = val->data;
std::promise<bool> p1;
std::promise<bool> p2;
node2.put(key, val, [&](bool ok){
p1.set_value(ok);
});
node2.put(key, val, [&](bool ok){
p2.set_value(ok);
});
auto p1ret = p1.get_future().get();
auto p2ret = p2.get_future().get();
CPPUNIT_ASSERT(p1ret);
CPPUNIT_ASSERT(p2ret);
auto vals = node1.get(key).get();
CPPUNIT_ASSERT(not vals.empty());
CPPUNIT_ASSERT(vals.size() == 1);
CPPUNIT_ASSERT(vals.front()->data == val_data);
}
void
DhtRunnerTester::testPutOverride() {
auto key = dht::InfoHash::get("123");
auto val = std::make_shared<dht::Value>("meh");
val->id = 42;
auto val2 = std::make_shared<dht::Value>("hey");
val2->id = 42;
CPPUNIT_ASSERT_EQUAL(val->id, val2->id);
auto val_data = val->data;
std::promise<bool> p1;
std::promise<bool> p2;
node2.put(key, val, [&](bool ok){
p1.set_value(ok);
});
node2.put(key, val2, [&](bool ok){
p2.set_value(ok);
});
auto p1ret = p1.get_future().get();
auto p2ret = p2.get_future().get();
CPPUNIT_ASSERT(!p1ret);
CPPUNIT_ASSERT(p2ret);
auto vals = node1.get(key).get();
CPPUNIT_ASSERT(not vals.empty());
CPPUNIT_ASSERT(vals.size() == 1);
CPPUNIT_ASSERT(vals.front()->data == val_data);
}
void
DhtRunnerTester::testListen() {
std::mutex mutex;
@@ -289,7 +342,7 @@ DhtRunnerTester::testIdOps() {
valueCountEdit++;
cv.notify_all();
});
node2.listen(key2, [&](const std::vector<std::shared_ptr<dht::Value>>& values, bool expired){
node2.listen(key2, [&](const std::vector<std::shared_ptr<dht::Value>>& values, bool /*expired*/){
for (const auto& v : values) {
if (v->seq == 0)
CPPUNIT_ASSERT_EQUAL("v1"s, dht::unpackMsg<std::string>(v->data));
+9
View File
@@ -31,6 +31,7 @@ class DhtRunnerTester : public CppUnit::TestFixture {
CPPUNIT_TEST_SUITE(DhtRunnerTester);
CPPUNIT_TEST(testConstructors);
CPPUNIT_TEST(testGetPut);
CPPUNIT_TEST(testPutDuplicate);
CPPUNIT_TEST(testListen);
CPPUNIT_TEST(testListenLotOfBytes);
CPPUNIT_TEST(testIdOps);
@@ -55,6 +56,14 @@ class DhtRunnerTester : public CppUnit::TestFixture {
* Test get and put methods
*/
void testGetPut();
/**
* Test get and multiple put
*/
void testPutDuplicate();
/**
* Test get and multiple put with changing value
*/
void testPutOverride();
/**
* Test listen method
*/