Fragments carried 1280 bytes of value data, plus the msgpack framing and,
for encrypted peers, the envelope overhead: packets exceeded the 1280-byte
IPv6 minimum MTU. Derive both fragment sizes from the 1232-byte UDP payload
that fits in it, less an upper bound of the fragment framing.
Value data fragments carry no node ID and were merged into a partial
message by source address and transaction ID alone, first come first
served. Transaction IDs increment from a random start and show in any
plaintext packet of a pair, such as the first-contact ping. An attacker
spoofing the sender's address could thus pre-seed forged plaintext
fragments, or a plaintext header, for an encrypted reply: the real parts
were then dropped as overlapping, forging unsigned values or losing the
message.
Parts of a message must now all be plaintext, or all come in envelopes
from the same sender key as its first part. An encrypted part replaces
plaintext state, which anyone can forge from a spoofed address; plaintext
or foreign-key parts never join an encrypted message.
Value data fragments carry no node ID. Parts of a message received in
envelopes must all come under the sender key of the envelopes, so that
plaintext or foreign-key parts spoofed from the sender's address, which
would otherwise be merged first, are ignored.
A session was cached for an address whenever we answered a peer there,
including replies to Initial and Hello envelopes, whose source address is
not verified. A spoofed Initial or Hello therefore replaced the session of
the real owner of the address, whose Data envelopes were then dropped until
a retry, and spoofed Initials from many addresses filled the cache and
evicted legitimate sessions, which were picked by address order.
Only cache a session for a peer whose ephemeral key arrived in a Session
or Data envelope, i.e. opened with our key for its address. Replies to
Initial and Hello envelopes derive their keys without caching them, so a
node that sent an Initial only confirms the session, and switches to Data
envelopes, once the peer answered a Session envelope. Evict the least
recently used session when the cache is full.
Replies to Initial and Hello envelopes must not cache a session at the
source address, which can be spoofed, and the cache must evict the least
recently used session rather than the lowest address. A node that started
with an Initial envelope only switches to Data envelopes once the peer
answered a Session envelope, which is when the peer caches the session.
Sessions are keyed by peer address, so the receiver finds the keys from the source address: Data envelopes drop both keys from the header (30 bytes of overhead instead of 94) and need no key derivation to reject. A session is confirmed when the peer answers our envelope request with the session keys, or sends Data itself. Credentials only travel in key-carrying envelopes, which replaces Peer::acked. A received Initial envelope unconfirms the session, and full session caches evict instead of refusing, so a peer that derived keys keeps them.
Steady-state traffic uses Data envelopes, without sender keys nor credential, once the peer answered a request. Data envelopes only open from the session address, also authorize writes, and survive the receiver's epoch rotation.
Ephemeral keys derive from a per-epoch secret and the peer address, so only that address learns them. A Session envelope then proves return routability: encrypted put, refresh and listen are accepted without token, and messages to encrypted peers no longer carry one (40 bytes saved). Initial envelopes cannot write. Tokens for plaintext peers move from Dht to NetworkEngine and derive from the same epoch secret. Session caches are keyed by address, and cached keys of both epochs are tried before any derivation.
Encrypted writes must come in a Session envelope, whose ephemeral key is bound to the sender address. Plaintext writes still need a token, now issued by the network engine.
Each node rotates an ephemeral X25519 key every 30 minutes and keeps the
previous one for another period. Envelopes carry the sender static and
ephemeral keys. Session envelopes use keys derived from all four DH
combinations of static and ephemeral keys, erased with the epoch.
Initial envelopes, sent before the peer's ephemeral key is known, mix
the sender ephemeral and static keys with the receiver static key.
A hello is unauthenticated: it is only answered, without adding or
moving its claimed node. A peer is acknowledged only by a reply to a
request sent in an envelope, and a request retry restarts with an
Initial envelope and our credential.
The previous ephemeral key is accepted for one more period, then erased.
A request retry restarts with an Initial envelope and our credential.
A hello is only answered and doesn't change what is known of its sender.
With persist_path, a transport key generated by the node is saved in the
state (owner-only file permissions) and restored on start, keeping the
node ID across restarts. Configured keys are never written to the state.
--no-encryption, --transport-key, --network-authority and
--network-credential configure the node; --gen-network-authority and
--issue-credential manage a private network.
Nodes using transport encryption derive their node ID from their X25519
transport key, advertise it in plaintext messages ("k") and in node
lists ("k4"/"k6"), and encrypt all traffic with nodes whose key is
known. Keys are bound to node IDs, so they can't be forged, and
plaintext messages never move a node known to support encryption.
Legacy nodes keep talking in plaintext.
Private networks (TransportConfig::authorities) only accept nodes with
a credential issued by one of the network authorities, never send or
accept plaintext, and introduce themselves to addresses with a Hello.
X25519 transport keys, node ID derivation from the public key,
HKDF-SHA256 key agreement, ChaCha20-Poly1305 envelopes that legacy
msgpack parsers reject, and Ed25519 network authority credentials.
Adrien Beraud