mirrors/scoutfs
1
0
Fork 0
mirror of https://github.com/versity/scoutfs.git synced 2026-02-12 13:31:12 +00:00
Code Issues Packages Projects Releases Wiki Activity

scoutfs: make networking more reliable

Browse Source 
The current networking code has loose reliability guarantees.  If a
connection between the client and server is broken then the client
reconnects as though its an entirely new connection.  The client resends
requests but no responses are resent.  A client's requests could be
processed twice on the same server.  The server throws away disconnected
client state.

This was fine, sort of, for the simple requests we had implemented so
far.  It's not good enough for the locking service which would prefer to
let networking worry about reliable message delivery so it doesn't have
to track and replay partial state across reconnection between the same
client and server.

This adds the infrastructure to ensure that requests and responses
between a given client and server will be delivered across reconnected
sockets and will only be processed once.

The server keeps track of disconnected clients and restores state if the
same client reconnects.  This required some work around the greetings so
that clients and servers can recognize each other.  Now that the server
remembers disconnected clients we add a farewell request so that servers
can forget about clients that are shutting down and won't be
reconnecting.

Now that connections between the client and server are preserved we can
resend responses across reconnection.  We add outgoing message sequence
numbers which are used to drop duplicates and communicate the received
sequence back to the sender to free responses once they're received.

When the client is reconnecting to a new server it resets its receive
state that was dependent on the old server and it drops responses which
were being sent to a server instance which no longer exists.

This stronger reliable messaging guarantee will make it much easier
to implement lock recovery which can now rewind state relative to
requests that are in flight and replay existing state on a new server
instance.

Signed-off-by: Zach Brown <zab@versity.com>
This commit is contained in:
Zach Brown
2019-02-04 14:29:10 -08:00
committed by Zach Brown
parent 20f4e1c338
commit 74366f0df1
8 changed files with 656 additions and 138 deletions
Download Patch File Download Diff File Expand all files Collapse all files

23
kmod/src/server.h
View File

@@ -22,11 +22,16 @@ do { \
__entry->name##_addr & 255, \
__entry->name##_port
#define SNH_FMT "id %llu data_len %u cmd %u flags 0x%x error %u"
#define SNH_ARG(nh) le64_to_cpu((nh)->id), le16_to_cpu((nh)->data_len), \
(nh)->cmd, (nh)->flags, (nh)->error
#define SNH_FMT \
"seq %llu recv_seq %llu id %llu data_len %u cmd %u flags 0x%x error %u"
#define SNH_ARG(nh) \
le64_to_cpu((nh)->seq), le64_to_cpu((nh)->recv_seq), \
le64_to_cpu((nh)->id), le16_to_cpu((nh)->data_len), (nh)->cmd, \
(nh)->flags, (nh)->error
#define snh_trace_define(name) \
__field(__u64, name##_seq) \
__field(__u64, name##_recv_seq) \
__field(__u64, name##_id) \
__field(__u16, name##_data_len) \
__field(__u8, name##_cmd) \
@@ -37,6 +42,8 @@ do { \
do { \
__typeof__(nh) _nh = (nh); \
\
__entry->name##_seq = le64_to_cpu(_nh->seq); \
__entry->name##_recv_seq = le64_to_cpu(_nh->recv_seq); \
__entry->name##_id = le64_to_cpu(_nh->id); \
__entry->name##_data_len = le16_to_cpu(_nh->data_len); \
__entry->name##_cmd = _nh->cmd; \
@@ -44,9 +51,10 @@ do { \
__entry->name##_error = _nh->error; \
} while (0)
#define snh_trace_args(name) \
__entry->name##_id, __entry->name##_data_len, __entry->name##_cmd, \
__entry->name##_flags, __entry->name##_error
#define snh_trace_args(name) \
__entry->name##_seq, __entry->name##_recv_seq, __entry->name##_id, \
__entry->name##_data_len, __entry->name##_cmd, __entry->name##_flags, \
__entry->name##_error
struct scoutfs_net_manifest_entry;
struct scoutfs_manifest_entry;
@@ -62,7 +70,8 @@ int scoutfs_server_lock_response(struct super_block *sb, u64 node_id,
u64 id, struct scoutfs_net_lock *nl);
struct sockaddr_in;
int scoutfs_server_start(struct super_block *sb, struct sockaddr_in *sin);
int scoutfs_server_start(struct super_block *sb, struct sockaddr_in *sin,
u64 term);
void scoutfs_server_stop(struct super_block *sb);
int scoutfs_server_setup(struct super_block *sb);