[p2p/pex] connect to more than 10 peers (#2169)

* [p2p/pex] connect to more than 10 peers

also, remove DefaultMinNumOutboundPeers because a) I am not sure it's
needed b) it's super confusing

look closely

```
maxPeers := sw.config.MaxNumPeers - DefaultMinNumOutboundPeers
if maxPeers <= sw.peers.Size() {
  sw.Logger.Info("Ignoring inbound connection: already have enough peers", "address", inConn.RemoteAddr().String(), "numPeers", sw.peers.Size(), "max", maxPeers)
```

we print maxPeers = config.MaxPeers - DefaultMinNumOutboundPeers. So we
may not have enough peers even though we say we have enough.

Refs #2130

* update spec

* replace MaxNumPeers with MaxNumInboundPeers/MaxNumOutboundPeers

Refs #2130

* update changelog

* make max rpc conns formula visible to users

* update spec

* docs: note max outbound peers excludes persistent

docs/spec/reactors/pex/pex.md

@@ -15,6 +15,9 @@ we will not put them in the address book or gossip them to others.
 All peers except private peers and peers coming from them are tracked using the
 address book.
 
+The rest of our peers are only distinguished by being either
+inbound (they dialed our public address) or outbound (we dialed them).
+
 ## Discovery
 
 Peer discovery begins with a list of seeds.
@@ -26,15 +29,15 @@ and will attempt to maintain persistent connections with them. If the connection
 we will redial every 5s for a few minutes, then switch to an exponential backoff schedule,
 and after about a day of trying, stop dialing the peer.
 
-So long as we have less than `MinNumOutboundPeers`, we periodically request additional peers
+So long as we have less than `MaxNumOutboundPeers`, we periodically request additional peers
 from each of our own. If sufficient time goes by and we still can't find enough peers,
 we try the seeds again.
 
 ## Listening
 
 Peers listen on a configurable ListenAddr that they self-report in their
-NodeInfo during handshakes with other peers. Peers accept up to (MaxNumPeers -
-MinNumOutboundPeers) incoming peers.
+NodeInfo during handshakes with other peers. Peers accept up to
+`MaxNumInboundPeers` incoming peers.
 
 ## Address Book
 
@@ -73,10 +76,11 @@ a trust metric (see below), but it's best to start with something simple.
 
 ## Select Peers to Dial
 
-When we need more peers, we pick them randomly from the addrbook with some
-configurable bias for unvetted peers. The bias should be lower when we have fewer peers
-and can increase as we obtain more, ensuring that our first peers are more trustworthy,
-but always giving us the chance to discover new good peers.
+When we need more peers, we pick addresses randomly from the addrbook with some
+configurable bias for unvetted peers. The bias should be lower when we have
+fewer peers and can increase as we obtain more, ensuring that our first peers
+are more trustworthy, but always giving us the chance to discover new good
+peers.
 
 We track the last time we dialed a peer and the number of unsuccessful attempts
 we've made. If too many attempts are made, we mark the peer as bad.
@@ -85,7 +89,8 @@ Connection attempts are made with exponential backoff (plus jitter). Because
 the selection process happens every `ensurePeersPeriod`, we might not end up
 dialing a peer for much longer than the backoff duration.
 
-If we fail to connect to the peer after 16 tries (with exponential backoff), we remove from address book completely.
+If we fail to connect to the peer after 16 tries (with exponential backoff), we
+remove from address book completely.
 
 ## Select Peers to Exchange
 

docs/tendermint-core/configuration.md

@@ -77,6 +77,8 @@ grpc_laddr = ""
 # If you want to accept more significant number than the default, make sure
 # you increase your OS limits.
 # 0 - unlimited.
+# Should be < {ulimit -Sn} - {MaxNumInboundPeers} - {MaxNumOutboundPeers} - {N of wal, db and other open files}
+# 1024 - 40 - 10 - 50 = 924 = ~900
 grpc_max_open_connections = 900
 
 # Activate unsafe RPC commands like /dial_seeds and /unsafe_flush_mempool
@@ -87,7 +89,9 @@ unsafe = false
 # If you want to accept more significant number than the default, make sure
 # you increase your OS limits.
 # 0 - unlimited.
-max_open_connections = 450
+# Should be < {ulimit -Sn} - {MaxNumInboundPeers} - {MaxNumOutboundPeers} - {N of wal, db and other open files}
+# 1024 - 40 - 10 - 50 = 924 = ~900
+max_open_connections = 900
 
 ##### peer to peer configuration options #####
 [p2p]
@@ -113,8 +117,11 @@ addr_book_strict = true
 # Time to wait before flushing messages out on the connection, in ms
 flush_throttle_timeout = 100
 
-# Maximum number of peers to connect to
-max_num_peers = 50
+# Maximum number of inbound peers
+max_num_inbound_peers = 40
+
+# Maximum number of outbound peers to connect to
+max_num_outbound_peers = 10
 
 # Maximum size of a message packet payload, in bytes
 max_packet_msg_payload_size = 1024