docs: cleanup (#5252)

This commit is contained in:
Marko
2020-08-18 12:07:13 +02:00
committed by GitHub
parent 42e4e8b58e
commit 4a38bd216e
27 changed files with 516 additions and 924 deletions

View File

@@ -0,0 +1,651 @@
<!---
order: 2
--->
# Creating a built-in application in Go
## Guide assumptions
This guide is designed for beginners who want to get started with a Tendermint
Core application from scratch. It does not assume that you have any prior
experience with Tendermint Core.
Tendermint Core is Byzantine Fault Tolerant (BFT) middleware that takes a state
transition machine - written in any programming language - and securely
replicates it on many machines.
Although Tendermint Core is written in the Golang programming language, prior
knowledge of it is not required for this guide. You can learn it as we go due
to it's simplicity. However, you may want to go through [Learn X in Y minutes
Where X=Go](https://learnxinyminutes.com/docs/go/) first to familiarize
yourself with the syntax.
By following along with this guide, you'll create a Tendermint Core project
called kvstore, a (very) simple distributed BFT key-value store.
## Built-in app vs external app
Running your application inside the same process as Tendermint Core will give
you the best possible performance.
For other languages, your application have to communicate with Tendermint Core
through a TCP, Unix domain socket or gRPC.
## 1.1 Installing Go
Please refer to [the official guide for installing
Go](https://golang.org/doc/install).
Verify that you have the latest version of Go installed:
```bash
$ go version
go version go1.13.1 darwin/amd64
```
Make sure you have `$GOPATH` environment variable set:
```bash
$ echo $GOPATH
/Users/melekes/go
```
## 1.2 Creating a new Go project
We'll start by creating a new Go project.
```bash
mkdir kvstore
cd kvstore
```
Inside the example directory create a `main.go` file with the following content:
```go
package main
import (
"fmt"
)
func main() {
fmt.Println("Hello, Tendermint Core")
}
```
When run, this should print "Hello, Tendermint Core" to the standard output.
```bash
$ go run main.go
Hello, Tendermint Core
```
## 1.3 Writing a Tendermint Core application
Tendermint Core communicates with the application through the Application
BlockChain Interface (ABCI). All message types are defined in the [protobuf
file](https://github.com/tendermint/tendermint/blob/master/abci/types/types.proto).
This allows Tendermint Core to run applications written in any programming
language.
Create a file called `app.go` with the following content:
```go
package main
import (
abcitypes "github.com/tendermint/tendermint/abci/types"
)
type KVStoreApplication struct {}
var _ abcitypes.Application = (*KVStoreApplication)(nil)
func NewKVStoreApplication() *KVStoreApplication {
return &KVStoreApplication{}
}
func (KVStoreApplication) Info(req abcitypes.RequestInfo) abcitypes.ResponseInfo {
return abcitypes.ResponseInfo{}
}
func (KVStoreApplication) SetOption(req abcitypes.RequestSetOption) abcitypes.ResponseSetOption {
return abcitypes.ResponseSetOption{}
}
func (KVStoreApplication) DeliverTx(req abcitypes.RequestDeliverTx) abcitypes.ResponseDeliverTx {
return abcitypes.ResponseDeliverTx{Code: 0}
}
func (KVStoreApplication) CheckTx(req abcitypes.RequestCheckTx) abcitypes.ResponseCheckTx {
return abcitypes.ResponseCheckTx{Code: 0}
}
func (KVStoreApplication) Commit() abcitypes.ResponseCommit {
return abcitypes.ResponseCommit{}
}
func (KVStoreApplication) Query(req abcitypes.RequestQuery) abcitypes.ResponseQuery {
return abcitypes.ResponseQuery{Code: 0}
}
func (KVStoreApplication) InitChain(req abcitypes.RequestInitChain) abcitypes.ResponseInitChain {
return abcitypes.ResponseInitChain{}
}
func (KVStoreApplication) BeginBlock(req abcitypes.RequestBeginBlock) abcitypes.ResponseBeginBlock {
return abcitypes.ResponseBeginBlock{}
}
func (KVStoreApplication) EndBlock(req abcitypes.RequestEndBlock) abcitypes.ResponseEndBlock {
return abcitypes.ResponseEndBlock{}
}
```
Now I will go through each method explaining when it's called and adding
required business logic.
### 1.3.1 CheckTx
When a new transaction is added to the Tendermint Core, it will ask the
application to check it (validate the format, signatures, etc.).
```go
import "bytes"
func (app *KVStoreApplication) isValid(tx []byte) (code uint32) {
// check format
parts := bytes.Split(tx, []byte("="))
if len(parts) != 2 {
return 1
}
key, value := parts[0], parts[1]
// check if the same key=value already exists
err := app.db.View(func(txn *badger.Txn) error {
item, err := txn.Get(key)
if err != nil && err != badger.ErrKeyNotFound {
return err
}
if err == nil {
return item.Value(func(val []byte) error {
if bytes.Equal(val, value) {
code = 2
}
return nil
})
}
return nil
})
if err != nil {
panic(err)
}
return code
}
func (app *KVStoreApplication) CheckTx(req abcitypes.RequestCheckTx) abcitypes.ResponseCheckTx {
code := app.isValid(req.Tx)
return abcitypes.ResponseCheckTx{Code: code, GasWanted: 1}
}
```
Don't worry if this does not compile yet.
If the transaction does not have a form of `{bytes}={bytes}`, we return `1`
code. When the same key=value already exist (same key and value), we return `2`
code. For others, we return a zero code indicating that they are valid.
Note that anything with non-zero code will be considered invalid (`-1`, `100`,
etc.) by Tendermint Core.
Valid transactions will eventually be committed given they are not too big and
have enough gas. To learn more about gas, check out ["the
specification"](https://docs.tendermint.com/master/spec/abci/apps.html#gas).
For the underlying key-value store we'll use
[badger](https://github.com/dgraph-io/badger), which is an embeddable,
persistent and fast key-value (KV) database.
```go
import "github.com/dgraph-io/badger"
type KVStoreApplication struct {
db *badger.DB
currentBatch *badger.Txn
}
func NewKVStoreApplication(db *badger.DB) *KVStoreApplication {
return &KVStoreApplication{
db: db,
}
}
```
### 1.3.2 BeginBlock -> DeliverTx -> EndBlock -> Commit
When Tendermint Core has decided on the block, it's transfered to the
application in 3 parts: `BeginBlock`, one `DeliverTx` per transaction and
`EndBlock` in the end. DeliverTx are being transfered asynchronously, but the
responses are expected to come in order.
```go
func (app *KVStoreApplication) BeginBlock(req abcitypes.RequestBeginBlock) abcitypes.ResponseBeginBlock {
app.currentBatch = app.db.NewTransaction(true)
return abcitypes.ResponseBeginBlock{}
}
```
Here we create a batch, which will store block's transactions.
```go
func (app *KVStoreApplication) DeliverTx(req abcitypes.RequestDeliverTx) abcitypes.ResponseDeliverTx {
code := app.isValid(req.Tx)
if code != 0 {
return abcitypes.ResponseDeliverTx{Code: code}
}
parts := bytes.Split(req.Tx, []byte("="))
key, value := parts[0], parts[1]
err := app.currentBatch.Set(key, value)
if err != nil {
panic(err)
}
return abcitypes.ResponseDeliverTx{Code: 0}
}
```
If the transaction is badly formatted or the same key=value already exist, we
again return the non-zero code. Otherwise, we add it to the current batch.
In the current design, a block can include incorrect transactions (those who
passed CheckTx, but failed DeliverTx or transactions included by the proposer
directly). This is done for performance reasons.
Note we can't commit transactions inside the `DeliverTx` because in such case
`Query`, which may be called in parallel, will return inconsistent data (i.e.
it will report that some value already exist even when the actual block was not
yet committed).
`Commit` instructs the application to persist the new state.
```go
func (app *KVStoreApplication) Commit() abcitypes.ResponseCommit {
app.currentBatch.Commit()
return abcitypes.ResponseCommit{Data: []byte{}}
}
```
### 1.3.3 Query
Now, when the client wants to know whenever a particular key/value exist, it
will call Tendermint Core RPC `/abci_query` endpoint, which in turn will call
the application's `Query` method.
Applications are free to provide their own APIs. But by using Tendermint Core
as a proxy, clients (including [light client
package](https://godoc.org/github.com/tendermint/tendermint/light)) can leverage
the unified API across different applications. Plus they won't have to call the
otherwise separate Tendermint Core API for additional proofs.
Note we don't include a proof here.
```go
func (app *KVStoreApplication) Query(reqQuery abcitypes.RequestQuery) (resQuery abcitypes.ResponseQuery) {
resQuery.Key = reqQuery.Data
err := app.db.View(func(txn *badger.Txn) error {
item, err := txn.Get(reqQuery.Data)
if err != nil && err != badger.ErrKeyNotFound {
return err
}
if err == badger.ErrKeyNotFound {
resQuery.Log = "does not exist"
} else {
return item.Value(func(val []byte) error {
resQuery.Log = "exists"
resQuery.Value = val
return nil
})
}
return nil
})
if err != nil {
panic(err)
}
return
}
```
The complete specification can be found
[here](https://docs.tendermint.com/master/spec/abci/).
## 1.4 Starting an application and a Tendermint Core instance in the same process
Put the following code into the "main.go" file:
```go
package main
import (
"errors"
"flag"
"fmt"
"os"
"os/signal"
"path/filepath"
"syscall"
"github.com/dgraph-io/badger"
"github.com/spf13/viper"
abci "github.com/tendermint/tendermint/abci/types"
cfg "github.com/tendermint/tendermint/config"
tmflags "github.com/tendermint/tendermint/libs/cli/flags"
"github.com/tendermint/tendermint/libs/log"
nm "github.com/tendermint/tendermint/node"
"github.com/tendermint/tendermint/p2p"
"github.com/tendermint/tendermint/privval"
"github.com/tendermint/tendermint/proxy"
)
var configFile string
func init() {
flag.StringVar(&configFile, "config", "$HOME/.tendermint/config/config.toml", "Path to config.toml")
}
func main() {
db, err := badger.Open(badger.DefaultOptions("/tmp/badger"))
if err != nil {
fmt.Fprintf(os.Stderr, "failed to open badger db: %v", err)
os.Exit(1)
}
defer db.Close()
app := NewKVStoreApplication(db)
flag.Parse()
node, err := newTendermint(app, configFile)
if err != nil {
fmt.Fprintf(os.Stderr, "%v", err)
os.Exit(2)
}
node.Start()
defer func() {
node.Stop()
node.Wait()
}()
c := make(chan os.Signal, 1)
signal.Notify(c, os.Interrupt, syscall.SIGTERM)
<-c
os.Exit(0)
}
func newTendermint(app abci.Application, configFile string) (*nm.Node, error) {
// read config
config := cfg.DefaultConfig()
config.RootDir = filepath.Dir(filepath.Dir(configFile))
viper.SetConfigFile(configFile)
if err := viper.ReadInConfig(); err != nil {
return nil, fmt.Errorf("viper failed to read config file: %w", err)
}
if err := viper.Unmarshal(config); err != nil {
return nil, fmt.Errorf("viper failed to unmarshal config: %w", err)
}
if err := config.ValidateBasic(); err != nil {
return nil, fmt.Errorf("config is invalid: %w", err)
}
// create logger
logger := log.NewTMLogger(log.NewSyncWriter(os.Stdout))
var err error
logger, err = tmflags.ParseLogLevel(config.LogLevel, logger, cfg.DefaultLogLevel())
if err != nil {
return nil, fmt.Errorf("failed to parse log level: %w", err)
}
// read private validator
pv := privval.LoadFilePV(
config.PrivValidatorKeyFile(),
config.PrivValidatorStateFile(),
)
// read node key
nodeKey, err := p2p.LoadNodeKey(config.NodeKeyFile())
if err != nil {
return nil, fmt.Errorf("failed to load node's key: %w", err)
}
// create node
node, err := nm.NewNode(
config,
pv,
nodeKey,
proxy.NewLocalClientCreator(app),
nm.DefaultGenesisDocProviderFunc(config),
nm.DefaultDBProvider,
nm.DefaultMetricsProvider(config.Instrumentation),
logger)
if err != nil {
return nil, fmt.Errorf("failed to create new Tendermint node: %w", err)
}
return node, nil
}
```
This is a huge blob of code, so let's break it down into pieces.
First, we initialize the Badger database and create an app instance:
```go
db, err := badger.Open(badger.DefaultOptions("/tmp/badger"))
if err != nil {
fmt.Fprintf(os.Stderr, "failed to open badger db: %v", err)
os.Exit(1)
}
defer db.Close()
app := NewKVStoreApplication(db)
```
For **Windows** users, restarting this app will make badger throw an error as it requires value log to be truncated. For more information on this, visit [here](https://github.com/dgraph-io/badger/issues/744).
This can be avoided by setting the truncate option to true, like this:
```go
db, err := badger.Open(badger.DefaultOptions("/tmp/badger").WithTruncate(true))
```
Then we use it to create a Tendermint Core `Node` instance:
```go
flag.Parse()
node, err := newTendermint(app, configFile)
if err != nil {
fmt.Fprintf(os.Stderr, "%v", err)
os.Exit(2)
}
...
// create node
node, err := nm.NewNode(
config,
pv,
nodeKey,
proxy.NewLocalClientCreator(app),
nm.DefaultGenesisDocProviderFunc(config),
nm.DefaultDBProvider,
nm.DefaultMetricsProvider(config.Instrumentation),
logger)
if err != nil {
return nil, fmt.Errorf("failed to create new Tendermint node: %w", err)
}
```
`NewNode` requires a few things including a configuration file, a private
validator, a node key and a few others in order to construct the full node.
Note we use `proxy.NewLocalClientCreator` here to create a local client instead
of one communicating through a socket or gRPC.
[viper](https://github.com/spf13/viper) is being used for reading the config,
which we will generate later using the `tendermint init` command.
```go
config := cfg.DefaultConfig()
config.RootDir = filepath.Dir(filepath.Dir(configFile))
viper.SetConfigFile(configFile)
if err := viper.ReadInConfig(); err != nil {
return nil, fmt.Errorf("viper failed to read config file: %w", err)
}
if err := viper.Unmarshal(config); err != nil {
return nil, fmt.Errorf("viper failed to unmarshal config: %w", err)
}
if err := config.ValidateBasic(); err != nil {
return nil, fmt.Errorf("config is invalid: %w", err)
}
```
We use `FilePV`, which is a private validator (i.e. thing which signs consensus
messages). Normally, you would use `SignerRemote` to connect to an external
[HSM](https://kb.certus.one/hsm.html).
```go
pv := privval.LoadFilePV(
config.PrivValidatorKeyFile(),
config.PrivValidatorStateFile(),
)
```
`nodeKey` is needed to identify the node in a p2p network.
```go
nodeKey, err := p2p.LoadNodeKey(config.NodeKeyFile())
if err != nil {
return nil, fmt.Errorf("failed to load node's key: %w", err)
}
```
As for the logger, we use the build-in library, which provides a nice
abstraction over [go-kit's
logger](https://github.com/go-kit/kit/tree/master/log).
```go
logger := log.NewTMLogger(log.NewSyncWriter(os.Stdout))
var err error
logger, err = tmflags.ParseLogLevel(config.LogLevel, logger, cfg.DefaultLogLevel())
if err != nil {
return nil, fmt.Errorf("failed to parse log level: %w", err)
}
```
Finally, we start the node and add some signal handling to gracefully stop it
upon receiving SIGTERM or Ctrl-C.
```go
node.Start()
defer func() {
node.Stop()
node.Wait()
}()
c := make(chan os.Signal, 1)
signal.Notify(c, os.Interrupt, syscall.SIGTERM)
<-c
os.Exit(0)
```
## 1.5 Getting Up and Running
We are going to use [Go modules](https://github.com/golang/go/wiki/Modules) for
dependency management.
```bash
go mod init github.com/me/example
go build
```
This should build the binary.
To create a default configuration, nodeKey and private validator files, let's
execute `tendermint init`. But before we do that, we will need to install
Tendermint Core. Please refer to [the official
guide](https://docs.tendermint.com/master/introduction/install.html). If you're
installing from source, don't forget to checkout the latest release (`git checkout vX.Y.Z`).
```bash
$ rm -rf /tmp/example
$ TMHOME="/tmp/example" tendermint init
I[2019-07-16|18:40:36.480] Generated private validator module=main keyFile=/tmp/example/config/priv_validator_key.json stateFile=/tmp/example2/data/priv_validator_state.json
I[2019-07-16|18:40:36.481] Generated node key module=main path=/tmp/example/config/node_key.json
I[2019-07-16|18:40:36.482] Generated genesis file module=main path=/tmp/example/config/genesis.json
```
We are ready to start our application:
```bash
$ ./example -config "/tmp/example/config/config.toml"
badger 2019/07/16 18:42:25 INFO: All 0 tables opened in 0s
badger 2019/07/16 18:42:25 INFO: Replaying file id: 0 at offset: 0
badger 2019/07/16 18:42:25 INFO: Replay took: 695.227s
E[2019-07-16|18:42:25.818] Couldn't connect to any seeds module=p2p
I[2019-07-16|18:42:26.853] Executed block module=state height=1 validTxs=0 invalidTxs=0
I[2019-07-16|18:42:26.865] Committed state module=state height=1 txs=0 appHash=
```
Now open another tab in your terminal and try sending a transaction:
```bash
$ curl -s 'localhost:26657/broadcast_tx_commit?tx="tendermint=rocks"'
{
"jsonrpc": "2.0",
"id": "",
"result": {
"check_tx": {
"gasWanted": "1"
},
"deliver_tx": {},
"hash": "1B3C5A1093DB952C331B1749A21DCCBB0F6C7F4E0055CD04D16346472FC60EC6",
"height": "128"
}
}
```
Response should contain the height where this transaction was committed.
Now let's check if the given key now exists and its value:
```json
$ curl -s 'localhost:26657/abci_query?data="tendermint"'
{
"jsonrpc": "2.0",
"id": "",
"result": {
"response": {
"log": "exists",
"key": "dGVuZGVybWludA==",
"value": "cm9ja3M="
}
}
}
```
"dGVuZGVybWludA==" and "cm9ja3M=" are the base64-encoding of the ASCII of
"tendermint" and "rocks" accordingly.
## Outro
I hope everything went smoothly and your first, but hopefully not the last,
Tendermint Core application is up and running. If not, please [open an issue on
Github](https://github.com/tendermint/tendermint/issues/new/choose). To dig
deeper, read [the docs](https://docs.tendermint.com/master/).

536
docs/tutorials/go.md Normal file
View File

@@ -0,0 +1,536 @@
<!---
order: 1
--->
# Creating an application in Go
## Guide Assumptions
This guide is designed for beginners who want to get started with a Tendermint
Core application from scratch. It does not assume that you have any prior
experience with Tendermint Core.
Tendermint Core is Byzantine Fault Tolerant (BFT) middleware that takes a state
transition machine - written in any programming language - and securely
replicates it on many machines.
Although Tendermint Core is written in the Golang programming language, prior
knowledge of it is not required for this guide. You can learn it as we go due
to it's simplicity. However, you may want to go through [Learn X in Y minutes
Where X=Go](https://learnxinyminutes.com/docs/go/) first to familiarize
yourself with the syntax.
By following along with this guide, you'll create a Tendermint Core project
called kvstore, a (very) simple distributed BFT key-value store.
## Built-in app vs external app
To get maximum performance it is better to run your application alongside
Tendermint Core. [Cosmos SDK](https://github.com/cosmos/cosmos-sdk) is written
this way. Please refer to [Writing a built-in Tendermint Core application in
Go](./go-built-in.md) guide for details.
Having a separate application might give you better security guarantees as two
processes would be communicating via established binary protocol. Tendermint
Core will not have access to application's state.
## 1.1 Installing Go
Please refer to [the official guide for installing
Go](https://golang.org/doc/install).
Verify that you have the latest version of Go installed:
```bash
$ go version
go version go1.14.x darwin/amd64
```
Make sure you have `$GOPATH` environment variable set:
```bash
echo $GOPATH
/Users/melekes/go
```
## 1.2 Creating a new Go project
We'll start by creating a new Go project.
```bash
mkdir kvstore
cd kvstore
```
Inside the example directory create a `main.go` file with the following content:
```go
package main
import (
"fmt"
)
func main() {
fmt.Println("Hello, Tendermint Core")
}
```
When run, this should print "Hello, Tendermint Core" to the standard output.
```bash
go run main.go
Hello, Tendermint Core
```
## 1.3 Writing a Tendermint Core application
Tendermint Core communicates with the application through the Application
BlockChain Interface (ABCI). All message types are defined in the [protobuf
file](https://github.com/tendermint/tendermint/blob/master/abci/types/types.proto).
This allows Tendermint Core to run applications written in any programming
language.
Create a file called `app.go` with the following content:
```go
package main
import (
abcitypes "github.com/tendermint/tendermint/abci/types"
)
type KVStoreApplication struct {}
var _ abcitypes.Application = (*KVStoreApplication)(nil)
func NewKVStoreApplication() *KVStoreApplication {
return &KVStoreApplication{}
}
func (KVStoreApplication) Info(req abcitypes.RequestInfo) abcitypes.ResponseInfo {
return abcitypes.ResponseInfo{}
}
func (KVStoreApplication) SetOption(req abcitypes.RequestSetOption) abcitypes.ResponseSetOption {
return abcitypes.ResponseSetOption{}
}
func (KVStoreApplication) DeliverTx(req abcitypes.RequestDeliverTx) abcitypes.ResponseDeliverTx {
return abcitypes.ResponseDeliverTx{Code: 0}
}
func (KVStoreApplication) CheckTx(req abcitypes.RequestCheckTx) abcitypes.ResponseCheckTx {
return abcitypes.ResponseCheckTx{Code: 0}
}
func (KVStoreApplication) Commit() abcitypes.ResponseCommit {
return abcitypes.ResponseCommit{}
}
func (KVStoreApplication) Query(req abcitypes.RequestQuery) abcitypes.ResponseQuery {
return abcitypes.ResponseQuery{Code: 0}
}
func (KVStoreApplication) InitChain(req abcitypes.RequestInitChain) abcitypes.ResponseInitChain {
return abcitypes.ResponseInitChain{}
}
func (KVStoreApplication) BeginBlock(req abcitypes.RequestBeginBlock) abcitypes.ResponseBeginBlock {
return abcitypes.ResponseBeginBlock{}
}
func (KVStoreApplication) EndBlock(req abcitypes.RequestEndBlock) abcitypes.ResponseEndBlock {
return abcitypes.ResponseEndBlock{}
}
```
Now I will go through each method explaining when it's called and adding
required business logic.
### 1.3.1 CheckTx
When a new transaction is added to the Tendermint Core, it will ask the
application to check it (validate the format, signatures, etc.).
```go
import "bytes"
func (app *KVStoreApplication) isValid(tx []byte) (code uint32) {
// check format
parts := bytes.Split(tx, []byte("="))
if len(parts) != 2 {
return 1
}
key, value := parts[0], parts[1]
// check if the same key=value already exists
err := app.db.View(func(txn *badger.Txn) error {
item, err := txn.Get(key)
if err != nil && err != badger.ErrKeyNotFound {
return err
}
if err == nil {
return item.Value(func(val []byte) error {
if bytes.Equal(val, value) {
code = 2
}
return nil
})
}
return nil
})
if err != nil {
panic(err)
}
return code
}
func (app *KVStoreApplication) CheckTx(req abcitypes.RequestCheckTx) abcitypes.ResponseCheckTx {
code := app.isValid(req.Tx)
return abcitypes.ResponseCheckTx{Code: code, GasWanted: 1}
}
```
Don't worry if this does not compile yet.
If the transaction does not have a form of `{bytes}={bytes}`, we return `1`
code. When the same key=value already exist (same key and value), we return `2`
code. For others, we return a zero code indicating that they are valid.
Note that anything with non-zero code will be considered invalid (`-1`, `100`,
etc.) by Tendermint Core.
Valid transactions will eventually be committed given they are not too big and
have enough gas. To learn more about gas, check out ["the
specification"](https://docs.tendermint.com/master/spec/abci/apps.html#gas).
For the underlying key-value store we'll use
[badger](https://github.com/dgraph-io/badger), which is an embeddable,
persistent and fast key-value (KV) database.
```go
import "github.com/dgraph-io/badger"
type KVStoreApplication struct {
db *badger.DB
currentBatch *badger.Txn
}
func NewKVStoreApplication(db *badger.DB) *KVStoreApplication {
return &KVStoreApplication{
db: db,
}
}
```
### 1.3.2 BeginBlock -> DeliverTx -> EndBlock -> Commit
When Tendermint Core has decided on the block, it's transferred to the
application in 3 parts: `BeginBlock`, one `DeliverTx` per transaction and
`EndBlock` in the end. DeliverTx are being transferred asynchronously, but the
responses are expected to come in order.
```go
func (app *KVStoreApplication) BeginBlock(req abcitypes.RequestBeginBlock) abcitypes.ResponseBeginBlock {
app.currentBatch = app.db.NewTransaction(true)
return abcitypes.ResponseBeginBlock{}
}
```
Here we create a batch, which will store block's transactions.
```go
func (app *KVStoreApplication) DeliverTx(req abcitypes.RequestDeliverTx) abcitypes.ResponseDeliverTx {
code := app.isValid(req.Tx)
if code != 0 {
return abcitypes.ResponseDeliverTx{Code: code}
}
parts := bytes.Split(req.Tx, []byte("="))
key, value := parts[0], parts[1]
err := app.currentBatch.Set(key, value)
if err != nil {
panic(err)
}
return abcitypes.ResponseDeliverTx{Code: 0}
}
```
If the transaction is badly formatted or the same key=value already exist, we
again return the non-zero code. Otherwise, we add it to the current batch.
In the current design, a block can include incorrect transactions (those who
passed CheckTx, but failed DeliverTx or transactions included by the proposer
directly). This is done for performance reasons.
Note we can't commit transactions inside the `DeliverTx` because in such case
`Query`, which may be called in parallel, will return inconsistent data (i.e.
it will report that some value already exist even when the actual block was not
yet committed).
`Commit` instructs the application to persist the new state.
```go
func (app *KVStoreApplication) Commit() abcitypes.ResponseCommit {
app.currentBatch.Commit()
return abcitypes.ResponseCommit{Data: []byte{}}
}
```
### 1.3.3 Query
Now, when the client wants to know whenever a particular key/value exist, it
will call Tendermint Core RPC `/abci_query` endpoint, which in turn will call
the application's `Query` method.
Applications are free to provide their own APIs. But by using Tendermint Core
as a proxy, clients (including [light client
package](https://godoc.org/github.com/tendermint/tendermint/lite2)) can leverage
the unified API across different applications. Plus they won't have to call the
otherwise separate Tendermint Core API for additional proofs.
Note we don't include a proof here.
```go
func (app *KVStoreApplication) Query(reqQuery abcitypes.RequestQuery) (resQuery abcitypes.ResponseQuery) {
resQuery.Key = reqQuery.Data
err := app.db.View(func(txn *badger.Txn) error {
item, err := txn.Get(reqQuery.Data)
if err != nil && err != badger.ErrKeyNotFound {
return err
}
if err == badger.ErrKeyNotFound {
resQuery.Log = "does not exist"
} else {
return item.Value(func(val []byte) error {
resQuery.Log = "exists"
resQuery.Value = val
return nil
})
}
return nil
})
if err != nil {
panic(err)
}
return
}
```
The complete specification can be found
[here](https://docs.tendermint.com/master/spec/abci/).
## 1.4 Starting an application and a Tendermint Core instances
Put the following code into the "main.go" file:
```go
package main
import (
"flag"
"fmt"
"os"
"os/signal"
"syscall"
"github.com/dgraph-io/badger"
abciserver "github.com/tendermint/tendermint/abci/server"
"github.com/tendermint/tendermint/libs/log"
)
var socketAddr string
func init() {
flag.StringVar(&socketAddr, "socket-addr", "unix://example.sock", "Unix domain socket address")
}
func main() {
db, err := badger.Open(badger.DefaultOptions("/tmp/badger"))
if err != nil {
fmt.Fprintf(os.Stderr, "failed to open badger db: %v", err)
os.Exit(1)
}
defer db.Close()
app := NewKVStoreApplication(db)
flag.Parse()
logger := log.NewTMLogger(log.NewSyncWriter(os.Stdout))
server := abciserver.NewSocketServer(socketAddr, app)
server.SetLogger(logger)
if err := server.Start(); err != nil {
fmt.Fprintf(os.Stderr, "error starting socket server: %v", err)
os.Exit(1)
}
defer server.Stop()
c := make(chan os.Signal, 1)
signal.Notify(c, os.Interrupt, syscall.SIGTERM)
<-c
os.Exit(0)
}
```
This is a huge blob of code, so let's break it down into pieces.
First, we initialize the Badger database and create an app instance:
```go
db, err := badger.Open(badger.DefaultOptions("/tmp/badger"))
if err != nil {
fmt.Fprintf(os.Stderr, "failed to open badger db: %v", err)
os.Exit(1)
}
defer db.Close()
app := NewKVStoreApplication(db)
```
For **Windows** users, restarting this app will make badger throw an error as it requires value log to be truncated. For more information on this, visit [here](https://github.com/dgraph-io/badger/issues/744).
This can be avoided by setting the truncate option to true, like this:
```go
db, err := badger.Open(badger.DefaultOptions("/tmp/badger").WithTruncate(true))
```
Then we start the ABCI server and add some signal handling to gracefully stop
it upon receiving SIGTERM or Ctrl-C. Tendermint Core will act as a client,
which connects to our server and send us transactions and other messages.
```go
server := abciserver.NewSocketServer(socketAddr, app)
server.SetLogger(logger)
if err := server.Start(); err != nil {
fmt.Fprintf(os.Stderr, "error starting socket server: %v", err)
os.Exit(1)
}
defer server.Stop()
c := make(chan os.Signal, 1)
signal.Notify(c, os.Interrupt, syscall.SIGTERM)
<-c
os.Exit(0)
```
## 1.5 Getting Up and Running
We are going to use [Go modules](https://github.com/golang/go/wiki/Modules) for
dependency management.
```bash
export GO111MODULE=on
go mod init github.com/me/example
go build
```
This should build the binary.
To create a default configuration, nodeKey and private validator files, let's
execute `tendermint init`. But before we do that, we will need to install
Tendermint Core. Please refer to [the official
guide](https://docs.tendermint.com/master/introduction/install.html). If you're
installing from source, don't forget to checkout the latest release (`git checkout vX.Y.Z`).
```bash
rm -rf /tmp/example
TMHOME="/tmp/example" tendermint init
I[2019-07-16|18:20:36.480] Generated private validator module=main keyFile=/tmp/example/config/priv_validator_key.json stateFile=/tmp/example2/data/priv_validator_state.json
I[2019-07-16|18:20:36.481] Generated node key module=main path=/tmp/example/config/node_key.json
I[2019-07-16|18:20:36.482] Generated genesis file module=main path=/tmp/example/config/genesis.json
```
Feel free to explore the generated files, which can be found at
`/tmp/example/config` directory. Documentation on the config can be found
[here](https://docs.tendermint.com/master/tendermint-core/configuration.html).
We are ready to start our application:
```bash
rm example.sock
./example
badger 2019/07/16 18:25:11 INFO: All 0 tables opened in 0s
badger 2019/07/16 18:25:11 INFO: Replaying file id: 0 at offset: 0
badger 2019/07/16 18:25:11 INFO: Replay took: 300.4s
I[2019-07-16|18:25:11.523] Starting ABCIServer impl=ABCIServ
```
Then we need to start Tendermint Core and point it to our application. Staying
within the application directory execute:
```bash
TMHOME="/tmp/example" tendermint node --proxy_app=unix://example.sock
I[2019-07-16|18:26:20.362] Version info module=main software=0.32.1 block=10 p2p=7
I[2019-07-16|18:26:20.383] Starting Node module=main impl=Node
E[2019-07-16|18:26:20.392] Couldn't connect to any seeds module=p2p
I[2019-07-16|18:26:20.394] Started node module=main nodeInfo="{ProtocolVersion:{P2P:7 Block:10 App:0} ID_:8dab80770ae8e295d4ce905d86af78c4ff634b79 ListenAddr:tcp://0.0.0.0:26656 Network:test-chain-nIO96P Version:0.32.1 Channels:4020212223303800 Moniker:app48.fun-box.ru Other:{TxIndex:on RPCAddress:tcp://127.0.0.1:26657}}"
I[2019-07-16|18:26:21.440] Executed block module=state height=1 validTxs=0 invalidTxs=0
I[2019-07-16|18:26:21.446] Committed state module=state height=1 txs=0 appHash=
```
This should start the full node and connect to our ABCI application.
```sh
I[2019-07-16|18:25:11.525] Waiting for new connection...
I[2019-07-16|18:26:20.329] Accepted a new connection
I[2019-07-16|18:26:20.329] Waiting for new connection...
I[2019-07-16|18:26:20.330] Accepted a new connection
I[2019-07-16|18:26:20.330] Waiting for new connection...
I[2019-07-16|18:26:20.330] Accepted a new connection
```
Now open another tab in your terminal and try sending a transaction:
```json
curl -s 'localhost:26657/broadcast_tx_commit?tx="tendermint=rocks"'
{
"jsonrpc": "2.0",
"id": "",
"result": {
"check_tx": {
"gasWanted": "1"
},
"deliver_tx": {},
"hash": "CDD3C6DFA0A08CAEDF546F9938A2EEC232209C24AA0E4201194E0AFB78A2C2BB",
"height": "33"
}
```
Response should contain the height where this transaction was committed.
Now let's check if the given key now exists and its value:
```json
curl -s 'localhost:26657/abci_query?data="tendermint"'
{
"jsonrpc": "2.0",
"id": "",
"result": {
"response": {
"log": "exists",
"key": "dGVuZGVybWludA==",
"value": "cm9ja3My"
}
}
}
```
"dGVuZGVybWludA==" and "cm9ja3M=" are the base64-encoding of the ASCII of
"tendermint" and "rocks" accordingly.
## Outro
I hope everything went smoothly and your first, but hopefully not the last,
Tendermint Core application is up and running. If not, please [open an issue on
Github](https://github.com/tendermint/tendermint/issues/new/choose). To dig
deeper, read [the docs](https://docs.tendermint.com/master/).

630
docs/tutorials/java.md Normal file
View File

@@ -0,0 +1,630 @@
<!---
order: 3
--->
# Creating an application in Java
## Guide Assumptions
This guide is designed for beginners who want to get started with a Tendermint
Core application from scratch. It does not assume that you have any prior
experience with Tendermint Core.
Tendermint Core is Byzantine Fault Tolerant (BFT) middleware that takes a state
transition machine (your application) - written in any programming language - and securely
replicates it on many machines.
By following along with this guide, you'll create a Tendermint Core project
called kvstore, a (very) simple distributed BFT key-value store. The application (which should
implementing the blockchain interface (ABCI)) will be written in Java.
This guide assumes that you are not new to JVM world. If you are new please see [JVM Minimal Survival Guide](https://hadihariri.com/2013/12/29/jvm-minimal-survival-guide-for-the-dotnet-developer/#java-the-language-java-the-ecosystem-java-the-jvm) and [Gradle Docs](https://docs.gradle.org/current/userguide/userguide.html).
## Built-in app vs external app
If you use Golang, you can run your app and Tendermint Core in the same process to get maximum performance.
[Cosmos SDK](https://github.com/cosmos/cosmos-sdk) is written this way.
Please refer to [Writing a built-in Tendermint Core application in Go](./go-built-in.md) guide for details.
If you choose another language, like we did in this guide, you have to write a separate app,
which will communicate with Tendermint Core via a socket (UNIX or TCP) or gRPC.
This guide will show you how to build external application using RPC server.
Having a separate application might give you better security guarantees as two
processes would be communicating via established binary protocol. Tendermint
Core will not have access to application's state.
## 1.1 Installing Java and Gradle
Please refer to [the Oracle's guide for installing JDK](https://www.oracle.com/technetwork/java/javase/downloads/index.html).
Verify that you have installed Java successfully:
```bash
$ java -version
java version "12.0.2" 2019-07-16
Java(TM) SE Runtime Environment (build 12.0.2+10)
Java HotSpot(TM) 64-Bit Server VM (build 12.0.2+10, mixed mode, sharing)
```
You can choose any version of Java higher or equal to 8.
This guide is written using Java SE Development Kit 12.
Make sure you have `$JAVA_HOME` environment variable set:
```bash
$ echo $JAVA_HOME
/Library/Java/JavaVirtualMachines/jdk-12.0.2.jdk/Contents/Home
```
For Gradle installation, please refer to [their official guide](https://gradle.org/install/).
## 1.2 Creating a new Java project
We'll start by creating a new Gradle project.
```bash
export KVSTORE_HOME=~/kvstore
mkdir $KVSTORE_HOME
cd $KVSTORE_HOME
```
Inside the example directory run:
```bash
gradle init --dsl groovy --package io.example --project-name example --type java-application --test-framework junit
```
This will create a new project for you. The tree of files should look like:
```bash
$ tree
.
|-- build.gradle
|-- gradle
| `-- wrapper
| |-- gradle-wrapper.jar
| `-- gradle-wrapper.properties
|-- gradlew
|-- gradlew.bat
|-- settings.gradle
`-- src
|-- main
| |-- java
| | `-- io
| | `-- example
| | `-- App.java
| `-- resources
`-- test
|-- java
| `-- io
| `-- example
| `-- AppTest.java
`-- resources
```
When run, this should print "Hello world." to the standard output.
```bash
$ ./gradlew run
> Task :run
Hello world.
```
## 1.3 Writing a Tendermint Core application
Tendermint Core communicates with the application through the Application
BlockChain Interface (ABCI). All message types are defined in the [protobuf
file](https://github.com/tendermint/tendermint/blob/master/abci/types/types.proto).
This allows Tendermint Core to run applications written in any programming
language.
### 1.3.1 Compile .proto files
Add the following piece to the top of the `build.gradle`:
```groovy
buildscript {
repositories {
mavenCentral()
}
dependencies {
classpath 'com.google.protobuf:protobuf-gradle-plugin:0.8.8'
}
}
```
Enable the protobuf plugin in the `plugins` section of the `build.gradle`:
```groovy
plugins {
id 'com.google.protobuf' version '0.8.8'
}
```
Add the following code to `build.gradle`:
```groovy
protobuf {
protoc {
artifact = "com.google.protobuf:protoc:3.7.1"
}
plugins {
grpc {
artifact = 'io.grpc:protoc-gen-grpc-java:1.22.1'
}
}
generateProtoTasks {
all()*.plugins {
grpc {}
}
}
}
```
Now we should be ready to compile the `*.proto` files.
Copy the necessary `.proto` files to your project:
```bash
mkdir -p \
$KVSTORE_HOME/src/main/proto/github.com/tendermint/tendermint/proto/tendermint/abci \
$KVSTORE_HOME/src/main/proto/github.com/tendermint/tendermint/proto/tendermint/version \
$KVSTORE_HOME/src/main/proto/github.com/tendermint/tendermint/proto/tendermint/types \
$KVSTORE_HOME/src/main/proto/github.com/tendermint/tendermint/proto/tendermint/crypto \
$KVSTORE_HOME/src/main/proto/github.com/tendermint/tendermint/proto/tendermint/libs \
$KVSTORE_HOME/src/main/proto/github.com/gogo/protobuf/gogoproto
cp $GOPATH/src/github.com/tendermint/tendermint/proto/tendermint/abci/types.proto \
$KVSTORE_HOME/src/main/proto/github.com/tendermint/tendermint/proto/tendermint/abci/types.proto
cp $GOPATH/src/github.com/tendermint/tendermint/proto/tendermint/version/version.proto \
$KVSTORE_HOME/src/main/proto/github.com/tendermint/tendermint/proto/tendermint/version/version.proto
cp $GOPATH/src/github.com/tendermint/tendermint/proto/tendermint/types/types.proto \
$KVSTORE_HOME/src/main/proto/github.com/tendermint/tendermint/proto/tendermint/types/types.proto
cp $GOPATH/src/github.com/tendermint/tendermint/proto/tendermint/types/evidence.proto \
$KVSTORE_HOME/src/main/proto/github.com/tendermint/tendermint/proto/tendermint/types/evidence.proto
cp $GOPATH/src/github.com/tendermint/tendermint/proto/tendermint/types/params.proto \
$KVSTORE_HOME/src/main/proto/github.com/tendermint/tendermint/proto/tendermint/types/params.proto
cp $GOPATH/src/github.com/tendermint/tendermint/proto/tendermint/crypto/merkle.proto \
$KVSTORE_HOME/src/main/proto/github.com/tendermint/tendermint/proto/tendermint/crypto/merkle.proto
cp $GOPATH/src/github.com/tendermint/tendermint/proto/tendermint/crypto/keys.proto \
$KVSTORE_HOME/src/main/proto/github.com/tendermint/tendermint/proto/tendermint/crypto/keys.proto
cp $GOPATH/src/github.com/tendermint/tendermint/proto/tendermint/libs/types.proto \
$KVSTORE_HOME/src/main/proto/github.com/tendermint/tendermint/proto/tendermint/libs/types.proto
cp $GOPATH/src/github.com/gogo/protobuf/gogoproto/gogo.proto \
$KVSTORE_HOME/src/main/proto/github.com/gogo/protobuf/gogoproto/gogo.proto
```
Add these dependencies to `build.gradle`:
```groovy
dependencies {
implementation 'io.grpc:grpc-protobuf:1.22.1'
implementation 'io.grpc:grpc-netty-shaded:1.22.1'
implementation 'io.grpc:grpc-stub:1.22.1'
}
```
To generate all protobuf-type classes run:
```bash
./gradlew generateProto
```
To verify that everything went smoothly, you can inspect the `build/generated/` directory:
```bash
$ tree build/generated/
build/generated/
|-- source
| `-- proto
| `-- main
| |-- grpc
| | `-- types
| | `-- ABCIApplicationGrpc.java
| `-- java
| |-- com
| | `-- google
| | `-- protobuf
| | `-- GoGoProtos.java
| |-- common
| | `-- Types.java
| |-- merkle
| | `-- Merkle.java
| `-- types
| `-- Types.java
```
### 1.3.2 Implementing ABCI
The resulting `$KVSTORE_HOME/build/generated/source/proto/main/grpc/types/ABCIApplicationGrpc.java` file
contains the abstract class `ABCIApplicationImplBase`, which is an interface we'll need to implement.
Create `$KVSTORE_HOME/src/main/java/io/example/KVStoreApp.java` file with the following content:
```java
package io.example;
import io.grpc.stub.StreamObserver;
import types.ABCIApplicationGrpc;
import types.Types.*;
class KVStoreApp extends ABCIApplicationGrpc.ABCIApplicationImplBase {
// methods implementation
}
```
Now I will go through each method of `ABCIApplicationImplBase` explaining when it's called and adding
required business logic.
### 1.3.3 CheckTx
When a new transaction is added to the Tendermint Core, it will ask the
application to check it (validate the format, signatures, etc.).
```java
@Override
public void checkTx(RequestCheckTx req, StreamObserver<ResponseCheckTx> responseObserver) {
var tx = req.getTx();
int code = validate(tx);
var resp = ResponseCheckTx.newBuilder()
.setCode(code)
.setGasWanted(1)
.build();
responseObserver.onNext(resp);
responseObserver.onCompleted();
}
private int validate(ByteString tx) {
List<byte[]> parts = split(tx, '=');
if (parts.size() != 2) {
return 1;
}
byte[] key = parts.get(0);
byte[] value = parts.get(1);
// check if the same key=value already exists
var stored = getPersistedValue(key);
if (stored != null && Arrays.equals(stored, value)) {
return 2;
}
return 0;
}
private List<byte[]> split(ByteString tx, char separator) {
var arr = tx.toByteArray();
int i;
for (i = 0; i < tx.size(); i++) {
if (arr[i] == (byte)separator) {
break;
}
}
if (i == tx.size()) {
return Collections.emptyList();
}
return List.of(
tx.substring(0, i).toByteArray(),
tx.substring(i + 1).toByteArray()
);
}
```
Don't worry if this does not compile yet.
If the transaction does not have a form of `{bytes}={bytes}`, we return `1`
code. When the same key=value already exist (same key and value), we return `2`
code. For others, we return a zero code indicating that they are valid.
Note that anything with non-zero code will be considered invalid (`-1`, `100`,
etc.) by Tendermint Core.
Valid transactions will eventually be committed given they are not too big and
have enough gas. To learn more about gas, check out ["the
specification"](https://docs.tendermint.com/master/spec/abci/apps.html#gas).
For the underlying key-value store we'll use
[JetBrains Xodus](https://github.com/JetBrains/xodus), which is a transactional schema-less embedded high-performance database written in Java.
`build.gradle`:
```groovy
dependencies {
implementation 'org.jetbrains.xodus:xodus-environment:1.3.91'
}
```
```java
...
import jetbrains.exodus.ArrayByteIterable;
import jetbrains.exodus.ByteIterable;
import jetbrains.exodus.env.Environment;
import jetbrains.exodus.env.Store;
import jetbrains.exodus.env.StoreConfig;
import jetbrains.exodus.env.Transaction;
class KVStoreApp extends ABCIApplicationGrpc.ABCIApplicationImplBase {
private Environment env;
private Transaction txn = null;
private Store store = null;
KVStoreApp(Environment env) {
this.env = env;
}
...
private byte[] getPersistedValue(byte[] k) {
return env.computeInReadonlyTransaction(txn -> {
var store = env.openStore("store", StoreConfig.WITHOUT_DUPLICATES, txn);
ByteIterable byteIterable = store.get(txn, new ArrayByteIterable(k));
if (byteIterable == null) {
return null;
}
return byteIterable.getBytesUnsafe();
});
}
}
```
### 1.3.4 BeginBlock -> DeliverTx -> EndBlock -> Commit
When Tendermint Core has decided on the block, it's transferred to the
application in 3 parts: `BeginBlock`, one `DeliverTx` per transaction and
`EndBlock` in the end. `DeliverTx` are being transferred asynchronously, but the
responses are expected to come in order.
```java
@Override
public void beginBlock(RequestBeginBlock req, StreamObserver<ResponseBeginBlock> responseObserver) {
txn = env.beginTransaction();
store = env.openStore("store", StoreConfig.WITHOUT_DUPLICATES, txn);
var resp = ResponseBeginBlock.newBuilder().build();
responseObserver.onNext(resp);
responseObserver.onCompleted();
}
```
Here we begin a new transaction, which will accumulate the block's transactions and open the corresponding store.
```java
@Override
public void deliverTx(RequestDeliverTx req, StreamObserver<ResponseDeliverTx> responseObserver) {
var tx = req.getTx();
int code = validate(tx);
if (code == 0) {
List<byte[]> parts = split(tx, '=');
var key = new ArrayByteIterable(parts.get(0));
var value = new ArrayByteIterable(parts.get(1));
store.put(txn, key, value);
}
var resp = ResponseDeliverTx.newBuilder()
.setCode(code)
.build();
responseObserver.onNext(resp);
responseObserver.onCompleted();
}
```
If the transaction is badly formatted or the same key=value already exist, we
again return the non-zero code. Otherwise, we add it to the store.
In the current design, a block can include incorrect transactions (those who
passed `CheckTx`, but failed `DeliverTx` or transactions included by the proposer
directly). This is done for performance reasons.
Note we can't commit transactions inside the `DeliverTx` because in such case
`Query`, which may be called in parallel, will return inconsistent data (i.e.
it will report that some value already exist even when the actual block was not
yet committed).
`Commit` instructs the application to persist the new state.
```java
@Override
public void commit(RequestCommit req, StreamObserver<ResponseCommit> responseObserver) {
txn.commit();
var resp = ResponseCommit.newBuilder()
.setData(ByteString.copyFrom(new byte[8]))
.build();
responseObserver.onNext(resp);
responseObserver.onCompleted();
}
```
### 1.3.5 Query
Now, when the client wants to know whenever a particular key/value exist, it
will call Tendermint Core RPC `/abci_query` endpoint, which in turn will call
the application's `Query` method.
Applications are free to provide their own APIs. But by using Tendermint Core
as a proxy, clients (including [light client
package](https://godoc.org/github.com/tendermint/tendermint/light)) can leverage
the unified API across different applications. Plus they won't have to call the
otherwise separate Tendermint Core API for additional proofs.
Note we don't include a proof here.
```java
@Override
public void query(RequestQuery req, StreamObserver<ResponseQuery> responseObserver) {
var k = req.getData().toByteArray();
var v = getPersistedValue(k);
var builder = ResponseQuery.newBuilder();
if (v == null) {
builder.setLog("does not exist");
} else {
builder.setLog("exists");
builder.setKey(ByteString.copyFrom(k));
builder.setValue(ByteString.copyFrom(v));
}
responseObserver.onNext(builder.build());
responseObserver.onCompleted();
}
```
The complete specification can be found
[here](https://docs.tendermint.com/master/spec/abci/).
## 1.4 Starting an application and a Tendermint Core instances
Put the following code into the `$KVSTORE_HOME/src/main/java/io/example/App.java` file:
```java
package io.example;
import jetbrains.exodus.env.Environment;
import jetbrains.exodus.env.Environments;
import java.io.IOException;
public class App {
public static void main(String[] args) throws IOException, InterruptedException {
try (Environment env = Environments.newInstance("tmp/storage")) {
var app = new KVStoreApp(env);
var server = new GrpcServer(app, 26658);
server.start();
server.blockUntilShutdown();
}
}
}
```
It is the entry point of the application.
Here we create a special object `Environment`, which knows where to store the application state.
Then we create and start the gRPC server to handle Tendermint Core requests.
Create the `$KVSTORE_HOME/src/main/java/io/example/GrpcServer.java` file with the following content:
```java
package io.example;
import io.grpc.BindableService;
import io.grpc.Server;
import io.grpc.ServerBuilder;
import java.io.IOException;
class GrpcServer {
private Server server;
GrpcServer(BindableService service, int port) {
this.server = ServerBuilder.forPort(port)
.addService(service)
.build();
}
void start() throws IOException {
server.start();
System.out.println("gRPC server started, listening on $port");
Runtime.getRuntime().addShutdownHook(new Thread(() -> {
System.out.println("shutting down gRPC server since JVM is shutting down");
GrpcServer.this.stop();
System.out.println("server shut down");
}));
}
private void stop() {
server.shutdown();
}
/**
* Await termination on the main thread since the grpc library uses daemon threads.
*/
void blockUntilShutdown() throws InterruptedException {
server.awaitTermination();
}
}
```
## 1.5 Getting Up and Running
To create a default configuration, nodeKey and private validator files, let's
execute `tendermint init`. But before we do that, we will need to install
Tendermint Core.
```bash
$ rm -rf /tmp/example
$ cd $GOPATH/src/github.com/tendermint/tendermint
$ make install
$ TMHOME="/tmp/example" tendermint init
I[2019-07-16|18:20:36.480] Generated private validator module=main keyFile=/tmp/example/config/priv_validator_key.json stateFile=/tmp/example2/data/priv_validator_state.json
I[2019-07-16|18:20:36.481] Generated node key module=main path=/tmp/example/config/node_key.json
I[2019-07-16|18:20:36.482] Generated genesis file module=main path=/tmp/example/config/genesis.json
```
Feel free to explore the generated files, which can be found at
`/tmp/example/config` directory. Documentation on the config can be found
[here](https://docs.tendermint.com/master/tendermint-core/configuration.html).
We are ready to start our application:
```bash
./gradlew run
gRPC server started, listening on 26658
```
Then we need to start Tendermint Core and point it to our application. Staying
within the application directory execute:
```bash
$ TMHOME="/tmp/example" tendermint node --abci grpc --proxy_app tcp://127.0.0.1:26658
I[2019-07-28|15:44:53.632] Version info module=main software=0.32.1 block=10 p2p=7
I[2019-07-28|15:44:53.677] Starting Node module=main impl=Node
I[2019-07-28|15:44:53.681] Started node module=main nodeInfo="{ProtocolVersion:{P2P:7 Block:10 App:0} ID_:7639e2841ccd47d5ae0f5aad3011b14049d3f452 ListenAddr:tcp://0.0.0.0:26656 Network:test-chain-Nhl3zk Version:0.32.1 Channels:4020212223303800 Moniker:Ivans-MacBook-Pro.local Other:{TxIndex:on RPCAddress:tcp://127.0.0.1:26657}}"
I[2019-07-28|15:44:54.801] Executed block module=state height=8 validTxs=0 invalidTxs=0
I[2019-07-28|15:44:54.814] Committed state module=state height=8 txs=0 appHash=0000000000000000
```
Now open another tab in your terminal and try sending a transaction:
```bash
$ curl -s 'localhost:26657/broadcast_tx_commit?tx="tendermint=rocks"'
{
"jsonrpc": "2.0",
"id": "",
"result": {
"check_tx": {
"gasWanted": "1"
},
"deliver_tx": {},
"hash": "CDD3C6DFA0A08CAEDF546F9938A2EEC232209C24AA0E4201194E0AFB78A2C2BB",
"height": "33"
}
```
Response should contain the height where this transaction was committed.
Now let's check if the given key now exists and its value:
```bash
$ curl -s 'localhost:26657/abci_query?data="tendermint"'
{
"jsonrpc": "2.0",
"id": "",
"result": {
"response": {
"log": "exists",
"key": "dGVuZGVybWludA==",
"value": "cm9ja3My"
}
}
}
```
`dGVuZGVybWludA==` and `cm9ja3M=` are the base64-encoding of the ASCII of `tendermint` and `rocks` accordingly.
## Outro
I hope everything went smoothly and your first, but hopefully not the last,
Tendermint Core application is up and running. If not, please [open an issue on
Github](https://github.com/tendermint/tendermint/issues/new/choose). To dig
deeper, read [the docs](https://docs.tendermint.com/master/).
The full source code of this example project can be found [here](https://github.com/climber73/tendermint-abci-grpc-java).

604
docs/tutorials/kotlin.md Normal file
View File

@@ -0,0 +1,604 @@
<!---
order: 4
--->
# Creating an application in Kotlin
## Guide Assumptions
This guide is designed for beginners who want to get started with a Tendermint
Core application from scratch. It does not assume that you have any prior
experience with Tendermint Core.
Tendermint Core is Byzantine Fault Tolerant (BFT) middleware that takes a state
transition machine (your application) - written in any programming language - and securely
replicates it on many machines.
By following along with this guide, you'll create a Tendermint Core project
called kvstore, a (very) simple distributed BFT key-value store. The application (which should
implementing the blockchain interface (ABCI)) will be written in Kotlin.
This guide assumes that you are not new to JVM world. If you are new please see [JVM Minimal Survival Guide](https://hadihariri.com/2013/12/29/jvm-minimal-survival-guide-for-the-dotnet-developer/#java-the-language-java-the-ecosystem-java-the-jvm) and [Gradle Docs](https://docs.gradle.org/current/userguide/userguide.html).
## Built-in app vs external app
If you use Golang, you can run your app and Tendermint Core in the same process to get maximum performance.
[Cosmos SDK](https://github.com/cosmos/cosmos-sdk) is written this way.
Please refer to [Writing a built-in Tendermint Core application in Go](./go-built-in.md) guide for details.
If you choose another language, like we did in this guide, you have to write a separate app,
which will communicate with Tendermint Core via a socket (UNIX or TCP) or gRPC.
This guide will show you how to build external application using RPC server.
Having a separate application might give you better security guarantees as two
processes would be communicating via established binary protocol. Tendermint
Core will not have access to application's state.
## 1.1 Installing Java and Gradle
Please refer to [the Oracle's guide for installing JDK](https://www.oracle.com/technetwork/java/javase/downloads/index.html).
Verify that you have installed Java successfully:
```bash
java -version
java version "1.8.0_162"
Java(TM) SE Runtime Environment (build 1.8.0_162-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.162-b12, mixed mode)
```
You can choose any version of Java higher or equal to 8.
In my case it is Java SE Development Kit 8.
Make sure you have `$JAVA_HOME` environment variable set:
```bash
echo $JAVA_HOME
/Library/Java/JavaVirtualMachines/jdk1.8.0_162.jdk/Contents/Home
```
For Gradle installation, please refer to [their official guide](https://gradle.org/install/).
## 1.2 Creating a new Kotlin project
We'll start by creating a new Gradle project.
```bash
export KVSTORE_HOME=~/kvstore
mkdir $KVSTORE_HOME
cd $KVSTORE_HOME
```
Inside the example directory run:
```bash
gradle init --dsl groovy --package io.example --project-name example --type kotlin-application
```
This will create a new project for you. The tree of files should look like:
```bash
tree
.
|-- build.gradle
|-- gradle
| `-- wrapper
| |-- gradle-wrapper.jar
| `-- gradle-wrapper.properties
|-- gradlew
|-- gradlew.bat
|-- settings.gradle
`-- src
|-- main
| |-- kotlin
| | `-- io
| | `-- example
| | `-- App.kt
| `-- resources
`-- test
|-- kotlin
| `-- io
| `-- example
| `-- AppTest.kt
`-- resources
```
When run, this should print "Hello world." to the standard output.
```bash
./gradlew run
> Task :run
Hello world.
```
## 1.3 Writing a Tendermint Core application
Tendermint Core communicates with the application through the Application
BlockChain Interface (ABCI). All message types are defined in the [protobuf
file](https://github.com/tendermint/tendermint/blob/master/abci/types/types.proto).
This allows Tendermint Core to run applications written in any programming
language.
### 1.3.1 Compile .proto files
Add the following piece to the top of the `build.gradle`:
```groovy
buildscript {
repositories {
mavenCentral()
}
dependencies {
classpath 'com.google.protobuf:protobuf-gradle-plugin:0.8.8'
}
}
```
Enable the protobuf plugin in the `plugins` section of the `build.gradle`:
```groovy
plugins {
id 'com.google.protobuf' version '0.8.8'
}
```
Add the following code to `build.gradle`:
```groovy
protobuf {
protoc {
artifact = "com.google.protobuf:protoc:3.7.1"
}
plugins {
grpc {
artifact = 'io.grpc:protoc-gen-grpc-java:1.22.1'
}
}
generateProtoTasks {
all()*.plugins {
grpc {}
}
}
}
```
Now we should be ready to compile the `*.proto` files.
Copy the necessary `.proto` files to your project:
```bash
mkdir -p \
$KVSTORE_HOME/src/main/proto/github.com/tendermint/tendermint/proto/tendermint/abci \
$KVSTORE_HOME/src/main/proto/github.com/tendermint/tendermint/proto/tendermint/version \
$KVSTORE_HOME/src/main/proto/github.com/tendermint/tendermint/proto/tendermint/types \
$KVSTORE_HOME/src/main/proto/github.com/tendermint/tendermint/proto/tendermint/crypto \
$KVSTORE_HOME/src/main/proto/github.com/tendermint/tendermint/proto/tendermint/libs \
$KVSTORE_HOME/src/main/proto/github.com/gogo/protobuf/gogoproto
cp $GOPATH/src/github.com/tendermint/tendermint/proto/tendermint/abci/types.proto \
$KVSTORE_HOME/src/main/proto/github.com/tendermint/tendermint/proto/tendermint/abci/types.proto
cp $GOPATH/src/github.com/tendermint/tendermint/proto/tendermint/version/version.proto \
$KVSTORE_HOME/src/main/proto/github.com/tendermint/tendermint/proto/tendermint/version/version.proto
cp $GOPATH/src/github.com/tendermint/tendermint/proto/tendermint/types/types.proto \
$KVSTORE_HOME/src/main/proto/github.com/tendermint/tendermint/proto/tendermint/types/types.proto
cp $GOPATH/src/github.com/tendermint/tendermint/proto/tendermint/types/evidence.proto \
$KVSTORE_HOME/src/main/proto/github.com/tendermint/tendermint/proto/tendermint/types/evidence.proto
cp $GOPATH/src/github.com/tendermint/tendermint/proto/tendermint/types/params.proto \
$KVSTORE_HOME/src/main/proto/github.com/tendermint/tendermint/proto/tendermint/types/params.proto
cp $GOPATH/src/github.com/tendermint/tendermint/proto/tendermint/crypto/merkle.proto \
$KVSTORE_HOME/src/main/proto/github.com/tendermint/tendermint/proto/tendermint/crypto/merkle.proto
cp $GOPATH/src/github.com/tendermint/tendermint/proto/tendermint/crypto/keys.proto \
$KVSTORE_HOME/src/main/proto/github.com/tendermint/tendermint/proto/tendermint/crypto/keys.proto
cp $GOPATH/src/github.com/tendermint/tendermint/proto/tendermint/libs/types.proto \
$KVSTORE_HOME/src/main/proto/github.com/tendermint/tendermint/proto/tendermint/libs/types.proto
cp $GOPATH/src/github.com/gogo/protobuf/gogoproto/gogo.proto \
$KVSTORE_HOME/src/main/proto/github.com/gogo/protobuf/gogoproto/gogo.proto
```
Add these dependencies to `build.gradle`:
```groovy
dependencies {
implementation 'io.grpc:grpc-protobuf:1.22.1'
implementation 'io.grpc:grpc-netty-shaded:1.22.1'
implementation 'io.grpc:grpc-stub:1.22.1'
}
```
To generate all protobuf-type classes run:
```bash
./gradlew generateProto
```
To verify that everything went smoothly, you can inspect the `build/generated/` directory:
```bash
tree build/generated/
build/generated/
`-- source
`-- proto
`-- main
|-- grpc
| `-- types
| `-- ABCIApplicationGrpc.java
`-- java
|-- com
| `-- google
| `-- protobuf
| `-- GoGoProtos.java
|-- common
| `-- Types.java
|-- merkle
| `-- Merkle.java
`-- types
`-- Types.java
```
### 1.3.2 Implementing ABCI
The resulting `$KVSTORE_HOME/build/generated/source/proto/main/grpc/types/ABCIApplicationGrpc.java` file
contains the abstract class `ABCIApplicationImplBase`, which is an interface we'll need to implement.
Create `$KVSTORE_HOME/src/main/kotlin/io/example/KVStoreApp.kt` file with the following content:
```kotlin
package io.example
import io.grpc.stub.StreamObserver
import types.ABCIApplicationGrpc
import types.Types.*
class KVStoreApp : ABCIApplicationGrpc.ABCIApplicationImplBase() {
// methods implementation
}
```
Now I will go through each method of `ABCIApplicationImplBase` explaining when it's called and adding
required business logic.
### 1.3.3 CheckTx
When a new transaction is added to the Tendermint Core, it will ask the
application to check it (validate the format, signatures, etc.).
```kotlin
override fun checkTx(req: RequestCheckTx, responseObserver: StreamObserver<ResponseCheckTx>) {
val code = req.tx.validate()
val resp = ResponseCheckTx.newBuilder()
.setCode(code)
.setGasWanted(1)
.build()
responseObserver.onNext(resp)
responseObserver.onCompleted()
}
private fun ByteString.validate(): Int {
val parts = this.split('=')
if (parts.size != 2) {
return 1
}
val key = parts[0]
val value = parts[1]
// check if the same key=value already exists
val stored = getPersistedValue(key)
if (stored != null && stored.contentEquals(value)) {
return 2
}
return 0
}
private fun ByteString.split(separator: Char): List<ByteArray> {
val arr = this.toByteArray()
val i = (0 until this.size()).firstOrNull { arr[it] == separator.toByte() }
?: return emptyList()
return listOf(
this.substring(0, i).toByteArray(),
this.substring(i + 1).toByteArray()
)
}
```
Don't worry if this does not compile yet.
If the transaction does not have a form of `{bytes}={bytes}`, we return `1`
code. When the same key=value already exist (same key and value), we return `2`
code. For others, we return a zero code indicating that they are valid.
Note that anything with non-zero code will be considered invalid (`-1`, `100`,
etc.) by Tendermint Core.
Valid transactions will eventually be committed given they are not too big and
have enough gas. To learn more about gas, check out ["the
specification"](https://docs.tendermint.com/master/spec/abci/apps.html#gas).
For the underlying key-value store we'll use
[JetBrains Xodus](https://github.com/JetBrains/xodus), which is a transactional schema-less embedded high-performance database written in Java.
`build.gradle`:
```groovy
dependencies {
implementation 'org.jetbrains.xodus:xodus-environment:1.3.91'
}
```
```kotlin
...
import jetbrains.exodus.ArrayByteIterable
import jetbrains.exodus.env.Environment
import jetbrains.exodus.env.Store
import jetbrains.exodus.env.StoreConfig
import jetbrains.exodus.env.Transaction
class KVStoreApp(
private val env: Environment
) : ABCIApplicationGrpc.ABCIApplicationImplBase() {
private var txn: Transaction? = null
private var store: Store? = null
...
private fun getPersistedValue(k: ByteArray): ByteArray? {
return env.computeInReadonlyTransaction { txn ->
val store = env.openStore("store", StoreConfig.WITHOUT_DUPLICATES, txn)
store.get(txn, ArrayByteIterable(k))?.bytesUnsafe
}
}
}
```
### 1.3.4 BeginBlock -> DeliverTx -> EndBlock -> Commit
When Tendermint Core has decided on the block, it's transferred to the
application in 3 parts: `BeginBlock`, one `DeliverTx` per transaction and
`EndBlock` in the end. `DeliverTx` are being transferred asynchronously, but the
responses are expected to come in order.
```kotlin
override fun beginBlock(req: RequestBeginBlock, responseObserver: StreamObserver<ResponseBeginBlock>) {
txn = env.beginTransaction()
store = env.openStore("store", StoreConfig.WITHOUT_DUPLICATES, txn!!)
val resp = ResponseBeginBlock.newBuilder().build()
responseObserver.onNext(resp)
responseObserver.onCompleted()
}
```
Here we begin a new transaction, which will accumulate the block's transactions and open the corresponding store.
```kotlin
override fun deliverTx(req: RequestDeliverTx, responseObserver: StreamObserver<ResponseDeliverTx>) {
val code = req.tx.validate()
if (code == 0) {
val parts = req.tx.split('=')
val key = ArrayByteIterable(parts[0])
val value = ArrayByteIterable(parts[1])
store!!.put(txn!!, key, value)
}
val resp = ResponseDeliverTx.newBuilder()
.setCode(code)
.build()
responseObserver.onNext(resp)
responseObserver.onCompleted()
}
```
If the transaction is badly formatted or the same key=value already exist, we
again return the non-zero code. Otherwise, we add it to the store.
In the current design, a block can include incorrect transactions (those who
passed `CheckTx`, but failed `DeliverTx` or transactions included by the proposer
directly). This is done for performance reasons.
Note we can't commit transactions inside the `DeliverTx` because in such case
`Query`, which may be called in parallel, will return inconsistent data (i.e.
it will report that some value already exist even when the actual block was not
yet committed).
`Commit` instructs the application to persist the new state.
```kotlin
override fun commit(req: RequestCommit, responseObserver: StreamObserver<ResponseCommit>) {
txn!!.commit()
val resp = ResponseCommit.newBuilder()
.setData(ByteString.copyFrom(ByteArray(8)))
.build()
responseObserver.onNext(resp)
responseObserver.onCompleted()
}
```
### 1.3.5 Query
Now, when the client wants to know whenever a particular key/value exist, it
will call Tendermint Core RPC `/abci_query` endpoint, which in turn will call
the application's `Query` method.
Applications are free to provide their own APIs. But by using Tendermint Core
as a proxy, clients (including [light client
package](https://godoc.org/github.com/tendermint/tendermint/light)) can leverage
the unified API across different applications. Plus they won't have to call the
otherwise separate Tendermint Core API for additional proofs.
Note we don't include a proof here.
```kotlin
override fun query(req: RequestQuery, responseObserver: StreamObserver<ResponseQuery>) {
val k = req.data.toByteArray()
val v = getPersistedValue(k)
val builder = ResponseQuery.newBuilder()
if (v == null) {
builder.log = "does not exist"
} else {
builder.log = "exists"
builder.key = ByteString.copyFrom(k)
builder.value = ByteString.copyFrom(v)
}
responseObserver.onNext(builder.build())
responseObserver.onCompleted()
}
```
The complete specification can be found
[here](https://docs.tendermint.com/master/spec/abci/).
## 1.4 Starting an application and a Tendermint Core instances
Put the following code into the `$KVSTORE_HOME/src/main/kotlin/io/example/App.kt` file:
```kotlin
package io.example
import jetbrains.exodus.env.Environments
fun main() {
Environments.newInstance("tmp/storage").use { env ->
val app = KVStoreApp(env)
val server = GrpcServer(app, 26658)
server.start()
server.blockUntilShutdown()
}
}
```
It is the entry point of the application.
Here we create a special object `Environment`, which knows where to store the application state.
Then we create and start the gRPC server to handle Tendermint Core requests.
Create `$KVSTORE_HOME/src/main/kotlin/io/example/GrpcServer.kt` file with the following content:
```kotlin
package io.example
import io.grpc.BindableService
import io.grpc.ServerBuilder
class GrpcServer(
private val service: BindableService,
private val port: Int
) {
private val server = ServerBuilder
.forPort(port)
.addService(service)
.build()
fun start() {
server.start()
println("gRPC server started, listening on $port")
Runtime.getRuntime().addShutdownHook(object : Thread() {
override fun run() {
println("shutting down gRPC server since JVM is shutting down")
this@GrpcServer.stop()
println("server shut down")
}
})
}
fun stop() {
server.shutdown()
}
/**
* Await termination on the main thread since the grpc library uses daemon threads.
*/
fun blockUntilShutdown() {
server.awaitTermination()
}
}
```
## 1.5 Getting Up and Running
To create a default configuration, nodeKey and private validator files, let's
execute `tendermint init`. But before we do that, we will need to install
Tendermint Core.
```bash
rm -rf /tmp/example
cd $GOPATH/src/github.com/tendermint/tendermint
make install
TMHOME="/tmp/example" tendermint init
I[2019-07-16|18:20:36.480] Generated private validator module=main keyFile=/tmp/example/config/priv_validator_key.json stateFile=/tmp/example2/data/priv_validator_state.json
I[2019-07-16|18:20:36.481] Generated node key module=main path=/tmp/example/config/node_key.json
I[2019-07-16|18:20:36.482] Generated genesis file module=main path=/tmp/example/config/genesis.json
```
Feel free to explore the generated files, which can be found at
`/tmp/example/config` directory. Documentation on the config can be found
[here](https://docs.tendermint.com/master/tendermint-core/configuration.html).
We are ready to start our application:
```bash
./gradlew run
gRPC server started, listening on 26658
```
Then we need to start Tendermint Core and point it to our application. Staying
within the application directory execute:
```bash
TMHOME="/tmp/example" tendermint node --abci grpc --proxy_app tcp://127.0.0.1:26658
I[2019-07-28|15:44:53.632] Version info module=main software=0.32.1 block=10 p2p=7
I[2019-07-28|15:44:53.677] Starting Node module=main impl=Node
I[2019-07-28|15:44:53.681] Started node module=main nodeInfo="{ProtocolVersion:{P2P:7 Block:10 App:0} ID_:7639e2841ccd47d5ae0f5aad3011b14049d3f452 ListenAddr:tcp://0.0.0.0:26656 Network:test-chain-Nhl3zk Version:0.32.1 Channels:4020212223303800 Moniker:Ivans-MacBook-Pro.local Other:{TxIndex:on RPCAddress:tcp://127.0.0.1:26657}}"
I[2019-07-28|15:44:54.801] Executed block module=state height=8 validTxs=0 invalidTxs=0
I[2019-07-28|15:44:54.814] Committed state module=state height=8 txs=0 appHash=0000000000000000
```
Now open another tab in your terminal and try sending a transaction:
```bash
curl -s 'localhost:26657/broadcast_tx_commit?tx="tendermint=rocks"'
{
"jsonrpc": "2.0",
"id": "",
"result": {
"check_tx": {
"gasWanted": "1"
},
"deliver_tx": {},
"hash": "CDD3C6DFA0A08CAEDF546F9938A2EEC232209C24AA0E4201194E0AFB78A2C2BB",
"height": "33"
}
```
Response should contain the height where this transaction was committed.
Now let's check if the given key now exists and its value:
```bash
curl -s 'localhost:26657/abci_query?data="tendermint"'
{
"jsonrpc": "2.0",
"id": "",
"result": {
"response": {
"log": "exists",
"key": "dGVuZGVybWludA==",
"value": "cm9ja3My"
}
}
}
```
`dGVuZGVybWludA==` and `cm9ja3M=` are the base64-encoding of the ASCII of `tendermint` and `rocks` accordingly.
## Outro
I hope everything went smoothly and your first, but hopefully not the last,
Tendermint Core application is up and running. If not, please [open an issue on
Github](https://github.com/tendermint/tendermint/issues/new/choose). To dig
deeper, read [the docs](https://docs.tendermint.com/master/).
The full source code of this example project can be found [here](https://github.com/climber73/tendermint-abci-grpc-kotlin).

7
docs/tutorials/readme.md Normal file
View File

@@ -0,0 +1,7 @@
---
order: false
parent:
order: 2
---
# Guides