2.53

skip cookie checking if from grpc api
add block and mutex profiling
2026-10-01 20:26:27 +00:00 · 2021-06-13 17:12:34 -07:00 · 2021-06-13 16:16:11 -07:00 · 2021-06-13 16:15:54 -07:00 · 2021-06-13 07:31:56 -07:00 · 2021-06-13 07:09:06 -07:00
113 changed files with 1590 additions and 713 deletions
@@ -3,10 +3,10 @@

 [![Slack](https://img.shields.io/badge/slack-purple)](https://join.slack.com/t/seaweedfs/shared_invite/enQtMzI4MTMwMjU2MzA3LTEyYzZmZWYzOGQ3MDJlZWMzYmI0OTE4OTJiZjJjODBmMzUxNmYwODg0YjY3MTNlMjBmZDQ1NzQ5NDJhZWI2ZmY)
 [![Twitter](https://img.shields.io/twitter/follow/seaweedfs.svg?style=social&label=Follow)](https://twitter.com/intent/follow?screen_name=seaweedfs)
-[![Build Status](https://travis-ci.org/chrislusf/seaweedfs.svg?branch=master)](https://travis-ci.org/chrislusf/seaweedfs)
+[![Build Status](https://travis-ci.com/chrislusf/seaweedfs.svg?branch=master)](https://travis-ci.org/chrislusf/seaweedfs)
 [![GoDoc](https://godoc.org/github.com/chrislusf/seaweedfs/weed?status.svg)](https://godoc.org/github.com/chrislusf/seaweedfs/weed)
 [![Wiki](https://img.shields.io/badge/docs-wiki-blue.svg)](https://github.com/chrislusf/seaweedfs/wiki)
-[![Docker Pulls](https://img.shields.io/docker/pulls/chrislusf/seaweedfs.svg?maxAge=4800)](https://hub.docker.com/r/chrislusf/seaweedfs/)
+[![Docker Pulls](https://img.shields.io/docker/pulls/chrislusf/seaweedfs?maxAge=4800)](https://hub.docker.com/r/chrislusf/seaweedfs/)
 [![SeaweedFS on Maven Central](https://img.shields.io/maven-central/v/com.github.chrislusf/seaweedfs-client)](https://search.maven.org/search?q=g:com.github.chrislusf)


@@ -44,7 +44,8 @@ Your support will be really appreciated by me and other supporters!
 - [SeaweedFS Mailing List](https://groups.google.com/d/forum/seaweedfs)
 - [Wiki Documentation](https://github.com/chrislusf/seaweedfs/wiki)
 - [SeaweedFS White Paper](https://github.com/chrislusf/seaweedfs/wiki/SeaweedFS_Architecture.pdf)
- [SeaweedFS Introduction Slides](https://www.slideshare.net/chrislusf/seaweedfs-introduction)
+- [SeaweedFS Introduction Slides 2021.5](https://docs.google.com/presentation/d/1DcxKWlINc-HNCjhYeERkpGXXm6nTCES8mi2W5G0Z4Ts/edit?usp=sharing)
+- [SeaweedFS Introduction Slides 2019.3](https://www.slideshare.net/chrislusf/seaweedfs-introduction)

 Table of Contents
 =================
@@ -92,13 +93,6 @@ and these volume servers manage files and their metadata.
 This relieves concurrency pressure from the central master and spreads file metadata into volume servers, 
 allowing faster file access (O(1), usually just one disk read operation).

-SeaweedFS can transparently integrate with the cloud. 
-With hot data on local cluster, and warm data on the cloud with O(1) access time, 
-SeaweedFS can achieve both fast local access time and elastic cloud storage capacity. 
-What's more, the cloud storage access API cost is minimized. 
-Faster and Cheaper than direct cloud storage!
-Signup for future managed SeaweedFS cluster offering at "seaweedfilesystem at gmail dot com".
-
 There is only 40 bytes of disk storage overhead for each file's metadata. 
 It is so simple with O(1) disk reads that you are welcome to challenge the performance with your actual use cases.

@@ -108,12 +102,18 @@ Also, SeaweedFS implements erasure coding with ideas from

 On top of the object store, optional [Filer] can support directories and POSIX attributes. 
 Filer is a separate linearly-scalable stateless server with customizable metadata stores, 
-e.g., MySql, Postgres, Redis, Cassandra, HBase, Mongodb, Elastic Search, LevelDB, RocksDB, MemSql, TiDB, Etcd, CockroachDB, etc.
+e.g., MySql, Postgres, Redis, Cassandra, HBase, Mongodb, Elastic Search, LevelDB, RocksDB, Sqlite, MemSql, TiDB, Etcd, CockroachDB, etc.

 For any distributed key value stores, the large values can be offloaded to SeaweedFS. 
 With the fast access speed and linearly scalable capacity, 
 SeaweedFS can work as a distributed [Key-Large-Value store][KeyLargeValueStore].

+SeaweedFS can transparently integrate with the cloud. 
+With hot data on local cluster, and warm data on the cloud with O(1) access time, 
+SeaweedFS can achieve both fast local access time and elastic cloud storage capacity.
+What's more, the cloud storage access API cost is minimized. 
+Faster and Cheaper than direct cloud storage!
+
 [Back to TOC](#table-of-contents)

 ## Additional Features ##
@@ -399,7 +399,7 @@ The architectures are mostly the same. SeaweedFS aims to store and read files fa

 * SeaweedFS optimizes for small files, ensuring O(1) disk seek operation, and can also handle large files.
 * SeaweedFS statically assigns a volume id for a file. Locating file content becomes just a lookup of the volume id, which can be easily cached.
-* SeaweedFS Filer metadata store can be any well-known and proven data stores, e.g., Redis, Cassandra, HBase, Mongodb, Elastic Search, MySql, Postgres, MemSql, TiDB, CockroachDB, Etcd etc, and is easy to customized.
+* SeaweedFS Filer metadata store can be any well-known and proven data stores, e.g., Redis, Cassandra, HBase, Mongodb, Elastic Search, MySql, Postgres, Sqlite, MemSql, TiDB, CockroachDB, Etcd etc, and is easy to customized.
 * SeaweedFS Volume server also communicates directly with clients via HTTP, supporting range queries, direct uploads, etc.

 | System         | File Metadata                   | File Content Read| POSIX  | REST API | Optimized for large number of small files |
@@ -441,7 +441,7 @@ Ceph uses CRUSH hashing to automatically manage the data placement, which is eff

 SeaweedFS is optimized for small files. Small files are stored as one continuous block of content, with at most 8 unused bytes between files. Small file access is O(1) disk read.

-SeaweedFS Filer uses off-the-shelf stores, such as MySql, Postgres, Mongodb, Redis, Elastic Search, Cassandra, HBase, MemSql, TiDB, CockroachCB, Etcd, to manage file directories. These stores are proven, scalable, and easier to manage.
+SeaweedFS Filer uses off-the-shelf stores, such as MySql, Postgres, Sqlite, Mongodb, Redis, Elastic Search, Cassandra, HBase, MemSql, TiDB, CockroachCB, Etcd, to manage file directories. These stores are proven, scalable, and easier to manage.

 | SeaweedFS         | comparable to Ceph | advantage |
 | -------------  | ------------- | ---------------- |
@@ -38,7 +38,7 @@ services:
      S3TEST_CONF: "s3tests.conf"
      NOSETESTS_OPTIONS: "--verbose --logging-level=ERROR --with-xunit --failure-detail s3tests_boto3.functional.test_s3"
      NOSETESTS_ATTR: "!tagging,!fails_on_aws,!encryption,!bucket-policy,!versioning,!fails_on_rgw,!bucket-policy,!fails_with_subdomain,!policy_status,!object-lock,!lifecycle,!cors,!user-policy"
-      NOSETESTS_EXCLUDE: "(bucket_list_delimiter_basic|bucket_listv2_delimiter_basic|bucket_listv2_encoding_basic|bucket_list_encoding_basic|bucket_list_delimiter_prefix|bucket_listv2_delimiter_prefix_ends_with_delimiter|bucket_list_delimiter_prefix_ends_with_delimiter|bucket_list_delimiter_alt|bucket_listv2_delimiter_alt|bucket_list_delimiter_prefix_underscore|bucket_list_delimiter_percentage|bucket_listv2_delimiter_percentage|bucket_list_delimiter_whitespace|bucket_listv2_delimiter_whitespace|bucket_list_delimiter_dot|bucket_listv2_delimiter_dot|bucket_list_delimiter_unreadable|bucket_listv2_delimiter_unreadable|bucket_listv2_fetchowner_defaultempty|bucket_listv2_fetchowner_empty|bucket_list_prefix_delimiter_alt|bucket_listv2_prefix_delimiter_alt|bucket_list_prefix_delimiter_prefix_not_exist|bucket_listv2_prefix_delimiter_prefix_not_exist|bucket_list_prefix_delimiter_delimiter_not_exist|bucket_listv2_prefix_delimiter_delimiter_not_exist|bucket_list_prefix_delimiter_prefix_delimiter_not_exist|bucket_listv2_prefix_delimiter_prefix_delimiter_not_exist|bucket_list_maxkeys_none|bucket_listv2_maxkeys_none|bucket_list_maxkeys_invalid|bucket_listv2_continuationtoken_empty|bucket_list_return_data|bucket_list_objects_anonymous|bucket_listv2_objects_anonymous|bucket_notexist|bucketv2_notexist|bucket_delete_nonempty|bucket_concurrent_set_canned_acl|object_write_to_nonexist_bucket|object_requestid_matches_header_on_error|object_write_cache_control|object_write_expires|object_set_get_metadata_none_to_good|object_set_get_metadata_none_to_empty|object_set_get_metadata_overwrite_to_empty|post_object_anonymous_request|post_object_authenticated_request|post_object_authenticated_no_content_type|post_object_authenticated_request_bad_access_key|post_object_set_success_code|post_object_set_invalid_success_code|post_object_upload_larger_than_chunk|post_object_set_key_from_filename|post_object_ignored_header|post_object_case_insensitive_condition_fields|post_object_escaped_field_values|post_object_success_redirect_action|post_object_invalid_signature|post_object_invalid_access_key|post_object_missing_policy_condition|post_object_user_specified_header|post_object_request_missing_policy_specified_field|post_object_expired_policy|post_object_invalid_request_field_value|get_object_ifmatch_failed|get_object_ifunmodifiedsince_good|put_object_ifmatch_failed|object_raw_get_bucket_gone|object_delete_key_bucket_gone|object_raw_get_bucket_acl|object_raw_get_object_acl|object_raw_response_headers|object_raw_authenticated_bucket_gone|object_raw_get_x_amz_expires_out_max_range|object_raw_get_x_amz_expires_out_positive_range|object_anon_put_write_access|object_raw_put_authenticated_expired|bucket_create_exists|bucket_create_naming_bad_short_one|bucket_create_naming_bad_short_two|bucket_get_location|bucket_acl_default|bucket_acl_canned|bucket_acl_canned_publicreadwrite|bucket_acl_canned_authenticatedread|object_acl_default|object_acl_canned_during_create|object_acl_canned|object_acl_canned_publicreadwrite|object_acl_canned_authenticatedread|object_acl_canned_bucketownerread|object_acl_canned_bucketownerfullcontrol|object_acl_full_control_verify_attributes|bucket_acl_canned_private_to_private|bucket_acl_grant_nonexist_user|bucket_acl_no_grants|bucket_acl_grant_email_not_exist|bucket_acl_revoke_all|bucket_recreate_not_overriding|object_copy_verify_contenttype|object_copy_to_itself_with_metadata|object_copy_not_owned_bucket|object_copy_not_owned_object_bucket|object_copy_retaining_metadata|object_copy_replacing_metadata|multipart_upload_empty|multipart_copy_invalid_range|multipart_copy_special_names|multipart_upload_resend_part|multipart_upload_size_too_small|abort_multipart_upload_not_found|multipart_upload_missing_part|multipart_upload_incorrect_etag|100_continue|ranged_request_invalid_range|ranged_request_empty_object|access_bucket)"
+      NOSETESTS_EXCLUDE: "(bucket_list_delimiter_basic|bucket_listv2_delimiter_basic|bucket_listv2_encoding_basic|bucket_list_encoding_basic|bucket_list_delimiter_prefix|bucket_listv2_delimiter_prefix_ends_with_delimiter|bucket_list_delimiter_prefix_ends_with_delimiter|bucket_list_delimiter_alt|bucket_listv2_delimiter_alt|bucket_list_delimiter_prefix_underscore|bucket_list_delimiter_percentage|bucket_listv2_delimiter_percentage|bucket_list_delimiter_whitespace|bucket_listv2_delimiter_whitespace|bucket_list_delimiter_dot|bucket_listv2_delimiter_dot|bucket_list_delimiter_unreadable|bucket_listv2_delimiter_unreadable|bucket_listv2_fetchowner_defaultempty|bucket_listv2_fetchowner_empty|bucket_list_prefix_delimiter_alt|bucket_listv2_prefix_delimiter_alt|bucket_list_prefix_delimiter_prefix_not_exist|bucket_listv2_prefix_delimiter_prefix_not_exist|bucket_list_prefix_delimiter_delimiter_not_exist|bucket_listv2_prefix_delimiter_delimiter_not_exist|bucket_list_prefix_delimiter_prefix_delimiter_not_exist|bucket_listv2_prefix_delimiter_prefix_delimiter_not_exist|bucket_list_maxkeys_none|bucket_listv2_maxkeys_none|bucket_list_maxkeys_invalid|bucket_listv2_continuationtoken_empty|bucket_list_return_data|bucket_list_objects_anonymous|bucket_listv2_objects_anonymous|bucket_notexist|bucketv2_notexist|bucket_delete_nonempty|bucket_concurrent_set_canned_acl|object_write_to_nonexist_bucket|object_requestid_matches_header_on_error|object_set_get_metadata_none_to_good|object_set_get_metadata_none_to_empty|object_set_get_metadata_overwrite_to_empty|post_object_anonymous_request|post_object_authenticated_request|post_object_authenticated_no_content_type|post_object_authenticated_request_bad_access_key|post_object_set_success_code|post_object_set_invalid_success_code|post_object_upload_larger_than_chunk|post_object_set_key_from_filename|post_object_ignored_header|post_object_case_insensitive_condition_fields|post_object_escaped_field_values|post_object_success_redirect_action|post_object_invalid_signature|post_object_invalid_access_key|post_object_missing_policy_condition|post_object_user_specified_header|post_object_request_missing_policy_specified_field|post_object_expired_policy|post_object_invalid_request_field_value|get_object_ifunmodifiedsince_good|put_object_ifmatch_failed|object_raw_get_bucket_gone|object_delete_key_bucket_gone|object_raw_get_bucket_acl|object_raw_get_object_acl|object_raw_response_headers|object_raw_authenticated_bucket_gone|object_raw_get_x_amz_expires_out_max_range|object_raw_get_x_amz_expires_out_positive_range|object_anon_put_write_access|object_raw_put_authenticated_expired|bucket_create_exists|bucket_create_naming_bad_short_one|bucket_create_naming_bad_short_two|bucket_get_location|bucket_acl_default|bucket_acl_canned|bucket_acl_canned_publicreadwrite|bucket_acl_canned_authenticatedread|object_acl_default|object_acl_canned_during_create|object_acl_canned|object_acl_canned_publicreadwrite|object_acl_canned_authenticatedread|object_acl_canned_bucketownerread|object_acl_canned_bucketownerfullcontrol|object_acl_full_control_verify_attributes|bucket_acl_canned_private_to_private|bucket_acl_grant_nonexist_user|bucket_acl_no_grants|bucket_acl_grant_email_not_exist|bucket_acl_revoke_all|bucket_recreate_not_overriding|object_copy_verify_contenttype|object_copy_to_itself_with_metadata|object_copy_not_owned_bucket|object_copy_not_owned_object_bucket|object_copy_retaining_metadata|object_copy_replacing_metadata|multipart_upload_empty|multipart_copy_invalid_range|multipart_copy_special_names|multipart_upload_resend_part|multipart_upload_size_too_small|abort_multipart_upload_not_found|multipart_upload_missing_part|multipart_upload_incorrect_etag|100_continue|ranged_request_invalid_range|ranged_request_empty_object|access_bucket)"
    depends_on:
      - master
      - volume
@@ -6,7 +6,6 @@ require (
 	cloud.google.com/go v0.58.0 // indirect
 	cloud.google.com/go/pubsub v1.3.1
 	cloud.google.com/go/storage v1.9.0
-	github.com/Azure/azure-amqp-common-go/v2 v2.1.0 // indirect
 	github.com/Azure/azure-pipeline-go v0.2.2 // indirect
 	github.com/Azure/azure-storage-blob-go v0.9.0
 	github.com/OneOfOne/xxhash v1.2.2
@@ -39,7 +38,7 @@ require (
 	github.com/google/uuid v1.1.1
 	github.com/gorilla/mux v1.7.4
 	github.com/gorilla/websocket v1.4.1 // indirect
-	github.com/grpc-ecosystem/go-grpc-middleware v1.0.1-0.20190118093823-f849b5445de4 // indirect
+	github.com/grpc-ecosystem/go-grpc-middleware v1.0.1-0.20190118093823-f849b5445de4
 	github.com/grpc-ecosystem/grpc-gateway v1.11.0 // indirect
 	github.com/jcmturner/gofork v1.0.0 // indirect
 	github.com/jinzhu/copier v0.2.8
@@ -59,9 +58,10 @@ require (
 	github.com/olivere/elastic/v7 v7.0.19
 	github.com/peterh/liner v1.1.0
 	github.com/pierrec/lz4 v2.2.7+incompatible // indirect
+	github.com/pquerna/cachecontrol v0.1.0
 	github.com/prometheus/client_golang v1.3.0
 	github.com/rcrowley/go-metrics v0.0.0-20190826022208-cac0b30c2563 // indirect
-	github.com/seaweedfs/fuse v1.1.6
+	github.com/seaweedfs/fuse v1.1.7
 	github.com/seaweedfs/goexif v1.0.2
 	github.com/skip2/go-qrcode v0.0.0-20200617195104-da1b6568686e
 	github.com/spaolacci/murmur3 v1.1.0 // indirect
@@ -88,9 +88,8 @@ require (
 	gocloud.dev/pubsub/rabbitpubsub v0.20.0
 	golang.org/x/image v0.0.0-20200119044424-58c23975cae1 // indirect
 	golang.org/x/net v0.0.0-20201202161906-c7110b5ffcbb
-	golang.org/x/sync v0.0.0-20200930132711-30421366ff76 // indirect
-	golang.org/x/sys v0.0.0-20201022201747-fb209a7c41cd
-	golang.org/x/tools v0.0.0-20200608174601-1b747fd94509
+	golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c
+	golang.org/x/tools v0.0.0-20201124115921-2c860bdd6e78
 	google.golang.org/api v0.26.0
 	google.golang.org/grpc v1.29.1
 	google.golang.org/protobuf v1.24.0
@@ -98,6 +97,7 @@ require (
 	gopkg.in/jcmturner/goidentity.v3 v3.0.0 // indirect
 	gopkg.in/jcmturner/gokrb5.v7 v7.3.0 // indirect
 	gopkg.in/karlseguin/expect.v1 v1.0.1 // indirect
+	modernc.org/sqlite v1.10.7
 )

 // replace github.com/seaweedfs/fuse => /Users/chris/go/src/github.com/seaweedfs/fuse
@@ -350,6 +350,7 @@ github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/
 github.com/google/go-cmp v0.4.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
 github.com/google/go-cmp v0.5.0 h1:/QaMHBdZ26BB3SSst0Iwl10Epc+xhTquomWX0oZEB6w=
 github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
+github.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
 github.com/google/go-cmp v0.5.4 h1:L8R9j+yAqZuZjsqh/z+F1NCffTKKLShY6zXTItVIZ8M=
 github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
 github.com/google/go-replayers/grpcreplay v0.1.0 h1:eNb1y9rZFmY4ax45uEEECSa8fsxGRU+8Bil52ASAwic=
@@ -476,6 +477,8 @@ github.com/karlseguin/expect v1.0.2-0.20190806010014-778a5f0c6003 h1:vJ0Snvo+SLM
 github.com/karlseguin/expect v1.0.2-0.20190806010014-778a5f0c6003/go.mod h1:zNBxMY8P21owkeogJELCLeHIt+voOSduHYTFUbwRAV8=
 github.com/karrick/godirwalk v1.8.0/go.mod h1:H5KPZjojv4lE+QYImBI8xVtrBRgYrIVsaRPx4tDPEn4=
 github.com/karrick/godirwalk v1.10.3/go.mod h1:RoGL9dQei4vP9ilrpETWE8CLOZ1kiN0LhBygSwrAsHA=
+github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 h1:Z9n2FFNUXsshfwJMBgNA0RU6/i7WVaAegv3PtuIHPMs=
+github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51/go.mod h1:CzGEWj7cYgsdH8dAjBGEr58BoE7ScuLd+fwFZ44+/x8=
 github.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvWXihfKN4Q=
 github.com/kisielk/errcheck v1.2.0/go.mod h1:/BMXB+zMLi60iA8Vv6Ksmxu/1UDYcXs4uQLJ+jE2L00=
 github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
@@ -533,11 +536,14 @@ github.com/mattn/go-ieproxy v0.0.1/go.mod h1:pYabZ6IHcRpFh7vIaLfK7rdcWgFEb3SFJ6/
 github.com/mattn/go-isatty v0.0.3/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4=
 github.com/mattn/go-isatty v0.0.4/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4=
 github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
+github.com/mattn/go-isatty v0.0.12 h1:wuysRhFDzyxgEmMf5xjvJ2M9dZoWAXNNr5LSBS7uHXY=
+github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU=
 github.com/mattn/go-runewidth v0.0.2/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU=
 github.com/mattn/go-runewidth v0.0.3 h1:a+kO+98RDGEfo6asOGMmpodZq4FNtnGP54yps8BzLR4=
 github.com/mattn/go-runewidth v0.0.3/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU=
 github.com/mattn/go-runewidth v0.0.4 h1:2BvfKmzob6Bmd4YsL0zygOqfdFnK7GR4QL06Do4/p7Y=
 github.com/mattn/go-runewidth v0.0.4/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU=
+github.com/mattn/go-sqlite3 v1.14.6/go.mod h1:NyWgC/yNuGj7Q9rpYnZvas74GogHl5/Z4A/KQRfk6bU=
 github.com/matttproud/golang_protobuf_extensions v1.0.1 h1:4hp9jkHxhMHkqkrB3Ix0jegS5sx/RkqARlsWZ6pIwiU=
 github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
 github.com/miekg/dns v1.0.14/go.mod h1:W1PPwlIAgtquWBMBEV9nkV9Cazfe8ScdGz/Lj7v3Nrg=
@@ -639,6 +645,8 @@ github.com/pkg/sftp v1.10.1/go.mod h1:lYOWFsE0bwd1+KfKJaKeuokY15vzFx25BLbzYYoAxZ
 github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
 github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
 github.com/posener/complete v1.1.1/go.mod h1:em0nMJCgc9GFtwrmVmEMR/ZL6WyhyjMBndrE9hABlRI=
+github.com/pquerna/cachecontrol v0.1.0 h1:yJMy84ti9h/+OEWa752kBTKv4XC30OtVVHYv/8cTqKc=
+github.com/pquerna/cachecontrol v0.1.0/go.mod h1:NrUG3Z7Rdu85UNR3vm7SOsl1nFIeSiQnrHV5K9mBcUI=
 github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=
 github.com/prometheus/client_golang v0.9.3-0.20190127221311-3c4408c8b829/go.mod h1:p2iRAGwDERtqlqzRXnrOVns+ignqQo//hLXqYxZYVNs=
 github.com/prometheus/client_golang v0.9.3/go.mod h1:/TN21ttK/J9q6uSwhBd54HahCDft0ttaMvbicHlPoso=
@@ -701,6 +709,8 @@ github.com/seaweedfs/fuse v1.1.5 h1:wyuRh/mDvrvt8ZLDS7YdPSe6nczniSx4sQFs/Jonveo=
 github.com/seaweedfs/fuse v1.1.5/go.mod h1:+PP6WlkrRUG6KPE+Th2EX5To/PjHaFsvqg/UgQ39aj8=
 github.com/seaweedfs/fuse v1.1.6 h1:kvCqaIsCEaYOBw5r8kJPUs9GcbwlIKcScnkPLT7HLuQ=
 github.com/seaweedfs/fuse v1.1.6/go.mod h1:+PP6WlkrRUG6KPE+Th2EX5To/PjHaFsvqg/UgQ39aj8=
+github.com/seaweedfs/fuse v1.1.7 h1:T4L5c/Sn+q8lE+0zCmH2MWvIO+B5TttWOSqK5KQPRMQ=
+github.com/seaweedfs/fuse v1.1.7/go.mod h1:+PP6WlkrRUG6KPE+Th2EX5To/PjHaFsvqg/UgQ39aj8=
 github.com/seaweedfs/goexif v1.0.2 h1:p+rTXYdQ2mgxd+1JaTrQ9N8DvYuw9UH9xgYmJ+Bb29E=
 github.com/seaweedfs/goexif v1.0.2/go.mod h1:MrKs5LK0HXdffrdCZrW3OIMegL2xXpC6ThLyXMyjdrk=
 github.com/secsy/goftp v0.0.0-20190720192957-f31499d7c79a h1:C6IhVTxNkhlb0tlCB6JfHOUv1f0xHPK7V8X4HlJZEJw=
@@ -810,6 +820,7 @@ github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2/go.mod h1:UETIi67q
 github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77/go.mod h1:aYKd//L2LvnjZzWKhF00oedf4jCCReLcmhLdhm1A27Q=
 github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
 github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
+github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
 go.etcd.io/bbolt v1.3.2 h1:Z/90sZLPOeCy2PwprqkFa25PdkusRzaj9P8zm/KNyvk=
 go.etcd.io/bbolt v1.3.2/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU=
 go.etcd.io/bbolt v1.3.4 h1:hi1bXHMVrlQh6WwxAy+qZCV/SYIlqo+Ushwdpa4tAKg=
@@ -954,6 +965,7 @@ golang.org/x/net v0.0.0-20200520004742-59133d7f0dd7/go.mod h1:qpuaurCH72eLCgpAm/
 golang.org/x/net v0.0.0-20200520182314-0ba52f642ac2/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
 golang.org/x/net v0.0.0-20200602114024-627f9648deb9/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
 golang.org/x/net v0.0.0-20201016165138-7b1cca2348c0/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
+golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
 golang.org/x/net v0.0.0-20201202161906-c7110b5ffcbb h1:eBmm0M9fYhWpKZLjQUUKka/LtIxf46G4fxeEz5KJr9U=
 golang.org/x/net v0.0.0-20201202161906-c7110b5ffcbb/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
 golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
@@ -975,6 +987,8 @@ golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJ
 golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20200930132711-30421366ff76 h1:JnxiSYT3Nm0BT2a8CyvYyM6cnrWpidecD1UuSYbhKm0=
 golang.org/x/sync v0.0.0-20200930132711-30421366ff76/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9 h1:SQFwaSi55rU7vdNs9Yr0Z324VNlrF+0wMqRXT4St8ck=
+golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
@@ -1007,6 +1021,7 @@ golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7w
 golang.org/x/sys v0.0.0-20191220142924-d4481acd189f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200113162924-86b910548bc1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200122134326-e047566fdf82/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5 h1:LfCXLvNmTYH9kEmVgqbnsWfruoXZIrh4YBgqVHtDvw0=
 golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
@@ -1026,6 +1041,9 @@ golang.org/x/sys v0.0.0-20200602225109-6fdc65e7d980/go.mod h1:h1NjWce9XRLGQEsW7w
 golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20201022201747-fb209a7c41cd h1:WgqgiQvkiZWz7XLhphjt2GI2GcGCTIZs9jqXMWmH+oc=
 golang.org/x/sys v0.0.0-20201022201747-fb209a7c41cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20201126233918-771906719818/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c h1:VwygUrnw9jn88c4u8GD3rZQbqrP/tgas88tPUbBxQrk=
+golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
 golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
 golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
@@ -1095,12 +1113,16 @@ golang.org/x/tools v0.0.0-20200601175630-2caf76543d99/go.mod h1:EkVYQZoAsY45+roY
 golang.org/x/tools v0.0.0-20200606014950-c42cb6316fb6/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
 golang.org/x/tools v0.0.0-20200608174601-1b747fd94509 h1:MI14dOfl3OG6Zd32w3ugsrvcUO810fDZdWakTq39dH4=
 golang.org/x/tools v0.0.0-20200608174601-1b747fd94509/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
+golang.org/x/tools v0.0.0-20201124115921-2c860bdd6e78 h1:M8tBwCtWD/cZV9DZpFYRUgaymAYAr+aIUTWzDaM3uPs=
+golang.org/x/tools v0.0.0-20201124115921-2c860bdd6e78/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
 golang.org/x/xerrors v0.0.0-20190513163551-3ee3066db522/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7 h1:9zdDQZ7Thm29KFXgAX/+yaf3eVbP7djjWp/dXAppNCc=
 golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 h1:E7g+9GITq07hpfrRu66IVDexMakfv52eLZ2CXBWiKr4=
 golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 h1:go1bK/D/BFZV2I8cIQd1NKEZ+0owSTG1fDTci4IqFcE=
+golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
 google.golang.org/api v0.3.1/go.mod h1:6wY9I6uQWHQ8EM57III9mq/AjF+i8G65rmVagqKMtkk=
 google.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE=
 google.golang.org/api v0.5.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE=
@@ -1252,10 +1274,33 @@ honnef.co/go/tools v0.0.1-2020.1.4 h1:UoveltGrhghAA7ePc+e+QYDHXrBps2PqFZiHkGR/xK
 honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
 modernc.org/b v1.0.0 h1:vpvqeyp17ddcQWF29Czawql4lDdABCDRbXRAS4+aF2o=
 modernc.org/b v1.0.0/go.mod h1:uZWcZfRj1BpYzfN9JTerzlNUnnPsV9O2ZA8JsRcubNg=
+modernc.org/cc/v3 v3.32.4/go.mod h1:0R6jl1aZlIl2avnYfbfHBS1QB6/f+16mihBObaBC878=
+modernc.org/cc/v3 v3.33.5 h1:gfsIOmcv80EelyQyOHn/Xhlzex8xunhQxWiJRMYmPrI=
+modernc.org/cc/v3 v3.33.5/go.mod h1:0R6jl1aZlIl2avnYfbfHBS1QB6/f+16mihBObaBC878=
+modernc.org/ccgo/v3 v3.9.2/go.mod h1:gnJpy6NIVqkETT+L5zPsQFj7L2kkhfPMzOghRNv/CFo=
+modernc.org/ccgo/v3 v3.9.4 h1:mt2+HyTZKxva27O6T4C9//0xiNQ/MornL3i8itM5cCs=
+modernc.org/ccgo/v3 v3.9.4/go.mod h1:19XAY9uOrYnDhOgfHwCABasBvK69jgC4I8+rizbk3Bc=
+modernc.org/httpfs v1.0.6/go.mod h1:7dosgurJGp0sPaRanU53W4xZYKh14wfzX420oZADeHM=
+modernc.org/libc v1.7.13-0.20210308123627-12f642a52bb8/go.mod h1:U1eq8YWr/Kc1RWCMFUWEdkTg8OTcfLw2kY8EDwl039w=
+modernc.org/libc v1.9.5 h1:zv111ldxmP7DJ5mOIqzRbza7ZDl3kh4ncKfASB2jIYY=
+modernc.org/libc v1.9.5/go.mod h1:U1eq8YWr/Kc1RWCMFUWEdkTg8OTcfLw2kY8EDwl039w=
 modernc.org/mathutil v1.1.1 h1:FeylZSVX8S+58VsyJlkEj2bcpdytmp9MmDKZkKx8OIE=
 modernc.org/mathutil v1.1.1/go.mod h1:mZW8CKdRPY1v87qxC/wUdX5O1qDzXMP5TH3wjfpga6E=
+modernc.org/mathutil v1.2.2 h1:+yFk8hBprV+4c0U9GjFtL+dV3N8hOJ8JCituQcMShFY=
+modernc.org/mathutil v1.2.2/go.mod h1:mZW8CKdRPY1v87qxC/wUdX5O1qDzXMP5TH3wjfpga6E=
+modernc.org/memory v1.0.4 h1:utMBrFcpnQDdNsmM6asmyH/FM9TqLPS7XF7otpJmrwM=
+modernc.org/memory v1.0.4/go.mod h1:nV2OApxradM3/OVbs2/0OsP6nPfakXpi50C7dcoHXlc=
+modernc.org/opt v0.1.1 h1:/0RX92k9vwVeDXj+Xn23DKp2VJubL7k8qNffND6qn3A=
+modernc.org/opt v0.1.1/go.mod h1:WdSiB5evDcignE70guQKxYUl14mgWtbClRi5wmkkTX0=
+modernc.org/sqlite v1.10.7 h1:B4ITfAx3HxSxOOKZqKhw4vnrhM+kTY1HoJf2L7PQBCQ=
+modernc.org/sqlite v1.10.7/go.mod h1:GXpJIZPNgRGqG0inyYDW18j9YpBpFUBn/weGI63hLLs=
 modernc.org/strutil v1.1.0 h1:+1/yCzZxY2pZwwrsbH+4T7BQMoLQ9QiBshRC9eicYsc=
 modernc.org/strutil v1.1.0/go.mod h1:lstksw84oURvj9y3tn8lGvRxyRC1S2+g5uuIzNfIOBs=
+modernc.org/tcl v1.5.2/go.mod h1:pmJYOLgpiys3oI4AeAafkcUfE+TKKilminxNyU/+Zlo=
+modernc.org/token v1.0.0 h1:a0jaWiNMDhDUtqOj09wvjWWAqd3q7WpBulmL9H2egsk=
+modernc.org/token v1.0.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM=
+modernc.org/z v1.0.1-0.20210308123920-1f282aa71362/go.mod h1:8/SRk5C/HgiQWCgXdfpb+1RvhORdkz5sw72d3jjtyqA=
+modernc.org/z v1.0.1/go.mod h1:8/SRk5C/HgiQWCgXdfpb+1RvhORdkz5sw72d3jjtyqA=
 pack.ag/amqp v0.11.2/go.mod h1:4/cbmt4EJXSKlG6LCfWHoqmN0uFdy5i/+YFz+fTfhV4=
 rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8=
 rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0=
@@ -1,5 +1,5 @@
 apiVersion: v1
 description: SeaweedFS
 name: seaweedfs
-appVersion: "2.48"
-version: 2.48
+appVersion: "2.53"
+version: "2.53"
@@ -4,7 +4,7 @@ global:
  registry: ""
  repository: ""
  imageName: chrislusf/seaweedfs
-  # imageTag: "2.48" - started using {.Chart.appVersion}
+  # imageTag: "2.53" - started using {.Chart.appVersion}
  imagePullPolicy: IfNotPresent
  imagePullSecrets: imagepullsecret
  restartPolicy: Always
@@ -5,7 +5,7 @@

    <groupId>com.github.chrislusf</groupId>
    <artifactId>seaweedfs-client</artifactId>
-    <version>1.6.4</version>
+    <version>1.6.6</version>

    <parent>
        <groupId>org.sonatype.oss</groupId>
@@ -5,7 +5,7 @@

    <groupId>com.github.chrislusf</groupId>
    <artifactId>seaweedfs-client</artifactId>
-    <version>1.6.4</version>
+    <version>1.6.6</version>

    <parent>
        <groupId>org.sonatype.oss</groupId>
@@ -5,7 +5,7 @@

    <groupId>com.github.chrislusf</groupId>
    <artifactId>seaweedfs-client</artifactId>
-    <version>1.6.4</version>
+    <version>1.6.6</version>

    <parent>
        <groupId>org.sonatype.oss</groupId>
@@ -39,7 +39,7 @@ public class SeaweedOutputStream extends OutputStream {
    }

    public SeaweedOutputStream(FilerClient filerClient, final String fullpath, final String replication) {
-        this(filerClient, fullpath, null, 0, 8 * 1024 * 1024, "000");
+        this(filerClient, fullpath, null, 0, 8 * 1024 * 1024, replication);
    }

    public SeaweedOutputStream(FilerClient filerClient, final String path, FilerProto.Entry.Builder entry,
@@ -1,5 +1,6 @@
 package seaweedfs.client;

+import com.google.common.base.Strings;
 import com.google.protobuf.ByteString;
 import org.apache.http.client.methods.CloseableHttpResponse;
 import org.apache.http.client.methods.HttpPost;
@@ -45,11 +46,16 @@ public class SeaweedWrite {
        FilerProto.AssignVolumeResponse response = filerClient.getBlockingStub().assignVolume(
                FilerProto.AssignVolumeRequest.newBuilder()
                        .setCollection(filerClient.getCollection())
-                        .setReplication(replication == null ? filerClient.getReplication() : replication)
+                        .setReplication(Strings.isNullOrEmpty(replication) ? filerClient.getReplication() : replication)
                        .setDataCenter("")
                        .setTtlSec(0)
                        .setPath(path)
                        .build());
+
+        if (!Strings.isNullOrEmpty(response.getError())) {
+            throw new IOException(response.getError());
+        }
+
        String fileId = response.getFileId();
        String auth = response.getAuth();

@@ -361,6 +361,7 @@ message FilerConf {
        string disk_type = 5;
        bool fsync = 6;
        uint32 volume_growth_count = 7;
+        bool read_only = 8;
    }
    repeated PathConf locations = 2;
 }
@@ -11,13 +11,13 @@
        <dependency>
            <groupId>com.github.chrislusf</groupId>
            <artifactId>seaweedfs-client</artifactId>
-            <version>1.6.4</version>
+            <version>1.6.6</version>
            <scope>compile</scope>
        </dependency>
        <dependency>
            <groupId>com.github.chrislusf</groupId>
            <artifactId>seaweedfs-hadoop2-client</artifactId>
-            <version>1.6.4</version>
+            <version>1.6.6</version>
            <scope>compile</scope>
        </dependency>
        <dependency>
@@ -301,7 +301,7 @@
    </snapshotRepository>
  </distributionManagement>
  <properties>
-    <seaweedfs.client.version>1.6.4</seaweedfs.client.version>
+    <seaweedfs.client.version>1.6.6</seaweedfs.client.version>
    <hadoop.version>2.9.2</hadoop.version>
  </properties>
 </project>
@@ -5,7 +5,7 @@
    <modelVersion>4.0.0</modelVersion>

    <properties>
-        <seaweedfs.client.version>1.6.4</seaweedfs.client.version>
+        <seaweedfs.client.version>1.6.6</seaweedfs.client.version>
        <hadoop.version>2.9.2</hadoop.version>
    </properties>

@@ -309,7 +309,7 @@
    </snapshotRepository>
  </distributionManagement>
  <properties>
-    <seaweedfs.client.version>1.6.4</seaweedfs.client.version>
+    <seaweedfs.client.version>1.6.6</seaweedfs.client.version>
    <hadoop.version>3.1.1</hadoop.version>
  </properties>
 </project>
@@ -5,7 +5,7 @@
    <modelVersion>4.0.0</modelVersion>

    <properties>
-        <seaweedfs.client.version>1.6.4</seaweedfs.client.version>
+        <seaweedfs.client.version>1.6.6</seaweedfs.client.version>
        <hadoop.version>3.1.1</hadoop.version>
    </properties>

@@ -1,4 +1,4 @@
-package load_test_leveldb
+package main

 import (
 	"crypto/md5"
@@ -0,0 +1,108 @@
+package main
+
+import (
+	"context"
+	"flag"
+	"fmt"
+	"github.com/chrislusf/seaweedfs/weed/pb"
+	"github.com/chrislusf/seaweedfs/weed/pb/filer_pb"
+	"google.golang.org/grpc"
+	"io"
+	"strconv"
+)
+
+var (
+	dir       = flag.String("dir", "/tmp", "directory to create files")
+	n         = flag.Int("n", 100, "the number of metadata")
+	tailFiler = flag.String("filer", "localhost:8888", "the filer address")
+	isWrite = flag.Bool("write", false, "only write")
+)
+
+func main() {
+
+	flag.Parse()
+
+	if *isWrite {
+		startGenerateMetadata()
+		return
+	}
+
+	expected := 0
+	startSubscribeMetadata(func(event *filer_pb.SubscribeMetadataResponse) error {
+		if event.Directory != *dir {
+			return nil
+		}
+		name := event.EventNotification.NewEntry.Name
+		fmt.Printf("=> %s\n", name)
+		id := name[4:]
+		if x, err := strconv.Atoi(id); err == nil {
+			if x != expected {
+				return fmt.Errorf("Expected file%d Actual %s\n", expected, name)
+			}
+			expected++
+		} else {
+			return err
+		}
+		return nil
+	})
+
+}
+
+func startGenerateMetadata() {
+	pb.WithFilerClient(*tailFiler, grpc.WithInsecure(), func(client filer_pb.SeaweedFilerClient) error {
+
+		for i := 0; i < *n; i++ {
+			name := fmt.Sprintf("file%d", i)
+			if err := filer_pb.CreateEntry(client, &filer_pb.CreateEntryRequest{
+				Directory: *dir,
+				Entry: &filer_pb.Entry{
+					Name: name,
+				},
+			}); err != nil {
+				fmt.Printf("create entry %s: %v\n", name, err)
+				return err
+			}
+		}
+
+		return nil
+
+	})
+}
+
+func startSubscribeMetadata(eachEntryFunc func(event *filer_pb.SubscribeMetadataResponse) error) {
+
+	lastTsNs := int64(0)
+
+	tailErr := pb.WithFilerClient(*tailFiler, grpc.WithInsecure(), func(client filer_pb.SeaweedFilerClient) error {
+
+		ctx, cancel := context.WithCancel(context.Background())
+		defer cancel()
+
+		stream, err := client.SubscribeMetadata(ctx, &filer_pb.SubscribeMetadataRequest{
+			ClientName: "tail",
+			PathPrefix: *dir,
+			SinceNs:    lastTsNs,
+		})
+		if err != nil {
+			return fmt.Errorf("listen: %v", err)
+		}
+
+		for {
+			resp, listenErr := stream.Recv()
+			if listenErr == io.EOF {
+				return nil
+			}
+			if listenErr != nil {
+				return listenErr
+			}
+			if err = eachEntryFunc(resp); err != nil {
+				return err
+			}
+			lastTsNs = resp.TsNs
+		}
+
+	})
+	if tailErr != nil {
+		fmt.Printf("tail %s: %v\n", *tailFiler, tailErr)
+	}
+}
@@ -22,6 +22,7 @@ var Commands = []*Command{
 	cmdFilerReplicate,
 	cmdFilerSynchronize,
 	cmdFix,
+	cmdFuse,
 	cmdGateway,
 	cmdMaster,
 	cmdMount,
@@ -213,11 +213,15 @@ func genFileCopyTask(fileOrDir string, destPath string, fileCopyTaskChan chan Fi

 	mode := fi.Mode()
 	uid, gid := util.GetFileUidGid(fi)
+	fileSize := fi.Size()
+	if mode.IsDir() {
+		fileSize = 0
+	}

 	fileCopyTaskChan <- FileCopyTask{
 		sourceLocation:     fileOrDir,
 		destinationUrlPath: destPath,
-		fileSize:           fi.Size(),
+		fileSize:           fileSize,
 		fileMode:           fi.Mode(),
 		uid:                uid,
 		gid:                gid,
@@ -359,16 +359,19 @@ func genProcessFunction(sourcePath string, targetPath string, dataSink sink.Repl
 	return processEventFn
 }

-func buildKey(dataSink sink.ReplicationSink, message *filer_pb.EventNotification, targetPath string, sourceKey util.FullPath, sourcePath string) string {
+func buildKey(dataSink sink.ReplicationSink, message *filer_pb.EventNotification, targetPath string, sourceKey util.FullPath, sourcePath string) (key string) {
 	if !dataSink.IsIncremental() {
-		return util.Join(targetPath, string(sourceKey)[len(sourcePath):])
+		key = util.Join(targetPath, string(sourceKey)[len(sourcePath):])
+	} else {
+		var mTime int64
+		if message.NewEntry != nil {
+			mTime = message.NewEntry.Attributes.Mtime
+		} else if message.OldEntry != nil {
+			mTime = message.OldEntry.Attributes.Mtime
+		}
+		dateKey := time.Unix(mTime, 0).Format("2006-01-02")
+		key = util.Join(targetPath, dateKey, string(sourceKey)[len(sourcePath):])
 	}
-	var mTime int64
-	if message.NewEntry != nil {
-		mTime = message.NewEntry.Attributes.Mtime
-	} else if message.OldEntry != nil {
-		mTime = message.OldEntry.Attributes.Mtime
-	}
-	dateKey := time.Unix(mTime, 0).Format("2006-01-02")
-	return util.Join(targetPath, dateKey, string(sourceKey)[len(sourcePath):])
+
+	return escapeKey(key)
 }
@@ -0,0 +1,7 @@
+// +build !windows
+
+package command
+
+func escapeKey(key string) string {
+	return key
+}
@@ -0,0 +1,12 @@
+package command
+
+import (
+	"strings"
+)
+
+func escapeKey(key string) string {
+	if strings.Contains(key, ":") {
+		return strings.ReplaceAll(key, ":", "")
+	}
+	return key
+}
@@ -0,0 +1,222 @@
+package command
+
+import (
+	"fmt"
+	"strings"
+	"strconv"
+	"time"
+	"os"
+)
+
+func init() {
+	cmdFuse.Run = runFuse // break init cycle
+}
+
+type parameter struct {
+	name string
+	value string
+}
+
+func runFuse(cmd *Command, args []string) bool {
+	rawArgs := strings.Join(args, " ")
+	rawArgsLen := len(rawArgs)
+	option := strings.Builder{}
+	options := []parameter{}
+
+	// first parameter
+	i := 0
+	for i = 0; i < rawArgsLen && rawArgs[i] != ' '; i++ {
+		option.WriteByte(rawArgs[i])
+	}
+	options = append(options, parameter{"arg0", option.String()})
+	option.Reset()
+
+	for i++; i < rawArgsLen; i++ {
+
+		// space separator check for filled option
+		if rawArgs[i] == ' ' {
+			if option.Len() > 0 {
+				options = append(options, parameter{option.String(), "true"})
+				option.Reset()
+			}
+
+		// dash separator read option until next space
+		} else if rawArgs[i] == '-' {
+			for i++; i < rawArgsLen && rawArgs[i] != ' '; i++ {
+				option.WriteByte(rawArgs[i])
+			}
+			options = append(options, parameter{option.String(), "true"})
+			option.Reset()
+
+		// equal separator start option with pending value
+		} else if rawArgs[i] == '=' {
+			name := option.String()
+			option.Reset()
+
+			for i++; i < rawArgsLen && rawArgs[i] != ','; i++ {
+				// double quote separator read option until next double quote
+				if rawArgs[i] == '"' {
+					for i++; i < rawArgsLen && rawArgs[i] != '"'; i++ {
+						option.WriteByte(rawArgs[i])
+					}
+
+				// single quote separator read option until next single quote
+				} else if rawArgs[i] == '\'' {
+					for i++; i < rawArgsLen && rawArgs[i] != '\''; i++ {
+						option.WriteByte(rawArgs[i])
+					}
+
+				// add chars before comma
+				} else if rawArgs[i] != ' ' {
+					option.WriteByte(rawArgs[i])
+				}
+			}
+
+			options = append(options, parameter{name, option.String()})
+			option.Reset()
+
+		// comma separator just read current option
+		} else if rawArgs[i] == ',' {
+			options = append(options, parameter{option.String(), "true"})
+			option.Reset()
+
+		// what is not a separator fill option buffer
+		} else {
+			option.WriteByte(rawArgs[i])
+		}
+	}
+
+	// get residual option data
+	if option.Len() > 0 {
+		// add value to pending option
+		options = append(options, parameter{option.String(), "true"})
+		option.Reset()
+	}
+
+	// scan each parameter
+	for i := 0; i < len(options); i++ {
+		parameter := options[i]
+
+		switch parameter.name  {
+		case "arg0":
+			mountOptions.dir = &parameter.value
+		case "filer":
+			mountOptions.filer = &parameter.value
+		case "filer.path":
+			mountOptions.filerMountRootPath = &parameter.value
+		case "dirAutoCreate":
+			if parsed, err := strconv.ParseBool(parameter.value); err != nil {
+				mountOptions.dirAutoCreate = &parsed
+			} else {
+				panic(fmt.Errorf("dirAutoCreate: %s", err))
+			}
+		case "collection":
+			mountOptions.collection = &parameter.value
+		case "replication":
+			mountOptions.replication = &parameter.value
+		case "disk":
+			mountOptions.diskType = &parameter.value
+		case "ttl":
+			if parsed, err := strconv.ParseInt(parameter.value, 0, 32); err != nil {
+				intValue := int(parsed)
+				mountOptions.ttlSec = &intValue
+			} else {
+				panic(fmt.Errorf("ttl: %s", err))
+			}
+		case "chunkSizeLimitMB":
+			if parsed, err := strconv.ParseInt(parameter.value, 0, 32); err != nil {
+				intValue := int(parsed)
+				mountOptions.chunkSizeLimitMB = &intValue
+			} else {
+				panic(fmt.Errorf("chunkSizeLimitMB: %s", err))
+			}
+		case "concurrentWriters":
+			i++
+			if parsed, err := strconv.ParseInt(parameter.value, 0, 32); err != nil {
+				intValue := int(parsed)
+				mountOptions.concurrentWriters = &intValue
+			} else {
+				panic(fmt.Errorf("concurrentWriters: %s", err))
+			}
+		case "cacheDir":
+			mountOptions.cacheDir = &parameter.value
+		case "cacheCapacityMB":
+			if parsed, err := strconv.ParseInt(parameter.value, 0, 64); err != nil {
+				mountOptions.cacheSizeMB = &parsed
+			} else {
+				panic(fmt.Errorf("cacheCapacityMB: %s", err))
+			}
+		case "dataCenter":
+			mountOptions.dataCenter = &parameter.value
+		case "allowOthers":
+			if parsed, err := strconv.ParseBool(parameter.value); err != nil {
+				mountOptions.allowOthers = &parsed
+			} else {
+				panic(fmt.Errorf("allowOthers: %s", err))
+			}
+		case "umask":
+			mountOptions.umaskString = &parameter.value
+		case "nonempty":
+			if parsed, err := strconv.ParseBool(parameter.value); err != nil {
+				mountOptions.nonempty = &parsed
+			} else {
+				panic(fmt.Errorf("nonempty: %s", err))
+			}
+		case "volumeServerAccess":
+			mountOptions.volumeServerAccess = &parameter.value
+		case "map.uid":
+			mountOptions.uidMap = &parameter.value
+		case "map.gid":
+			mountOptions.gidMap = &parameter.value
+		case "readOnly":
+			if parsed, err := strconv.ParseBool(parameter.value); err != nil {
+				mountOptions.readOnly = &parsed
+			} else {
+				panic(fmt.Errorf("readOnly: %s", err))
+			}
+		case "cpuprofile":
+			mountCpuProfile = &parameter.value
+		case "memprofile":
+			mountMemProfile = &parameter.value
+		case "readRetryTime":
+			if parsed, err := time.ParseDuration(parameter.value); err != nil {
+				mountReadRetryTime = &parsed
+			} else {
+				panic(fmt.Errorf("readRetryTime: %s", err))
+			}
+		}
+	}
+
+	// I don't know why PATH environment variable is lost
+	if err := os.Setenv("PATH", "/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin"); err != nil {
+		panic(fmt.Errorf("setenv: %s", err))
+	}
+
+	// just call "weed mount" command
+	return runMount(cmdMount, []string{})
+}
+
+var cmdFuse = &Command{
+	UsageLine: "fuse /mnt/mount/point -o \"filer=localhost:8888,filer.path=/\"",
+	Short: "Allow use weed with linux's mount command",
+	Long: `Allow use weed with linux's mount command
+
+  You can use -t weed on mount command:
+  mv weed /sbin/mount.weed
+  mount -t weed fuse /mnt -o "filer=localhost:8888,filer.path=/"
+
+  Or you can use -t fuse on mount command:
+  mv weed /sbin/weed
+  mount -t fuse.weed fuse /mnt -o "filer=localhost:8888,filer.path=/"
+  mount -t fuse "weed#fuse" /mnt -o "filer=localhost:8888,filer.path=/"
+
+  To use without mess with your /sbin:
+  mount -t fuse./home/user/bin/weed fuse /mnt -o "filer=localhost:8888,filer.path=/"
+  mount -t fuse "/home/user/bin/weed#fuse" /mnt -o "filer=localhost:8888,filer.path=/"
+
+  To pass more than one parameter use quotes, example:
+  mount -t weed fuse /mnt -o "filer='192.168.0.1:8888,192.168.0.2:8888',filer.path=/"
+
+  To check valid options look "weed mount --help"
+  `,
+}
@@ -37,7 +37,7 @@ var (

 func init() {
 	cmdMount.Run = runMount // break init cycle
-	mountOptions.filer = cmdMount.Flag.String("filer", "localhost:8888", "weed filer location")
+	mountOptions.filer = cmdMount.Flag.String("filer", "localhost:8888", "comma-separated weed filer location")
 	mountOptions.filerMountRootPath = cmdMount.Flag.String("filer.path", "/", "mount this remote path from filer server")
 	mountOptions.dir = cmdMount.Flag.String("dir", ".", "mount weed filer to this directory")
 	mountOptions.dirAutoCreate = cmdMount.Flag.Bool("dirAutoCreate", false, "auto create the directory to mount to")
@@ -5,15 +5,18 @@ package command
 import (
 	"context"
 	"fmt"
-	"github.com/chrislusf/seaweedfs/weed/storage/types"
 	"os"
 	"os/user"
 	"path"
+	"path/filepath"
 	"runtime"
 	"strconv"
 	"strings"
+	"syscall"
 	"time"

+	"github.com/chrislusf/seaweedfs/weed/storage/types"
+
 	"github.com/chrislusf/seaweedfs/weed/filesys/meta_cache"

 	"github.com/seaweedfs/fuse"
@@ -49,11 +52,26 @@ func runMount(cmd *Command, args []string) bool {
 	return RunMount(&mountOptions, os.FileMode(umask))
 }

+func getParentInode(mountDir string) (uint64, error) {
+	parentDir := filepath.Clean(filepath.Join(mountDir, ".."))
+	fi, err := os.Stat(parentDir)
+	if err != nil {
+		return 0, err
+	}
+
+	stat, ok := fi.Sys().(*syscall.Stat_t)
+	if !ok {
+		return 0, nil
+	}
+
+	return stat.Ino, nil
+}
+
 func RunMount(option *MountOptions, umask os.FileMode) bool {

-	filer := *option.filer
+	filers := strings.Split(*option.filer, ",")
 	// parse filer grpc address
-	filerGrpcAddress, err := pb.ParseServerToGrpcAddress(filer)
+	filerGrpcAddresses, err := pb.ParseServersToGrpcAddresses(filers)
 	if err != nil {
 		glog.V(0).Infof("ParseFilerGrpcAddress: %v", err)
 		return true
@@ -64,34 +82,40 @@ func RunMount(option *MountOptions, umask os.FileMode) bool {
 	grpcDialOption := security.LoadClientTLS(util.GetViper(), "grpc.client")
 	var cipher bool
 	for i := 0; i < 10; i++ {
-		err = pb.WithGrpcFilerClient(filerGrpcAddress, grpcDialOption, func(client filer_pb.SeaweedFilerClient) error {
+		err = pb.WithOneOfGrpcFilerClients(filerGrpcAddresses, grpcDialOption, func(client filer_pb.SeaweedFilerClient) error {
 			resp, err := client.GetFilerConfiguration(context.Background(), &filer_pb.GetFilerConfigurationRequest{})
 			if err != nil {
-				return fmt.Errorf("get filer grpc address %s configuration: %v", filerGrpcAddress, err)
+				return fmt.Errorf("get filer grpc address %v configuration: %v", filerGrpcAddresses, err)
 			}
 			cipher = resp.Cipher
 			return nil
 		})
 		if err != nil {
-			glog.V(0).Infof("failed to talk to filer %s: %v", filerGrpcAddress, err)
+			glog.V(0).Infof("failed to talk to filer %v: %v", filerGrpcAddresses, err)
 			glog.V(0).Infof("wait for %d seconds ...", i+1)
 			time.Sleep(time.Duration(i+1) * time.Second)
 		}
 	}
 	if err != nil {
-		glog.Errorf("failed to talk to filer %s: %v", filerGrpcAddress, err)
+		glog.Errorf("failed to talk to filer %v: %v", filerGrpcAddresses, err)
 		return true
 	}

 	filerMountRootPath := *option.filerMountRootPath
 	dir := util.ResolvePath(*option.dir)
-	chunkSizeLimitMB := *mountOptions.chunkSizeLimitMB
+	parentInode, err := getParentInode(dir)
+	if err != nil {
+		glog.Errorf("failed to retrieve inode for parent directory of %s: %v", dir, err)
+		return true
+	}

 	fmt.Printf("This is SeaweedFS version %s %s %s\n", util.Version(), runtime.GOOS, runtime.GOARCH)
 	if dir == "" {
 		fmt.Printf("Please specify the mount directory via \"-dir\"")
 		return false
 	}
+
+	chunkSizeLimitMB := *mountOptions.chunkSizeLimitMB
 	if chunkSizeLimitMB <= 0 {
 		fmt.Printf("Please specify a reasonable buffer size.")
 		return false
@@ -145,7 +169,7 @@ func RunMount(option *MountOptions, umask os.FileMode) bool {

 	options := []fuse.MountOption{
 		fuse.VolumeName(mountName),
-		fuse.FSName(filer + ":" + filerMountRootPath),
+		fuse.FSName(*option.filer + ":" + filerMountRootPath),
 		fuse.Subtype("seaweedfs"),
 		// fuse.NoAppleDouble(), // include .DS_Store, otherwise can not delete non-empty folders
 		fuse.NoAppleXattr(),
@@ -181,8 +205,8 @@ func RunMount(option *MountOptions, umask os.FileMode) bool {

 	seaweedFileSystem := filesys.NewSeaweedFileSystem(&filesys.Option{
 		MountDirectory:     dir,
-		FilerAddress:       filer,
-		FilerGrpcAddress:   filerGrpcAddress,
+		FilerAddresses:     filers,
+		FilerGrpcAddresses: filerGrpcAddresses,
 		GrpcDialOption:     grpcDialOption,
 		FilerMountRootPath: mountRoot,
 		Collection:         *option.collection,
@@ -199,6 +223,7 @@ func RunMount(option *MountOptions, umask os.FileMode) bool {
 		MountMode:          mountMode,
 		MountCtime:         fileInfo.ModTime(),
 		MountMtime:         time.Now(),
+		MountParentInode:   parentInode,
 		Umask:              umask,
 		VolumeServerAccess: *mountOptions.volumeServerAccess,
 		Cipher:             cipher,
@@ -218,9 +243,10 @@ func RunMount(option *MountOptions, umask os.FileMode) bool {
 		c.Close()
 	})

-	glog.V(0).Infof("mounted %s%s to %s", filer, mountRoot, dir)
+	glog.V(0).Infof("mounted %s%s to %v", *option.filer, mountRoot, dir)
 	server := fs.New(c, nil)
 	seaweedFileSystem.Server = server
+	seaweedFileSystem.StartBackgroundTasks()
 	err = server.Serve(seaweedFileSystem)

 	// check if the mount process has an error to report
@@ -101,6 +101,11 @@ dir = "./filerldb3"					# directory to store level db files
 enabled = false
 dir = "./filerrdb"					# directory to store rocksdb files

+[sqlite]
+# local on disk, similar to leveldb
+enabled = false
+dbFile = "./filer.db"				# sqlite db file
+
 [mysql]  # or memsql, tidb
 # CREATE TABLE IF NOT EXISTS filemeta (
 #   dirhash     BIGINT               COMMENT 'first 64 bits of MD5 hash value of directory field',
@@ -351,13 +356,14 @@ sub_url = "rabbit://myqueue"

 	REPLICATION_TOML_EXAMPLE = `
 # A sample TOML config file for replicating SeaweedFS filer
-# Used with "weed filer.replicate"
+# Used with "weed filer.backup"
+# Using with "weed filer.replicate" is deprecated.
 # Put this file to one of the location, with descending priority
 #    ./replication.toml
 #    $HOME/.seaweedfs/replication.toml
 #    /etc/seaweedfs/replication.toml

-[source.filer]
+[source.filer]  # deprecated. Only useful with "weed filer.replicate"
 enabled = true
 grpcAddress = "localhost:18888"
 # all files under this directory tree are replicated.
@@ -372,12 +378,6 @@ directory = "/data"
 # so each date directory contains all new and updated files.
 is_incremental = false

-[sink.local_incremental]
-# all replicated files are under modified time as yyyy-mm-dd directories
-# so each date directory contains all new and updated files.
-enabled = false
-directory = "/backup"
-
 [sink.filer]
 enabled = false
 grpcAddress = "localhost:18888"
@@ -245,7 +245,7 @@ func runServer(cmd *Command, args []string) bool {

 	// start volume server
 	if *isStartingVolumeServer {
-		minFreeSpaces := util.MustParseMinFreeSpace(*minFreeSpace, *minFreeSpacePercent)
+		minFreeSpaces := util.MustParseMinFreeSpace(*volumeMinFreeSpace, *volumeMinFreeSpacePercent)
 		go serverOptions.v.startVolumeServer(*volumeDataFolders, *volumeMaxDataVolumeCounts, *serverWhiteListOption, minFreeSpaces)
 	}

@@ -84,7 +84,7 @@ func runUpload(cmd *Command, args []string) bool {
 		if *upload.dir == "" {
 			return false
 		}
-		filepath.Walk(util.ResolvePath(*upload.dir), func(path string, info os.FileInfo, err error) error {
+		err = filepath.Walk(util.ResolvePath(*upload.dir), func(path string, info os.FileInfo, err error) error {
 			if err == nil {
 				if !info.IsDir() {
 					if *upload.include != "" {
@@ -108,12 +108,21 @@ func runUpload(cmd *Command, args []string) bool {
 			}
 			return err
 		})
+		if err != nil {
+			fmt.Println(err.Error())
+			return false;
+		}
 	} else {
 		parts, e := operation.NewFileParts(args)
 		if e != nil {
 			fmt.Println(e.Error())
+			return false
+		}
+		results, err := operation.SubmitFiles(func() string { return *upload.master }, grpcDialOption, parts, *upload.replication, *upload.collection, *upload.dataCenter, *upload.ttl, *upload.diskType, *upload.maxMB, *upload.usePublicUrl)
+		if err != nil {
+			fmt.Println(err.Error())
+			return false
 		}
-		results, _ := operation.SubmitFiles(func() string { return *upload.master }, grpcDialOption, parts, *upload.replication, *upload.collection, *upload.dataCenter, *upload.ttl, *upload.diskType, *upload.maxMB, *upload.usePublicUrl)
 		bytes, _ := json.Marshal(results)
 		fmt.Println(string(bytes))
 	}
@@ -32,6 +32,9 @@ type AbstractSqlStore struct {
 	dbsLock            sync.Mutex
 }

+func (store *AbstractSqlStore) CanDropWholeBucket() bool {
+	return store.SupportBucketTable
+}
 func (store *AbstractSqlStore) OnBucketCreation(bucket string) {
 	store.dbsLock.Lock()
 	defer store.dbsLock.Unlock()
@@ -126,6 +126,9 @@ func mergePathConf(a, b *filer_pb.FilerConf_PathConf) {
 	if b.VolumeGrowthCount > 0 {
 		a.VolumeGrowthCount = b.VolumeGrowthCount
 	}
+	if b.ReadOnly {
+		a.ReadOnly = b.ReadOnly
+	}
 }

 func (fc *FilerConf) ToProto() *filer_pb.FilerConf {
@@ -24,6 +24,18 @@ func TestFilerConf(t *testing.T) {
 			LocationPrefix: "/buckets/",
 			Replication:    "001",
 		},
+		{
+			LocationPrefix: "/buckets",
+			ReadOnly:       false,
+		},
+		{
+			LocationPrefix: "/buckets/xxx",
+			ReadOnly:       true,
+		},
+		{
+			LocationPrefix: "/buckets/xxx/yyy",
+			ReadOnly:       false,
+		},
 	}}
 	fc.doLoadConf(conf)

@@ -31,4 +43,7 @@ func TestFilerConf(t *testing.T) {
 	assert.Equal(t, "abcd", fc.MatchStorageRule("/buckets/abcd/jasdf").Collection)
 	assert.Equal(t, "001", fc.MatchStorageRule("/buckets/abc/jasdf").Replication)

+	assert.Equal(t, true, fc.MatchStorageRule("/buckets/xxx/yyy/zzz").ReadOnly)
+	assert.Equal(t, false, fc.MatchStorageRule("/buckets/other").ReadOnly)
+
 }
@@ -71,7 +71,7 @@ func (f *Filer) doBatchDeleteFolderMetaAndData(ctx context.Context, entry *Entry

 	lastFileName := ""
 	includeLastFile := false
-	if !isDeletingBucket {
+	if !isDeletingBucket || !f.Store.CanDropWholeBucket() {
 		for {
 			entries, _, err := f.ListDirectoryEntries(ctx, entry.FullPath, lastFileName, includeLastFile, PaginationSize, "", "", "")
 			if err != nil {
@@ -43,4 +43,5 @@ type FilerStore interface {
 type BucketAware interface {
 	OnBucketCreation(bucket string)
 	OnBucketDeletion(bucket string)
+	CanDropWholeBucket() bool
 }
@@ -23,6 +23,7 @@ type VirtualFilerStore interface {
 	AddPathSpecificStore(path string, storeId string, store FilerStore)
 	OnBucketCreation(bucket string)
 	OnBucketDeletion(bucket string)
+	CanDropWholeBucket() bool
 }

 type FilerStoreWrapper struct {
@@ -42,6 +43,13 @@ func NewFilerStoreWrapper(store FilerStore) *FilerStoreWrapper {
 	}
 }

+func (fsw *FilerStoreWrapper) CanDropWholeBucket() bool {
+	if ba, ok := fsw.defaultStore.(BucketAware); ok {
+		return ba.CanDropWholeBucket()
+	}
+	return false
+}
+
 func (fsw *FilerStoreWrapper) OnBucketCreation(bucket string) {
 	for _, store := range fsw.storeIdToStore {
 		if ba, ok := store.(BucketAware); ok {
@@ -71,6 +71,7 @@ func (ma *MetaAggregator) subscribeToOneFiler(f *Filer, self string, peer string

 	// when filer store is not shared by multiple filers
 	if peerSignature != f.Signature {
+		lastTsNs = 0
 		if prevTsNs, err := ma.readOffset(f, peer, peerSignature); err == nil {
 			lastTsNs = prevTsNs
 		}
@@ -7,6 +7,7 @@ import (
 	"github.com/chrislusf/seaweedfs/weed/glog"
 	"go.mongodb.org/mongo-driver/bson"
 	"go.mongodb.org/mongo-driver/mongo"
+	"go.mongodb.org/mongo-driver/mongo/options"
 )

 func (store *MongodbStore) KvPut(ctx context.Context, key []byte, value []byte) (err error) {
@@ -15,11 +16,11 @@ func (store *MongodbStore) KvPut(ctx context.Context, key []byte, value []byte)

 	c := store.connect.Database(store.database).Collection(store.collectionName)

-	_, err = c.InsertOne(ctx, Model{
-		Directory: dir,
-		Name:      name,
-		Meta:      value,
-	})
+	opts := options.Update().SetUpsert(true)
+	filter := bson.D{{"directory", dir}, {"name", name}}
+	update := bson.D{{"$set", bson.D{{"meta", value}}}}
+
+	_, err = c.UpdateOne(ctx, filter, update, opts)

 	if err != nil {
 		return fmt.Errorf("kv put: %v", err)
@@ -0,0 +1,78 @@
+// +build linux darwin windows
+
+// limited GOOS due to modernc.org/libc/unistd
+
+package sqlite
+
+import (
+	"context"
+	"database/sql"
+	"fmt"
+
+	"github.com/chrislusf/seaweedfs/weed/filer"
+	"github.com/chrislusf/seaweedfs/weed/filer/abstract_sql"
+	"github.com/chrislusf/seaweedfs/weed/filer/mysql"
+	"github.com/chrislusf/seaweedfs/weed/util"
+	_ "modernc.org/sqlite"
+)
+
+func init() {
+	filer.Stores = append(filer.Stores, &SqliteStore{})
+}
+
+type SqliteStore struct {
+	abstract_sql.AbstractSqlStore
+}
+
+func (store *SqliteStore) GetName() string {
+	return "sqlite"
+}
+
+func (store *SqliteStore) Initialize(configuration util.Configuration, prefix string) (err error) {
+	dbFile := configuration.GetString(prefix + "dbFile")
+	createTable := `CREATE TABLE IF NOT EXISTS "%s" (
+		dirhash BIGINT,
+		name VARCHAR(1000),
+		directory TEXT,
+		meta BLOB,
+		PRIMARY KEY (dirhash, name)
+	) WITHOUT ROWID;`
+	upsertQuery := `INSERT INTO "%s"(dirhash,name,directory,meta)VALUES(?,?,?,?)
+	ON CONFLICT(dirhash,name) DO UPDATE SET
+		directory=excluded.directory,
+		meta=excluded.meta;
+	`
+	return store.initialize(
+		dbFile,
+		createTable,
+		upsertQuery,
+	)
+}
+
+func (store *SqliteStore) initialize(dbFile, createTable, upsertQuery string) (err error) {
+
+	store.SupportBucketTable = true
+	store.SqlGenerator = &mysql.SqlGenMysql{
+		CreateTableSqlTemplate: createTable,
+		DropTableSqlTemplate:   "drop table `%s`",
+		UpsertQueryTemplate:    upsertQuery,
+	}
+
+	var dbErr error
+	store.DB, dbErr = sql.Open("sqlite", dbFile)
+	if dbErr != nil {
+		store.DB.Close()
+		store.DB = nil
+		return fmt.Errorf("can not connect to %s error:%v", dbFile, err)
+	}
+
+	if err = store.DB.Ping(); err != nil {
+		return fmt.Errorf("connect to %s error:%v", dbFile, err)
+	}
+
+	if err = store.CreateTable(context.Background(), abstract_sql.DEFAULT_TABLE); err != nil {
+		return fmt.Errorf("init table %s: %v", abstract_sql.DEFAULT_TABLE, err)
+	}
+
+	return nil
+}
@@ -0,0 +1,9 @@
+// +build !linux,!darwin,!windows
+
+// limited GOOS due to modernc.org/libc/unistd
+
+package sqlite
+
+func init() {
+	// filer.Stores = append(filer.Stores, &SqliteStore{})
+}
@@ -6,9 +6,11 @@ import (
 	"io"
 	"math"
 	"strings"
+	"time"

 	"github.com/chrislusf/seaweedfs/weed/glog"
 	"github.com/chrislusf/seaweedfs/weed/pb/filer_pb"
+	"github.com/chrislusf/seaweedfs/weed/stats"
 	"github.com/chrislusf/seaweedfs/weed/util"
 	"github.com/chrislusf/seaweedfs/weed/wdclient"
 )
@@ -36,16 +38,20 @@ func StreamContent(masterClient wdclient.HasLookupFileIdFunction, w io.Writer, c
 	for _, chunkView := range chunkViews {

 		urlStrings := fileId2Url[chunkView.FileId]
+		start := time.Now()
 		data, err := retriedFetchChunkData(urlStrings, chunkView.CipherKey, chunkView.IsGzipped, chunkView.IsFullChunk(), chunkView.Offset, int(chunkView.Size))
+		stats.FilerRequestHistogram.WithLabelValues("chunkDownload").Observe(time.Since(start).Seconds())
 		if err != nil {
-			glog.Errorf("read chunk: %v", err)
+			stats.FilerRequestCounter.WithLabelValues("chunkDownloadError").Inc()
 			return fmt.Errorf("read chunk: %v", err)
 		}
+
 		_, err = w.Write(data)
 		if err != nil {
-			glog.Errorf("write chunk: %v", err)
+			stats.FilerRequestCounter.WithLabelValues("chunkDownloadedError").Inc()
 			return fmt.Errorf("write chunk: %v", err)
 		}
+		stats.FilerRequestCounter.WithLabelValues("chunkDownload").Inc()
 	}

 	return nil
@@ -54,28 +54,27 @@ func (dir *Dir) Id() uint64 {

 func (dir *Dir) Attr(ctx context.Context, attr *fuse.Attr) error {

-	// https://github.com/bazil/fuse/issues/196
-	attr.Valid = time.Second
-
-	if dir.FullPath() == dir.wfs.option.FilerMountRootPath {
-		dir.setRootDirAttributes(attr)
-		glog.V(3).Infof("root dir Attr %s, attr: %+v", dir.FullPath(), attr)
-		return nil
-	}
-
 	entry, err := dir.maybeLoadEntry()
 	if err != nil {
-		glog.V(3).Infof("dir Attr %s,err: %+v", dir.FullPath(), err)
+		glog.V(3).Infof("dir Attr %s, err: %+v", dir.FullPath(), err)
 		return err
 	}

+	// https://github.com/bazil/fuse/issues/196
+	attr.Valid = time.Second
 	attr.Inode = dir.Id()
 	attr.Mode = os.FileMode(entry.Attributes.FileMode) | os.ModeDir
 	attr.Mtime = time.Unix(entry.Attributes.Mtime, 0)
 	attr.Crtime = time.Unix(entry.Attributes.Crtime, 0)
+	attr.Ctime = time.Unix(entry.Attributes.Crtime, 0)
+	attr.Atime = time.Unix(entry.Attributes.Mtime, 0)
 	attr.Gid = entry.Attributes.Gid
 	attr.Uid = entry.Attributes.Uid

+	if dir.FullPath() == dir.wfs.option.FilerMountRootPath {
+		attr.BlockSize = blockSize
+	}
+
 	glog.V(4).Infof("dir Attr %s, attr: %+v", dir.FullPath(), attr)

 	return nil
@@ -93,20 +92,6 @@ func (dir *Dir) Getxattr(ctx context.Context, req *fuse.GetxattrRequest, resp *f
 	return getxattr(entry, req, resp)
 }

-func (dir *Dir) setRootDirAttributes(attr *fuse.Attr) {
-	// attr.Inode = 1 // filer2.FullPath(dir.Path).AsInode()
-	attr.Valid = time.Second
-	attr.Inode = dir.Id()
-	attr.Uid = dir.wfs.option.MountUid
-	attr.Gid = dir.wfs.option.MountGid
-	attr.Mode = dir.wfs.option.MountMode
-	attr.Crtime = dir.wfs.option.MountCtime
-	attr.Ctime = dir.wfs.option.MountCtime
-	attr.Mtime = dir.wfs.option.MountMtime
-	attr.Atime = dir.wfs.option.MountMtime
-	attr.BlockSize = blockSize
-}
-
 func (dir *Dir) Fsync(ctx context.Context, req *fuse.FsyncRequest) error {
 	// fsync works at OS level
 	// write the file chunks to the filerGrpcAddress
@@ -375,6 +360,28 @@ func (dir *Dir) ReadDirAll(ctx context.Context) (ret []fuse.Dirent, err error) {
 		glog.Errorf("list meta cache: %v", listErr)
 		return nil, fuse.EIO
 	}
+
+	// create proper . and .. directories
+	ret = append(ret, fuse.Dirent{
+		Inode: dirPath.AsInode(),
+		Name:  ".",
+		Type:  fuse.DT_Dir,
+	})
+
+	// return the correct parent inode for the mount root
+	var inode uint64
+	if string(dirPath) == dir.wfs.option.FilerMountRootPath {
+		inode = dir.wfs.option.MountParentInode
+	} else {
+		inode = util.FullPath(dir.parent.FullPath()).AsInode()
+	}
+
+	ret = append(ret, fuse.Dirent{
+		Inode: inode,
+		Name:  "..",
+		Type:  fuse.DT_Dir,
+	})
+
 	return
 }

@@ -6,7 +6,6 @@ import (
 	"github.com/chrislusf/seaweedfs/weed/pb/filer_pb"
 	"io"
 	"os"
-	"path/filepath"
 	"sync"
 	"time"
 )
@@ -24,14 +23,6 @@ type TempFileDirtyPages struct {
 	replication      string
 }

-var (
-	tmpDir = filepath.Join(os.TempDir(), "sw")
-)
-
-func init() {
-	os.Mkdir(tmpDir, 0755)
-}
-
 func newTempFileDirtyPages(file *File, writeOnly bool) *TempFileDirtyPages {

 	tempFile := &TempFileDirtyPages{
@@ -49,7 +40,7 @@ func (pages *TempFileDirtyPages) AddPage(offset int64, data []byte) {
 	defer pages.pageAddLock.Unlock()

 	if pages.tf == nil {
-		tf, err := os.CreateTemp(tmpDir, "")
+		tf, err := os.CreateTemp(pages.f.wfs.option.getTempFilePageDir(), "")
 		if err != nil {
 			glog.Errorf("create temp file: %v", err)
 			pages.lastErr = err
@@ -115,16 +115,6 @@ func (file *File) Setattr(ctx context.Context, req *fuse.SetattrRequest, resp *f
 	if err != nil {
 		return err
 	}
-	if file.isOpen > 0 {
-		file.wfs.handlesLock.Lock()
-		fileHandle := file.wfs.handles[file.Id()]
-		file.wfs.handlesLock.Unlock()
-
-		if fileHandle != nil {
-			fileHandle.Lock()
-			defer fileHandle.Unlock()
-		}
-	}

 	if req.Valid.Size() {

@@ -207,6 +197,11 @@ func (file *File) Setxattr(ctx context.Context, req *fuse.SetxattrRequest) error
 	if err := setxattr(entry, req); err != nil {
 		return err
 	}
+	file.dirtyMetadata = true
+
+	if file.isOpen > 0 {
+		return nil
+	}

 	return file.saveEntry(entry)

@@ -224,6 +219,11 @@ func (file *File) Removexattr(ctx context.Context, req *fuse.RemovexattrRequest)
 	if err := removexattr(entry, req); err != nil {
 		return err
 	}
+	file.dirtyMetadata = true
+
+	if file.isOpen > 0 {
+		return nil
+	}

 	return file.saveEntry(entry)

@@ -351,6 +351,8 @@ func (file *File) saveEntry(entry *filer_pb.Entry) error {

 		file.wfs.metaCache.InsertEntry(context.Background(), filer.FromPbEntry(request.Directory, request.Entry))

+		file.dirtyMetadata = false
+
 		return nil
 	})
 }
@@ -40,9 +40,22 @@ func SubscribeMetaEvents(mc *MetaCache, selfSignature int32, client filer_pb.Fil
 			newEntry = filer.FromPbEntry(dir, message.NewEntry)
 		}
 		err := mc.AtomicUpdateEntryFromFiler(context.Background(), oldPath, newEntry)
-		if err == nil && message.OldEntry != nil && message.NewEntry != nil {
-			key := util.NewFullPath(dir, message.NewEntry.Name)
-			mc.invalidateFunc(key)
+		if err == nil {
+			if message.OldEntry != nil && message.NewEntry != nil {
+				if message.OldEntry.Name == message.NewEntry.Name {
+					// no need to invalidate
+				} else {
+					oldKey := util.NewFullPath(resp.Directory, message.OldEntry.Name)
+					mc.invalidateFunc(oldKey)
+					newKey := util.NewFullPath(dir, message.NewEntry.Name)
+					mc.invalidateFunc(newKey)
+				}
+			} else if message.OldEntry == nil && message.NewEntry != nil {
+				// no need to invaalidate
+			} else if message.OldEntry != nil && message.NewEntry == nil {
+				oldKey := util.NewFullPath(resp.Directory, message.OldEntry.Name)
+				mc.invalidateFunc(oldKey)
+			}
 		}

 		return err
@@ -3,14 +3,17 @@ package filesys
 import (
 	"context"
 	"fmt"
+	"math"
+	"math/rand"
+	"os"
+	"path"
+	"path/filepath"
+	"sync"
+	"time"
+
 	"github.com/chrislusf/seaweedfs/weed/filer"
 	"github.com/chrislusf/seaweedfs/weed/storage/types"
 	"github.com/chrislusf/seaweedfs/weed/wdclient"
-	"math"
-	"os"
-	"path"
-	"sync"
-	"time"

 	"google.golang.org/grpc"

@@ -28,8 +31,9 @@ import (

 type Option struct {
 	MountDirectory     string
-	FilerAddress       string
-	FilerGrpcAddress   string
+	FilerAddresses     []string
+	filerIndex         int
+	FilerGrpcAddresses []string
 	GrpcDialOption     grpc.DialOption
 	FilerMountRootPath string
 	Collection         string
@@ -43,15 +47,19 @@ type Option struct {
 	DataCenter         string
 	Umask              os.FileMode

-	MountUid   uint32
-	MountGid   uint32
-	MountMode  os.FileMode
-	MountCtime time.Time
-	MountMtime time.Time
+	MountUid         uint32
+	MountGid         uint32
+	MountMode        os.FileMode
+	MountCtime       time.Time
+	MountMtime       time.Time
+	MountParentInode uint64

 	VolumeServerAccess string // how to access volume servers
 	Cipher             bool   // whether encrypt data on volume server
 	UidGidMapper       *meta_cache.UidGidMapper
+
+	uniqueCacheDir         string
+	uniqueCacheTempPageDir string
 }

 var _ = fs.FS(&WFS{})
@@ -95,14 +103,13 @@ func NewSeaweedFileSystem(option *Option) *WFS {
 		},
 		signature: util.RandomInt32(),
 	}
-	cacheUniqueId := util.Md5String([]byte(option.MountDirectory + option.FilerGrpcAddress + option.FilerMountRootPath + util.Version()))[0:8]
-	cacheDir := path.Join(option.CacheDir, cacheUniqueId)
+	wfs.option.filerIndex = rand.Intn(len(option.FilerAddresses))
+	wfs.option.setupUniqueCacheDirectory()
 	if option.CacheSizeMB > 0 {
-		os.MkdirAll(cacheDir, os.FileMode(0777)&^option.Umask)
-		wfs.chunkCache = chunk_cache.NewTieredChunkCache(256, cacheDir, option.CacheSizeMB, 1024*1024)
+		wfs.chunkCache = chunk_cache.NewTieredChunkCache(256, option.getUniqueCacheDir(), option.CacheSizeMB, 1024*1024)
 	}

-	wfs.metaCache = meta_cache.NewMetaCache(path.Join(cacheDir, "meta"), util.FullPath(option.FilerMountRootPath), option.UidGidMapper, func(filePath util.FullPath) {
+	wfs.metaCache = meta_cache.NewMetaCache(path.Join(option.getUniqueCacheDir(), "meta"), util.FullPath(option.FilerMountRootPath), option.UidGidMapper, func(filePath util.FullPath) {

 		fsNode := NodeWithId(filePath.AsInode())
 		if err := wfs.Server.InvalidateNodeData(fsNode); err != nil {
@@ -118,8 +125,6 @@ func NewSeaweedFileSystem(option *Option) *WFS {
 			glog.V(4).Infof("InvalidateEntry %s : %v", filePath, err)
 		}
 	})
-	startTime := time.Now()
-	go meta_cache.SubscribeMetaEvents(wfs.metaCache, wfs.signature, wfs, wfs.option.FilerMountRootPath, startTime.UnixNano())
 	grace.OnInterrupt(func() {
 		wfs.metaCache.Shutdown()
 	})
@@ -134,6 +139,11 @@ func NewSeaweedFileSystem(option *Option) *WFS {
 	return wfs
 }

+func (wfs *WFS) StartBackgroundTasks() {
+	startTime := time.Now()
+	go meta_cache.SubscribeMetaEvents(wfs.metaCache, wfs.signature, wfs, wfs.option.FilerMountRootPath, startTime.UnixNano())
+}
+
 func (wfs *WFS) Root() (fs.Node, error) {
 	return wfs.root, nil
 }
@@ -259,11 +269,27 @@ func (wfs *WFS) mapPbIdFromLocalToFiler(entry *filer_pb.Entry) {
 func (wfs *WFS) LookupFn() wdclient.LookupFileIdFunctionType {
 	if wfs.option.VolumeServerAccess == "filerProxy" {
 		return func(fileId string) (targetUrls []string, err error) {
-			return []string{"http://" + wfs.option.FilerAddress + "/?proxyChunkId=" + fileId}, nil
+			return []string{"http://" + wfs.getCurrentFiler() + "/?proxyChunkId=" + fileId}, nil
 		}
 	}
 	return filer.LookupFn(wfs)
+}
+func (wfs *WFS) getCurrentFiler() string {
+	return wfs.option.FilerAddresses[wfs.option.filerIndex]
+}

+func (option *Option) setupUniqueCacheDirectory() {
+	cacheUniqueId := util.Md5String([]byte(option.MountDirectory + option.FilerGrpcAddresses[0] + option.FilerMountRootPath + util.Version()))[0:8]
+	option.uniqueCacheDir = path.Join(option.CacheDir, cacheUniqueId)
+	option.uniqueCacheTempPageDir = filepath.Join(option.uniqueCacheDir, "sw")
+	os.MkdirAll(option.uniqueCacheTempPageDir, os.FileMode(0777)&^option.Umask)
+}
+
+func (option *Option) getTempFilePageDir() string {
+	return option.uniqueCacheTempPageDir
+}
+func (option *Option) getUniqueCacheDir() string {
+	return option.uniqueCacheDir
 }

 type NodeWithId uint64
@@ -1,6 +1,7 @@
 package filesys

 import (
+	"github.com/chrislusf/seaweedfs/weed/glog"
 	"github.com/chrislusf/seaweedfs/weed/util"
 	"google.golang.org/grpc"

@@ -10,20 +11,36 @@ import (

 var _ = filer_pb.FilerClient(&WFS{})

-func (wfs *WFS) WithFilerClient(fn func(filer_pb.SeaweedFilerClient) error) error {
+func (wfs *WFS) WithFilerClient(fn func(filer_pb.SeaweedFilerClient) error) (err error) {

-	err := util.Retry("filer grpc "+wfs.option.FilerGrpcAddress, func() error {
-		return pb.WithCachedGrpcClient(func(grpcConnection *grpc.ClientConn) error {
-			client := filer_pb.NewSeaweedFilerClient(grpcConnection)
-			return fn(client)
-		}, wfs.option.FilerGrpcAddress, wfs.option.GrpcDialOption)
+	return util.Retry("filer grpc", func() error {
+
+		i := wfs.option.filerIndex
+		n := len(wfs.option.FilerGrpcAddresses)
+		for x := 0; x < n; x++ {
+
+			filerGrpcAddress := wfs.option.FilerGrpcAddresses[i]
+			err = pb.WithCachedGrpcClient(func(grpcConnection *grpc.ClientConn) error {
+				client := filer_pb.NewSeaweedFilerClient(grpcConnection)
+				return fn(client)
+			}, filerGrpcAddress, wfs.option.GrpcDialOption)
+
+			if err != nil {
+				glog.V(0).Infof("WithFilerClient %d %v: %v", x, filerGrpcAddress, err)
+			} else {
+				wfs.option.filerIndex = i
+				return nil
+			}
+
+			i++
+			if i >= n {
+				i = 0
+			}
+
+		}
+		return err
 	})

-	if err == nil {
-		return nil
-	}
-	return err
-
 }

 func (wfs *WFS) AdjustedUrl(location *filer_pb.Location) string {
@@ -56,7 +56,7 @@ func (wfs *WFS) saveDataAsChunk(fullPath util.FullPath, writeOnly bool) filer.Sa

 		fileUrl := fmt.Sprintf("http://%s/%s", host, fileId)
 		if wfs.option.VolumeServerAccess == "filerProxy" {
-			fileUrl = fmt.Sprintf("http://%s/?proxyChunkId=%s", wfs.option.FilerAddress, fileId)
+			fileUrl = fmt.Sprintf("http://%s/?proxyChunkId=%s", wfs.getCurrentFiler(), fileId)
 		}
 		uploadResult, err, data := operation.Upload(fileUrl, filename, wfs.option.Cipher, reader, false, "", nil, auth)
 		if err != nil {
@@ -113,6 +113,21 @@ func (wfs *WFS) maybeLoadEntry(dir, name string) (entry *filer_pb.Entry, err err
 	fullpath := util.NewFullPath(dir, name)
 	// glog.V(3).Infof("read entry cache miss %s", fullpath)

+	// return a valid entry for the mount root
+	if string(fullpath) == wfs.option.FilerMountRootPath {
+		return &filer_pb.Entry{
+			Name:        wfs.option.FilerMountRootPath,
+			IsDirectory: true,
+			Attributes: &filer_pb.FuseAttributes{
+				Mtime:    wfs.option.MountMtime.Unix(),
+				FileMode: uint32(wfs.option.MountMode),
+				Uid:      wfs.option.MountUid,
+				Gid:      wfs.option.MountGid,
+				Crtime:   wfs.option.MountCtime.Unix(),
+			},
+		}, nil
+	}
+
 	// read from async meta cache
 	meta_cache.EnsureVisited(wfs.metaCache, wfs, util.FullPath(dir))
 	cachedEntry, cacheErr := wfs.metaCache.FindEntry(context.Background(), fullpath)
@@ -1,55 +1,13 @@
 package iamapi

 import (
-	"bytes"
-	"encoding/xml"
 	"fmt"
-	"strconv"
-
-	"net/http"
-	"net/url"
-	"time"
-
+	"github.com/aws/aws-sdk-go/service/iam"
 	"github.com/chrislusf/seaweedfs/weed/glog"
 	"github.com/chrislusf/seaweedfs/weed/s3api/s3err"
-
-	"github.com/aws/aws-sdk-go/service/iam"
+	"net/http"
 )

-type mimeType string
-
-const (
-	mimeNone mimeType = ""
-	mimeXML  mimeType = "application/xml"
-)
-
-func setCommonHeaders(w http.ResponseWriter) {
-	w.Header().Set("x-amz-request-id", fmt.Sprintf("%d", time.Now().UnixNano()))
-	w.Header().Set("Accept-Ranges", "bytes")
-}
-
-// Encodes the response headers into XML format.
-func encodeResponse(response interface{}) []byte {
-	var bytesBuffer bytes.Buffer
-	bytesBuffer.WriteString(xml.Header)
-	e := xml.NewEncoder(&bytesBuffer)
-	e.Encode(response)
-	return bytesBuffer.Bytes()
-}
-
-// If none of the http routes match respond with MethodNotAllowed
-func notFoundHandler(w http.ResponseWriter, r *http.Request) {
-	glog.V(0).Infof("unsupported %s %s", r.Method, r.RequestURI)
-	writeErrorResponse(w, s3err.ErrMethodNotAllowed, r.URL)
-}
-
-func writeErrorResponse(w http.ResponseWriter, errorCode s3err.ErrorCode, reqURL *url.URL) {
-	apiError := s3err.GetAPIError(errorCode)
-	errorResponse := getRESTErrorResponse(apiError, reqURL.Path)
-	encodedErrorResponse := encodeResponse(errorResponse)
-	writeResponse(w, apiError.HTTPStatusCode, encodedErrorResponse, mimeXML)
-}
-
 func writeIamErrorResponse(w http.ResponseWriter, err error, object string, value string, msg error) {
 	errCode := err.Error()
 	errorResp := ErrorResponse{}
@@ -64,42 +22,10 @@ func writeIamErrorResponse(w http.ResponseWriter, err error, object string, valu
 	case iam.ErrCodeNoSuchEntityException:
 		msg := fmt.Sprintf("The %s with name %s cannot be found.", object, value)
 		errorResp.Error.Message = &msg
-		writeResponse(w, http.StatusNotFound, encodeResponse(errorResp), mimeXML)
+		s3err.WriteXMLResponse(w, http.StatusNotFound, errorResp)
 	case iam.ErrCodeServiceFailureException:
-		writeResponse(w, http.StatusInternalServerError, encodeResponse(errorResp), mimeXML)
+		s3err.WriteXMLResponse(w, http.StatusInternalServerError, errorResp)
 	default:
-		writeResponse(w, http.StatusInternalServerError, encodeResponse(errorResp), mimeXML)
+		s3err.WriteXMLResponse(w, http.StatusInternalServerError, errorResp)
 	}
 }
-
-func getRESTErrorResponse(err s3err.APIError, resource string) s3err.RESTErrorResponse {
-	return s3err.RESTErrorResponse{
-		Code:      err.Code,
-		Message:   err.Description,
-		Resource:  resource,
-		RequestID: fmt.Sprintf("%d", time.Now().UnixNano()),
-	}
-}
-
-func writeResponse(w http.ResponseWriter, statusCode int, response []byte, mType mimeType) {
-	setCommonHeaders(w)
-	if response != nil {
-		w.Header().Set("Content-Length", strconv.Itoa(len(response)))
-	}
-	if mType != mimeNone {
-		w.Header().Set("Content-Type", string(mType))
-	}
-	w.WriteHeader(statusCode)
-	if response != nil {
-		glog.V(4).Infof("status %d %s: %s", statusCode, mType, string(response))
-		_, err := w.Write(response)
-		if err != nil {
-			glog.V(0).Infof("write err: %v", err)
-		}
-		w.(http.Flusher).Flush()
-	}
-}
-
-func writeSuccessResponseXML(w http.ResponseWriter, response []byte) {
-	writeResponse(w, http.StatusOK, response, mimeXML)
-}
@@ -362,7 +362,7 @@ func (iama *IamApiServer) DeleteAccessKey(s3cfg *iam_pb.S3ApiConfiguration, valu

 func (iama *IamApiServer) DoActions(w http.ResponseWriter, r *http.Request) {
 	if err := r.ParseForm(); err != nil {
-		writeErrorResponse(w, s3err.ErrInvalidRequest, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrInvalidRequest, r)
 		return
 	}
 	values := r.PostForm
@@ -370,7 +370,7 @@ func (iama *IamApiServer) DoActions(w http.ResponseWriter, r *http.Request) {
 	s3cfgLock.RLock()
 	s3cfg := &iam_pb.S3ApiConfiguration{}
 	if err := iama.s3ApiConfig.GetS3ApiConfiguration(s3cfg); err != nil {
-		writeErrorResponse(w, s3err.ErrInternalError, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrInternalError, r)
 		return
 	}
 	s3cfgLock.RUnlock()
@@ -411,14 +411,14 @@ func (iama *IamApiServer) DoActions(w http.ResponseWriter, r *http.Request) {
 		response, err = iama.CreatePolicy(s3cfg, values)
 		if err != nil {
 			glog.Errorf("CreatePolicy:  %+v", err)
-			writeErrorResponse(w, s3err.ErrInvalidRequest, r.URL)
+			s3err.WriteErrorResponse(w, s3err.ErrInvalidRequest, r)
 			return
 		}
 	case "PutUserPolicy":
 		response, err = iama.PutUserPolicy(s3cfg, values)
 		if err != nil {
 			glog.Errorf("PutUserPolicy:  %+v", err)
-			writeErrorResponse(w, s3err.ErrInvalidRequest, r.URL)
+			s3err.WriteErrorResponse(w, s3err.ErrInvalidRequest, r)
 			return
 		}
 	case "GetUserPolicy":
@@ -437,7 +437,7 @@ func (iama *IamApiServer) DoActions(w http.ResponseWriter, r *http.Request) {
 		errorResponse := ErrorResponse{}
 		errorResponse.Error.Code = &errNotImplemented.Code
 		errorResponse.Error.Message = &errNotImplemented.Description
-		writeResponse(w, errNotImplemented.HTTPStatusCode, encodeResponse(errorResponse), mimeXML)
+		s3err.WriteXMLResponse(w, errNotImplemented.HTTPStatusCode, errorResponse)
 		return
 	}
 	if changed {
@@ -449,5 +449,5 @@ func (iama *IamApiServer) DoActions(w http.ResponseWriter, r *http.Request) {
 			return
 		}
 	}
-	writeSuccessResponseXML(w, encodeResponse(response))
+	s3err.WriteXMLResponse(w, http.StatusOK, response)
 }
@@ -12,6 +12,7 @@ import (
 	"github.com/chrislusf/seaweedfs/weed/pb/iam_pb"
 	"github.com/chrislusf/seaweedfs/weed/s3api"
 	. "github.com/chrislusf/seaweedfs/weed/s3api/s3_constants"
+	"github.com/chrislusf/seaweedfs/weed/s3api/s3err"
 	"github.com/chrislusf/seaweedfs/weed/wdclient"
 	"github.com/gorilla/mux"
 	"google.golang.org/grpc"
@@ -71,7 +72,7 @@ func (iama *IamApiServer) registerRouter(router *mux.Router) {
 	apiRouter.Methods("POST").Path("/").HandlerFunc(iama.iam.Auth(iama.DoActions, ACTION_ADMIN))
 	//
 	// NotFound
-	apiRouter.NotFoundHandler = http.HandlerFunc(notFoundHandler)
+	apiRouter.NotFoundHandler = http.HandlerFunc(s3err.NotFoundHandler)
 }

 func (iam IamS3ApiConfigure) GetS3ApiConfiguration(s3cfg *iam_pb.S3ApiConfiguration) (err error) {
@@ -100,7 +100,7 @@ func DeleteFilesWithLookupVolumeId(grpcDialOption grpc.DialOption, fileIds []str
 		go func(server string, fidList []string) {
 			defer wg.Done()

-			if deleteResults, deleteErr := DeleteFilesAtOneVolumeServer(server, grpcDialOption, fidList, true); deleteErr != nil {
+			if deleteResults, deleteErr := DeleteFilesAtOneVolumeServer(server, grpcDialOption, fidList, false); deleteErr != nil {
 				err = deleteErr
 			} else if deleteResults != nil {
 				resultChan <- deleteResults
@@ -18,7 +18,7 @@ type MockClient struct {
 }

 func (m *MockClient) Do(req *http.Request) (*http.Response, error) {
-	n, originalSize, _, err := needle.CreateNeedleFromRequest(req, false, 1024*1024)
+	n, originalSize, _, err := needle.CreateNeedleFromRequest(req, false, 1024*1024, &bytes.Buffer{})
 	if m.needleHandling != nil {
 		m.needleHandling(n, originalSize, err)
 	}
@@ -361,6 +361,7 @@ message FilerConf {
        string disk_type = 5;
        bool fsync = 6;
        uint32 volume_growth_count = 7;
+        bool read_only = 8;
    }
    repeated PathConf locations = 2;
 }
@@ -3135,6 +3135,7 @@ type FilerConf_PathConf struct {
 	DiskType          string `protobuf:"bytes,5,opt,name=disk_type,json=diskType,proto3" json:"disk_type,omitempty"`
 	Fsync             bool   `protobuf:"varint,6,opt,name=fsync,proto3" json:"fsync,omitempty"`
 	VolumeGrowthCount uint32 `protobuf:"varint,7,opt,name=volume_growth_count,json=volumeGrowthCount,proto3" json:"volume_growth_count,omitempty"`
+	ReadOnly          bool   `protobuf:"varint,8,opt,name=read_only,json=readOnly,proto3" json:"read_only,omitempty"`
 }

 func (x *FilerConf_PathConf) Reset() {
@@ -3218,6 +3219,13 @@ func (x *FilerConf_PathConf) GetVolumeGrowthCount() uint32 {
 	return 0
 }

+func (x *FilerConf_PathConf) GetReadOnly() bool {
+	if x != nil {
+		return x.ReadOnly
+	}
+	return false
+}
+
 var File_filer_proto protoreflect.FileDescriptor

 var file_filer_proto_rawDesc = []byte{
@@ -3594,13 +3602,13 @@ var file_filer_proto_rawDesc = []byte{
 	0x75, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x05, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x22,
 	0x25, 0x0a, 0x0d, 0x4b, 0x76, 0x50, 0x75, 0x74, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65,
 	0x12, 0x14, 0x0a, 0x05, 0x65, 0x72, 0x72, 0x6f, 0x72, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52,
-	0x05, 0x65, 0x72, 0x72, 0x6f, 0x72, 0x22, 0xce, 0x02, 0x0a, 0x09, 0x46, 0x69, 0x6c, 0x65, 0x72,
+	0x05, 0x65, 0x72, 0x72, 0x6f, 0x72, 0x22, 0xeb, 0x02, 0x0a, 0x09, 0x46, 0x69, 0x6c, 0x65, 0x72,
 	0x43, 0x6f, 0x6e, 0x66, 0x12, 0x18, 0x0a, 0x07, 0x76, 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x18,
 	0x01, 0x20, 0x01, 0x28, 0x05, 0x52, 0x07, 0x76, 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x12, 0x3a,
 	0x0a, 0x09, 0x6c, 0x6f, 0x63, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x73, 0x18, 0x02, 0x20, 0x03, 0x28,
 	0x0b, 0x32, 0x1c, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x46, 0x69, 0x6c,
 	0x65, 0x72, 0x43, 0x6f, 0x6e, 0x66, 0x2e, 0x50, 0x61, 0x74, 0x68, 0x43, 0x6f, 0x6e, 0x66, 0x52,
-	0x09, 0x6c, 0x6f, 0x63, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x73, 0x1a, 0xea, 0x01, 0x0a, 0x08, 0x50,
+	0x09, 0x6c, 0x6f, 0x63, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x73, 0x1a, 0x87, 0x02, 0x0a, 0x08, 0x50,
 	0x61, 0x74, 0x68, 0x43, 0x6f, 0x6e, 0x66, 0x12, 0x27, 0x0a, 0x0f, 0x6c, 0x6f, 0x63, 0x61, 0x74,
 	0x69, 0x6f, 0x6e, 0x5f, 0x70, 0x72, 0x65, 0x66, 0x69, 0x78, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09,
 	0x52, 0x0e, 0x6c, 0x6f, 0x63, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x50, 0x72, 0x65, 0x66, 0x69, 0x78,
@@ -3615,114 +3623,116 @@ var file_filer_proto_rawDesc = []byte{
 	0x52, 0x05, 0x66, 0x73, 0x79, 0x6e, 0x63, 0x12, 0x2e, 0x0a, 0x13, 0x76, 0x6f, 0x6c, 0x75, 0x6d,
 	0x65, 0x5f, 0x67, 0x72, 0x6f, 0x77, 0x74, 0x68, 0x5f, 0x63, 0x6f, 0x75, 0x6e, 0x74, 0x18, 0x07,
 	0x20, 0x01, 0x28, 0x0d, 0x52, 0x11, 0x76, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x47, 0x72, 0x6f, 0x77,
-	0x74, 0x68, 0x43, 0x6f, 0x75, 0x6e, 0x74, 0x32, 0xdc, 0x0c, 0x0a, 0x0c, 0x53, 0x65, 0x61, 0x77,
-	0x65, 0x65, 0x64, 0x46, 0x69, 0x6c, 0x65, 0x72, 0x12, 0x67, 0x0a, 0x14, 0x4c, 0x6f, 0x6f, 0x6b,
-	0x75, 0x70, 0x44, 0x69, 0x72, 0x65, 0x63, 0x74, 0x6f, 0x72, 0x79, 0x45, 0x6e, 0x74, 0x72, 0x79,
-	0x12, 0x25, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x4c, 0x6f, 0x6f, 0x6b,
-	0x75, 0x70, 0x44, 0x69, 0x72, 0x65, 0x63, 0x74, 0x6f, 0x72, 0x79, 0x45, 0x6e, 0x74, 0x72, 0x79,
-	0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x26, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f,
-	0x70, 0x62, 0x2e, 0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x44, 0x69, 0x72, 0x65, 0x63, 0x74, 0x6f,
-	0x72, 0x79, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22,
-	0x00, 0x12, 0x4e, 0x0a, 0x0b, 0x4c, 0x69, 0x73, 0x74, 0x45, 0x6e, 0x74, 0x72, 0x69, 0x65, 0x73,
-	0x12, 0x1c, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x4c, 0x69, 0x73, 0x74,
-	0x45, 0x6e, 0x74, 0x72, 0x69, 0x65, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1d,
-	0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x4c, 0x69, 0x73, 0x74, 0x45, 0x6e,
-	0x74, 0x72, 0x69, 0x65, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x30,
-	0x01, 0x12, 0x4c, 0x0a, 0x0b, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x45, 0x6e, 0x74, 0x72, 0x79,
-	0x12, 0x1c, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x43, 0x72, 0x65, 0x61,
-	0x74, 0x65, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1d,
-	0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65,
-	0x45, 0x6e, 0x74, 0x72, 0x79, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x12,
-	0x4c, 0x0a, 0x0b, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x12, 0x1c,
-	0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65,
-	0x45, 0x6e, 0x74, 0x72, 0x79, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1d, 0x2e, 0x66,
-	0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x45, 0x6e,
-	0x74, 0x72, 0x79, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x12, 0x52, 0x0a,
-	0x0d, 0x41, 0x70, 0x70, 0x65, 0x6e, 0x64, 0x54, 0x6f, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x12, 0x1e,
-	0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x41, 0x70, 0x70, 0x65, 0x6e, 0x64,
-	0x54, 0x6f, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1f,
-	0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x41, 0x70, 0x70, 0x65, 0x6e, 0x64,
-	0x54, 0x6f, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22,
-	0x00, 0x12, 0x4c, 0x0a, 0x0b, 0x44, 0x65, 0x6c, 0x65, 0x74, 0x65, 0x45, 0x6e, 0x74, 0x72, 0x79,
-	0x12, 0x1c, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x44, 0x65, 0x6c, 0x65,
-	0x74, 0x65, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1d,
-	0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x44, 0x65, 0x6c, 0x65, 0x74, 0x65,
-	0x45, 0x6e, 0x74, 0x72, 0x79, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x12,
-	0x5e, 0x0a, 0x11, 0x41, 0x74, 0x6f, 0x6d, 0x69, 0x63, 0x52, 0x65, 0x6e, 0x61, 0x6d, 0x65, 0x45,
-	0x6e, 0x74, 0x72, 0x79, 0x12, 0x22, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e,
+	0x74, 0x68, 0x43, 0x6f, 0x75, 0x6e, 0x74, 0x12, 0x1b, 0x0a, 0x09, 0x72, 0x65, 0x61, 0x64, 0x5f,
+	0x6f, 0x6e, 0x6c, 0x79, 0x18, 0x08, 0x20, 0x01, 0x28, 0x08, 0x52, 0x08, 0x72, 0x65, 0x61, 0x64,
+	0x4f, 0x6e, 0x6c, 0x79, 0x32, 0xdc, 0x0c, 0x0a, 0x0c, 0x53, 0x65, 0x61, 0x77, 0x65, 0x65, 0x64,
+	0x46, 0x69, 0x6c, 0x65, 0x72, 0x12, 0x67, 0x0a, 0x14, 0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x44,
+	0x69, 0x72, 0x65, 0x63, 0x74, 0x6f, 0x72, 0x79, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x12, 0x25, 0x2e,
+	0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x44,
+	0x69, 0x72, 0x65, 0x63, 0x74, 0x6f, 0x72, 0x79, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x52, 0x65, 0x71,
+	0x75, 0x65, 0x73, 0x74, 0x1a, 0x26, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e,
+	0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x44, 0x69, 0x72, 0x65, 0x63, 0x74, 0x6f, 0x72, 0x79, 0x45,
+	0x6e, 0x74, 0x72, 0x79, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x12, 0x4e,
+	0x0a, 0x0b, 0x4c, 0x69, 0x73, 0x74, 0x45, 0x6e, 0x74, 0x72, 0x69, 0x65, 0x73, 0x12, 0x1c, 0x2e,
+	0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x4c, 0x69, 0x73, 0x74, 0x45, 0x6e, 0x74,
+	0x72, 0x69, 0x65, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1d, 0x2e, 0x66, 0x69,
+	0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x4c, 0x69, 0x73, 0x74, 0x45, 0x6e, 0x74, 0x72, 0x69,
+	0x65, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x30, 0x01, 0x12, 0x4c,
+	0x0a, 0x0b, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x12, 0x1c, 0x2e,
+	0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x45,
+	0x6e, 0x74, 0x72, 0x79, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1d, 0x2e, 0x66, 0x69,
+	0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x45, 0x6e, 0x74,
+	0x72, 0x79, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x12, 0x4c, 0x0a, 0x0b,
+	0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x12, 0x1c, 0x2e, 0x66, 0x69,
+	0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x45, 0x6e, 0x74,
+	0x72, 0x79, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1d, 0x2e, 0x66, 0x69, 0x6c, 0x65,
+	0x72, 0x5f, 0x70, 0x62, 0x2e, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x45, 0x6e, 0x74, 0x72, 0x79,
+	0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x12, 0x52, 0x0a, 0x0d, 0x41, 0x70,
+	0x70, 0x65, 0x6e, 0x64, 0x54, 0x6f, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x12, 0x1e, 0x2e, 0x66, 0x69,
+	0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x41, 0x70, 0x70, 0x65, 0x6e, 0x64, 0x54, 0x6f, 0x45,
+	0x6e, 0x74, 0x72, 0x79, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1f, 0x2e, 0x66, 0x69,
+	0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x41, 0x70, 0x70, 0x65, 0x6e, 0x64, 0x54, 0x6f, 0x45,
+	0x6e, 0x74, 0x72, 0x79, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x12, 0x4c,
+	0x0a, 0x0b, 0x44, 0x65, 0x6c, 0x65, 0x74, 0x65, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x12, 0x1c, 0x2e,
+	0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x44, 0x65, 0x6c, 0x65, 0x74, 0x65, 0x45,
+	0x6e, 0x74, 0x72, 0x79, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1d, 0x2e, 0x66, 0x69,
+	0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x44, 0x65, 0x6c, 0x65, 0x74, 0x65, 0x45, 0x6e, 0x74,
+	0x72, 0x79, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x12, 0x5e, 0x0a, 0x11,
 	0x41, 0x74, 0x6f, 0x6d, 0x69, 0x63, 0x52, 0x65, 0x6e, 0x61, 0x6d, 0x65, 0x45, 0x6e, 0x74, 0x72,
-	0x79, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x23, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72,
-	0x5f, 0x70, 0x62, 0x2e, 0x41, 0x74, 0x6f, 0x6d, 0x69, 0x63, 0x52, 0x65, 0x6e, 0x61, 0x6d, 0x65,
-	0x45, 0x6e, 0x74, 0x72, 0x79, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x12,
-	0x4f, 0x0a, 0x0c, 0x41, 0x73, 0x73, 0x69, 0x67, 0x6e, 0x56, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x12,
-	0x1d, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x41, 0x73, 0x73, 0x69, 0x67,
-	0x6e, 0x56, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1e,
-	0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x41, 0x73, 0x73, 0x69, 0x67, 0x6e,
-	0x56, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00,
-	0x12, 0x4f, 0x0a, 0x0c, 0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x56, 0x6f, 0x6c, 0x75, 0x6d, 0x65,
-	0x12, 0x1d, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x4c, 0x6f, 0x6f, 0x6b,
-	0x75, 0x70, 0x56, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a,
-	0x1e, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x4c, 0x6f, 0x6f, 0x6b, 0x75,
-	0x70, 0x56, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22,
-	0x00, 0x12, 0x55, 0x0a, 0x0e, 0x43, 0x6f, 0x6c, 0x6c, 0x65, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x4c,
-	0x69, 0x73, 0x74, 0x12, 0x1f, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x43,
-	0x6f, 0x6c, 0x6c, 0x65, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x4c, 0x69, 0x73, 0x74, 0x52, 0x65, 0x71,
-	0x75, 0x65, 0x73, 0x74, 0x1a, 0x20, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e,
-	0x43, 0x6f, 0x6c, 0x6c, 0x65, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x4c, 0x69, 0x73, 0x74, 0x52, 0x65,
-	0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x12, 0x5b, 0x0a, 0x10, 0x44, 0x65, 0x6c, 0x65,
-	0x74, 0x65, 0x43, 0x6f, 0x6c, 0x6c, 0x65, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x12, 0x21, 0x2e, 0x66,
-	0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x44, 0x65, 0x6c, 0x65, 0x74, 0x65, 0x43, 0x6f,
-	0x6c, 0x6c, 0x65, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a,
-	0x22, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x44, 0x65, 0x6c, 0x65, 0x74,
-	0x65, 0x43, 0x6f, 0x6c, 0x6c, 0x65, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x52, 0x65, 0x73, 0x70, 0x6f,
-	0x6e, 0x73, 0x65, 0x22, 0x00, 0x12, 0x49, 0x0a, 0x0a, 0x53, 0x74, 0x61, 0x74, 0x69, 0x73, 0x74,
-	0x69, 0x63, 0x73, 0x12, 0x1b, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x53,
-	0x74, 0x61, 0x74, 0x69, 0x73, 0x74, 0x69, 0x63, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74,
-	0x1a, 0x1c, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x53, 0x74, 0x61, 0x74,
-	0x69, 0x73, 0x74, 0x69, 0x63, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00,
-	0x12, 0x6a, 0x0a, 0x15, 0x47, 0x65, 0x74, 0x46, 0x69, 0x6c, 0x65, 0x72, 0x43, 0x6f, 0x6e, 0x66,
-	0x69, 0x67, 0x75, 0x72, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x12, 0x26, 0x2e, 0x66, 0x69, 0x6c, 0x65,
-	0x72, 0x5f, 0x70, 0x62, 0x2e, 0x47, 0x65, 0x74, 0x46, 0x69, 0x6c, 0x65, 0x72, 0x43, 0x6f, 0x6e,
-	0x66, 0x69, 0x67, 0x75, 0x72, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73,
-	0x74, 0x1a, 0x27, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x47, 0x65, 0x74,
-	0x46, 0x69, 0x6c, 0x65, 0x72, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x75, 0x72, 0x61, 0x74, 0x69,
-	0x6f, 0x6e, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x12, 0x60, 0x0a, 0x11,
-	0x53, 0x75, 0x62, 0x73, 0x63, 0x72, 0x69, 0x62, 0x65, 0x4d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74,
-	0x61, 0x12, 0x22, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x53, 0x75, 0x62,
-	0x73, 0x63, 0x72, 0x69, 0x62, 0x65, 0x4d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x52, 0x65,
+	0x79, 0x12, 0x22, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x41, 0x74, 0x6f,
+	0x6d, 0x69, 0x63, 0x52, 0x65, 0x6e, 0x61, 0x6d, 0x65, 0x45, 0x6e, 0x74, 0x72, 0x79, 0x52, 0x65,
 	0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x23, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62,
+	0x2e, 0x41, 0x74, 0x6f, 0x6d, 0x69, 0x63, 0x52, 0x65, 0x6e, 0x61, 0x6d, 0x65, 0x45, 0x6e, 0x74,
+	0x72, 0x79, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x12, 0x4f, 0x0a, 0x0c,
+	0x41, 0x73, 0x73, 0x69, 0x67, 0x6e, 0x56, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x12, 0x1d, 0x2e, 0x66,
+	0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x41, 0x73, 0x73, 0x69, 0x67, 0x6e, 0x56, 0x6f,
+	0x6c, 0x75, 0x6d, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1e, 0x2e, 0x66, 0x69,
+	0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x41, 0x73, 0x73, 0x69, 0x67, 0x6e, 0x56, 0x6f, 0x6c,
+	0x75, 0x6d, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x12, 0x4f, 0x0a,
+	0x0c, 0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x56, 0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x12, 0x1d, 0x2e,
+	0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x56,
+	0x6f, 0x6c, 0x75, 0x6d, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1e, 0x2e, 0x66,
+	0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x4c, 0x6f, 0x6f, 0x6b, 0x75, 0x70, 0x56, 0x6f,
+	0x6c, 0x75, 0x6d, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x12, 0x55,
+	0x0a, 0x0e, 0x43, 0x6f, 0x6c, 0x6c, 0x65, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x4c, 0x69, 0x73, 0x74,
+	0x12, 0x1f, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x43, 0x6f, 0x6c, 0x6c,
+	0x65, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x4c, 0x69, 0x73, 0x74, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73,
+	0x74, 0x1a, 0x20, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x43, 0x6f, 0x6c,
+	0x6c, 0x65, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x4c, 0x69, 0x73, 0x74, 0x52, 0x65, 0x73, 0x70, 0x6f,
+	0x6e, 0x73, 0x65, 0x22, 0x00, 0x12, 0x5b, 0x0a, 0x10, 0x44, 0x65, 0x6c, 0x65, 0x74, 0x65, 0x43,
+	0x6f, 0x6c, 0x6c, 0x65, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x12, 0x21, 0x2e, 0x66, 0x69, 0x6c, 0x65,
+	0x72, 0x5f, 0x70, 0x62, 0x2e, 0x44, 0x65, 0x6c, 0x65, 0x74, 0x65, 0x43, 0x6f, 0x6c, 0x6c, 0x65,
+	0x63, 0x74, 0x69, 0x6f, 0x6e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x22, 0x2e, 0x66,
+	0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x44, 0x65, 0x6c, 0x65, 0x74, 0x65, 0x43, 0x6f,
+	0x6c, 0x6c, 0x65, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65,
+	0x22, 0x00, 0x12, 0x49, 0x0a, 0x0a, 0x53, 0x74, 0x61, 0x74, 0x69, 0x73, 0x74, 0x69, 0x63, 0x73,
+	0x12, 0x1b, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x53, 0x74, 0x61, 0x74,
+	0x69, 0x73, 0x74, 0x69, 0x63, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1c, 0x2e,
+	0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x53, 0x74, 0x61, 0x74, 0x69, 0x73, 0x74,
+	0x69, 0x63, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x12, 0x6a, 0x0a,
+	0x15, 0x47, 0x65, 0x74, 0x46, 0x69, 0x6c, 0x65, 0x72, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x75,
+	0x72, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x12, 0x26, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70,
+	0x62, 0x2e, 0x47, 0x65, 0x74, 0x46, 0x69, 0x6c, 0x65, 0x72, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67,
+	0x75, 0x72, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x27,
+	0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x47, 0x65, 0x74, 0x46, 0x69, 0x6c,
+	0x65, 0x72, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x75, 0x72, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x52,
+	0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x12, 0x60, 0x0a, 0x11, 0x53, 0x75, 0x62,
+	0x73, 0x63, 0x72, 0x69, 0x62, 0x65, 0x4d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x12, 0x22,
+	0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x53, 0x75, 0x62, 0x73, 0x63, 0x72,
+	0x69, 0x62, 0x65, 0x4d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x52, 0x65, 0x71, 0x75, 0x65,
+	0x73, 0x74, 0x1a, 0x23, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x53, 0x75,
+	0x62, 0x73, 0x63, 0x72, 0x69, 0x62, 0x65, 0x4d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x52,
+	0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x30, 0x01, 0x12, 0x65, 0x0a, 0x16, 0x53,
+	0x75, 0x62, 0x73, 0x63, 0x72, 0x69, 0x62, 0x65, 0x4c, 0x6f, 0x63, 0x61, 0x6c, 0x4d, 0x65, 0x74,
+	0x61, 0x64, 0x61, 0x74, 0x61, 0x12, 0x22, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62,
 	0x2e, 0x53, 0x75, 0x62, 0x73, 0x63, 0x72, 0x69, 0x62, 0x65, 0x4d, 0x65, 0x74, 0x61, 0x64, 0x61,
-	0x74, 0x61, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x30, 0x01, 0x12, 0x65,
-	0x0a, 0x16, 0x53, 0x75, 0x62, 0x73, 0x63, 0x72, 0x69, 0x62, 0x65, 0x4c, 0x6f, 0x63, 0x61, 0x6c,
-	0x4d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x12, 0x22, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72,
-	0x5f, 0x70, 0x62, 0x2e, 0x53, 0x75, 0x62, 0x73, 0x63, 0x72, 0x69, 0x62, 0x65, 0x4d, 0x65, 0x74,
-	0x61, 0x64, 0x61, 0x74, 0x61, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x23, 0x2e, 0x66,
-	0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x53, 0x75, 0x62, 0x73, 0x63, 0x72, 0x69, 0x62,
-	0x65, 0x4d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73,
-	0x65, 0x22, 0x00, 0x30, 0x01, 0x12, 0x56, 0x0a, 0x0d, 0x4b, 0x65, 0x65, 0x70, 0x43, 0x6f, 0x6e,
-	0x6e, 0x65, 0x63, 0x74, 0x65, 0x64, 0x12, 0x1e, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70,
-	0x62, 0x2e, 0x4b, 0x65, 0x65, 0x70, 0x43, 0x6f, 0x6e, 0x6e, 0x65, 0x63, 0x74, 0x65, 0x64, 0x52,
-	0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1f, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70,
-	0x62, 0x2e, 0x4b, 0x65, 0x65, 0x70, 0x43, 0x6f, 0x6e, 0x6e, 0x65, 0x63, 0x74, 0x65, 0x64, 0x52,
-	0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x28, 0x01, 0x30, 0x01, 0x12, 0x4f, 0x0a,
-	0x0c, 0x4c, 0x6f, 0x63, 0x61, 0x74, 0x65, 0x42, 0x72, 0x6f, 0x6b, 0x65, 0x72, 0x12, 0x1d, 0x2e,
-	0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x4c, 0x6f, 0x63, 0x61, 0x74, 0x65, 0x42,
-	0x72, 0x6f, 0x6b, 0x65, 0x72, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1e, 0x2e, 0x66,
-	0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x4c, 0x6f, 0x63, 0x61, 0x74, 0x65, 0x42, 0x72,
-	0x6f, 0x6b, 0x65, 0x72, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x12, 0x3a,
-	0x0a, 0x05, 0x4b, 0x76, 0x47, 0x65, 0x74, 0x12, 0x16, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f,
-	0x70, 0x62, 0x2e, 0x4b, 0x76, 0x47, 0x65, 0x74, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a,
-	0x17, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x4b, 0x76, 0x47, 0x65, 0x74,
-	0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x12, 0x3a, 0x0a, 0x05, 0x4b, 0x76,
-	0x50, 0x75, 0x74, 0x12, 0x16, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x4b,
-	0x76, 0x50, 0x75, 0x74, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x17, 0x2e, 0x66, 0x69,
-	0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x4b, 0x76, 0x50, 0x75, 0x74, 0x52, 0x65, 0x73, 0x70,
-	0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x42, 0x4f, 0x0a, 0x10, 0x73, 0x65, 0x61, 0x77, 0x65, 0x65,
-	0x64, 0x66, 0x73, 0x2e, 0x63, 0x6c, 0x69, 0x65, 0x6e, 0x74, 0x42, 0x0a, 0x46, 0x69, 0x6c, 0x65,
-	0x72, 0x50, 0x72, 0x6f, 0x74, 0x6f, 0x5a, 0x2f, 0x67, 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63,
-	0x6f, 0x6d, 0x2f, 0x63, 0x68, 0x72, 0x69, 0x73, 0x6c, 0x75, 0x73, 0x66, 0x2f, 0x73, 0x65, 0x61,
-	0x77, 0x65, 0x65, 0x64, 0x66, 0x73, 0x2f, 0x77, 0x65, 0x65, 0x64, 0x2f, 0x70, 0x62, 0x2f, 0x66,
-	0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33,
+	0x74, 0x61, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x23, 0x2e, 0x66, 0x69, 0x6c, 0x65,
+	0x72, 0x5f, 0x70, 0x62, 0x2e, 0x53, 0x75, 0x62, 0x73, 0x63, 0x72, 0x69, 0x62, 0x65, 0x4d, 0x65,
+	0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00,
+	0x30, 0x01, 0x12, 0x56, 0x0a, 0x0d, 0x4b, 0x65, 0x65, 0x70, 0x43, 0x6f, 0x6e, 0x6e, 0x65, 0x63,
+	0x74, 0x65, 0x64, 0x12, 0x1e, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x4b,
+	0x65, 0x65, 0x70, 0x43, 0x6f, 0x6e, 0x6e, 0x65, 0x63, 0x74, 0x65, 0x64, 0x52, 0x65, 0x71, 0x75,
+	0x65, 0x73, 0x74, 0x1a, 0x1f, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x4b,
+	0x65, 0x65, 0x70, 0x43, 0x6f, 0x6e, 0x6e, 0x65, 0x63, 0x74, 0x65, 0x64, 0x52, 0x65, 0x73, 0x70,
+	0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x28, 0x01, 0x30, 0x01, 0x12, 0x4f, 0x0a, 0x0c, 0x4c, 0x6f,
+	0x63, 0x61, 0x74, 0x65, 0x42, 0x72, 0x6f, 0x6b, 0x65, 0x72, 0x12, 0x1d, 0x2e, 0x66, 0x69, 0x6c,
+	0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x4c, 0x6f, 0x63, 0x61, 0x74, 0x65, 0x42, 0x72, 0x6f, 0x6b,
+	0x65, 0x72, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1e, 0x2e, 0x66, 0x69, 0x6c, 0x65,
+	0x72, 0x5f, 0x70, 0x62, 0x2e, 0x4c, 0x6f, 0x63, 0x61, 0x74, 0x65, 0x42, 0x72, 0x6f, 0x6b, 0x65,
+	0x72, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x12, 0x3a, 0x0a, 0x05, 0x4b,
+	0x76, 0x47, 0x65, 0x74, 0x12, 0x16, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e,
+	0x4b, 0x76, 0x47, 0x65, 0x74, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x17, 0x2e, 0x66,
+	0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x4b, 0x76, 0x47, 0x65, 0x74, 0x52, 0x65, 0x73,
+	0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x00, 0x12, 0x3a, 0x0a, 0x05, 0x4b, 0x76, 0x50, 0x75, 0x74,
+	0x12, 0x16, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72, 0x5f, 0x70, 0x62, 0x2e, 0x4b, 0x76, 0x50, 0x75,
+	0x74, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x17, 0x2e, 0x66, 0x69, 0x6c, 0x65, 0x72,
+	0x5f, 0x70, 0x62, 0x2e, 0x4b, 0x76, 0x50, 0x75, 0x74, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73,
+	0x65, 0x22, 0x00, 0x42, 0x4f, 0x0a, 0x10, 0x73, 0x65, 0x61, 0x77, 0x65, 0x65, 0x64, 0x66, 0x73,
+	0x2e, 0x63, 0x6c, 0x69, 0x65, 0x6e, 0x74, 0x42, 0x0a, 0x46, 0x69, 0x6c, 0x65, 0x72, 0x50, 0x72,
+	0x6f, 0x74, 0x6f, 0x5a, 0x2f, 0x67, 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f,
+	0x63, 0x68, 0x72, 0x69, 0x73, 0x6c, 0x75, 0x73, 0x66, 0x2f, 0x73, 0x65, 0x61, 0x77, 0x65, 0x65,
+	0x64, 0x66, 0x73, 0x2f, 0x77, 0x65, 0x65, 0x64, 0x2f, 0x70, 0x62, 0x2f, 0x66, 0x69, 0x6c, 0x65,
+	0x72, 0x5f, 0x70, 0x62, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33,
 }

 var (
@@ -4,6 +4,7 @@ import (
 	"context"
 	"fmt"
 	"github.com/chrislusf/seaweedfs/weed/glog"
+	"math/rand"
 	"net/http"
 	"strconv"
 	"strings"
@@ -24,10 +25,16 @@ const (

 var (
 	// cache grpc connections
-	grpcClients     = make(map[string]*grpc.ClientConn)
+	grpcClients     = make(map[string]*versionedGrpcClient)
 	grpcClientsLock sync.Mutex
 )

+type versionedGrpcClient struct {
+	*grpc.ClientConn
+	version  int
+	errCount int
+}
+
 func init() {
 	http.DefaultTransport.(*http.Transport).MaxIdleConnsPerHost = 1024
 	http.DefaultTransport.(*http.Transport).MaxIdleConns = 1024
@@ -43,7 +50,7 @@ func NewGrpcServer(opts ...grpc.ServerOption) *grpc.Server {
 		}),
 		grpc.KeepaliveEnforcementPolicy(keepalive.EnforcementPolicy{
 			MinTime:             60 * time.Second, // min time a client should wait before sending a ping
-			PermitWithoutStream: false,
+			PermitWithoutStream: true,
 		}),
 		grpc.MaxRecvMsgSize(Max_Message_Size),
 		grpc.MaxSendMsgSize(Max_Message_Size),
@@ -69,7 +76,7 @@ func GrpcDial(ctx context.Context, address string, opts ...grpc.DialOption) (*gr
 		grpc.WithKeepaliveParams(keepalive.ClientParameters{
 			Time:                30 * time.Second, // client ping server if no activity for this long
 			Timeout:             20 * time.Second,
-			PermitWithoutStream: false,
+			PermitWithoutStream: true,
 		}))
 	for _, opt := range opts {
 		if opt != nil {
@@ -79,7 +86,7 @@ func GrpcDial(ctx context.Context, address string, opts ...grpc.DialOption) (*gr
 	return grpc.DialContext(ctx, address, options...)
 }

-func getOrCreateConnection(address string, opts ...grpc.DialOption) (*grpc.ClientConn, error) {
+func getOrCreateConnection(address string, opts ...grpc.DialOption) (*versionedGrpcClient, error) {

 	grpcClientsLock.Lock()
 	defer grpcClientsLock.Unlock()
@@ -94,23 +101,55 @@ func getOrCreateConnection(address string, opts ...grpc.DialOption) (*grpc.Clien
 		return nil, fmt.Errorf("fail to dial %s: %v", address, err)
 	}

-	grpcClients[address] = grpcConnection
+	vgc := &versionedGrpcClient{
+		grpcConnection,
+		rand.Int(),
+		0,
+	}
+	grpcClients[address] = vgc

-	return grpcConnection, nil
+	return vgc, nil
 }

 func WithCachedGrpcClient(fn func(*grpc.ClientConn) error, address string, opts ...grpc.DialOption) error {

-	grpcConnection, err := getOrCreateConnection(address, opts...)
+	vgc, err := getOrCreateConnection(address, opts...)
 	if err != nil {
 		return fmt.Errorf("getOrCreateConnection %s: %v", address, err)
 	}
-	return fn(grpcConnection)
+	executionErr := fn(vgc.ClientConn)
+	if executionErr != nil {
+		vgc.errCount++
+		if vgc.errCount > 3 ||
+			strings.Contains(executionErr.Error(), "transport") ||
+			strings.Contains(executionErr.Error(), "connection closed") {
+			grpcClientsLock.Lock()
+			if t, ok := grpcClients[address]; ok {
+				if t.version == vgc.version {
+					vgc.Close()
+					delete(grpcClients, address)
+				}
+			}
+			grpcClientsLock.Unlock()
+		}
+	}
+
+	return executionErr
 }

 func ParseServerToGrpcAddress(server string) (serverGrpcAddress string, err error) {
 	return ParseServerAddress(server, 10000)
 }
+func ParseServersToGrpcAddresses(servers []string) (serverGrpcAddresses []string, err error) {
+	for _, server := range servers {
+		if serverGrpcAddress, parseErr := ParseServerToGrpcAddress(server); parseErr == nil {
+			serverGrpcAddresses = append(serverGrpcAddresses, serverGrpcAddress)
+		} else {
+			return nil, parseErr
+		}
+	}
+	return
+}

 func ParseServerAddress(server string, deltaPort int) (newServerAddress string, err error) {

@@ -202,3 +241,18 @@ func WithGrpcFilerClient(filerGrpcAddress string, grpcDialOption grpc.DialOption
 	}, filerGrpcAddress, grpcDialOption)

 }
+
+func WithOneOfGrpcFilerClients(filerGrpcAddresses []string, grpcDialOption grpc.DialOption, fn func(client filer_pb.SeaweedFilerClient) error) (err error) {
+
+	for _, filerGrpcAddress := range filerGrpcAddresses {
+		err = WithCachedGrpcClient(func(grpcConnection *grpc.ClientConn) error {
+			client := filer_pb.NewSeaweedFilerClient(grpcConnection)
+			return fn(client)
+		}, filerGrpcAddress, grpcDialOption)
+		if err == nil {
+			return nil
+		}
+	}
+
+	return err
+}
@@ -129,8 +129,7 @@ func (g *AzureSink) CreateEntry(key string, entry *filer_pb.Entry, signatures []

 func (g *AzureSink) UpdateEntry(key string, oldEntry *filer_pb.Entry, newParentPath string, newEntry *filer_pb.Entry, deleteIncludeChunks bool, signatures []int32) (foundExistingEntry bool, err error) {
 	key = cleanKey(key)
-	// TODO improve efficiency
-	return false, nil
+	return true, g.CreateEntry(key, newEntry, signatures)
 }

 func cleanKey(key string) string {
@@ -118,11 +118,8 @@ func (g *B2Sink) CreateEntry(key string, entry *filer_pb.Entry, signatures []int
 }

 func (g *B2Sink) UpdateEntry(key string, oldEntry *filer_pb.Entry, newParentPath string, newEntry *filer_pb.Entry, deleteIncludeChunks bool, signatures []int32) (foundExistingEntry bool, err error) {
-
 	key = cleanKey(key)
-
-	// TODO improve efficiency
-	return false, nil
+	return true, g.CreateEntry(key, newEntry, signatures)
 }

 func cleanKey(key string) string {
@@ -116,6 +116,5 @@ func (g *GcsSink) CreateEntry(key string, entry *filer_pb.Entry, signatures []in
 }

 func (g *GcsSink) UpdateEntry(key string, oldEntry *filer_pb.Entry, newParentPath string, newEntry *filer_pb.Entry, deleteIncludeChunks bool, signatures []int32) (foundExistingEntry bool, err error) {
-	// TODO improve efficiency
-	return false, nil
+	return true, g.CreateEntry(key, newEntry, signatures)
 }
@@ -8,15 +8,15 @@ import (
 	"github.com/chrislusf/seaweedfs/weed/replication/sink"
 	"github.com/chrislusf/seaweedfs/weed/replication/source"
 	"github.com/chrislusf/seaweedfs/weed/util"
-	"io/ioutil"
 	"os"
 	"path/filepath"
 	"strings"
 )

 type LocalSink struct {
-	Dir         string
-	filerSource *source.FilerSource
+	Dir           string
+	filerSource   *source.FilerSource
+	isIncremental bool
 }

 func init() {
@@ -35,15 +35,17 @@ func (localsink *LocalSink) isMultiPartEntry(key string) bool {
 	return strings.HasSuffix(key, ".part") && strings.Contains(key, "/.uploads/")
 }

-func (localsink *LocalSink) initialize(dir string) error {
+func (localsink *LocalSink) initialize(dir string, isIncremental bool) error {
 	localsink.Dir = dir
+	localsink.isIncremental = isIncremental
 	return nil
 }

 func (localsink *LocalSink) Initialize(configuration util.Configuration, prefix string) error {
 	dir := configuration.GetString(prefix + "directory")
+	isIncremental := configuration.GetBool(prefix + "is_incremental")
 	glog.V(4).Infof("sink.local.directory: %v", dir)
-	return localsink.initialize(dir)
+	return localsink.initialize(dir, isIncremental)
 }

 func (localsink *LocalSink) GetSinkToDirectory() string {
@@ -51,7 +53,7 @@ func (localsink *LocalSink) GetSinkToDirectory() string {
 }

 func (localsink *LocalSink) IsIncremental() bool {
-	return true
+	return localsink.isIncremental
 }

 func (localsink *LocalSink) DeleteEntry(key string, isDirectory, deleteIncludeChunks bool, signatures []int32) error {
@@ -83,8 +85,18 @@ func (localsink *LocalSink) CreateEntry(key string, entry *filer_pb.Entry, signa
 		}
 	}

+	if entry.IsDirectory {
+		return os.Mkdir(key, os.FileMode(entry.Attributes.FileMode))
+	}
+
+	dstFile, err := os.OpenFile(key, os.O_RDWR|os.O_CREATE|os.O_TRUNC, os.FileMode(entry.Attributes.FileMode))
+	if err != nil {
+		return err
+	}
+	defer dstFile.Close()
+
 	writeFunc := func(data []byte) error {
-		writeErr := ioutil.WriteFile(key, data, 0755)
+		_, writeErr := dstFile.Write(data)
 		return writeErr
 	}

@@ -101,5 +113,7 @@ func (localsink *LocalSink) UpdateEntry(key string, oldEntry *filer_pb.Entry, ne
 	}
 	glog.V(4).Infof("Update Entry key: %s", key)
 	// do delete and create
-	return false, nil
+	foundExistingEntry = util.FileExists(key)
+	err = localsink.CreateEntry(key, newEntry, signatures)
+	return
 }
@@ -147,8 +147,7 @@ func (s3sink *S3Sink) CreateEntry(key string, entry *filer_pb.Entry, signatures

 func (s3sink *S3Sink) UpdateEntry(key string, oldEntry *filer_pb.Entry, newParentPath string, newEntry *filer_pb.Entry, deleteIncludeChunks bool, signatures []int32) (foundExistingEntry bool, err error) {
 	key = cleanKey(key)
-	// TODO improve efficiency
-	return false, nil
+	return true, s3sink.CreateEntry(key, newEntry, signatures)
 }

 func cleanKey(key string) string {
@@ -58,7 +58,6 @@ func (fs *FilerSource) LookupFileId(part string) (fileUrls []string, err error)

 	err = fs.WithFilerClient(func(client filer_pb.SeaweedFilerClient) error {

-		glog.V(4).Infof("read lookup volume id locations: %v", vid)
 		resp, err := client.LookupVolume(context.Background(), &filer_pb.LookupVolumeRequest{
 			VolumeIds: []string{vid},
 		})
@@ -150,7 +150,7 @@ func (iam *IdentityAccessManagement) Auth(f http.HandlerFunc, action Action) htt
 			f(w, r)
 			return
 		}
-		writeErrorResponse(w, errCode, r.URL)
+		s3err.WriteErrorResponse(w, errCode, r)
 	}
 }

@@ -3,6 +3,7 @@ package s3api
 import (
 	"github.com/aws/aws-sdk-go/aws"
 	"github.com/aws/aws-sdk-go/service/s3"
+	"github.com/chrislusf/seaweedfs/weed/s3api/s3err"
 	"testing"
 	"time"
 )
@@ -19,7 +20,7 @@ func TestInitiateMultipartUploadResult(t *testing.T) {
 		},
 	}

-	encoded := string(encodeResponse(response))
+	encoded := string(s3err.EncodeXMLResponse(response))
 	if encoded != expected {
 		t.Errorf("unexpected output: %s\nexpecting:%s", encoded, expected)
 	}
@@ -41,7 +42,7 @@ func TestListPartsResult(t *testing.T) {
 		},
 	}

-	encoded := string(encodeResponse(response))
+	encoded := string(s3err.EncodeXMLResponse(response))
 	if encoded != expected {
 		t.Errorf("unexpected output: %s\nexpecting:%s", encoded, expected)
 	}
@@ -32,7 +32,7 @@ func (s3a *S3ApiServer) ListBucketsHandler(w http.ResponseWriter, r *http.Reques
 	if s3a.iam.isEnabled() {
 		identity, s3Err = s3a.iam.authUser(r)
 		if s3Err != s3err.ErrNone {
-			writeErrorResponse(w, s3Err, r.URL)
+			s3err.WriteErrorResponse(w, s3Err, r)
 			return
 		}
 	}
@@ -42,7 +42,7 @@ func (s3a *S3ApiServer) ListBucketsHandler(w http.ResponseWriter, r *http.Reques
 	entries, _, err := s3a.list(s3a.option.BucketsPath, "", "", false, math.MaxInt32)

 	if err != nil {
-		writeErrorResponse(w, s3err.ErrInternalError, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrInternalError, r)
 		return
 	}

@@ -69,7 +69,7 @@ func (s3a *S3ApiServer) ListBucketsHandler(w http.ResponseWriter, r *http.Reques
 		Buckets: buckets,
 	}

-	writeSuccessResponseXML(w, encodeResponse(response))
+	writeSuccessResponseXML(w, response)
 }

 func (s3a *S3ApiServer) PutBucketHandler(w http.ResponseWriter, r *http.Request) {
@@ -95,14 +95,14 @@ func (s3a *S3ApiServer) PutBucketHandler(w http.ResponseWriter, r *http.Request)
 		}
 		return nil
 	}); err != nil {
-		writeErrorResponse(w, s3err.ErrInternalError, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrInternalError, r)
 		return
 	}
 	if exist, err := s3a.exists(s3a.option.BucketsPath, bucket, true); err == nil && exist {
 		errCode = s3err.ErrBucketAlreadyExists
 	}
 	if errCode != s3err.ErrNone {
-		writeErrorResponse(w, errCode, r.URL)
+		s3err.WriteErrorResponse(w, errCode, r)
 		return
 	}

@@ -118,7 +118,7 @@ func (s3a *S3ApiServer) PutBucketHandler(w http.ResponseWriter, r *http.Request)
 	// create the folder for bucket, but lazily create actual collection
 	if err := s3a.mkdir(s3a.option.BucketsPath, bucket, fn); err != nil {
 		glog.Errorf("PutBucketHandler mkdir: %v", err)
-		writeErrorResponse(w, s3err.ErrInternalError, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrInternalError, r)
 		return
 	}

@@ -130,7 +130,7 @@ func (s3a *S3ApiServer) DeleteBucketHandler(w http.ResponseWriter, r *http.Reque
 	bucket, _ := getBucketAndObject(r)

 	if err := s3a.checkBucket(r, bucket); err != s3err.ErrNone {
-		writeErrorResponse(w, err, r.URL)
+		s3err.WriteErrorResponse(w, err, r)
 		return
 	}

@@ -152,11 +152,11 @@ func (s3a *S3ApiServer) DeleteBucketHandler(w http.ResponseWriter, r *http.Reque
 	err = s3a.rm(s3a.option.BucketsPath, bucket, false, true)

 	if err != nil {
-		writeErrorResponse(w, s3err.ErrInternalError, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrInternalError, r)
 		return
 	}

-	writeResponse(w, http.StatusNoContent, nil, mimeNone)
+	s3err.WriteEmptyResponse(w, http.StatusNoContent)
 }

 func (s3a *S3ApiServer) HeadBucketHandler(w http.ResponseWriter, r *http.Request) {
@@ -164,7 +164,7 @@ func (s3a *S3ApiServer) HeadBucketHandler(w http.ResponseWriter, r *http.Request
 	bucket, _ := getBucketAndObject(r)

 	if err := s3a.checkBucket(r, bucket); err != s3err.ErrNone {
-		writeErrorResponse(w, err, r.URL)
+		s3err.WriteErrorResponse(w, err, r)
 		return
 	}

@@ -1,6 +1,7 @@
 package s3api

 import (
+	"github.com/chrislusf/seaweedfs/weed/s3api/s3err"
 	"testing"
 	"time"

@@ -32,7 +33,7 @@ func TestListBucketsHandler(t *testing.T) {
 		Buckets: buckets,
 	}

-	encoded := string(encodeResponse(response))
+	encoded := string(s3err.EncodeXMLResponse(response))
 	if encoded != expected {
 		t.Errorf("unexpected output: %s\nexpecting:%s", encoded, expected)
 	}
@@ -1,45 +1,16 @@
 package s3api

 import (
-	"bytes"
 	"encoding/base64"
-	"encoding/xml"
 	"fmt"
 	"github.com/chrislusf/seaweedfs/weed/s3api/s3err"
-	"net/http"
-	"net/url"
-	"strconv"
-	"time"
-
 	"google.golang.org/grpc"
+	"net/http"

-	"github.com/chrislusf/seaweedfs/weed/glog"
 	"github.com/chrislusf/seaweedfs/weed/pb"
 	"github.com/chrislusf/seaweedfs/weed/pb/filer_pb"
 )

-type mimeType string
-
-const (
-	mimeNone mimeType = ""
-	mimeJSON mimeType = "application/json"
-	mimeXML  mimeType = "application/xml"
-)
-
-func setCommonHeaders(w http.ResponseWriter) {
-	w.Header().Set("x-amz-request-id", fmt.Sprintf("%d", time.Now().UnixNano()))
-	w.Header().Set("Accept-Ranges", "bytes")
-}
-
-// Encodes the response headers into XML format.
-func encodeResponse(response interface{}) []byte {
-	var bytesBuffer bytes.Buffer
-	bytesBuffer.WriteString(xml.Header)
-	e := xml.NewEncoder(&bytesBuffer)
-	e.Encode(response)
-	return bytesBuffer.Bytes()
-}
-
 var _ = filer_pb.FilerClient(&S3ApiServer{})

 func (s3a *S3ApiServer) WithFilerClient(fn func(filer_pb.SeaweedFilerClient) error) error {
@@ -54,53 +25,12 @@ func (s3a *S3ApiServer) AdjustedUrl(location *filer_pb.Location) string {
 	return location.Url
 }

-// If none of the http routes match respond with MethodNotAllowed
-func notFoundHandler(w http.ResponseWriter, r *http.Request) {
-	glog.V(0).Infof("unsupported %s %s", r.Method, r.RequestURI)
-	writeErrorResponse(w, s3err.ErrMethodNotAllowed, r.URL)
-}
-
-func writeErrorResponse(w http.ResponseWriter, errorCode s3err.ErrorCode, reqURL *url.URL) {
-	apiError := s3err.GetAPIError(errorCode)
-	errorResponse := getRESTErrorResponse(apiError, reqURL.Path)
-	encodedErrorResponse := encodeResponse(errorResponse)
-	writeResponse(w, apiError.HTTPStatusCode, encodedErrorResponse, mimeXML)
-}
-
-func getRESTErrorResponse(err s3err.APIError, resource string) s3err.RESTErrorResponse {
-	return s3err.RESTErrorResponse{
-		Code:      err.Code,
-		Message:   err.Description,
-		Resource:  resource,
-		RequestID: fmt.Sprintf("%d", time.Now().UnixNano()),
-	}
-}
-
-func writeResponse(w http.ResponseWriter, statusCode int, response []byte, mType mimeType) {
-	setCommonHeaders(w)
-	if response != nil {
-		w.Header().Set("Content-Length", strconv.Itoa(len(response)))
-	}
-	if mType != mimeNone {
-		w.Header().Set("Content-Type", string(mType))
-	}
-	w.WriteHeader(statusCode)
-	if response != nil {
-		glog.V(4).Infof("status %d %s: %s", statusCode, mType, string(response))
-		_, err := w.Write(response)
-		if err != nil {
-			glog.V(0).Infof("write err: %v", err)
-		}
-		w.(http.Flusher).Flush()
-	}
-}
-
-func writeSuccessResponseXML(w http.ResponseWriter, response []byte) {
-	writeResponse(w, http.StatusOK, response, mimeXML)
+func writeSuccessResponseXML(w http.ResponseWriter, response interface{}) {
+	s3err.WriteXMLResponse(w, http.StatusOK, response)
 }

 func writeSuccessResponseEmpty(w http.ResponseWriter) {
-	writeResponse(w, http.StatusOK, nil, mimeNone)
+	s3err.WriteEmptyResponse(w, http.StatusOK)
 }

 func validateContentMd5(h http.Header) ([]byte, error) {
@@ -32,28 +32,28 @@ func (s3a *S3ApiServer) CopyObjectHandler(w http.ResponseWriter, r *http.Request
 		dir, name := fullPath.DirAndName()
 		entry, err := s3a.getEntry(dir, name)
 		if err != nil {
-			writeErrorResponse(w, s3err.ErrInvalidCopySource, r.URL)
+			s3err.WriteErrorResponse(w, s3err.ErrInvalidCopySource, r)
 		}
 		entry.Extended = weed_server.SaveAmzMetaData(r, entry.Extended, isReplace(r))
 		err = s3a.touch(dir, name, entry)
 		if err != nil {
-			writeErrorResponse(w, s3err.ErrInvalidCopySource, r.URL)
+			s3err.WriteErrorResponse(w, s3err.ErrInvalidCopySource, r)
 		}
-		writeSuccessResponseXML(w, encodeResponse(CopyObjectResult{
+		writeSuccessResponseXML(w, CopyObjectResult{
 			ETag:         fmt.Sprintf("%x", entry.Attributes.Md5),
 			LastModified: time.Now().UTC(),
-		}))
+		})
 		return
 	}

 	// If source object is empty or bucket is empty, reply back invalid copy source.
 	if srcObject == "" || srcBucket == "" {
-		writeErrorResponse(w, s3err.ErrInvalidCopySource, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrInvalidCopySource, r)
 		return
 	}

 	if srcBucket == dstBucket && srcObject == dstObject {
-		writeErrorResponse(w, s3err.ErrInvalidCopyDest, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrInvalidCopyDest, r)
 		return
 	}

@@ -64,7 +64,7 @@ func (s3a *S3ApiServer) CopyObjectHandler(w http.ResponseWriter, r *http.Request

 	_, _, resp, err := util.DownloadFile(srcUrl)
 	if err != nil {
-		writeErrorResponse(w, s3err.ErrInvalidCopySource, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrInvalidCopySource, r)
 		return
 	}
 	defer util.CloseResponse(resp)
@@ -73,7 +73,7 @@ func (s3a *S3ApiServer) CopyObjectHandler(w http.ResponseWriter, r *http.Request
 	etag, errCode := s3a.putToFiler(r, dstUrl, resp.Body)

 	if errCode != s3err.ErrNone {
-		writeErrorResponse(w, errCode, r.URL)
+		s3err.WriteErrorResponse(w, errCode, r)
 		return
 	}

@@ -84,7 +84,7 @@ func (s3a *S3ApiServer) CopyObjectHandler(w http.ResponseWriter, r *http.Request
 		LastModified: time.Now().UTC(),
 	}

-	writeSuccessResponseXML(w, encodeResponse(response))
+	writeSuccessResponseXML(w, response)

 }

@@ -117,7 +117,7 @@ func (s3a *S3ApiServer) CopyObjectPartHandler(w http.ResponseWriter, r *http.Req
 	srcBucket, srcObject := pathToBucketAndObject(cpSrcPath)
 	// If source object is empty or bucket is empty, reply back invalid copy source.
 	if srcObject == "" || srcBucket == "" {
-		writeErrorResponse(w, s3err.ErrInvalidCopySource, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrInvalidCopySource, r)
 		return
 	}

@@ -126,13 +126,13 @@ func (s3a *S3ApiServer) CopyObjectPartHandler(w http.ResponseWriter, r *http.Req

 	partID, err := strconv.Atoi(partIDString)
 	if err != nil {
-		writeErrorResponse(w, s3err.ErrInvalidPart, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrInvalidPart, r)
 		return
 	}

 	// check partID with maximum part ID for multipart objects
 	if partID > globalMaxPartID {
-		writeErrorResponse(w, s3err.ErrInvalidMaxParts, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrInvalidMaxParts, r)
 		return
 	}

@@ -145,7 +145,7 @@ func (s3a *S3ApiServer) CopyObjectPartHandler(w http.ResponseWriter, r *http.Req

 	dataReader, err := util.ReadUrlAsReaderCloser(srcUrl, rangeHeader)
 	if err != nil {
-		writeErrorResponse(w, s3err.ErrInvalidCopySource, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrInvalidCopySource, r)
 		return
 	}
 	defer dataReader.Close()
@@ -154,7 +154,7 @@ func (s3a *S3ApiServer) CopyObjectPartHandler(w http.ResponseWriter, r *http.Req
 	etag, errCode := s3a.putToFiler(r, dstUrl, dataReader)

 	if errCode != s3err.ErrNone {
-		writeErrorResponse(w, errCode, r.URL)
+		s3err.WriteErrorResponse(w, errCode, r)
 		return
 	}

@@ -165,7 +165,7 @@ func (s3a *S3ApiServer) CopyObjectPartHandler(w http.ResponseWriter, r *http.Req
 		LastModified: time.Now().UTC(),
 	}

-	writeSuccessResponseXML(w, encodeResponse(response))
+	writeSuccessResponseXML(w, response)

 }

@@ -6,12 +6,14 @@ import (
 	"encoding/xml"
 	"fmt"
 	"github.com/chrislusf/seaweedfs/weed/filer"
+	"github.com/pquerna/cachecontrol/cacheobject"
 	"io"
 	"io/ioutil"
 	"net/http"
 	"net/url"
 	"sort"
 	"strings"
+	"time"

 	"github.com/chrislusf/seaweedfs/weed/s3api/s3err"

@@ -42,13 +44,27 @@ func (s3a *S3ApiServer) PutObjectHandler(w http.ResponseWriter, r *http.Request)

 	_, err := validateContentMd5(r.Header)
 	if err != nil {
-		writeErrorResponse(w, s3err.ErrInvalidDigest, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrInvalidDigest, r)
 		return
 	}

+	if r.Header.Get("Cache-Control") != "" {
+		if _, err = cacheobject.ParseRequestCacheControl(r.Header.Get("Cache-Control")); err != nil {
+			s3err.WriteErrorResponse(w, s3err.ErrInvalidDigest, r)
+			return
+		}
+	}
+
+	if r.Header.Get("Expires") != "" {
+		if _, err = time.Parse(http.TimeFormat, r.Header.Get("Expires")); err != nil {
+			s3err.WriteErrorResponse(w, s3err.ErrInvalidDigest, r)
+			return
+		}
+	}
+
 	dataReader := r.Body
+	rAuthType := getRequestAuthType(r)
 	if s3a.iam.isEnabled() {
-		rAuthType := getRequestAuthType(r)
 		var s3ErrCode s3err.ErrorCode
 		switch rAuthType {
 		case authTypeStreamingSigned:
@@ -59,7 +75,12 @@ func (s3a *S3ApiServer) PutObjectHandler(w http.ResponseWriter, r *http.Request)
 			_, s3ErrCode = s3a.iam.reqSignatureV4Verify(r)
 		}
 		if s3ErrCode != s3err.ErrNone {
-			writeErrorResponse(w, s3ErrCode, r.URL)
+			s3err.WriteErrorResponse(w, s3ErrCode, r)
+			return
+		}
+	} else {
+		if authTypeStreamingSigned == rAuthType {
+			s3err.WriteErrorResponse(w, s3err.ErrAuthNotSetup, r)
 			return
 		}
 	}
@@ -67,7 +88,7 @@ func (s3a *S3ApiServer) PutObjectHandler(w http.ResponseWriter, r *http.Request)

 	if strings.HasSuffix(object, "/") {
 		if err := s3a.mkdir(s3a.option.BucketsPath, bucket+object, nil); err != nil {
-			writeErrorResponse(w, s3err.ErrInternalError, r.URL)
+			s3err.WriteErrorResponse(w, s3err.ErrInternalError, r)
 			return
 		}
 	} else {
@@ -76,7 +97,7 @@ func (s3a *S3ApiServer) PutObjectHandler(w http.ResponseWriter, r *http.Request)
 		etag, errCode := s3a.putToFiler(r, uploadUrl, dataReader)

 		if errCode != s3err.ErrNone {
-			writeErrorResponse(w, errCode, r.URL)
+			s3err.WriteErrorResponse(w, errCode, r)
 			return
 		}

@@ -99,7 +120,7 @@ func (s3a *S3ApiServer) GetObjectHandler(w http.ResponseWriter, r *http.Request)
 	bucket, object := getBucketAndObject(r)

 	if strings.HasSuffix(r.URL.Path, "/") {
-		writeErrorResponse(w, s3err.ErrNotImplemented, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrNotImplemented, r)
 		return
 	}

@@ -174,13 +195,13 @@ func (s3a *S3ApiServer) DeleteMultipleObjectsHandler(w http.ResponseWriter, r *h

 	deleteXMLBytes, err := ioutil.ReadAll(r.Body)
 	if err != nil {
-		writeErrorResponse(w, s3err.ErrInternalError, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrInternalError, r)
 		return
 	}

 	deleteObjects := &DeleteObjectsRequest{}
 	if err := xml.Unmarshal(deleteXMLBytes, deleteObjects); err != nil {
-		writeErrorResponse(w, s3err.ErrMalformedXML, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrMalformedXML, r)
 		return
 	}

@@ -232,7 +253,7 @@ func (s3a *S3ApiServer) DeleteMultipleObjectsHandler(w http.ResponseWriter, r *h
 	}
 	deleteResp.Errors = deleteErrors

-	writeSuccessResponseXML(w, encodeResponse(deleteResp))
+	writeSuccessResponseXML(w, deleteResp)

 }

@@ -276,7 +297,7 @@ func (s3a *S3ApiServer) proxyToFiler(w http.ResponseWriter, r *http.Request, des

 	if err != nil {
 		glog.Errorf("NewRequest %s: %v", destUrl, err)
-		writeErrorResponse(w, s3err.ErrInternalError, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrInternalError, r)
 		return
 	}

@@ -306,14 +327,19 @@ func (s3a *S3ApiServer) proxyToFiler(w http.ResponseWriter, r *http.Request, des

 	if postErr != nil {
 		glog.Errorf("post to filer: %v", postErr)
-		writeErrorResponse(w, s3err.ErrInternalError, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrInternalError, r)
 		return
 	}
 	defer util.CloseResponse(resp)

+	if resp.StatusCode == http.StatusPreconditionFailed {
+		s3err.WriteErrorResponse(w, s3err.ErrPreconditionFailed, r)
+		return
+	}
+
 	if (resp.ContentLength == -1 || resp.StatusCode == 404) && resp.StatusCode != 304 {
 		if r.Method != "DELETE" {
-			writeErrorResponse(w, s3err.ErrNoSuchKey, r.URL)
+			s3err.WriteErrorResponse(w, s3err.ErrNoSuchKey, r)
 			return
 		}
 	}
@@ -326,7 +352,11 @@ func passThroughResponse(proxyResponse *http.Response, w http.ResponseWriter) {
 	for k, v := range proxyResponse.Header {
 		w.Header()[k] = v
 	}
-	w.WriteHeader(proxyResponse.StatusCode)
+	if proxyResponse.Header.Get("Content-Range") != "" && proxyResponse.StatusCode == 200 {
+		w.WriteHeader(http.StatusPartialContent)
+	} else {
+		w.WriteHeader(proxyResponse.StatusCode)
+	}
 	io.Copy(w, proxyResponse.Body)
 }

@@ -26,23 +26,23 @@ func (s3a *S3ApiServer) PostPolicyBucketHandler(w http.ResponseWriter, r *http.R

 	reader, err := r.MultipartReader()
 	if err != nil {
-		writeErrorResponse(w, s3err.ErrMalformedPOSTRequest, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrMalformedPOSTRequest, r)
 		return
 	}
 	form, err := reader.ReadForm(int64(5 * humanize.MiByte))
 	if err != nil {
-		writeErrorResponse(w, s3err.ErrMalformedPOSTRequest, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrMalformedPOSTRequest, r)
 		return
 	}
 	defer form.RemoveAll()

 	fileBody, fileName, fileSize, formValues, err := extractPostPolicyFormValues(form)
 	if err != nil {
-		writeErrorResponse(w, s3err.ErrMalformedPOSTRequest, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrMalformedPOSTRequest, r)
 		return
 	}
 	if fileBody == nil {
-		writeErrorResponse(w, s3err.ErrPOSTFileRequired, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrPOSTFileRequired, r)
 		return
 	}
 	defer fileBody.Close()
@@ -60,7 +60,7 @@ func (s3a *S3ApiServer) PostPolicyBucketHandler(w http.ResponseWriter, r *http.R
 	if successRedirect != "" {
 		redirectURL, err = url.Parse(successRedirect)
 		if err != nil {
-			writeErrorResponse(w, s3err.ErrMalformedPOSTRequest, r.URL)
+			s3err.WriteErrorResponse(w, s3err.ErrMalformedPOSTRequest, r)
 			return
 		}
 	}
@@ -68,13 +68,13 @@ func (s3a *S3ApiServer) PostPolicyBucketHandler(w http.ResponseWriter, r *http.R
 	// Verify policy signature.
 	errCode := s3a.iam.doesPolicySignatureMatch(formValues)
 	if errCode != s3err.ErrNone {
-		writeErrorResponse(w, errCode, r.URL)
+		s3err.WriteErrorResponse(w, errCode, r)
 		return
 	}

 	policyBytes, err := base64.StdEncoding.DecodeString(formValues.Get("Policy"))
 	if err != nil {
-		writeErrorResponse(w, s3err.ErrMalformedPOSTRequest, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrMalformedPOSTRequest, r)
 		return
 	}

@@ -83,7 +83,7 @@ func (s3a *S3ApiServer) PostPolicyBucketHandler(w http.ResponseWriter, r *http.R

 		postPolicyForm, err := policy.ParsePostPolicyForm(string(policyBytes))
 		if err != nil {
-			writeErrorResponse(w, s3err.ErrPostPolicyConditionInvalidFormat, r.URL)
+			s3err.WriteErrorResponse(w, s3err.ErrPostPolicyConditionInvalidFormat, r)
 			return
 		}

@@ -99,12 +99,12 @@ func (s3a *S3ApiServer) PostPolicyBucketHandler(w http.ResponseWriter, r *http.R
 		lengthRange := postPolicyForm.Conditions.ContentLengthRange
 		if lengthRange.Valid {
 			if fileSize < lengthRange.Min {
-				writeErrorResponse(w, s3err.ErrEntityTooSmall, r.URL)
+				s3err.WriteErrorResponse(w, s3err.ErrEntityTooSmall, r)
 				return
 			}

 			if fileSize > lengthRange.Max {
-				writeErrorResponse(w, s3err.ErrEntityTooLarge, r.URL)
+				s3err.WriteErrorResponse(w, s3err.ErrEntityTooLarge, r)
 				return
 			}
 		}
@@ -115,7 +115,7 @@ func (s3a *S3ApiServer) PostPolicyBucketHandler(w http.ResponseWriter, r *http.R
 	etag, errCode := s3a.putToFiler(r, uploadUrl, fileBody)

 	if errCode != s3err.ErrNone {
-		writeErrorResponse(w, errCode, r.URL)
+		s3err.WriteErrorResponse(w, errCode, r)
 		return
 	}

@@ -123,7 +123,7 @@ func (s3a *S3ApiServer) PostPolicyBucketHandler(w http.ResponseWriter, r *http.R
 		// Replace raw query params..
 		redirectURL.RawQuery = getRedirectPostRawQuery(bucket, object, etag)
 		w.Header().Set("Location", redirectURL.String())
-		writeResponse(w, http.StatusSeeOther, nil, mimeNone)
+		s3err.WriteEmptyResponse(w, http.StatusSeeOther)
 		return
 	}

@@ -132,15 +132,15 @@ func (s3a *S3ApiServer) PostPolicyBucketHandler(w http.ResponseWriter, r *http.R
 	// Decide what http response to send depending on success_action_status parameter
 	switch successStatus {
 	case "201":
-		resp := encodeResponse(PostResponse{
+		resp := PostResponse{
 			Bucket:   bucket,
 			Key:      object,
 			ETag:     `"` + etag + `"`,
 			Location: w.Header().Get("Location"),
-		})
-		writeResponse(w, http.StatusCreated, resp, mimeXML)
+		}
+		s3err.WriteXMLResponse(w, http.StatusCreated, resp)
 	case "200":
-		writeResponse(w, http.StatusOK, nil, mimeNone)
+		s3err.WriteEmptyResponse(w, http.StatusOK)
 	default:
 		writeSuccessResponseEmpty(w)
 	}
@@ -29,14 +29,14 @@ func (s3a *S3ApiServer) NewMultipartUploadHandler(w http.ResponseWriter, r *http
 		Key:    objectKey(aws.String(object)),
 	})

-	glog.V(2).Info("NewMultipartUploadHandler", string(encodeResponse(response)), errCode)
+	glog.V(2).Info("NewMultipartUploadHandler", s3err.EncodeXMLResponse(response), errCode)

 	if errCode != s3err.ErrNone {
-		writeErrorResponse(w, errCode, r.URL)
+		s3err.WriteErrorResponse(w, errCode, r)
 		return
 	}

-	writeSuccessResponseXML(w, encodeResponse(response))
+	writeSuccessResponseXML(w, response)

 }

@@ -53,14 +53,14 @@ func (s3a *S3ApiServer) CompleteMultipartUploadHandler(w http.ResponseWriter, r
 		UploadId: aws.String(uploadID),
 	})

-	glog.V(2).Info("CompleteMultipartUploadHandler", string(encodeResponse(response)), errCode)
+	glog.V(2).Info("CompleteMultipartUploadHandler", s3err.EncodeXMLResponse(response), errCode)

 	if errCode != s3err.ErrNone {
-		writeErrorResponse(w, errCode, r.URL)
+		s3err.WriteErrorResponse(w, errCode, r)
 		return
 	}

-	writeSuccessResponseXML(w, encodeResponse(response))
+	writeSuccessResponseXML(w, response)

 }

@@ -78,13 +78,13 @@ func (s3a *S3ApiServer) AbortMultipartUploadHandler(w http.ResponseWriter, r *ht
 	})

 	if errCode != s3err.ErrNone {
-		writeErrorResponse(w, errCode, r.URL)
+		s3err.WriteErrorResponse(w, errCode, r)
 		return
 	}

-	glog.V(2).Info("AbortMultipartUploadHandler", string(encodeResponse(response)))
+	glog.V(2).Info("AbortMultipartUploadHandler", s3err.EncodeXMLResponse(response))

-	writeSuccessResponseXML(w, encodeResponse(response))
+	writeSuccessResponseXML(w, response)

 }

@@ -94,13 +94,13 @@ func (s3a *S3ApiServer) ListMultipartUploadsHandler(w http.ResponseWriter, r *ht

 	prefix, keyMarker, uploadIDMarker, delimiter, maxUploads, encodingType := getBucketMultipartResources(r.URL.Query())
 	if maxUploads < 0 {
-		writeErrorResponse(w, s3err.ErrInvalidMaxUploads, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrInvalidMaxUploads, r)
 		return
 	}
 	if keyMarker != "" {
 		// Marker not common with prefix is not implemented.
 		if !strings.HasPrefix(keyMarker, prefix) {
-			writeErrorResponse(w, s3err.ErrNotImplemented, r.URL)
+			s3err.WriteErrorResponse(w, s3err.ErrNotImplemented, r)
 			return
 		}
 	}
@@ -115,16 +115,16 @@ func (s3a *S3ApiServer) ListMultipartUploadsHandler(w http.ResponseWriter, r *ht
 		UploadIdMarker: aws.String(uploadIDMarker),
 	})

-	glog.V(2).Info("ListMultipartUploadsHandler", string(encodeResponse(response)), errCode)
+	glog.V(2).Info("ListMultipartUploadsHandler", s3err.EncodeXMLResponse(response), errCode)

 	if errCode != s3err.ErrNone {
-		writeErrorResponse(w, errCode, r.URL)
+		s3err.WriteErrorResponse(w, errCode, r)
 		return
 	}

 	// TODO handle encodingType

-	writeSuccessResponseXML(w, encodeResponse(response))
+	writeSuccessResponseXML(w, response)
 }

 // ListObjectPartsHandler - Lists object parts in a multipart upload.
@@ -133,11 +133,11 @@ func (s3a *S3ApiServer) ListObjectPartsHandler(w http.ResponseWriter, r *http.Re

 	uploadID, partNumberMarker, maxParts, _ := getObjectResources(r.URL.Query())
 	if partNumberMarker < 0 {
-		writeErrorResponse(w, s3err.ErrInvalidPartNumberMarker, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrInvalidPartNumberMarker, r)
 		return
 	}
 	if maxParts < 0 {
-		writeErrorResponse(w, s3err.ErrInvalidMaxParts, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrInvalidMaxParts, r)
 		return
 	}

@@ -149,14 +149,14 @@ func (s3a *S3ApiServer) ListObjectPartsHandler(w http.ResponseWriter, r *http.Re
 		UploadId:         aws.String(uploadID),
 	})

-	glog.V(2).Info("ListObjectPartsHandler", string(encodeResponse(response)), errCode)
+	glog.V(2).Info("ListObjectPartsHandler", s3err.EncodeXMLResponse(response), errCode)

 	if errCode != s3err.ErrNone {
-		writeErrorResponse(w, errCode, r.URL)
+		s3err.WriteErrorResponse(w, errCode, r)
 		return
 	}

-	writeSuccessResponseXML(w, encodeResponse(response))
+	writeSuccessResponseXML(w, response)

 }

@@ -167,18 +167,18 @@ func (s3a *S3ApiServer) PutObjectPartHandler(w http.ResponseWriter, r *http.Requ
 	uploadID := r.URL.Query().Get("uploadId")
 	exists, err := s3a.exists(s3a.genUploadsFolder(bucket), uploadID, true)
 	if !exists {
-		writeErrorResponse(w, s3err.ErrNoSuchUpload, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrNoSuchUpload, r)
 		return
 	}

 	partIDString := r.URL.Query().Get("partNumber")
 	partID, err := strconv.Atoi(partIDString)
 	if err != nil {
-		writeErrorResponse(w, s3err.ErrInvalidPart, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrInvalidPart, r)
 		return
 	}
 	if partID > globalMaxPartID {
-		writeErrorResponse(w, s3err.ErrInvalidMaxParts, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrInvalidMaxParts, r)
 		return
 	}

@@ -195,7 +195,7 @@ func (s3a *S3ApiServer) PutObjectPartHandler(w http.ResponseWriter, r *http.Requ
 			_, s3ErrCode = s3a.iam.reqSignatureV4Verify(r)
 		}
 		if s3ErrCode != s3err.ErrNone {
-			writeErrorResponse(w, s3ErrCode, r.URL)
+			s3err.WriteErrorResponse(w, s3ErrCode, r)
 			return
 		}
 	}
@@ -207,7 +207,7 @@ func (s3a *S3ApiServer) PutObjectPartHandler(w http.ResponseWriter, r *http.Requ
 	etag, errCode := s3a.putToFiler(r, uploadUrl, dataReader)

 	if errCode != s3err.ErrNone {
-		writeErrorResponse(w, errCode, r.URL)
+		s3err.WriteErrorResponse(w, errCode, r)
 		return
 	}

@@ -25,15 +25,15 @@ func (s3a *S3ApiServer) GetObjectTaggingHandler(w http.ResponseWriter, r *http.R
 	if err != nil {
 		if err == filer_pb.ErrNotFound {
 			glog.Errorf("GetObjectTaggingHandler %s: %v", r.URL, err)
-			writeErrorResponse(w, s3err.ErrNoSuchKey, r.URL)
+			s3err.WriteErrorResponse(w, s3err.ErrNoSuchKey, r)
 		} else {
 			glog.Errorf("GetObjectTaggingHandler %s: %v", r.URL, err)
-			writeErrorResponse(w, s3err.ErrInternalError, r.URL)
+			s3err.WriteErrorResponse(w, s3err.ErrInternalError, r)
 		}
 		return
 	}

-	writeSuccessResponseXML(w, encodeResponse(FromTags(tags)))
+	writeSuccessResponseXML(w, FromTags(tags))

 }

@@ -50,29 +50,29 @@ func (s3a *S3ApiServer) PutObjectTaggingHandler(w http.ResponseWriter, r *http.R
 	input, err := ioutil.ReadAll(io.LimitReader(r.Body, r.ContentLength))
 	if err != nil {
 		glog.Errorf("PutObjectTaggingHandler read input %s: %v", r.URL, err)
-		writeErrorResponse(w, s3err.ErrInternalError, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrInternalError, r)
 		return
 	}
 	if err = xml.Unmarshal(input, tagging); err != nil {
 		glog.Errorf("PutObjectTaggingHandler Unmarshal %s: %v", r.URL, err)
-		writeErrorResponse(w, s3err.ErrMalformedXML, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrMalformedXML, r)
 		return
 	}
 	tags := tagging.ToTags()
 	if len(tags) > 10 {
 		glog.Errorf("PutObjectTaggingHandler tags %s: %d tags more than 10", r.URL, len(tags))
-		writeErrorResponse(w, s3err.ErrInvalidTag, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrInvalidTag, r)
 		return
 	}
 	for k, v := range tags {
 		if len(k) > 128 {
 			glog.Errorf("PutObjectTaggingHandler tags %s: tag key %s longer than 128", r.URL, k)
-			writeErrorResponse(w, s3err.ErrInvalidTag, r.URL)
+			s3err.WriteErrorResponse(w, s3err.ErrInvalidTag, r)
 			return
 		}
 		if len(v) > 256 {
 			glog.Errorf("PutObjectTaggingHandler tags %s: tag value %s longer than 256", r.URL, v)
-			writeErrorResponse(w, s3err.ErrInvalidTag, r.URL)
+			s3err.WriteErrorResponse(w, s3err.ErrInvalidTag, r)
 			return
 		}
 	}
@@ -80,10 +80,10 @@ func (s3a *S3ApiServer) PutObjectTaggingHandler(w http.ResponseWriter, r *http.R
 	if err = s3a.setTags(dir, name, tagging.ToTags()); err != nil {
 		if err == filer_pb.ErrNotFound {
 			glog.Errorf("PutObjectTaggingHandler setTags %s: %v", r.URL, err)
-			writeErrorResponse(w, s3err.ErrNoSuchKey, r.URL)
+			s3err.WriteErrorResponse(w, s3err.ErrNoSuchKey, r)
 		} else {
 			glog.Errorf("PutObjectTaggingHandler setTags %s: %v", r.URL, err)
-			writeErrorResponse(w, s3err.ErrInternalError, r.URL)
+			s3err.WriteErrorResponse(w, s3err.ErrInternalError, r)
 		}
 		return
 	}
@@ -105,10 +105,10 @@ func (s3a *S3ApiServer) DeleteObjectTaggingHandler(w http.ResponseWriter, r *htt
 	if err != nil {
 		if err == filer_pb.ErrNotFound {
 			glog.Errorf("DeleteObjectTaggingHandler %s: %v", r.URL, err)
-			writeErrorResponse(w, s3err.ErrNoSuchKey, r.URL)
+			s3err.WriteErrorResponse(w, s3err.ErrNoSuchKey, r)
 		} else {
 			glog.Errorf("DeleteObjectTaggingHandler %s: %v", r.URL, err)
-			writeErrorResponse(w, s3err.ErrInternalError, r.URL)
+			s3err.WriteErrorResponse(w, s3err.ErrInternalError, r)
 		}
 		return
 	}
@@ -44,11 +44,11 @@ func (s3a *S3ApiServer) ListObjectsV2Handler(w http.ResponseWriter, r *http.Requ
 	originalPrefix, continuationToken, startAfter, delimiter, _, maxKeys := getListObjectsV2Args(r.URL.Query())

 	if maxKeys < 0 {
-		writeErrorResponse(w, s3err.ErrInvalidMaxKeys, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrInvalidMaxKeys, r)
 		return
 	}
 	if delimiter != "" && delimiter != "/" {
-		writeErrorResponse(w, s3err.ErrNotImplemented, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrNotImplemented, r)
 		return
 	}

@@ -60,13 +60,13 @@ func (s3a *S3ApiServer) ListObjectsV2Handler(w http.ResponseWriter, r *http.Requ
 	response, err := s3a.listFilerEntries(bucket, originalPrefix, maxKeys, marker, delimiter)

 	if err != nil {
-		writeErrorResponse(w, s3err.ErrInternalError, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrInternalError, r)
 		return
 	}

 	if len(response.Contents) == 0 {
 		if exists, existErr := s3a.exists(s3a.option.BucketsPath, bucket, true); existErr == nil && !exists {
-			writeErrorResponse(w, s3err.ErrNoSuchBucket, r.URL)
+			s3err.WriteErrorResponse(w, s3err.ErrNoSuchBucket, r)
 			return
 		}
 	}
@@ -86,7 +86,7 @@ func (s3a *S3ApiServer) ListObjectsV2Handler(w http.ResponseWriter, r *http.Requ
 		StartAfter:            startAfter,
 	}

-	writeSuccessResponseXML(w, encodeResponse(responseV2))
+	writeSuccessResponseXML(w, responseV2)
 }

 func (s3a *S3ApiServer) ListObjectsV1Handler(w http.ResponseWriter, r *http.Request) {
@@ -99,29 +99,29 @@ func (s3a *S3ApiServer) ListObjectsV1Handler(w http.ResponseWriter, r *http.Requ
 	originalPrefix, marker, delimiter, maxKeys := getListObjectsV1Args(r.URL.Query())

 	if maxKeys < 0 {
-		writeErrorResponse(w, s3err.ErrInvalidMaxKeys, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrInvalidMaxKeys, r)
 		return
 	}
 	if delimiter != "" && delimiter != "/" {
-		writeErrorResponse(w, s3err.ErrNotImplemented, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrNotImplemented, r)
 		return
 	}

 	response, err := s3a.listFilerEntries(bucket, originalPrefix, maxKeys, marker, delimiter)

 	if err != nil {
-		writeErrorResponse(w, s3err.ErrInternalError, r.URL)
+		s3err.WriteErrorResponse(w, s3err.ErrInternalError, r)
 		return
 	}

 	if len(response.Contents) == 0 {
 		if exists, existErr := s3a.exists(s3a.option.BucketsPath, bucket, true); existErr == nil && !exists {
-			writeErrorResponse(w, s3err.ErrNoSuchBucket, r.URL)
+			s3err.WriteErrorResponse(w, s3err.ErrNoSuchBucket, r)
 			return
 		}
 	}

-	writeSuccessResponseXML(w, encodeResponse(response))
+	writeSuccessResponseXML(w, response)
 }

 func (s3a *S3ApiServer) listFilerEntries(bucket string, originalPrefix string, maxKeys int, marker string, delimiter string) (response ListBucketResult, err error) {
@@ -1,6 +1,7 @@
 package s3api

 import (
+	"github.com/chrislusf/seaweedfs/weed/s3api/s3err"
 	"testing"
 	"time"
 )
@@ -31,7 +32,7 @@ func TestListObjectsHandler(t *testing.T) {
 		}},
 	}

-	encoded := string(encodeResponse(response))
+	encoded := string(s3err.EncodeXMLResponse(response))
 	if encoded != expected {
 		t.Errorf("unexpected output: %s\nexpecting:%s", encoded, expected)
 	}
@@ -4,6 +4,7 @@ import (
 	"fmt"
 	"github.com/chrislusf/seaweedfs/weed/filer"
 	. "github.com/chrislusf/seaweedfs/weed/s3api/s3_constants"
+	"github.com/chrislusf/seaweedfs/weed/s3api/s3err"
 	"net/http"
 	"strings"
 	"time"
@@ -132,6 +133,6 @@ func (s3a *S3ApiServer) registerRouter(router *mux.Router) {
 	apiRouter.Methods("GET").Path("/").HandlerFunc(track(s3a.ListBucketsHandler, "LIST"))

 	// NotFound
-	apiRouter.NotFoundHandler = http.HandlerFunc(notFoundHandler)
+	apiRouter.NotFoundHandler = http.HandlerFunc(s3err.NotFoundHandler)

 }
@@ -1,6 +1,7 @@
 package s3api

 import (
+	"github.com/chrislusf/seaweedfs/weed/s3api/s3err"
 	"testing"
 	"time"
 )
@@ -14,7 +15,7 @@ func TestCopyObjectResponse(t *testing.T) {
 		LastModified: time.Now(),
 	}

-	println(string(encodeResponse(response)))
+	println(string(s3err.EncodeXMLResponse(response)))

 }

@@ -27,6 +28,6 @@ func TestCopyPartResponse(t *testing.T) {
 		LastModified: time.Now(),
 	}

-	println(string(encodeResponse(response)))
+	println(string(s3err.EncodeXMLResponse(response)))

 }
@@ -0,0 +1,92 @@
+package s3err
+
+import (
+	"bytes"
+	"encoding/xml"
+	"fmt"
+	"github.com/chrislusf/seaweedfs/weed/glog"
+	"github.com/gorilla/mux"
+	"net/http"
+	"strconv"
+	"strings"
+	"time"
+)
+
+type mimeType string
+
+const (
+	mimeNone mimeType = ""
+	MimeXML  mimeType = "application/xml"
+)
+
+func WriteXMLResponse(w http.ResponseWriter, statusCode int, response interface{}) {
+	WriteResponse(w, statusCode, EncodeXMLResponse(response), MimeXML)
+}
+
+func WriteEmptyResponse(w http.ResponseWriter, statusCode int) {
+	WriteResponse(w, statusCode, []byte{}, mimeNone)
+}
+
+func WriteErrorResponse(w http.ResponseWriter, errorCode ErrorCode, r *http.Request) {
+	vars := mux.Vars(r)
+	bucket := vars["bucket"]
+	object := vars["object"]
+	if strings.HasPrefix(object, "/") {
+		object = object[1:]
+	}
+
+	apiError := GetAPIError(errorCode)
+	errorResponse := getRESTErrorResponse(apiError, r.URL.Path, bucket, object)
+	encodedErrorResponse := EncodeXMLResponse(errorResponse)
+	WriteResponse(w, apiError.HTTPStatusCode, encodedErrorResponse, MimeXML)
+}
+
+func getRESTErrorResponse(err APIError, resource string, bucket, object string) RESTErrorResponse {
+	return RESTErrorResponse{
+		Code:       err.Code,
+		BucketName: bucket,
+		Key:        object,
+		Message:    err.Description,
+		Resource:   resource,
+		RequestID:  fmt.Sprintf("%d", time.Now().UnixNano()),
+	}
+}
+
+// Encodes the response headers into XML format.
+func EncodeXMLResponse(response interface{}) []byte {
+	var bytesBuffer bytes.Buffer
+	bytesBuffer.WriteString(xml.Header)
+	e := xml.NewEncoder(&bytesBuffer)
+	e.Encode(response)
+	return bytesBuffer.Bytes()
+}
+
+func setCommonHeaders(w http.ResponseWriter) {
+	w.Header().Set("x-amz-request-id", fmt.Sprintf("%d", time.Now().UnixNano()))
+	w.Header().Set("Accept-Ranges", "bytes")
+}
+
+func WriteResponse(w http.ResponseWriter, statusCode int, response []byte, mType mimeType) {
+	setCommonHeaders(w)
+	if response != nil {
+		w.Header().Set("Content-Length", strconv.Itoa(len(response)))
+	}
+	if mType != mimeNone {
+		w.Header().Set("Content-Type", string(mType))
+	}
+	w.WriteHeader(statusCode)
+	if response != nil {
+		glog.V(4).Infof("status %d %s: %s", statusCode, mType, string(response))
+		_, err := w.Write(response)
+		if err != nil {
+			glog.V(0).Infof("write err: %v", err)
+		}
+		w.(http.Flusher).Flush()
+	}
+}
+
+// If none of the http routes match respond with MethodNotAllowed
+func NotFoundHandler(w http.ResponseWriter, r *http.Request) {
+	glog.V(0).Infof("unsupported %s %s", r.Method, r.RequestURI)
+	WriteErrorResponse(w, ErrMethodNotAllowed, r)
+}
@@ -15,11 +15,13 @@ type APIError struct {

 // RESTErrorResponse - error response format
 type RESTErrorResponse struct {
-	XMLName   xml.Name `xml:"Error" json:"-"`
-	Code      string   `xml:"Code" json:"Code"`
-	Message   string   `xml:"Message" json:"Message"`
-	Resource  string   `xml:"Resource" json:"Resource"`
-	RequestID string   `xml:"RequestId" json:"RequestId"`
+	XMLName    xml.Name `xml:"Error" json:"-"`
+	Code       string   `xml:"Code" json:"Code"`
+	Message    string   `xml:"Message" json:"Message"`
+	Resource   string   `xml:"Resource" json:"Resource"`
+	RequestID  string   `xml:"RequestId" json:"RequestId"`
+	Key        string   `xml:"Key,omitempty" json:"Key,omitempty"`
+	BucketName string   `xml:"BucketName,omitempty" json:"BucketName,omitempty"`

 	// Underlying HTTP status code for the returned error
 	StatusCode int `xml:"-" json:"-"`
@@ -91,7 +93,9 @@ const (
 	ErrRequestNotReadyYet
 	ErrMissingDateHeader
 	ErrInvalidRequest
+	ErrAuthNotSetup
 	ErrNotImplemented
+	ErrPreconditionFailed

 	ErrExistingObjectIsDirectory
 )
@@ -341,11 +345,21 @@ var errorCodeResponse = map[ErrorCode]APIError{
 		Description:    "Invalid Request",
 		HTTPStatusCode: http.StatusBadRequest,
 	},
+	ErrAuthNotSetup: {
+		Code:           "InvalidRequest",
+		Description:    "Signed request requires setting up SeaweedFS S3 authentication",
+		HTTPStatusCode: http.StatusBadRequest,
+	},
 	ErrNotImplemented: {
 		Code:           "NotImplemented",
 		Description:    "A header you provided implies functionality that is not implemented",
 		HTTPStatusCode: http.StatusNotImplemented,
 	},
+	ErrPreconditionFailed: {
+		Code:           "PreconditionFailed",
+		Description:    "At least one of the pre-conditions you specified did not hold",
+		HTTPStatusCode: http.StatusPreconditionFailed,
+	},
 	ErrExistingObjectIsDirectory: {
 		Code:           "ExistingObjectIsDirectory",
 		Description:    "Existing Object is a directory.",
@@ -2,6 +2,7 @@ package s3api

 import (
 	"encoding/xml"
+	"github.com/chrislusf/seaweedfs/weed/s3api/s3err"
 	"github.com/stretchr/testify/assert"
 	"testing"
 )
@@ -41,7 +42,7 @@ func TestXMLMarshall(t *testing.T) {
 		},
 	}

-	actual := string(encodeResponse(tags))
+	actual := string(s3err.EncodeXMLResponse(tags))

 	expected := `<?xml version="1.0" encoding="UTF-8"?>
 <Tagging xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><TagSet><Tag><Key>key1</Key><Value>value1</Value></Tag></TagSet></Tagging>`
@@ -1,6 +1,7 @@
 package weed_server

 import (
+	"bytes"
 	"encoding/json"
 	"errors"
 	"fmt"
@@ -104,7 +105,9 @@ func submitForClientHandler(w http.ResponseWriter, r *http.Request, masterFn ope
 	}

 	debug("parsing upload file...")
-	pu, pe := needle.ParseUpload(r, 256*1024*1024)
+	bytesBuffer := bufPool.Get().(*bytes.Buffer)
+	defer bufPool.Put(bytesBuffer)
+	pu, pe := needle.ParseUpload(r, 256*1024*1024, bytesBuffer)
 	if pe != nil {
 		writeJsonError(w, r, http.StatusBadRequest, pe)
 		return
@@ -232,12 +235,12 @@ func adjustHeaderContentDisposition(w http.ResponseWriter, r *http.Request, file
 	}
 }

-func processRangeRequest(r *http.Request, w http.ResponseWriter, totalSize int64, mimeType string, writeFn func(writer io.Writer, offset int64, size int64, httpStatusCode int) error) {
+func processRangeRequest(r *http.Request, w http.ResponseWriter, totalSize int64, mimeType string, writeFn func(writer io.Writer, offset int64, size int64) error) {
 	rangeReq := r.Header.Get("Range")

 	if rangeReq == "" {
 		w.Header().Set("Content-Length", strconv.FormatInt(totalSize, 10))
-		if err := writeFn(w, 0, totalSize, 0); err != nil {
+		if err := writeFn(w, 0, totalSize); err != nil {
 			http.Error(w, err.Error(), http.StatusInternalServerError)
 			return
 		}
@@ -277,7 +280,7 @@ func processRangeRequest(r *http.Request, w http.ResponseWriter, totalSize int64
 		w.Header().Set("Content-Length", strconv.FormatInt(ra.length, 10))
 		w.Header().Set("Content-Range", ra.contentRange(totalSize))

-		err = writeFn(w, ra.start, ra.length, http.StatusPartialContent)
+		err = writeFn(w, ra.start, ra.length)
 		if err != nil {
 			http.Error(w, err.Error(), http.StatusInternalServerError)
 			return
@@ -305,7 +308,7 @@ func processRangeRequest(r *http.Request, w http.ResponseWriter, totalSize int64
 				pw.CloseWithError(e)
 				return
 			}
-			if e = writeFn(part, ra.start, ra.length, 0); e != nil {
+			if e = writeFn(part, ra.start, ra.length); e != nil {
 				pw.CloseWithError(e)
 				return
 			}
@@ -259,14 +259,14 @@ func (fs *FilerServer) cleanupChunks(fullpath string, existingEntry *filer.Entry
 	garbage = append(garbage, coveredChunks...)

 	if newEntry.Attributes != nil {
-		so := fs.detectStorageOption(fullpath,
+		so, _ := fs.detectStorageOption(fullpath,
 			newEntry.Attributes.Collection,
 			newEntry.Attributes.Replication,
 			newEntry.Attributes.TtlSec,
 			newEntry.Attributes.DiskType,
 			"",
 			"",
-		)
+		) // ignore readonly error for capacity needed to manifestize
 		chunks, err = filer.MaybeManifestize(fs.saveAsChunk(so), chunks)
 		if err != nil {
 			// not good, but should be ok
@@ -307,7 +307,11 @@ func (fs *FilerServer) AppendToEntry(ctx context.Context, req *filer_pb.AppendTo
 	}

 	entry.Chunks = append(entry.Chunks, req.Chunks...)
-	so := fs.detectStorageOption(string(fullpath), entry.Collection, entry.Replication, entry.TtlSec, entry.DiskType, "", "")
+	so, err := fs.detectStorageOption(string(fullpath), entry.Collection, entry.Replication, entry.TtlSec, entry.DiskType, "", "")
+	if err != nil {
+		glog.Warningf("detectStorageOption: %v", err)
+		return &filer_pb.AppendToEntryResponse{}, err
+	}
 	entry.Chunks, err = filer.MaybeManifestize(fs.saveAsChunk(so), entry.Chunks)
 	if err != nil {
 		// not good, but should be ok
@@ -333,7 +337,11 @@ func (fs *FilerServer) DeleteEntry(ctx context.Context, req *filer_pb.DeleteEntr

 func (fs *FilerServer) AssignVolume(ctx context.Context, req *filer_pb.AssignVolumeRequest) (resp *filer_pb.AssignVolumeResponse, err error) {

-	so := fs.detectStorageOption(req.Path, req.Collection, req.Replication, req.TtlSec, req.DiskType, req.DataCenter, req.Rack)
+	so, err := fs.detectStorageOption(req.Path, req.Collection, req.Replication, req.TtlSec, req.DiskType, req.DataCenter, req.Rack)
+	if err != nil {
+		glog.V(3).Infof("AssignVolume: %v", err)
+		return &filer_pb.AssignVolumeResponse{Error: fmt.Sprintf("assign volume: %v", err)}, nil
+	}

 	assignRequest, altRequest := so.ToAssignRequests(int(req.Count))

@@ -115,8 +115,7 @@ func (fs *FilerServer) moveSelfEntry(ctx context.Context, oldParent util.FullPat
 		Extended: entry.Extended,
 		Content:  entry.Content,
 	}
-	createErr := fs.filer.CreateEntry(ctx, newEntry, false, false, nil)
-	if createErr != nil {
+	if createErr := fs.filer.CreateEntry(ctx, newEntry, false, false, nil); createErr != nil {
 		return createErr
 	}

@@ -30,6 +30,7 @@ import (
 	_ "github.com/chrislusf/seaweedfs/weed/filer/mongodb"
 	_ "github.com/chrislusf/seaweedfs/weed/filer/mysql"
 	_ "github.com/chrislusf/seaweedfs/weed/filer/mysql2"
+	_ "github.com/chrislusf/seaweedfs/weed/filer/sqlite"
 	_ "github.com/chrislusf/seaweedfs/weed/filer/postgres"
 	_ "github.com/chrislusf/seaweedfs/weed/filer/postgres2"
 	_ "github.com/chrislusf/seaweedfs/weed/filer/redis"
@@ -128,7 +129,7 @@ func NewFilerServer(defaultMux, readonlyMux *http.ServeMux, option *FilerOption)
 	fs.filer.DirBucketsPath = v.GetString("filer.options.buckets_folder")
 	// TODO deprecated, will be be removed after 2020-12-31
 	// replaced by https://github.com/chrislusf/seaweedfs/wiki/Path-Specific-Configuration
-	fs.filer.FsyncBuckets = v.GetStringSlice("filer.options.buckets_fsync")
+	// fs.filer.FsyncBuckets = v.GetStringSlice("filer.options.buckets_fsync")
 	fs.filer.LoadConfiguration(v)

 	notification.LoadConfiguration(v, "notification.")
@@ -1,6 +1,7 @@
 package weed_server

 import (
+	"github.com/chrislusf/seaweedfs/weed/glog"
 	"github.com/chrislusf/seaweedfs/weed/util"
 	"net/http"
 	"strings"
@@ -53,7 +54,8 @@ func (fs *FilerServer) filerHandler(w http.ResponseWriter, r *http.Request) {
 		// wait until in flight data is less than the limit
 		contentLength := getContentLength(r)
 		fs.inFlightDataLimitCond.L.Lock()
-		for atomic.LoadInt64(&fs.inFlightDataSize) > fs.option.ConcurrentUploadLimit {
+		for fs.option.ConcurrentUploadLimit != 0 && atomic.LoadInt64(&fs.inFlightDataSize) > fs.option.ConcurrentUploadLimit {
+			glog.V(4).Infof("wait because inflight data %d > %d", fs.inFlightDataSize, fs.option.ConcurrentUploadLimit)
 			fs.inFlightDataLimitCond.Wait()
 		}
 		atomic.AddInt64(&fs.inFlightDataSize, contentLength)
@@ -61,6 +61,13 @@ func (fs *FilerServer) GetOrHeadHandler(w http.ResponseWriter, r *http.Request,
 		return
 	}

+	// set etag
+	etag := filer.ETagEntry(entry)
+	if ifm := r.Header.Get("If-Match"); ifm != "" && ifm != "\""+etag+"\"" {
+		w.WriteHeader(http.StatusPreconditionFailed)
+		return
+	}
+
 	w.Header().Set("Accept-Ranges", "bytes")

 	// mime type
@@ -115,8 +122,6 @@ func (fs *FilerServer) GetOrHeadHandler(w http.ResponseWriter, r *http.Request,
 		}
 	}

-	// set etag
-	etag := filer.ETagEntry(entry)
 	if inm := r.Header.Get("If-None-Match"); inm == "\""+etag+"\"" {
 		w.WriteHeader(http.StatusNotModified)
 		return
@@ -150,10 +155,7 @@ func (fs *FilerServer) GetOrHeadHandler(w http.ResponseWriter, r *http.Request,
 		}
 	}

-	processRangeRequest(r, w, totalSize, mimeType, func(writer io.Writer, offset int64, size int64, httpStatusCode int) error {
-		if httpStatusCode != 0 {
-			w.WriteHeader(httpStatusCode)
-		}
+	processRangeRequest(r, w, totalSize, mimeType, func(writer io.Writer, offset int64, size int64) error {
 		if offset+size <= int64(len(entry.Content)) {
 			_, err := writer.Write(entry.Content[offset : offset+size])
 			if err != nil {
@@ -161,7 +163,10 @@ func (fs *FilerServer) GetOrHeadHandler(w http.ResponseWriter, r *http.Request,
 			}
 			return err
 		}
-		return filer.StreamContent(fs.filer.MasterClient, writer, entry.Chunks, offset, size)
+		err = filer.StreamContent(fs.filer.MasterClient, writer, entry.Chunks, offset, size)
+		if err != nil {
+			glog.Errorf("failed to stream content %s: %v", r.URL, err)
+		}
+		return err
 	})
-
 }
@@ -2,6 +2,7 @@ package weed_server

 import (
 	"context"
+	"errors"
 	"net/http"
 	"os"
 	"strings"
@@ -19,6 +20,8 @@ import (
 var (
 	OS_UID = uint32(os.Getuid())
 	OS_GID = uint32(os.Getgid())
+
+	ErrReadOnly = errors.New("read only")
 )

 type FilerPostResult struct {
@@ -57,7 +60,7 @@ func (fs *FilerServer) PostHandler(w http.ResponseWriter, r *http.Request, conte
 	ctx := context.Background()

 	query := r.URL.Query()
-	so := fs.detectStorageOption0(r.RequestURI,
+	so, err := fs.detectStorageOption0(r.RequestURI,
 		query.Get("collection"),
 		query.Get("replication"),
 		query.Get("ttl"),
@@ -65,6 +68,15 @@ func (fs *FilerServer) PostHandler(w http.ResponseWriter, r *http.Request, conte
 		query.Get("dataCenter"),
 		query.Get("rack"),
 	)
+	if err != nil {
+		if err == ErrReadOnly {
+			w.WriteHeader(http.StatusInsufficientStorage)
+		} else {
+			glog.V(1).Infoln("post", r.RequestURI, ":", err.Error())
+			w.WriteHeader(http.StatusInternalServerError)
+		}
+		return
+	}

 	fs.autoChunk(ctx, w, r, contentLength, so)
 	util.CloseRequest(r)
@@ -105,7 +117,7 @@ func (fs *FilerServer) DeleteHandler(w http.ResponseWriter, r *http.Request) {
 	w.WriteHeader(http.StatusNoContent)
 }

-func (fs *FilerServer) detectStorageOption(requestURI, qCollection, qReplication string, ttlSeconds int32, diskType string, dataCenter, rack string) *operation.StorageOption {
+func (fs *FilerServer) detectStorageOption(requestURI, qCollection, qReplication string, ttlSeconds int32, diskType, dataCenter, rack string) (*operation.StorageOption, error) {
 	collection := util.Nvl(qCollection, fs.option.Collection)
 	replication := util.Nvl(qReplication, fs.option.DefaultReplication)

@@ -121,6 +133,10 @@ func (fs *FilerServer) detectStorageOption(requestURI, qCollection, qReplication

 	rule := fs.filer.FilerConf.MatchStorageRule(requestURI)

+	if rule.ReadOnly {
+		return nil, ErrReadOnly
+	}
+
 	if ttlSeconds == 0 {
 		ttl, err := needle.ReadTTL(rule.GetTtl())
 		if err != nil {
@@ -138,10 +154,10 @@ func (fs *FilerServer) detectStorageOption(requestURI, qCollection, qReplication
 		DiskType:          util.Nvl(diskType, rule.DiskType),
 		Fsync:             fsync || rule.Fsync,
 		VolumeGrowthCount: rule.VolumeGrowthCount,
-	}
+	}, nil
 }

-func (fs *FilerServer) detectStorageOption0(requestURI, qCollection, qReplication string, qTtl string, diskType string, dataCenter, rack string) *operation.StorageOption {
+func (fs *FilerServer) detectStorageOption0(requestURI, qCollection, qReplication string, qTtl string, diskType string, dataCenter, rack string) (*operation.StorageOption, error) {

 	ttl, err := needle.ReadTTL(qTtl)
 	if err != nil {
@@ -218,10 +218,10 @@ func (fs *FilerServer) saveMetaData(ctx context.Context, r *http.Request, fileNa
 		entry.Extended = make(map[string][]byte)
 	}

-	SaveAmzMetaData(r, entry.Extended, false)
+	entry.Extended = SaveAmzMetaData(r, entry.Extended, false)

 	for k, v := range r.Header {
-		if len(v) > 0 && strings.HasPrefix(k, needle.PairNamePrefix) {
+		if len(v) > 0 && (strings.HasPrefix(k, needle.PairNamePrefix) || k == "Cache-Control" || k == "Expires") {
 			entry.Extended[k] = []byte(v[0])
 		}
 	}
@@ -1,6 +1,7 @@
 package weed_server

 import (
+	"bytes"
 	"context"
 	"fmt"
 	"net/http"
@@ -30,7 +31,10 @@ func (fs *FilerServer) encrypt(ctx context.Context, w http.ResponseWriter, r *ht

 	sizeLimit := int64(fs.option.MaxMB) * 1024 * 1024

-	pu, err := needle.ParseUpload(r, sizeLimit)
+	bytesBuffer := bufPool.Get().(*bytes.Buffer)
+	defer bufPool.Put(bytesBuffer)
+
+	pu, err := needle.ParseUpload(r, sizeLimit, bytesBuffer)
 	uncompressedData := pu.Data
 	if pu.IsGzipped {
 		uncompressedData = pu.UncompressedData
@@ -7,7 +7,10 @@ import (
 	"io"
 	"io/ioutil"
 	"net/http"
+	"sort"
 	"strings"
+	"sync"
+	"sync/atomic"
 	"time"

 	"github.com/chrislusf/seaweedfs/weed/filer"
@@ -19,85 +22,109 @@ import (
 	"github.com/chrislusf/seaweedfs/weed/util"
 )

+var bufPool = sync.Pool{
+	New: func() interface{} {
+		return new(bytes.Buffer)
+	},
+}
+
 func (fs *FilerServer) uploadReaderToChunks(w http.ResponseWriter, r *http.Request, reader io.Reader, chunkSize int32, fileName, contentType string, contentLength int64, so *operation.StorageOption) ([]*filer_pb.FileChunk, hash.Hash, int64, error, []byte) {
-	var fileChunks []*filer_pb.FileChunk

 	md5Hash := md5.New()
 	var partReader = ioutil.NopCloser(io.TeeReader(reader, md5Hash))

 	chunkOffset := int64(0)
 	var smallContent []byte
+	var uploadErr error

+	var wg sync.WaitGroup
+	var bytesBufferCounter int64
+	bytesBufferLimitCond := sync.NewCond(new(sync.Mutex))
+	var fileChunks []*filer_pb.FileChunk
+	var fileChunksLock sync.Mutex
 	for {
+
+		// need to throttle used byte buffer
+		bytesBufferLimitCond.L.Lock()
+		for atomic.LoadInt64(&bytesBufferCounter) >= 4 {
+			glog.V(4).Infof("waiting for byte buffer %d", bytesBufferCounter)
+			bytesBufferLimitCond.Wait()
+		}
+		atomic.AddInt64(&bytesBufferCounter, 1)
+		bytesBufferLimitCond.L.Unlock()
+
+		bytesBuffer := bufPool.Get().(*bytes.Buffer)
+		glog.V(4).Infof("received byte buffer %d", bytesBufferCounter)
+
 		limitedReader := io.LimitReader(partReader, int64(chunkSize))

-		data, err := ioutil.ReadAll(limitedReader)
-		if err != nil {
-			return nil, nil, 0, err, nil
+		bytesBuffer.Reset()
+
+		dataSize, err := bytesBuffer.ReadFrom(limitedReader)
+
+		// data, err := ioutil.ReadAll(limitedReader)
+		if err != nil || dataSize == 0 {
+			bufPool.Put(bytesBuffer)
+			atomic.AddInt64(&bytesBufferCounter, -1)
+			bytesBufferLimitCond.Signal()
+			return nil, md5Hash, 0, err, nil
 		}
 		if chunkOffset == 0 && !isAppend(r) {
-			if len(data) < int(fs.option.SaveToFilerLimit) || strings.HasPrefix(r.URL.Path, filer.DirectoryEtcRoot) && len(data) < 4*1024 {
-				smallContent = data
-				chunkOffset += int64(len(data))
+			if dataSize < fs.option.SaveToFilerLimit || strings.HasPrefix(r.URL.Path, filer.DirectoryEtcRoot) && dataSize < 4*1024 {
+				chunkOffset += dataSize
+				smallContent = make([]byte, dataSize)
+				bytesBuffer.Read(smallContent)
+				bufPool.Put(bytesBuffer)
+				atomic.AddInt64(&bytesBufferCounter, -1)
+				bytesBufferLimitCond.Signal()
 				break
 			}
 		}
-		dataReader := util.NewBytesReader(data)

-		// retry to assign a different file id
-		var fileId, urlLocation string
-		var auth security.EncodedJwt
-		var assignErr, uploadErr error
-		var uploadResult *operation.UploadResult
-		for i := 0; i < 3; i++ {
-			// assign one file id for one chunk
-			fileId, urlLocation, auth, assignErr = fs.assignNewFileInfo(so)
-			if assignErr != nil {
-				return nil, nil, 0, assignErr, nil
+		wg.Add(1)
+		go func(offset int64) {
+			defer func() {
+				bufPool.Put(bytesBuffer)
+				atomic.AddInt64(&bytesBufferCounter, -1)
+				bytesBufferLimitCond.Signal()
+				wg.Done()
+			}()
+
+			chunk, toChunkErr := fs.dataToChunk(fileName, contentType, bytesBuffer.Bytes(), offset, so)
+			if toChunkErr != nil {
+				uploadErr = toChunkErr
 			}
-
-			// upload the chunk to the volume server
-			uploadResult, uploadErr, _ = fs.doUpload(urlLocation, w, r, dataReader, fileName, contentType, nil, auth)
-			if uploadErr != nil {
-				time.Sleep(251 * time.Millisecond)
-				continue
+			if chunk != nil {
+				fileChunksLock.Lock()
+				fileChunks = append(fileChunks, chunk)
+				fileChunksLock.Unlock()
+				glog.V(4).Infof("uploaded %s chunk %d to %s [%d,%d)", fileName, len(fileChunks), chunk.FileId, offset, offset+int64(chunk.Size))
 			}
-			break
-		}
-		if uploadErr != nil {
-			return nil, nil, 0, uploadErr, nil
-		}
-
-		// if last chunk exhausted the reader exactly at the border
-		if uploadResult.Size == 0 {
-			break
-		}
-		if chunkOffset == 0 {
-			uploadedMd5 := util.Base64Md5ToBytes(uploadResult.ContentMd5)
-			readedMd5 := md5Hash.Sum(nil)
-			if !bytes.Equal(uploadedMd5, readedMd5) {
-				glog.Errorf("md5 %x does not match %x uploaded chunk %s to the volume server", readedMd5, uploadedMd5, uploadResult.Name)
-			}
-		}
-
-		// Save to chunk manifest structure
-		fileChunks = append(fileChunks, uploadResult.ToPbFileChunk(fileId, chunkOffset))
-
-		glog.V(4).Infof("uploaded %s chunk %d to %s [%d,%d)", fileName, len(fileChunks), fileId, chunkOffset, chunkOffset+int64(uploadResult.Size))
+		}(chunkOffset)

 		// reset variables for the next chunk
-		chunkOffset = chunkOffset + int64(uploadResult.Size)
+		chunkOffset = chunkOffset + dataSize

 		// if last chunk was not at full chunk size, but already exhausted the reader
-		if int64(uploadResult.Size) < int64(chunkSize) {
+		if dataSize < int64(chunkSize) {
 			break
 		}
 	}

+	wg.Wait()
+
+	if uploadErr != nil {
+		return nil, md5Hash, 0, uploadErr, nil
+	}
+
+	sort.Slice(fileChunks, func(i, j int) bool {
+		return fileChunks[i].Offset < fileChunks[j].Offset
+	})
+
 	return fileChunks, md5Hash, chunkOffset, nil, smallContent
 }

-func (fs *FilerServer) doUpload(urlLocation string, w http.ResponseWriter, r *http.Request, limitedReader io.Reader, fileName string, contentType string, pairMap map[string]string, auth security.EncodedJwt) (*operation.UploadResult, error, []byte) {
+func (fs *FilerServer) doUpload(urlLocation string, limitedReader io.Reader, fileName string, contentType string, pairMap map[string]string, auth security.EncodedJwt) (*operation.UploadResult, error, []byte) {

 	stats.FilerRequestCounter.WithLabelValues("chunkUpload").Inc()
 	start := time.Now()
@@ -111,3 +138,42 @@ func (fs *FilerServer) doUpload(urlLocation string, w http.ResponseWriter, r *ht
 	}
 	return uploadResult, err, data
 }
+
+func (fs *FilerServer) dataToChunk(fileName, contentType string, data []byte, chunkOffset int64, so *operation.StorageOption) (*filer_pb.FileChunk, error) {
+	dataReader := util.NewBytesReader(data)
+
+	// retry to assign a different file id
+	var fileId, urlLocation string
+	var auth security.EncodedJwt
+	var uploadErr error
+	var uploadResult *operation.UploadResult
+	for i := 0; i < 3; i++ {
+		// assign one file id for one chunk
+		fileId, urlLocation, auth, uploadErr = fs.assignNewFileInfo(so)
+		if uploadErr != nil {
+			glog.V(4).Infof("retry later due to assign error: %v", uploadErr)
+			time.Sleep(time.Duration(i+1) * 251 * time.Millisecond)
+			continue
+		}
+
+		// upload the chunk to the volume server
+		uploadResult, uploadErr, _ = fs.doUpload(urlLocation, dataReader, fileName, contentType, nil, auth)
+		if uploadErr != nil {
+			glog.V(4).Infof("retry later due to upload error: %v", uploadErr)
+			time.Sleep(time.Duration(i+1) * 251 * time.Millisecond)
+			continue
+		}
+		break
+	}
+	if uploadErr != nil {
+		glog.Errorf("upload error: %v", uploadErr)
+		return nil, uploadErr
+	}
+
+	// if last chunk exhausted the reader exactly at the border
+	if uploadResult.Size == 0 {
+		return nil, nil
+	}
+
+	return uploadResult.ToPbFileChunk(fileId, chunkOffset), nil
+}
@@ -205,8 +205,8 @@ func (ms *MasterServer) KeepConnected(stream master_pb.Seaweed_KeepConnectedServ
 			_, err := stream.Recv()
 			if err != nil {
 				glog.V(2).Infof("- client %v: %v", clientName, err)
-				stopChan <- true
-				break
+				close(stopChan)
+				return
 			}
 		}
 	}()
@@ -46,7 +46,8 @@ func (vs *VolumeServer) privateStoreHandler(w http.ResponseWriter, r *http.Reque
 		// wait until in flight data is less than the limit
 		contentLength := getContentLength(r)
 		vs.inFlightDataLimitCond.L.Lock()
-		for atomic.LoadInt64(&vs.inFlightDataSize) > vs.concurrentUploadLimit {
+		for vs.concurrentUploadLimit != 0 && atomic.LoadInt64(&vs.inFlightDataSize) > vs.concurrentUploadLimit {
+			glog.V(4).Infof("wait because inflight data %d > %d", vs.inFlightDataSize, vs.concurrentUploadLimit)
 			vs.inFlightDataLimitCond.Wait()
 		}
 		atomic.AddInt64(&vs.inFlightDataSize, contentLength)
@@ -27,7 +27,7 @@ var fileNameEscaper = strings.NewReplacer(`\`, `\\`, `"`, `\"`)

 func (vs *VolumeServer) GetOrHeadHandler(w http.ResponseWriter, r *http.Request) {

-	// println(r.Method + " " + r.URL.Path)
+	glog.V(9).Info(r.Method + " " + r.URL.Path + " " + r.Header.Get("Range"))

 	stats.VolumeServerRequestCounter.WithLabelValues("get").Inc()
 	start := time.Now()
@@ -261,13 +261,10 @@ func writeResponseContent(filename, mimeType string, rs io.ReadSeeker, w http.Re
 		return nil
 	}

-	processRangeRequest(r, w, totalSize, mimeType, func(writer io.Writer, offset int64, size int64, httpStatusCode int) error {
+	processRangeRequest(r, w, totalSize, mimeType, func(writer io.Writer, offset int64, size int64) error {
 		if _, e = rs.Seek(offset, 0); e != nil {
 			return e
 		}
-		if httpStatusCode != 0 {
-			w.WriteHeader(httpStatusCode)
-		}
 		_, e = io.CopyN(writer, rs, size)
 		return e
 	})
@@ -1,6 +1,7 @@
 package weed_server

 import (
+	"bytes"
 	"errors"
 	"fmt"
 	"net/http"
@@ -42,7 +43,10 @@ func (vs *VolumeServer) PostHandler(w http.ResponseWriter, r *http.Request) {
 		return
 	}

-	reqNeedle, originalSize, contentMd5, ne := needle.CreateNeedleFromRequest(r, vs.FixJpgOrientation, vs.fileSizeLimitBytes)
+	bytesBuffer := bufPool.Get().(*bytes.Buffer)
+	defer bufPool.Put(bytesBuffer)
+
+	reqNeedle, originalSize, contentMd5, ne := needle.CreateNeedleFromRequest(r, vs.FixJpgOrientation, vs.fileSizeLimitBytes, bytesBuffer)
 	if ne != nil {
 		writeJsonError(w, r, http.StatusBadRequest, ne)
 		return
@@ -31,7 +31,7 @@ var StatusTpl = template.Must(template.New("status").Funcs(funcMap).Parse(`<!DOC
  <head>
    <title>SeaweedFS {{ .Version }}</title>
 	<link rel="stylesheet" href="/seaweedfsstatic/bootstrap/3.3.1/css/bootstrap.min.css">
-	<script type="text/javascript" src="/seaweedfsstatic/javascript/jquery-2.1.3.min.js"></script>
+	<script type="text/javascript" src="/seaweedfsstatic/javascript/jquery-3.6.0.min.js"></script>
 	<script type="text/javascript" src="/seaweedfsstatic/javascript/jquery-sparklines/2.1.2/jquery.sparkline.min.js"></script>
    <script type="text/javascript">
    $(function() {
@@ -54,6 +54,7 @@ func (c *commandFsConfigure) Do(args []string, commandEnv *CommandEnv, writer io
 	ttl := fsConfigureCommand.String("ttl", "", "assign writes with this ttl")
 	diskType := fsConfigureCommand.String("disk", "", "[hdd|ssd|<tag>] hard drive or solid state drive or any tag")
 	fsync := fsConfigureCommand.Bool("fsync", false, "fsync for the writes")
+	isReadOnly := fsConfigureCommand.Bool("readOnly", false, "disable writes")
 	volumeGrowthCount := fsConfigureCommand.Int("volumeGrowthCount", 0, "the number of physical volumes to add if no writable volumes")
 	isDelete := fsConfigureCommand.Bool("delete", false, "delete the configuration by locationPrefix")
 	apply := fsConfigureCommand.Bool("apply", false, "update and apply filer configuration")
@@ -84,6 +85,7 @@ func (c *commandFsConfigure) Do(args []string, commandEnv *CommandEnv, writer io
 			Fsync:             *fsync,
 			DiskType:          *diskType,
 			VolumeGrowthCount: uint32(*volumeGrowthCount),
+			ReadOnly:          *isReadOnly,
 		}

 		// check collection
@@ -38,7 +38,7 @@ func (c *commandFsMetaSave) Help() string {
 	fs.meta.save                 # save from current directory

 	The meta data will be saved into a local <filer_host>-<port>-<time>.meta file.
-	These meta data can be later loaded by fs.meta.load command, 
+	These meta data can be later loaded by fs.meta.load command

 `
 }
@@ -46,6 +46,9 @@ func (c *commandS3BucketList) Do(args []string, commandEnv *CommandEnv, writer i
 	}

 	err = filer_pb.List(commandEnv, filerBucketsPath, "", func(entry *filer_pb.Entry, isLast bool) error {
+		if !entry.IsDirectory {
+			return nil
+		}
 		if entry.Attributes.Replication == "" || entry.Attributes.Replication == "000" {
 			fmt.Fprintf(writer, "  %s\n", entry.Name)
 		} else {
@@ -31,6 +31,7 @@ func RunShell(options ShellOptions) {
 	})

 	line.SetCtrlCAborts(true)
+	line.SetTabCompletionStyle(liner.TabPrints)

 	setCompletionHandler()
 	loadHistory()
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Chris Lu	7a81caa31e	2.53	2021-06-13 17:12:34 -07:00
Chris Lu	4d0cbd2700	skip cookie checking if from grpc api	2021-06-13 16:16:11 -07:00
Chris Lu	7225cb4ac5	add block and mutex profiling	2021-06-13 16:15:54 -07:00
Chris Lu	28a4a1f8d6	fix for mysql2 postgres2 on fast dropping buckets	2021-06-13 07:31:56 -07:00
Chris Lu	1e76fc994a	filer.copy: zero fileSize for directories	2021-06-13 07:09:06 -07:00
Chris Lu	ed6aa13520	minor	2021-06-12 18:36:25 -07:00
Chris Lu	ee6c67682c	minor	2021-06-12 02:52:41 -07:00
Chris Lu	9357911a95	remove all bucket metadata fix https://github.com/chrislusf/seaweedfs/issues/2118	2021-06-10 23:37:54 -07:00
Chris Lu	b71c3cfba4	avoid possible empty object	2021-06-10 22:17:53 -07:00
Chris Lu	310e31424e	adjust the error output fix https://github.com/chrislusf/seaweedfs/issues/2123	2021-06-10 21:55:13 -07:00
Chris Lu	8b382a8209	refactor	2021-06-10 21:50:21 -07:00
Chris Lu	33b87244ef	refactoring	2021-06-10 16:54:36 -07:00
Chris LuandGitHub	40a8cb9b0a	Update README.md	2021-06-09 13:14:54 -07:00
Chris LuandGitHub	492a91e0b0	Merge pull request #2115 from nagy/exitcode Return correct exitcode when `wheed upload` fails	2021-06-07 15:47:08 -07:00
Daniel Nagy	0e3adde47f	Return correct exitcode when `wheed upload` fails	2021-06-07 23:38:14 +02:00
Chris Lu	76f24af79a	2.52	2021-06-07 12:13:23 -07:00
Chris Lu	5e6dfbc25f	locks for data racing	2021-06-07 12:04:50 -07:00
Chris Lu	452c6ef183	limits concurrent uploads for one file	2021-06-06 23:05:17 -07:00
Chris Lu	8295e2feb6	skip md5 checking for now because of race condition	2021-06-06 22:16:32 -07:00
Chris Lu	c8dea3dd89	2.51	2021-06-06 21:54:00 -07:00
Chris Lu	19caeb7b02	fix writing the small file	2021-06-06 20:57:03 -07:00
Chris Lu	bb45dea15a	filer: parallel data upload	2021-06-06 20:23:36 -07:00
Chris Lu	e00443a940	mount: adjust starting order avoid possible nil wfs.Server	2021-06-06 20:22:42 -07:00
Chris Lu	44f1ba6894	refactor	2021-06-06 18:43:04 -07:00
Chris Lu	6c82326575	use bytes.Buffer to reduce memory allocation and gc	2021-06-06 13:42:36 -07:00
Chris Lu	9cba5cca0b	optionally disable concurrent upload limit	2021-06-06 13:13:33 -07:00
Chris Lu	21ad9a4ac2	filer: mongodb avoids E11000 duplicate key error collection: seaweedfs.filemeta index: directory_1_name_1 dup key	2021-06-06 13:12:01 -07:00
Chris Lu	e6ba2f9c37	verbose but cleaner logic to invalidate fuse cache	2021-06-05 13:09:37 -07:00
Chris LuandGitHub	d7638b73b5	Merge pull request #2112 from wenerme/patch-1	2021-06-05 11:20:07 -07:00
陈杨文andGitHub	5403ada150	use string version the helm package will build number version 2.50 to 2.5	2021-06-05 21:54:33 +08:00
Chris Lu	fadc1febdb	FUSE: invalidate FUSE cached entries fix https://github.com/chrislusf/seaweedfs/issues/2108	2021-06-05 02:23:07 -07:00
Chris Lu	ab606dec2a	filer: add path-specific option to enforce readonly	2021-06-04 01:03:41 -07:00
Chris Lu	f15d7a57f5	mount: support macfuse 4.x	2021-06-03 01:14:50 -07:00
Chris LuandGitHub	d350c114c8	Update README.md	2021-06-02 15:10:42 -07:00
Chris LuandGitHub	0a5388744c	Merge pull request #2105 from Woellchen/current_and_parent_dot_directories Return artificial . and .. directories	2021-06-02 12:54:44 -07:00
Patrick Schmidt	77100754e6	Return artificial . and .. directories	2021-06-02 21:28:02 +02:00
Chris LuandGitHub	62142ff1d2	Merge pull request #2104 from kmlebedev/sync_volume_on_close avoid data loss after restarting a container with a volum server	2021-06-02 11:47:57 -07:00
Chris Lu	cc34475012	remove file handle locking for setattr	2021-06-02 11:44:12 -07:00
Konstantin Lebedev	99ef280c7c	avoid data loss after restarting a container with a volum server	2021-06-02 17:07:19 +05:00
Chris Lu	3db1642392	mount: skip persisting metadata if file is open	2021-06-02 03:40:08 -07:00
Chris Lu	556cc3a4ca	mount: avoid exception if disk cache is not initialized related to https://github.com/chrislusf/seaweedfs/issues/2102	2021-05-31 16:42:55 -07:00
Chris Lu	615cb24ba6	shell: use tab print style for completion	2021-05-31 03:29:29 -07:00
Chris Lu	c5de97ddbe	keep alive for streaming connections fix https://github.com/chrislusf/seaweedfs/issues/2096 related to https://github.com/chrislusf/seaweedfs/issues/1277	2021-05-31 01:26:24 -07:00
Chris Lu	2b60e2abb1	only disallow streaming signed when no auth enabled fix https://github.com/chrislusf/seaweedfs/issues/2101	2021-05-31 01:03:04 -07:00
Chris Lu	9774cac3a1	Added SeaweedFS_Filer_Backup.png	2021-05-30 21:31:18 -07:00
Chris LuandGitHub	769fe4b772	Delete Filer Backup.png	2021-05-30 21:30:24 -07:00
Chris Lu	8f1f8c63e5	Update Filer Backup.png	2021-05-30 21:23:03 -07:00
Chris Lu	1fb1b7d7b8	Added Filer Backup.png	2021-05-30 21:13:01 -07:00
Chris Lu	4233ad3f07	2.50	2021-05-30 20:40:30 -07:00
Chris Lu	1456616a77	recreate grpc connections if too many errors address https://github.com/chrislusf/seaweedfs/issues/2098	2021-05-30 00:07:43 -07:00
Chris Lu	fb8036385a	s3: save metadata during put-object fix https://github.com/chrislusf/seaweedfs/issues/2092	2021-05-29 14:14:30 -07:00
Chris Lu	2d7b4e5bb6	filer.backup: escape colon from path on windows fix https://github.com/chrislusf/seaweedfs/issues/2084	2021-05-29 06:45:27 -07:00
Chris LuandGitHub	ae185b997f	Merge pull request #2099 from danielflira/mount-helper fix parameter multiple values	2021-05-28 21:57:15 -07:00
danielflira	849f36c1ac	fix parameter multiple values	2021-05-29 00:37:25 -03:00
Chris LuandGitHub	622fc2fa2f	Merge pull request #2097 from ekozlov-search/server_freediskspace	2021-05-28 09:43:25 -07:00
Eugeniy Kozlov	6e8bd16819	freespace params fix	2021-05-28 14:19:24 +03:00
Chris Lu	4d55132c7d	scaffold add comments	2021-05-27 14:40:25 -07:00
Chris Lu	45bffc92a8	filer.backup: fix cloud sinks when updating entry	2021-05-26 14:53:11 -07:00
Chris Lu	921e0d5008	remove verbose log	2021-05-26 14:43:34 -07:00
Chris Lu	1a70cb9b63	local sink: write and update files fix issues with https://github.com/chrislusf/seaweedfs/issues/2084	2021-05-26 14:42:21 -07:00
Chris LuandGitHub	e699d16c85	Merge pull request #2085 from danielflira/mount-helper create fuse subcommand to use weed with mount	2021-05-26 12:19:40 -07:00
danielflira	84488ebb33	replace filer.remote and parts[1]	2021-05-26 12:07:36 -03:00
Chris Lu	ac9bf71544	use jquery 3.6.0	2021-05-26 00:21:11 -07:00
danielflira	877c192af2	create fuse subcommand to use weed with mount	2021-05-25 23:32:35 -03:00
Chris Lu	67dd094b35	sqlite: does not support non-linux/darwin/windows	2021-05-25 17:36:15 -07:00
Chris Lu	dce1f02c9e	filer.backup: backup to local directory optionally is incremental fixed one issue with https://github.com/chrislusf/seaweedfs/issues/2084	2021-05-25 17:19:20 -07:00
Chris LuandGitHub	35ca9638c4	Merge pull request #2080 from kmlebedev/s3IfMatch	2021-05-24 06:59:03 -07:00
Konstantin Lebedev	b612d5aebd	s3 test get w/ If-Match: bogus ETag	2021-05-24 16:59:44 +05:00
Chris LuandGitHub	9079ce0e7f	Merge pull request #2079 from kmlebedev/s3CachingHeaders S3 write cache headers	2021-05-24 03:50:44 -07:00
Konstantin Lebedev	69c768870b	- object write cache control - object write expires	2021-05-24 15:43:55 +05:00
Chris LuandGitHub	15859a7daa	Merge pull request #2074 from kmlebedev/head_check_all_chunks avoid http error: superfluous response.WriteHeader	2021-05-24 00:39:55 -07:00
Konstantin LebedevandGitHub	84dce32a57	Merge branch 'master' into head_check_all_chunks	2021-05-24 12:28:19 +05:00
Chris LuandGitHub	359a08ecba	Merge pull request #2078 from kmlebedev/chunkDownloadStats Chunk download stats	2021-05-24 00:25:47 -07:00
Konstantin Lebedev	0e404cd84b	Chunk download stats	2021-05-24 12:14:50 +05:00
Chris Lu	759e4f1a75	add sqlite wiki	2021-05-24 00:09:45 -07:00
Konstantin Lebedev	bb42633522	rm chunk Download stats	2021-05-24 12:08:12 +05:00
Chris Lu	064269bb57	filer: Support sqlite as filer meta store	2021-05-23 23:58:28 -07:00
Chris Lu	b70aeb7585	bucket list only directories	2021-05-23 10:36:22 -07:00
Chris Lu	42fb03a66e	2.49	2021-05-23 00:51:47 -07:00
Chris Lu	2d85ffe7c5	java 1.6.6	2021-05-23 00:51:02 -07:00
Chris Lu	a852f8a3f5	some code to stress test the meta data events	2021-05-22 18:03:13 -07:00
Chris Lu	3d3fa43542	filer: re-create grpc connections if having transport error fix https://github.com/chrislusf/seaweedfs/issues/2070	2021-05-22 17:46:53 -07:00
Chris Lu	2d0a0733ec	filer: bootstrap from peer filer fix https://github.com/chrislusf/seaweedfs/issues/1861	2021-05-22 03:17:02 -07:00
Chris Lu	2586621362	minor	2021-05-22 01:02:31 -07:00
Chris Lu	e5a2bf1287	s3: deprecating filer.options.buckets_fsync	2021-05-22 00:24:23 -07:00
Chris Lu	431684798b	s3: add errors if requests are signed by no authentication is setup fix https://github.com/chrislusf/seaweedfs/issues/2075	2021-05-21 14:08:47 -07:00
Chris Lu	2f136a04a1	organize a bit better for temp file directory	2021-05-21 13:02:18 -07:00
Konstantin Lebedev	3325b850cc	rm func CheckAllChunkViews	2021-05-21 16:05:59 +05:00
Konstantin Lebedev	26a4f34a57	del checks url err logging stats chunk fetch	2021-05-21 15:59:12 +05:00
Chris Lu	e8b7d4ad64	simplify a bit	2021-05-21 02:10:44 -07:00
Chris Lu	0b00edfc3b	waits in case master lost connection	2021-05-21 02:09:22 -07:00
Chris Lu	085a61cb8f	minor	2021-05-21 01:50:20 -07:00
Chris Lu	4d0b894d37	Java Client: throw exception when assign fails	2021-05-21 01:49:25 -07:00
Chris Lu	2d237f7b50	Java client: fix passing replication value	2021-05-21 01:43:25 -07:00
Chris Lu	f37a4cbd0f	randomize initial filer	2021-05-21 01:41:34 -07:00
Chris Lu	5d77840cff	adjust help message	2021-05-21 01:38:57 -07:00
Chris Lu	dc1309f084	FUSE mount: support multiple filers fix https://github.com/chrislusf/seaweedfs/issues/2015 fix https://github.com/chrislusf/seaweedfs/issues/1531	2021-05-21 01:28:00 -07:00
Konstantin Lebedev	42f631f549	fix https://github.com/chrislusf/seaweedfs/issues/2046	2021-05-20 13:19:35 +05:00
Chris Lu	30c67e3652	minor	2021-05-19 23:59:39 -07:00
Konstantin Lebedev	03d1199d5f	Revert "revert PR #1903 avoid http error: superfluous response.WriteHeader" This reverts commit `ac71117e`	2021-05-20 11:45:21 +05:00
Chris Lu	87a32bfef4	avoid possible nil when node is disconnected from its parent fix https://github.com/chrislusf/seaweedfs/issues/2073	2021-05-19 10:02:01 -07:00
Chris Lu	6c1c72b1f4	java client 1.6.5	2021-05-17 23:18:37 -07:00
Chris LuandGitHub	33fe0b96e6	Update README.md	2021-05-17 02:10:20 -07:00
Chris LuandGitHub	aeeeee1e26	Update README.md	2021-05-17 01:22:01 -07:00
Chris Lu	1aa7e99a89	skip file not found error when deleting	2021-05-15 09:37:39 -07:00