Commit Graph
10942 Commits
Author SHA1 Message Date
Rain LiandGitHub 089446f5b1 Fix reload page before all file uploaded bug (#3534) 2022-08-28 07:51:47 -07:00
chrislu cc0c8c5f81 simplify 2022-08-27 00:21:57 -07:00
chrislu 3741779144 minor 2022-08-27 00:20:58 -07:00
chrislu 580a7fa685 adjust for data race
fix https://github.com/seaweedfs/seaweedfs/issues/3512
2022-08-27 00:19:13 -07:00
chrislu 87b70a6809 clean up 2022-08-27 00:09:04 -07:00
chrislu 27c7611ebf Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2022-08-26 23:47:39 -07:00
chrislu 9920d65bc0 gateway to remote object store: adjust upload concurrency 2022-08-26 23:47:37 -07:00
chrislu c839ce1b19 s3 sink use s3 upload manager
fix https://github.com/seaweedfs/seaweedfs/issues/3531
2022-08-26 23:47:12 -07:00
famosssandGitHub dc4037925d fix: Build DeletedVids before reset dn's children (#3530) 2022-08-26 22:52:08 -07:00
chrislu 3f3a1341d8 make CodeQL happy 2022-08-26 17:09:11 -07:00
chrislu 301b49b63f atomic operation 2022-08-26 17:04:11 -07:00
chrislu 57a46f46a0 nano level precision 2022-08-26 16:55:15 -07:00
chrislu 8dae81c5ed Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2022-08-26 16:49:51 -07:00
chrislu e0f4366f4c more detailed error
related to https://github.com/seaweedfs/seaweedfs/issues/3528
2022-08-26 16:49:48 -07:00
Patrick SchmidtandGitHub 5df105b1f9 Fix a few data races when reading files in mount (#3527) 2022-08-26 16:41:37 -07:00
chrislu f5156cf3a8 rename varaible 2022-08-26 10:23:42 -07:00
Konstantin LebedevandGitHub e90ab4ac60 avoid race conditions for OnPeerUpdate (#3525)
https://github.com/seaweedfs/seaweedfs/issues/3524
2022-08-26 10:18:49 -07:00
Konstantin LebedevandGitHub 4f7a1f67cd avoid race conditions for diskLocation.MaxVolumeCount (#3526) 2022-08-26 08:41:42 -07:00
Konstantin LebedevandGitHub 913a5d0dd4 add target dev_race to MakeFile (#3522) 2022-08-26 01:28:03 -07:00
chrislu 6f483a4d36 atomic add
fix https://github.com/seaweedfs/seaweedfs/issues/3514
2022-08-25 22:20:34 -07:00
chrislu 7394f7feee minor: adjust ordering 2022-08-25 00:20:23 -07:00
chrislu dbf0de4ce1 minor clean up 2022-08-25 00:19:08 -07:00
Patrick SchmidtandGitHub 2930263dfd Fix race conditions during in-flight size checks (#3505) 2022-08-24 20:03:34 -07:00
Patrick SchmidtandGitHub 7b424a54dc Add raft server access mutex to avoid races (#3503) 2022-08-24 09:49:05 -07:00
Chris LuandGitHub f7e4359b59 Temporarily enable cgo
Temporarily enable cgo following findings from https://github.com/seaweedfs/seaweedfs/discussions/3152 by https://github.com/dyus

Resolve build issue:
```
# github.com/mattn/go-ieproxy
/go/pkg/mod/github.com/mattn/go-ieproxy@v0.0.3/ieproxy.go:36:9: undefined: getConf
/go/pkg/mod/github.com/mattn/go-ieproxy@v0.0.3/ieproxy.go:41:9: undefined: reloadConf
/go/pkg/mod/github.com/mattn/go-ieproxy@v0.0.3/ieproxy.go:48:2: undefined: overrideEnvWithStaticProxy
/go/pkg/mod/github.com/mattn/go-ieproxy@v0.0.3/ieproxy.go:53:13: psc.findProxyForURL undefined (type *ProxyScriptConf has no field or method findProxyForURL, but does have FindProxyForURL)
```
2022-08-24 02:24:21 -07:00
Konstantin LebedevandGitHub f7aeb06544 s3: report metadata if the directory is explicitly created (#3498)
* replace mkdir to mkFile

* ContentLength must be zero

* revert mkDir

* Seaweedfs-Is-Directory-Key return metadata
2022-08-24 00:15:44 -07:00
Guo LeiandGitHub c57c79a0ab optimiz commitig compact (#3388)
* optimiz vacuuming volume

* fix bugx

* rename parameters

* fix conflict

* change copyDataBasedOnIndexFile to an instance method

* close needlemap

* optimiz commiting Vacuum volume for  leveldb index

* fix bugs

* fix leveldb loading bugs

* refactor

* fix leveldb loading bug

* add leveldb recovery

* add test case for levelDB

* modify test case to cover all the new branches

* use one tmpNm instead of two instances

* refactor

* refactor

* move setWatermark to the end

* add test for watermark and updating leveldb

* fix error logic

* refactor, add test

* check nil before close needlemapeer
add test case
fix metric bug

* add tests, fix bugs

* adjust log level
remove wrong test case
refactor

* avoid duplicate  updating metric for leveldb index
2022-08-23 23:53:35 -07:00
chrislu 10414fd81c ping timeout at 15 seconds
this 72 minute timeout setting seems unreasonably long

15 seconds is around the time when a new raft leader should be elected.
2022-08-23 23:28:16 -07:00
askeipxandGitHub 2e78a522ab remove old raft servers if they don't answer to pings for too long (#3398)
* remove old raft servers if they don't answer to pings for too long

add ping durations as options

rename ping fields

fix some todos

get masters through masterclient

raft remove server from leader

use raft servers to ping them

CheckMastersAlive for hashicorp raft only

* prepare blocking ping

* pass waitForReady as param

* pass waitForReady through all functions

* waitForReady works

* refactor

* remove unneeded params

* rollback unneeded changes

* fix
2022-08-23 23:18:21 -07:00
Guo LeiandGitHub 762dc219a6 fix deletecount bug (#3499) 2022-08-23 08:53:24 -07:00
chrislu 21a933c932 Merge branch 'master' of https://github.com/seaweedfs/seaweedfs 2022-08-23 01:52:32 -07:00
chrislu f0b4a7659a fix test 2022-08-23 01:52:29 -07:00
Patrick SchmidtandGitHub cda8cc22bc Implement lseek syscall in FUSE (#3491)
See the man page of lseek:
https://man7.org/linux/man-pages/man2/lseek.2.html
2022-08-23 01:47:18 -07:00
chrislu bf88006037 format 2022-08-23 01:20:45 -07:00
chrislu 9fce75607d s3: report http.StatusOK if the directory is explicitly created
fix https://github.com/seaweedfs/seaweedfs/issues/3457
2022-08-23 01:16:46 -07:00
famosssandGitHub 911475526c fix: TestCommandEcBalanceSmall Unit test fails when CommandEnv is nil (#3497) 2022-08-22 23:54:51 -07:00
famosssandGitHub 7eb15b1969 fix:Handle preflight cors requests (#3496) 2022-08-22 23:39:34 -07:00
chrislu 676e27c589 shell: stop long running jobs if lock is lost 2022-08-22 14:12:23 -07:00
chrislu 601ba5fb68 master: disable locking if not leader 2022-08-22 14:11:48 -07:00
chrislu 57e7582c36 refactoring 2022-08-22 14:11:13 -07:00
chrislu 7ea2c24c5d metadata processing use retry instead of retryForEver 2022-08-22 11:25:40 -07:00
dependabot[bot]andGitHub 741a013689 Bump github.com/Shopify/sarama from 1.35.0 to 1.36.0 (#3487) 2022-08-22 08:44:56 -07:00
dependabot[bot]andGitHub ebc1b478b1 Bump wangyoucao577/go-release-action from 1.22 to 1.30 (#3484) 2022-08-22 08:36:15 -07:00
dependabot[bot]andGitHub 66a1d6b50f Bump actions/dependency-review-action from 2.0.4 to 2.1.0 (#3485) 2022-08-22 08:35:59 -07:00
dependabot[bot]andGitHub 6ec81f09f7 Bump github.com/tidwall/gjson from 1.14.2 to 1.14.3 (#3486) 2022-08-22 08:35:36 -07:00
dependabot[bot]andGitHub 622521c247 Bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.33.0 to 3.35.1 (#3488) 2022-08-22 08:35:06 -07:00
dependabot[bot]andGitHub 6d4846a29a Bump github.com/fclairamb/ftpserverlib from 0.19.0 to 0.19.1 (#3489) 2022-08-22 08:34:43 -07:00
dependabot[bot]andGitHub d9991aeaef Bump modernc.org/sqlite from 1.18.0 to 1.18.1 (#3490) 2022-08-22 08:34:28 -07:00
dependabot[bot]andGitHub 9076a0051c Bump actions/checkout from 3 to 3.0.2 (#3483) 2022-08-22 08:33:53 -07:00
Patrick SchmidtandGitHub f875031f06 Reuse readDataByFileHandle in Read call (#3482) 2022-08-22 08:24:06 -07:00