mirrors/minio
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
10,801 Commits 4 Branches 517 Tags
d2f5c3621ff5e8428e592f7029ba0e18cc9b3bcc
Commit Graph

75 Commits

Author SHA1 Message Date
Harshavardhana
d2f5c3621f fix: add additional decommission traces for ILM expired content (#17522) 
current decommission traces were missing for

- Skipped ILM expired versions
- Skipped single DELETE marked version
- A success or failure in decommissioning DELETE marker
- allow additional info to be shared in DecomStatus() API
 2023-06-27 11:59:40 -07:00
Harshavardhana
eefa047974 fix: keep decommission in a go-routine (#17496) 
This was removed by mistake in #17491
 2023-06-23 12:29:32 -07:00
Harshavardhana
d315d012a4 decom: during multiple pool decom preserve current pool status (#17491) 
removal of completed pools must retain pool status of other
pools in draining, to resume any remaining draining operations.
 2023-06-23 07:44:18 -07:00
Harshavardhana
9af6c6ceef under rebalance look for expired versions v/s remaining versions (#17482) 
A continuation of PR #17479 for rebalance behavior must
also match the decommission behavior.

Fixes bug where rebalance would ignore rebalancing object
versions after one of the version returned "ObjectNotFound"
 2023-06-21 13:23:20 -07:00
Harshavardhana
ccc5801112 always look for expired versions v/s remaining versions (#17479) 
while decommissioning it can so happen that the non-current
versions are all expired but there is a DEL marker as the
latest version.

For such objects, we should not decommission them instead
calculate the remaining versions and if the remaining versions
is one and that version is a DEL marker consider such
an object not to be scheduled for decommissioning.
 2023-06-21 08:49:28 -07:00
Harshavardhana
15911c85f6 safely ignore out of band deletions while decommissioning (#17473) 2023-06-20 08:31:42 -07:00
Aditya Manthramurthy
5a1612fe32 Bump up madmin-go and pkg deps (#17469) 2023-06-19 17:53:08 -07:00
Anis Eleuch
38342b1df5 decom: Parallelize decommissining (#17364) 2023-06-07 14:27:51 -07:00
Anis Eleuch
1436858347 log: Add a log when saving pool.bin fails (#17338) 
Co-authored-by: Anis Elleuch <anis@min.io>
 2023-06-04 14:20:21 -07:00
Harshavardhana
b210ea79bc do not save MTime in newMultipartUpload() to avoid side-affects (#17340) 2023-06-02 14:38:09 -07:00
Harshavardhana
9b5829c16e avoid decommissioning DEL markers with single versions (#17274) 2023-05-25 09:18:49 -07:00
Krishnan Parthasarathi
3e128c116e Add lifecycle event source to audit log tags (#17248) 2023-05-22 15:28:56 -07:00
Harshavardhana
06557fe8be allow decommissioned pools to be removed while others are finishing (#17221) 2023-05-16 16:00:57 -07:00
Klaus Post
aaf1abc993 simplify HardLimitReader by using LimitReader for internal usage (#17218) 2023-05-16 13:14:37 -07:00
Poorna
e07c2ab868 Use hash.NewLimitReader for internal multipart calls (#17191) 2023-05-12 11:19:08 -07:00
Anis Eleuch
883c98e26f fix: remove objects when there are skipped versions due to ILM in decom (#17198) 2023-05-12 10:37:38 -07:00
Harshavardhana
3637aad36e do not count ILM expired objects and other skipped objects (#17184) 2023-05-11 13:35:16 -07:00
Harshavardhana
b92cdea578 fix: start using pkg/workers to spawn parallel workers (#17170) 2023-05-09 16:37:31 -07:00
Krishnan Parthasarathi
e7cac8acef Add tags to auditLogLifecycle (#17081) 2023-04-26 17:49:00 -07:00
Harshavardhana
6825bd7e75 fix: inlined objects don't need to honor long locks (#17039) 2023-04-17 12:16:37 -07:00
Poorna
d1e775313d support decommissioning of tiered objects (#16751) 2023-03-16 07:48:05 -07:00
Harshavardhana
b984bf8d1a allow expiration of all versions during Listing() (#16757) 2023-03-09 15:15:30 -08:00
Harshavardhana
31188e9327 add parallel workers in batch replication (#16609) 2023-02-13 12:07:58 -08:00
Florian Schwab
d67a846ec4 allow restarting of decommissioning if completed, failed or canceld (#16464) 2023-01-24 07:07:59 -08:00
Harshavardhana
095fc0561d feat: allow decom of multiple pools (#16416) 2023-01-16 21:36:34 +05:30
Anis Elleuch
475a88b555 fix: error out if an object is found after a full decom (#16277) 2023-01-12 05:52:51 +05:30
Harshavardhana
b882310e2b avoid locks for internal and invalid buckets in MakeBucket() (#16302) 2022-12-23 07:46:00 -08:00
Aditya Manthramurthy
a30cfdd88f Bump up madmin-go to v2 (#16162) 2022-12-06 13:46:50 -08:00
Harshavardhana
5a8df7efb3 re-implement StorageInfo to be a peer call (#16155) 2022-12-01 14:31:35 -08:00
jiuker
fe8eed963e fix: wrapped error will not equal in decommissioning (#16113) 2022-11-24 08:00:42 -08:00
Harshavardhana
853c4de75a allow changing endpoints in distributed setups (#16071) 2022-11-16 07:59:10 -08:00
Harshavardhana
1f3db03bf0 allow changing argument for path for SNSD setup (#16013) 2022-11-07 00:11:58 -08:00
Krishnan Parthasarathi
4523da6543 feat: introduce pool-level rebalance (#15483) 2022-10-25 12:36:57 -07:00
Anis Elleuch
fc6c794972 Audit dangling object removal (#15933) 2022-10-24 11:35:07 -07:00
Anis Elleuch
ac85c2af76 lifecycle: refactor rules filtering and tagging support (#15914) 2022-10-21 10:46:53 -07:00
Harshavardhana
928feb0889 remove unused debug param from evalActionFromLifecycle (#15813) 2022-10-07 10:24:12 -07:00
Anis Elleuch
158d0e26a2 decom: Ignore object/version error during deletion (#15806) 2022-10-06 09:41:58 -07:00
Klaus Post
a9f1ad7924 Add extended checksum support (#15433) 2022-08-29 16:57:16 -07:00
Harshavardhana
433b6fa8fe upgrade golang-lint to the latest (#15600) 2022-08-26 12:52:29 -07:00
Krishnan Parthasarathi
91e6af4470 Add trace support for decommissioning (#15502) 
* Add trace support for decommissioning
* Add support for tracing errors during decommission
 2022-08-10 12:46:45 -07:00
ebozduman
b57e7321e7 Replaces 'disk'=>'drive' visible to end user (#15464) 2022-08-04 16:10:08 -07:00
Poorna
426c902b87 site replication: fix healing of bucket deletes. (#15377) 
This PR changes the handling of bucket deletes for site 
replicated setups to hold on to deleted bucket state until 
it syncs to all the clusters participating in site replication.
 2022-07-25 17:51:32 -07:00
Harshavardhana
7da9e3a6f8 support encrypted/compressed objects properly during decommission (#15320) 
fixes #15314
 2022-07-16 19:35:24 -07:00
Harshavardhana
e7ac1ea54c allow decommission to continue when healing (#15312) 
Bonus:

- heal buckets in-case during startup the new
  pools have bucket missing.
 2022-07-15 21:03:23 -07:00
Harshavardhana
1b339ea062 allow force delete on decom pool (#15302) 
Bonus:

- skip suspended pool from being
  considered for multipart uploads

- add more context for decomErrors()
 2022-07-14 20:44:22 -07:00
Harshavardhana
236ef03dbd fix: skip objects expired via lifecycle rules during decommission (#15300) 2022-07-14 16:47:09 -07:00
Harshavardhana
0a8b78cb84 fix: simplify passing auditLog eventType (#15278) 
Rename Trigger -> Event to be a more appropriate
name for the audit event.

Bonus: fixes a bug in AddMRFWorker() it did not
cancel the waitgroup, leading to waitgroup leaks.
 2022-07-12 10:43:32 -07:00
Harshavardhana
ae92521310 remove unnecessary nAgreed value in partial() func (#15242) 2022-07-07 13:45:34 -07:00
Harshavardhana
5802df4365 retry and resume decom operation upon retriable failures (#15244) 
it is possible in a k8s-like system reading pool.bin
might not have quorum during startup, however, add
a way to retry after this failure.
 2022-07-07 12:31:44 -07:00
Harshavardhana
9d80ff5a05 fix: decommission delete markers for non-current objects (#15225) 
versioned buckets were not creating the delete markers
present in the versioned stack of an object, this essentially
would stop decommission to succeed.

This PR fixes creating such delete markers properly during
a decommissioning process, adds tests as well.
 2022-07-05 07:37:24 -07:00