Commit Graph

12089 Commits

Author SHA1 Message Date
chrislu
eec179ef02 fmt 2025-11-05 22:14:31 -08:00
chrislu
50d81319a9 Merge branch 'master' into also-delete-parent-directory-if-empty 2025-11-05 22:13:01 -08:00
Konstantin Lebedev
084b377f87 do delete expired entries on s3 list request (#7426)
* do delete expired entries on s3 list request
https://github.com/seaweedfs/seaweedfs/issues/6837

* disable delete expires s3 entry in filer

* pass opt allowDeleteObjectsByTTL to all servers

* delete on get and head

* add lifecycle expiration s3 tests

* fix opt allowDeleteObjectsByTTL for server

* fix test lifecycle expiration

* fix IsExpired

* fix locationPrefix for updateEntriesTTL

* fix s3tests

* resolv  coderabbitai

* GetS3ExpireTime on filer

* go mod

* clear TtlSeconds for volume

* move s3 delete expired entry to filer

* filer delete meta and data

* del unusing func removeExpiredObject

* test s3 put

* test s3 put multipart

* allowDeleteObjectsByTTL by default

* fix pipline tests

* rm dublicate SeaweedFSExpiresS3

* revert expiration tests

* fix updateTTL

* rm log

* resolv comment

* fix delete version object

* fix S3Versioning

* fix delete on FindEntry

* fix delete chunks

* fix sqlite not support concurrent writes/reads

* move deletion out of listing transaction; delete entries and empty folders

* Revert "fix sqlite not support concurrent writes/reads"

This reverts commit 5d5da14e0e.

* clearer handling on recursive empty directory deletion

* handle listing errors

* strut copying

* reuse code to delete empty folders

* use iterative approach with a queue to avoid recursive WithFilerClient calls

* stop a gRPC stream from the client-side callback is to return a specific error, e.g., io.EOF

* still issue UpdateEntry when the flag must be added

* errors join

* join path

* cleaner

* add context, sort directories by depth (deepest first) to avoid redundant checks

* batched operation, refactoring

* prevent deleting bucket

* constant

* reuse code

* more logging

* refactoring

* s3 TTL time

* Safety check

---------

Co-authored-by: chrislu <chris.lu@gmail.com>
2025-11-05 22:05:54 -08:00
chrislu
282e0dcbc8 only check empty folder once when LC 2025-11-05 22:04:16 -08:00
chrislu
a8a4bc06e8 optionally delete empty parent directories 2025-11-05 21:29:06 -08:00
chrislu
9612457a32 Safety check 2025-11-05 20:44:22 -08:00
chrislu
9866287d8d s3 TTL time 2025-11-05 16:31:17 -08:00
chrislu
06e9ca70a6 refactoring 2025-11-05 16:03:07 -08:00
chrislu
c125060b51 more logging 2025-11-05 15:59:59 -08:00
chrislu
3e440d2145 reuse code 2025-11-05 15:34:14 -08:00
chrislu
7484fcb137 constant 2025-11-05 14:43:18 -08:00
chrislu
dc69c875a1 prevent deleting bucket 2025-11-05 14:42:27 -08:00
chrislu
3cff8846c2 batched operation, refactoring 2025-11-05 14:32:20 -08:00
chrislu
ec8ca216a5 add context, sort directories by depth (deepest first) to avoid redundant checks 2025-11-05 14:17:38 -08:00
chrislu
55ee4513e7 cleaner 2025-11-05 14:12:20 -08:00
chrislu
d8bef68752 join path 2025-11-05 13:50:20 -08:00
chrislu
c087a47d38 errors join 2025-11-05 13:50:08 -08:00
chrislu
835e5696d9 still issue UpdateEntry when the flag must be added 2025-11-05 13:49:48 -08:00
chrislu
92525d78ce stop a gRPC stream from the client-side callback is to return a specific error, e.g., io.EOF 2025-11-05 13:38:40 -08:00
chrislu
629b520edf use iterative approach with a queue to avoid recursive WithFilerClient calls 2025-11-05 13:37:40 -08:00
chrislu
0cad84ee36 reuse code to delete empty folders 2025-11-05 13:27:47 -08:00
chrislu
45e0a661da strut copying 2025-11-05 13:11:16 -08:00
chrislu
636540aba2 handle listing errors 2025-11-05 13:10:58 -08:00
chrislu
671de48369 clearer handling on recursive empty directory deletion 2025-11-05 13:10:40 -08:00
chrislu
672488c828 Revert "fix sqlite not support concurrent writes/reads"
This reverts commit 5d5da14e0e.
2025-11-05 12:44:07 -08:00
chrislu
543e70c511 move deletion out of listing transaction; delete entries and empty folders 2025-11-05 12:42:30 -08:00
Konstantin Lebedev
5d5da14e0e fix sqlite not support concurrent writes/reads 2025-11-06 00:14:37 +05:00
Konstantin Lebedev
455dec12f4 fix delete chunks 2025-11-05 23:04:00 +05:00
Konstantin Lebedev
454964353a fix delete on FindEntry 2025-11-05 22:56:49 +05:00
Konstantin Lebedev
50e1cf568e fix S3Versioning 2025-11-05 22:50:09 +05:00
Konstantin Lebedev
53e9d408ab fix delete version object 2025-11-05 20:06:50 +05:00
Konstantin Lebedev
cf75abb408 resolv comment 2025-11-05 18:39:21 +05:00
Konstantin Lebedev
360c2387db rm log 2025-11-05 18:16:04 +05:00
Konstantin Lebedev
20fb1ead77 fix updateTTL 2025-11-05 18:14:55 +05:00
Konstantin Lebedev
a88eab0b97 revert expiration tests 2025-11-05 15:05:34 +05:00
Konstantin Lebedev
c18004f9f6 rm dublicate SeaweedFSExpiresS3 2025-11-05 14:16:07 +05:00
Konstantin Lebedev
20254cd2db fix pipline tests 2025-11-05 14:10:29 +05:00
Konstantin Lebedev
558f4be73b allowDeleteObjectsByTTL by default 2025-11-05 14:06:02 +05:00
Konstantin Lebedev
7c41795078 Merge branch 'master' into allow_delete_objects_by_TTL 2025-11-05 13:44:27 +05:00
chrislu
cc444b1868 muted texts 2025-11-04 22:17:21 -08:00
chrislu
ca8cd631ff Update admin.css 2025-11-04 22:11:19 -08:00
chrislu
82f2c3757f muted admin UI color 2025-11-04 22:09:32 -08:00
Chris Lu
ecdbe572ca master: fix negative active volumes (#7440)
* fix negative active volumes

* address comments

* simplify
2025-11-04 21:50:04 -08:00
Federico A. Corazza
17b23f61e1 Don't make nginx the default ingress controller (#7436) 2025-11-04 13:44:29 -08:00
Lisandro Pin
f466ff1412 Nit: use time.Durations instead of constants in seconds. (#7438)
Nit: use `time.Durations` instead of constants in seconds. Makes for slightly more readable code.
2025-11-04 13:02:22 -08:00
Konstantin Lebedev
bbd7546cea test s3 put multipart 2025-11-04 20:26:07 +05:00
Konstantin Lebedev
1abea0d9b5 test s3 put 2025-11-04 20:03:36 +05:00
Konstantin Lebedev
e8e080b4fa del unusing func removeExpiredObject 2025-11-04 15:29:42 +05:00
Konstantin Lebedev
6a0d1e0b6f filer delete meta and data 2025-11-04 14:54:49 +05:00
Konstantin Lebedev
8d768885c5 move s3 delete expired entry to filer 2025-11-04 13:17:36 +05:00