Commit Graph

  • ebdd24ce64 adjust log level chrislu 2025-08-14 23:36:59 -07:00
  • 1c09f313cc day-advance fix chrislu 2025-08-14 23:36:52 -07:00
  • 76c74ad3de err != filer_pb.ErrNotFound chrislu 2025-08-14 23:34:05 -07:00
  • b7f78d8c49 imports chrislu 2025-08-14 21:28:47 -07:00
  • fdfa089754 fix ListAllMyBucketsResult xmlns chrislu 2025-08-14 20:37:32 -07:00
  • 80db6f4d79 reduce lock scope to improve log buffer performance chrislu 2025-08-14 08:16:00 -07:00
  • 76f81ff236 fix ListAllMyBucketsResult xmlns chrislu 2025-08-14 20:37:32 -07:00
  • 543e074d60 reduce lock scope to improve log buffer performance chrislu 2025-08-14 08:16:00 -07:00
  • 18a22177b9 Fix volume server's status code returned for missing needles on EC-encoded shards (#7137) Lisandro Pin 2025-08-14 15:35:55 +02:00
  • c39d18c1da setup benchmarking chrislu 2025-08-13 21:59:54 -07:00
  • 1de251d575 sum from all shards chrislu 2025-08-13 18:48:05 -07:00
  • 722aebdf11 fix loading generational ec volumes chrislu 2025-08-13 18:31:03 -07:00
  • 66c688c656 collects metrics from ALL 6 servers/14 shards instead of just one chrislu 2025-08-13 18:11:38 -07:00
  • 5d1712e309 delete in ecj, add logs chrislu 2025-08-13 18:05:52 -07:00
  • b2d45f1f2d fix overflow chrislu 2025-08-13 18:04:48 -07:00
  • 2766504948 -v2 for volume servers chrislu 2025-08-13 18:03:03 -07:00
  • e3b45c9cbb Update .gitignore chrislu 2025-08-13 18:02:52 -07:00
  • 256b1c9c28 async persistence chrislu 2025-08-13 09:55:48 -07:00
  • 52f26187ca find master address earlier chrislu 2025-08-13 01:52:03 -07:00
  • e44a6189fa Update ec_decoder.go chrislu 2025-08-13 01:41:27 -07:00
  • 30a88615ba do not use .tmp chrislu 2025-08-13 01:27:45 -07:00
  • d7d19ea9ba ec volume deletion is generation-aware chrislu 2025-08-13 01:11:37 -07:00
  • 04b141749f explicit generation value chrislu 2025-08-13 00:57:36 -07:00
  • b616dc3d20 adjust comments chrislu 2025-08-13 00:34:41 -07:00
  • 117bba794c WriteDatFileAndVacuum chrislu 2025-08-13 00:28:12 -07:00
  • 1f8ba5c958 task ask admin for master address chrislu 2025-08-12 23:47:12 -07:00
  • c0e6d00bd3 collect ec volume total volume size chrislu 2025-08-12 22:47:10 -07:00
  • 3729e9ba25 chore(deps): bump golang.org/x/image from 0.29.0 to 0.30.0 (#7129) dependabot[bot] 2025-08-12 10:20:23 -07:00
  • 890b51eaef chore(deps): bump golang.org/x/tools from 0.35.0 to 0.36.0 (#7128) dependabot[bot] 2025-08-12 10:20:09 -07:00
  • 1485330547 Update ec_vacuum_task.go chrislu 2025-08-12 09:50:17 -07:00
  • 8213a4a2b6 use the EcVacuumLogic chrislu 2025-08-12 09:48:17 -07:00
  • 425a54019b adding EcVacuumLogic chrislu 2025-08-12 09:45:30 -07:00
  • 82907ad6ca determineGenerations() Removed chrislu 2025-08-12 09:33:40 -07:00
  • 729268d065 detection with generation info chrislu 2025-08-12 09:32:07 -07:00
  • 8a5c4a4972 add generation to TaskSource chrislu 2025-08-12 09:14:16 -07:00
  • 80bfce59ad Create NOTES.md chrislu 2025-08-12 09:13:57 -07:00
  • 35f3da108b distribute index files to all volume servers that will have shards mounted, not just one chrislu 2025-08-12 00:33:49 -07:00
  • 6bf5a6871c fix presigned signature chrislu 2025-08-11 23:57:57 -07:00
  • c8c758e639 fix hanging task detail page chrislu 2025-08-11 23:55:24 -07:00
  • 56db0a081a distribute .ecx files chrislu 2025-08-11 23:47:42 -07:00
  • 0d6ea416a7 fix retry link chrislu 2025-08-11 23:37:17 -07:00
  • 3864f89f92 chore(deps): bump golang.org/x/crypto from 0.40.0 to 0.41.0 (#7132) dependabot[bot] 2025-08-11 23:35:09 -07:00
  • d1f18f08d3 chore(deps): bump google.golang.org/protobuf from 1.36.6 to 1.36.7 (#7131) dependabot[bot] 2025-08-11 23:34:58 -07:00
  • 4f9f9ec971 fix routing chrislu 2025-08-11 22:36:48 -07:00
  • 3bbe4481f2 remove unused code chrislu 2025-08-11 21:54:21 -07:00
  • cac6a51cbf implement todos chrislu 2025-08-11 21:48:45 -07:00
  • 7780a7d652 ec vacuum workflow is correct now chrislu 2025-08-11 21:41:18 -07:00
  • 83b6a94bfe change to correct ec vacuum workflow chrislu 2025-08-11 21:14:36 -07:00
  • 0f1c734626 fix mounting ec volumes chrislu 2025-08-11 20:34:02 -07:00
  • 2a61cda2cd adding retry button chrislu 2025-08-11 18:28:02 -07:00
  • 5c93557314 detect shard sources chrislu 2025-08-11 17:36:27 -07:00
  • 7cf5ddf8ac more self contained tasks chrislu 2025-08-11 17:16:16 -07:00
  • 7622d14580 remove unused config chrislu 2025-08-11 17:08:48 -07:00
  • bddb2cd5cc keep only ec config chrislu 2025-08-11 17:02:15 -07:00
  • 96a2749355 adding EcVacuumTaskConfig chrislu 2025-08-11 16:59:55 -07:00
  • f9e7ca4308 Merge branch 'master' into add-ec-vacuum chrislu 2025-08-11 16:45:45 -07:00
  • 7889e78f4d use one http client chrislu 2025-08-11 16:44:51 -07:00
  • af3300e063 filer: server side copying (#7121) Chris Lu 2025-08-11 16:40:46 -07:00
  • 7e86045e22 chore(deps): bump golang.org/x/net from 0.42.0 to 0.43.0 (#7126) dependabot[bot] 2025-08-11 15:21:46 -07:00
  • 529c68a798 chore(deps): bump google.golang.org/api from 0.244.0 to 0.246.0 (#7127) dependabot[bot] 2025-08-11 15:06:25 -07:00
  • e3b15073a0 chore(deps): bump actions/checkout from 4 to 5 (#7125) dependabot[bot] 2025-08-11 15:06:14 -07:00
  • 5f62b11889 chore(deps): bump golang.org/x/sys from 0.34.0 to 0.35.0 (#7124) dependabot[bot] 2025-08-11 15:06:01 -07:00
  • 524c1916b6 chore(deps): bump github.com/redis/go-redis/v9 from 9.11.0 to 9.12.0 (#7123) dependabot[bot] 2025-08-11 15:05:53 -07:00
  • 412fd6ee86 chore(deps): bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.113.4 to 3.113.5 (#7122) dependabot[bot] 2025-08-11 15:05:44 -07:00
  • 605b3333c1 fix sftp start with filer (#7120) Chris Lu 2025-08-11 10:56:40 -07:00
  • 9265e81fe9 S3 API: unsigned streaming (no cred) but chunks contain signatures (#7118) Chris Lu 2025-08-11 10:31:01 -07:00
  • a265a07922 fix ttl objects chrislu 2025-08-11 02:20:02 -07:00
  • f03f4383af fix missing grpc dial option chrislu 2025-08-11 02:19:22 -07:00
  • c57ccd554a worker_pb.TaskParams_VacuumParams chrislu 2025-08-11 02:11:39 -07:00
  • 9df006b49d fix min volume age chrislu 2025-08-11 01:54:10 -07:00
  • 424bb7fe11 showing criteria chrislu 2025-08-11 01:34:33 -07:00
  • 97d58e77c6 getting ec volume deletions chrislu 2025-08-11 01:03:21 -07:00
  • 553a229fd3 reduce lock scope chrislu 2025-08-11 00:21:04 -07:00
  • 57d025910d cancel context chrislu 2025-08-11 00:20:52 -07:00
  • 213dd9b470 testing chrislu 2025-08-11 00:20:37 -07:00
  • a31fc1ac01 Bypass compaction revision check to handle volumes compacted after task creation chrislu 2025-08-10 20:39:48 -07:00
  • 91d641e685 avoid dead lock chrislu 2025-08-10 20:23:18 -07:00
  • 740983d69e Delete ec_test_files.json chrislu 2025-08-10 20:15:36 -07:00
  • 0e649f710a collect deletion for ec shards chrislu 2025-08-10 20:08:45 -07:00
  • 772ee0f967 address comments chrislu 2025-08-10 19:58:52 -07:00
  • 0c6980182c ec shards with generation chrislu 2025-08-10 19:40:35 -07:00
  • 47ea1ac228 unmount chrislu 2025-08-10 19:39:24 -07:00
  • 6446893e3c metrics chrislu 2025-08-10 19:39:11 -07:00
  • 06c012ea60 Update maintenance_scanner.go chrislu 2025-08-10 19:37:54 -07:00
  • 5a6954be1b sort chrislu 2025-08-10 18:42:27 -07:00
  • 0f1ca16457 more accurate estimation chrislu 2025-08-10 18:42:22 -07:00
  • 72f0a47563 CRITICAL: Check ALL task states for volume conflicts chrislu 2025-08-10 18:04:31 -07:00
  • 751cfac7d7 Implement volume-aware task conflict checking chrislu 2025-08-10 18:02:42 -07:00
  • 5c1e6e904d CRITICAL: Restore task conflict definitions to prevent data integrity issues chrislu 2025-08-10 17:56:13 -07:00
  • 04e3a68b61 Replace deprecated ioutil with os package functions chrislu 2025-08-10 17:53:48 -07:00
  • c220ad1e69 Replace bubble sort with idiomatic sort.Slice in EC shard management chrislu 2025-08-10 17:52:30 -07:00
  • c9ca213d4c Fix linter error in master_grpc_ec_generation_test.go chrislu 2025-08-10 17:51:28 -07:00
  • 4ec743583d Address PR #7116 review comments chrislu 2025-08-10 17:50:10 -07:00
  • 5bb475c572 Fixed CodeQL Security Issue chrislu 2025-08-10 17:48:29 -07:00
  • cd75202da8 Replaced goto with Idiomatic Map Approach chrislu 2025-08-10 17:48:18 -07:00
  • cdba261913 add safety check chrislu 2025-08-10 17:41:55 -07:00
  • 179bcb777e Delete ec_vacuum_integration_test.go chrislu 2025-08-10 17:34:03 -07:00
  • 802267b281 validates all critical upgrade scenarios for the generation-aware EC vacuum system chrislu 2025-08-10 17:33:08 -07:00
  • d6e335d6e2 Integration test: end-to-end vacuum G→G+1 with live reads (zero downtime) chrislu 2025-08-10 17:23:17 -07:00
  • 884da0496c testing mixed generation chrislu 2025-08-10 17:12:20 -07:00