2 Commits

Include renames

Author	SHA1	Message	Date
Chris LuandGitHub	1f6f473995	refactor(worker): co-locate plugin handlers with their task packages (#9301) ... * refactor(worker): co-locate plugin handlers with their task packages Move every per-task plugin handler from weed/plugin/worker/ into the matching weed/worker/tasks/<name>/ package, so each task owns its detection, scheduling, execution, and plugin handler in one place. Step 0 (within pluginworker, no behavior change): extract shared helpers that previously lived inside individual handler files into dedicated files and export the ones now consumed across packages. - activity.go: BuildExecutorActivity, BuildDetectorActivity - config.go: ReadStringConfig/Double/Int64/Bytes/StringList, MapTaskPriority - interval.go: ShouldSkipDetectionByInterval - volume_state.go: VolumeState + consts, FilterMetricsByVolumeState/Location - collection_filter.go: CollectionFilterMode + consts - volume_metrics.go: export CollectVolumeMetricsFromMasters, MasterAddressCandidates, FetchVolumeList - testing_senders_test.go: shared test stubs Phase 1: move the per-task plugin handlers (and the iceberg subpackage) into their task packages. weed/plugin/worker/vacuum_handler.go -> weed/worker/tasks/vacuum/plugin_handler.go weed/plugin/worker/ec_balance_handler.go -> weed/worker/tasks/ec_balance/plugin_handler.go weed/plugin/worker/erasure_coding_handler.go -> weed/worker/tasks/erasure_coding/plugin_handler.go weed/plugin/worker/volume_balance_handler.go -> weed/worker/tasks/balance/plugin_handler.go weed/plugin/worker/iceberg/ -> weed/worker/tasks/iceberg/ weed/plugin/worker/handlers/handlers.go now blank-imports all five task subpackages so their init() registrations fire. weed/command/mini.go and the worker tests construct the handler with vacuum.DefaultMaxExecutionConcurrency (the constant moved with the vacuum handler). admin_script remains in weed/plugin/worker/ because there is no underlying weed/worker/tasks/admin_script/ package to merge with. * refactor(worker): update test/plugin_workers imports for moved handlers Three handler constructors moved out of pluginworker into their task packages — update the integration test files in test/plugin_workers/ to import from the new locations: pluginworker.NewVacuumHandler -> vacuum.NewVacuumHandler pluginworker.NewVolumeBalanceHandler -> balance.NewVolumeBalanceHandler pluginworker.NewErasureCodingHandler -> erasure_coding.NewErasureCodingHandler The pluginworker import is kept where the file still uses pluginworker.WorkerOptions / pluginworker.JobHandler. * refactor(worker): update test/s3tables iceberg import path The iceberg subpackage moved from weed/plugin/worker/iceberg/ to weed/worker/tasks/iceberg/. test/s3tables/maintenance/maintenance_integration_test.go still imported the old path, breaking S3 Tables / RisingWave / Trino / Spark / Iceberg-catalog / STS integration test builds. Mirrors the OSS-side fix needed by every job in the run that transitively imports test/s3tables/maintenance. * chore: gofmt PR-touched files The S3 Tables Format Check job runs `gofmt -l` over weed/s3api/s3tables and test/s3tables, then fails if anything is unformatted. Files this PR moved or modified had import-grouping and trailing-spacing issues introduced by perl-based renames; reformat them with gofmt -w. Touched files: test/plugin_workers/erasure_coding/{detection,execution}_test.go test/s3tables/maintenance/maintenance_integration_test.go weed/plugin/worker/handlers/handlers.go weed/worker/tasks/{balance,ec_balance,erasure_coding,vacuum}/plugin_handler*.go * refactor(worker): bounds-checked int conversions for plugin config values CodeQL flagged 18 go/incorrect-integer-conversion warnings on the moved plugin handler files: results of pluginworker.ReadInt64Config (which ultimately calls strconv.ParseInt with bit size 64) were being narrowed to int32/uint32/int without an upper-bound check, so a malicious or malformed admin/worker config value could overflow the target type. Add three helpers in weed/plugin/worker/config.go that wrap ReadInt64Config and clamp out-of-range values back to the caller's fallback: ReadInt32Config (math.MinInt32 .. math.MaxInt32) ReadUint32Config (0 .. math.MaxUint32) ReadIntConfig (math.MinInt32 .. math.MaxInt32, platform-portable) Update each flagged call site in the four moved task packages to use the bounds-checked helper. For protobuf uint32 fields (volume IDs) the variable type also becomes uint32, removing the trailing uint32(volumeID) casts and changing the "missing volume_id" check from `<= 0` to `== 0`. Touched files: weed/plugin/worker/config.go weed/worker/tasks/balance/plugin_handler.go weed/worker/tasks/erasure_coding/plugin_handler.go weed/worker/tasks/vacuum/plugin_handler.go * refactor(worker): use ReadIntConfig for clamped derive-worker-config helpers CodeQL still flagged three call sites where ReadInt64Config was being narrowed to int after a value-range clamp (max_concurrent_moves <= 50, batch_size <= 100, min_server_count >= 2). The clamp is correct but CodeQL's flow analysis didn't recognize the bound, so it flagged them as unbounded narrowing. Switch to ReadIntConfig (already int32-bounded by the helper) for those three sites, drop the now-redundant int64 intermediate variables. Also drops the now-unused `> math.MaxInt32` clamp in ec_balance.deriveECBalanceWorkerConfig (the helper covers it).	2026-05-02 18:03:13 -07:00
3 people	587c24ec89	plugin worker: support job type categories (all, default, heavy) (#8547) ... * plugin worker: add handler registry with job categories Introduce a self-registration pattern for plugin worker job handlers. Each handler can register itself via init() with a HandlerFactory that declares its job type, category (default/heavy), CLI aliases, and a builder function. ResolveHandlerFactories accepts a mix of category names ("all", "default", "heavy") and explicit job type names/aliases, returning the matching factories. This enables workers to be configured by resource profile rather than requiring explicit job type enumeration. * plugin worker: register all handlers via init() Each job handler now self-registers into the global handler registry with its canonical job type, category, CLI aliases, and build function: - vacuum: category=default - volume_balance: category=default - admin_script: category=default - erasure_coding: category=heavy - iceberg_maintenance: category=heavy Adding a new job type now only requires adding the init() call in the handler file itself — no other files need to be touched. * plugin worker: replace hardcoded job type switch with registry Remove buildPluginWorkerHandler, parsePluginWorkerJobTypes, and canonicalPluginWorkerJobType from worker_runtime.go. The simplified buildPluginWorkerHandlers now delegates to pluginworker.ResolveHandlerFactories, which resolves category names ("all", "default", "heavy") and explicit job type names/aliases. The default job type is changed from an explicit list to "all", so new handlers registered via init() are automatically picked up. Update all tests to use the new API. * plugin worker: update CLI help text for job categories Update the -jobType flag description and command examples to document category support (all, default, heavy) alongside explicit job type names. * plugin worker: address review feedback - Add CategoryAll constant; use typed constants in tokenAsCategory - Pre-allocate result slice in ResolveHandlerFactories - Add vacuum aliases (vol.vacuum, volume.vacuum) - List alias examples (ec, balance, iceberg) in -jobType flag help - Create handlers aggregator package for subpackage blank imports so new handler subpackages only need to be added in one place - Make category tests relationship-based (subset/union checks) instead of asserting exact handler counts - Add clarifying comments to worker_test.go and mini_plugin_test.go listing expected handler names next to count assertions --------- Co-authored-by: Copilot <copilot@github.com>	2026-03-07 18:30:58 -08:00